Enum Class ColumnEncoding

java.lang.Object
java.lang.Enum<ColumnEncoding>
dev.hardwood.writer.ColumnEncoding
All Implemented Interfaces:
Serializable, Comparable<ColumnEncoding>, Constable
public enum ColumnEncoding extends Enum<ColumnEncoding>

How a column's values are encoded, set file-wide or per leaf column through WriterConfig.Builder.encoding(ColumnEncoding).

AUTO leaves the choice to the writer, which measures it per column chunk. Every other member names an encoding outright, and a column that names one builds no dictionary: its chunks carry that encoding in every row group of the file.

An encoding is legal only for some physical types, and a policy naming a column whose type cannot carry it fails when the writer is created rather than being quietly replaced:

Policy Physical types
AUTO every writable type
PLAIN every writable type
DELTA_BINARY_PACKED INT32, INT64
DELTA_LENGTH_BYTE_ARRAY BYTE_ARRAY
DELTA_BYTE_ARRAY BYTE_ARRAY, FIXED_LEN_BYTE_ARRAY
BYTE_STREAM_SPLIT INT32, INT64, FLOAT, DOUBLE, FIXED_LEN_BYTE_ARRAY

No policy demands a dictionary: dictionary encoding is an outcome AUTO may arrive at, not something to ask for.

  • Enum Constant Details

    • AUTO

      public static final ColumnEncoding AUTO

      Let the writer choose, per column chunk, between a dictionary and PLAIN.

      The chunk is dictionary-encoded where the dictionary page plus an index stream is smaller than the values written PLAIN, and PLAIN otherwise. The comparison is made once the row group is buffered, from the values the chunk actually holds, so one column may be dictionary-encoded in one row group and PLAIN in the next.

    • PLAIN

      public static final ColumnEncoding PLAIN
      Values as they are: fixed-width types little-endian, BYTE_ARRAY behind a 4-byte length prefix. Always available, never the smallest for data with structure to exploit.

    • DELTA_BINARY_PACKED

      public static final ColumnEncoding DELTA_BINARY_PACKED
      Differences between neighbouring values, bit-packed against the smallest difference in each block. Suits sorted or slow-moving integers — timestamps, identifiers, counters — and is larger than PLAIN on unordered data.

    • DELTA_LENGTH_BYTE_ARRAY

      public static final ColumnEncoding DELTA_LENGTH_BYTE_ARRAY
      The lengths delta-encoded ahead of the values, which is cheaper than PLAIN's per-value 4-byte length prefix wherever the lengths are similar.

    • DELTA_BYTE_ARRAY

      public static final ColumnEncoding DELTA_BYTE_ARRAY
      Each value carrying only what it does not share with the value before it. Suits sorted values with common starts — paths, URLs, keys — which are what a dictionary serves worst.

    • BYTE_STREAM_SPLIT

      public static final ColumnEncoding BYTE_STREAM_SPLIT
      A value's bytes scattered across one stream per byte position. This changes no page's size on its own; what it changes is how well the codec afterwards compresses floating point data, so it is worth choosing only alongside compression.

  • Method Details

    • values

      public static ColumnEncoding[] values()
      Returns an array containing the constants of this enum class, in the order they are declared.
      Returns:
      an array containing the constants of this enum class, in the order they are declared

    • valueOf

      public static ColumnEncoding valueOf(String name)
      Returns the enum constant of this class with the specified name. The string must match exactly an identifier used to declare an enum constant in this class. (Extraneous whitespace characters are not permitted.)
      Parameters:
      name - the name of the enum constant to be returned.
      Returns:
      the enum constant with the specified name
      Throws:
      IllegalArgumentException - if this enum class has no constant with the specified name
      NullPointerException - if the argument is null