Variant Columns

A Parquet column annotated with the VARIANT logical type carries semi-structured, JSON-like data in a self-describing binary encoding. Physically it is a group of two required BYTE_ARRAY children, metadata and value, whose bytes together define a Variant value with its own type tag (object, array, string, int, etc.). getVariant reads both children and surfaces them through the PqVariant API.

Try it yourself

Want to run it or explore the capabilities yourself? The Variant Columns example renders every Variant type through one recursive method, including nested objects and arrays.

try (RowReader rowReader = fileReader.rowReader()) {
    while (rowReader.hasNext()) {
        rowReader.next();
        PqVariant v = rowReader.getVariant("event");
        if (v == null) {
            continue;   // SQL NULL
        }

        // Type introspection
        VariantType tag = v.type();         // OBJECT, ARRAY, STRING, INT32, ...
        if (tag == VariantType.OBJECT) {
            PqVariantObject obj = v.asObject();
            String userId  = obj.getString("user_id");
            int    age     = obj.getInt("age");
            Instant ts     = obj.getTimestamp("ts");

            // Nested Variant OBJECT / ARRAY — same vocabulary all the way down
            PqVariantObject addr = obj.getObject("address");
            PqVariantArray  tags = obj.getArray("tags");
        }

        // Raw canonical bytes (for round-tripping or hashing)
        byte[] metadata = v.metadata();
        byte[] value    = v.value();
    }
}

The PqVariantObject view exposes the same primitive getters as a Parquet struct (getInt, getString, getTimestamp, …), but its complex navigation uses getObject and getArray (Variant-spec terminology) rather than getStruct / getList / getMap. A PqVariantArray is iterable and indexed; elements are heterogeneous PqVariants, each with its own type().

Primitive extraction on PqVariant: When you already hold a PqVariant (e.g. an array element) use the as*() methods — asInt, asString, asTimestamp, and so on. Each throws VariantTypeException if the variant's type tag doesn't match.

Timestamp tags: The Variant binary format carries four timestamp tags, split along the same isAdjustedToUTC boundary as the Parquet TIMESTAMP logical type. asTimestamp returns Instant and accepts the UTC-adjusted tags TIMESTAMP / TIMESTAMP_NANOS; asLocalTimestamp returns LocalDateTime and accepts the wall-clock tags TIMESTAMP_NTZ / TIMESTAMP_NTZ_NANOS. PqVariantObject.getTimestamp / getLocalTimestamp follow the same split.

Shredded Variants: Some writers store part of the payload in a typed sibling column (typed_value) alongside value. metadata() and value() return canonical bytes whether or not the file was shredded. A shredded Variant whose value and typed_value are both null reads as a PqVariant of type NULL; only a null Variant column reads as null. A typed_value column whose Parquet type the Variant shredding specification does not list (for example INT96 or TIMESTAMP(MILLIS)) makes building a row reader, or a column reader filtered on the Variant column, fail with an error.

Current limitations

• No Variant-aware predicate pushdown. Filter predicates against a Variant sub-path (e.g. WHERE v.age > 30) are not understood by the pushdown pipeline. The leaves below the group — metadata, value and a shredded variant's typed_value leaves — hold the encoded variant rather than values of their own, so a comparison or set predicate on one throws IllegalArgumentException at reader creation; they take isNull and isNotNull only. Filter on the Variant column itself for presence, and apply the value test per row against the PqVariant you read. Tracked as #309.
• No path projection optimization. Reading only v.age from a Variant column reassembles the whole Variant for each row rather than reading only the shredded typed_value.age column. Tracked as #700.