Parquet-Java Compatibility

The hardwood-parquet-java-compat module runs code written against Apache parquet-java's ParquetReader<Group> API on Hardwood's reader. It implements the same org.apache.parquet.* interfaces and includes Hadoop shims (Path, Configuration), so no Hadoop dependency is required.

Experimental

The hardwood-parquet-java-compat module is experimental; its API surface and behavior may change in future releases without prior deprecation. This module is not available from Maven Central; build it from source.

Mutually exclusive with parquet-java

This module provides its own type shims in the org.apache.parquet.* namespace. It cannot be used alongside parquet-java on the same classpath; pick one or the other.

Reading Local Files

import org.apache.hadoop.fs.Path;
import org.apache.parquet.example.data.Group;
import org.apache.parquet.hadoop.GroupReadSupport;
import org.apache.parquet.hadoop.ParquetReader;

Path path = new Path("data.parquet");

try (ParquetReader<Group> reader = ParquetReader.builder(new GroupReadSupport(), path).build()) {
    Group record;
    while ((record = reader.read()) != null) {
        long id = record.getLong("id", 0);
        String name = record.getString("name", 0);

        Group address = record.getGroup("address", 0);
        String city = address.getString("city", 0);
    }
}

Reading from S3

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.parquet.hadoop.util.HadoopInputFile;

Configuration conf = new Configuration();
conf.set("fs.s3a.access.key", "...");
conf.set("fs.s3a.secret.key", "...");
conf.set("fs.s3a.endpoint", "https://s3.us-east-1.amazonaws.com");

Path path = new Path("s3a://my-bucket/data.parquet");

try (ParquetReader<Group> reader = ParquetReader.builder(new GroupReadSupport(), path)
        .withConf(conf)
        .build()) {
    // read as usual
}

HadoopInputFile reads the same fs.s3a.* configuration properties as parquet-java. S3 support requires hardwood-s3 on the classpath. The compat layer loads it via reflection; if it is missing, the error message names the dependency to add.

Filter Pushdown

Filters use the standard FilterApi / FilterCompat classes:

import static org.apache.parquet.filter2.predicate.FilterApi.*;
import org.apache.parquet.filter2.compat.FilterCompat;

FilterPredicate pred = and(
    gtEq(longColumn("id"), 100L),
    lt(doubleColumn("amount"), 500.0)
);

try (ParquetReader<Group> reader = ParquetReader.builder(new GroupReadSupport(), path)
        .withFilter(FilterCompat.get(pred))
        .build()) {
    // only matching rows are returned
}

in and notIn take a Set of values:

FilterPredicate pred = in(binaryColumn("country"), Set.of(Binary.fromString("DE"), Binary.fromString("FR")));

parquet-java Compat Filters lists the supported predicates, how a Binary literal converts on each column type, and how nulls are handled.