Class ColumnReaders

java.lang.Object
dev.hardwood.reader.ColumnReaders
All Implemented Interfaces:
Closeable, AutoCloseable
public class ColumnReaders extends Object implements Closeable

Holds multiple ColumnReader instances backed by one shared decode pipeline for batch-oriented projection reads. Works for both single- and multi-file ParquetFileReader inputs; the pipeline transparently handles cross-file prefetching when more than one file is involved.

nextBatch() advances every underlying reader in lockstep: a single call drives every reader, returns false when the readers are exhausted, and validates that the readers report matching record counts. The readers show the group's current batch; they do not advance or close on their own.

try (ParquetFileReader parquet = ParquetFileReader.openAll(files);
     ColumnReaders columns = parquet.buildColumnReaders(
             ColumnProjection.columns("passenger_count", "trip_distance", "fare_amount"))
             .build()) {

    while (columns.nextBatch()) {
        int count = columns.getRecordCount();
        long[] v0 = columns.getColumnReader(0).getLongs();
        double[] v1 = columns.getColumnReader(1).getDoubles();
        double[] v2 = columns.getColumnReader(2).getDoubles();
        // ...
    }
}

  • Method Details

    • getColumnCount

      public int getColumnCount()
      Get the number of requested columns, counting a column once for every name in the projection that selects it.

    • getColumnReader

      public ColumnReader getColumnReader(String columnName)
      Get the ColumnReader for a named column. For nested columns, use the dot-separated field path (e.g. "address.zip").
      Parameters:
      columnName - the column name or dot-separated field path (must have been requested in the projection)
      Returns:
      the ColumnReader for the column
      Throws:
      IllegalArgumentException - if the column was not requested, or the name is the dot-separated path of more than one requested column

    • getColumnReader

      public ColumnReader getColumnReader(int index)

      Get the ColumnReader by index within the requested columns.

      Indices follow the order the projection names its columns in, one index for every leaf column a name selects. A name that selects several columns, such as a group or ColumnProjection.all(), contributes them in schema order, so a column listed after a group in columns(...) sits after all of that group's leaf columns and moves when the group gains or loses a field. A column that several names select appears at each of their positions, with the same reader at each.

      Parameters:
      index - index among the requested leaf columns (0-based)
      Returns:
      the ColumnReader at the given index

    • nextBatch

      public boolean nextBatch() throws IOException

      Advance every underlying ColumnReader to its next batch in lockstep. This is the only way to advance the group: ColumnReader.nextBatch() on one of its readers throws IllegalStateException.

      All readers share one pipeline, so they always publish batches at the same row boundaries. This method advances that pipeline once and returns:

      • true when a new batch is available — callers can then read values via the per-column accessors. The aligned record count is exposed through getRecordCount().
      • false when the readers are exhausted — partial advancement is impossible because all readers consume from the shared pipeline, so once one is done they all are.

      As a defensive guard, a mismatch between the columns' decoded record counts throws IllegalStateException. Under correct internal behavior this can't happen — the guard exists to detect future regressions in the per-column drain workers, not to be triggered in production.

      Returns:
      true if a new aligned batch is available across all readers, false if exhausted
      Throws:
      IOException - if the bytes could not be read
      ParquetReadException - if the file's bytes are not what a Parquet file can say: a footer or a page index that will not parse, a dictionary page the metadata places outside its column chunk, a page whose checksum fails, values that do not decode under the encoding declared for them. In a multi-file read this covers a later file that is not Parquet at all, or whose schema cannot be reconciled with the first file's
      IllegalStateException - if the readers report mismatched record counts, or the group was closed

    • getRecordCount

      public int getRecordCount()

      Number of records in the group's current batch.

      Equal to every underlying reader's ColumnReader.getRecordCount() — alignment is validated when the group advances.

      Throws:
      IllegalStateException - if no batch is currently available — call nextBatch() first

    • close

      public void close() throws IOException
      Releases the resources held by every reader of this group, after which nextBatch() throws IllegalStateException. Idempotent.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException