Class MultiFileColumnReaders

java.lang.Object
dev.hardwood.reader.MultiFileColumnReaders
All Implemented Interfaces:
AutoCloseable
public class MultiFileColumnReaders extends Object implements AutoCloseable
Holds multiple ColumnReader instances backed by a shared FileManager for cross-file prefetching across multiple Parquet files.

Usage:

try (Hardwood hardwood = Hardwood.create();
     MultiFileParquetReader parquet = hardwood.openAll(files);
     MultiFileColumnReaders columns = parquet.createColumnReaders(
         ColumnProjection.columns("passenger_count", "trip_distance", "fare_amount"))) {

    ColumnReader col0 = columns.getColumnReader("passenger_count");
    ColumnReader col1 = columns.getColumnReader("trip_distance");
    ColumnReader col2 = columns.getColumnReader("fare_amount");

    while (col0.nextBatch() & col1.nextBatch() & col2.nextBatch()) {
        int count = col0.getRecordCount();
        double[] v0 = col0.getDoubles();
        // ...
    }
}

  • Method Details

    • getColumnCount

      public int getColumnCount()
      Get the number of projected columns.

    • getColumnReader

      public ColumnReader getColumnReader(String columnName)
      Get the ColumnReader for a named column.
      Parameters:
      columnName - the column name (must have been requested in the projection)
      Returns:
      the ColumnReader for the column
      Throws:
      IllegalArgumentException - if the column was not requested

    • getColumnReader

      public ColumnReader getColumnReader(int index)
      Get the ColumnReader by index within the requested columns.
      Parameters:
      index - index within the requested column names (0-based)
      Returns:
      the ColumnReader at the given index

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable