Class MultiFileRowReader

java.lang.Object
dev.hardwood.reader.MultiFileRowReader
All Implemented Interfaces:
RowReader, StructAccessor, AutoCloseable
public class MultiFileRowReader extends Object

A RowReader that reads across multiple Parquet files with automatic file prefetching.

This reader uses a FileManager to handle file lifecycle and prefetching. The next file is automatically prepared while reading the current file, minimizing latency at file boundaries.

Usage:

try (Hardwood hardwood = Hardwood.create();
     MultiFileParquetReader parquet = hardwood.openAll(files);
     MultiFileRowReader reader = parquet.createRowReader()) {
    while (reader.hasNext()) {
        reader.next();
        // access data using same API as RowReader
    }
}

  • Field Details

    • dataView

      protected dev.hardwood.internal.reader.BatchDataView dataView

    • filterPredicate

      protected dev.hardwood.internal.predicate.ResolvedPredicate filterPredicate

    • projectedSchemaRef

      protected ProjectedSchema projectedSchemaRef

    • rowIndex

      protected int rowIndex

    • batchSize

      protected int batchSize

    • exhausted

      protected boolean exhausted

    • closed

      protected volatile boolean closed

    • initialized

      protected boolean initialized

    • maxRows

      protected long maxRows

  • Method Details

    • initialize

      protected void initialize()
      Ensures the reader is initialized. Called by metadata methods that may be invoked before iteration starts.

    • loadNextBatch

      protected boolean loadNextBatch()
      Loads the next batch of data.
      Returns:
      true if a batch was loaded, false if no more data

    • close

      public void close()

    • hasNext

      public boolean hasNext()
      Description copied from interface: RowReader
      Check if there are more rows to read.
      Specified by:
      hasNext in interface RowReader
      Returns:
      true if there are more rows available

    • next

      public void next()
      Description copied from interface: RowReader
      Advance to the next row. Must be called before accessing row data.
      Specified by:
      next in interface RowReader

    • getInt

      public int getInt(String name)
      Description copied from interface: StructAccessor
      Get an INT32 field value by name.
      Specified by:
      getInt in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the int value

    • getInt

      public int getInt(int columnIndex)
      Description copied from interface: RowReader
      Get an INT32 field value by field index.
      Specified by:
      getInt in interface RowReader

    • getLong

      public long getLong(String name)
      Description copied from interface: StructAccessor
      Get an INT64 field value by name.
      Specified by:
      getLong in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the long value

    • getLong

      public long getLong(int columnIndex)
      Description copied from interface: RowReader
      Get an INT64 field value by field index.
      Specified by:
      getLong in interface RowReader

    • getFloat

      public float getFloat(String name)
      Description copied from interface: StructAccessor
      Get a FLOAT field value by name.
      Specified by:
      getFloat in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the float value

    • getFloat

      public float getFloat(int columnIndex)
      Description copied from interface: RowReader
      Get a FLOAT field value by field index.
      Specified by:
      getFloat in interface RowReader

    • getDouble

      public double getDouble(String name)
      Description copied from interface: StructAccessor
      Get a DOUBLE field value by name.
      Specified by:
      getDouble in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the double value

    • getDouble

      public double getDouble(int columnIndex)
      Description copied from interface: RowReader
      Get a DOUBLE field value by field index.
      Specified by:
      getDouble in interface RowReader

    • getBoolean

      public boolean getBoolean(String name)
      Description copied from interface: StructAccessor
      Get a BOOLEAN field value by name.
      Specified by:
      getBoolean in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the boolean value

    • getBoolean

      public boolean getBoolean(int columnIndex)
      Description copied from interface: RowReader
      Get a BOOLEAN field value by field index.
      Specified by:
      getBoolean in interface RowReader

    • getString

      public String getString(String name)
      Description copied from interface: StructAccessor
      Get a STRING field value by name.
      Specified by:
      getString in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the string value, or null if the field is null

    • getString

      public String getString(int columnIndex)
      Description copied from interface: RowReader
      Get a STRING field value by field index.
      Specified by:
      getString in interface RowReader
      Returns:
      the string value, or null if the field is null

    • getBinary

      public byte[] getBinary(String name)
      Description copied from interface: StructAccessor
      Get a BINARY field value by name.
      Specified by:
      getBinary in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the byte array, or null if the field is null

    • getBinary

      public byte[] getBinary(int columnIndex)
      Description copied from interface: RowReader
      Get a BINARY field value by field index.
      Specified by:
      getBinary in interface RowReader
      Returns:
      the binary value, or null if the field is null

    • getDate

      public LocalDate getDate(String name)
      Description copied from interface: StructAccessor
      Get a DATE field value by name.
      Specified by:
      getDate in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the date value, or null if the field is null

    • getDate

      public LocalDate getDate(int columnIndex)
      Description copied from interface: RowReader
      Get a DATE field value by field index.
      Specified by:
      getDate in interface RowReader
      Returns:
      the date value, or null if the field is null

    • getTime

      public LocalTime getTime(String name)
      Description copied from interface: StructAccessor
      Get a TIME field value by name.
      Specified by:
      getTime in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the time value, or null if the field is null

    • getTime

      public LocalTime getTime(int columnIndex)
      Description copied from interface: RowReader
      Get a TIME field value by field index.
      Specified by:
      getTime in interface RowReader
      Returns:
      the time value, or null if the field is null

    • getTimestamp

      public Instant getTimestamp(String name)
      Description copied from interface: StructAccessor
      Get a TIMESTAMP field value by name.
      Specified by:
      getTimestamp in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the instant value, or null if the field is null

    • getTimestamp

      public Instant getTimestamp(int columnIndex)
      Description copied from interface: RowReader
      Get a TIMESTAMP field value by field index.
      Specified by:
      getTimestamp in interface RowReader
      Returns:
      the timestamp value, or null if the field is null

    • getDecimal

      public BigDecimal getDecimal(String name)
      Description copied from interface: StructAccessor
      Get a DECIMAL field value by name.
      Specified by:
      getDecimal in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the decimal value, or null if the field is null

    • getDecimal

      public BigDecimal getDecimal(int columnIndex)
      Description copied from interface: RowReader
      Get a DECIMAL field value by field index.
      Specified by:
      getDecimal in interface RowReader
      Returns:
      the decimal value, or null if the field is null

    • getUuid

      public UUID getUuid(String name)
      Description copied from interface: StructAccessor
      Get a UUID field value by name.
      Specified by:
      getUuid in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the UUID value, or null if the field is null

    • getUuid

      public UUID getUuid(int columnIndex)
      Description copied from interface: RowReader
      Get a UUID field value by field index.
      Specified by:
      getUuid in interface RowReader
      Returns:
      the UUID value, or null if the field is null

    • getStruct

      public PqStruct getStruct(String name)
      Description copied from interface: StructAccessor
      Get a nested struct field value by name.
      Specified by:
      getStruct in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the nested struct, or null if the field is null

    • getListOfInts

      public PqIntList getListOfInts(String name)
      Description copied from interface: StructAccessor
      Get an INT32 list field by name.
      Specified by:
      getListOfInts in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the int list, or null if the field is null

    • getListOfLongs

      public PqLongList getListOfLongs(String name)
      Description copied from interface: StructAccessor
      Get an INT64 list field by name.
      Specified by:
      getListOfLongs in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the long list, or null if the field is null

    • getListOfDoubles

      public PqDoubleList getListOfDoubles(String name)
      Description copied from interface: StructAccessor
      Get a DOUBLE list field by name.
      Specified by:
      getListOfDoubles in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the double list, or null if the field is null

    • getList

      public PqList getList(String name)
      Description copied from interface: StructAccessor
      Get a LIST field value by name.
      Specified by:
      getList in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the list, or null if the field is null

    • getMap

      public PqMap getMap(String name)
      Description copied from interface: StructAccessor
      Get a MAP field value by name.
      Specified by:
      getMap in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the map, or null if the field is null

    • getStruct

      public PqStruct getStruct(int columnIndex)
      Description copied from interface: RowReader
      Get a nested struct field value by field index.
      Specified by:
      getStruct in interface RowReader
      Returns:
      the struct value, or null if the field is null

    • getListOfInts

      public PqIntList getListOfInts(int columnIndex)
      Description copied from interface: RowReader
      Get an INT32 list field by field index.
      Specified by:
      getListOfInts in interface RowReader
      Returns:
      the list, or null if the field is null

    • getListOfLongs

      public PqLongList getListOfLongs(int columnIndex)
      Description copied from interface: RowReader
      Get an INT64 list field by field index.
      Specified by:
      getListOfLongs in interface RowReader
      Returns:
      the list, or null if the field is null

    • getListOfDoubles

      public PqDoubleList getListOfDoubles(int columnIndex)
      Description copied from interface: RowReader
      Get a DOUBLE list field by field index.
      Specified by:
      getListOfDoubles in interface RowReader
      Returns:
      the list, or null if the field is null

    • getList

      public PqList getList(int columnIndex)
      Description copied from interface: RowReader
      Get a LIST field value by field index.
      Specified by:
      getList in interface RowReader
      Returns:
      the list, or null if the field is null

    • getMap

      public PqMap getMap(int columnIndex)
      Description copied from interface: RowReader
      Get a MAP field value by field index.
      Specified by:
      getMap in interface RowReader
      Returns:
      the map, or null if the field is null

    • getValue

      public Object getValue(String name)
      Description copied from interface: StructAccessor
      Get a field value by name without type conversion. Returns the raw value as stored internally.
      Specified by:
      getValue in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      the raw value, or null if the field is null

    • getValue

      public Object getValue(int columnIndex)
      Description copied from interface: RowReader
      Get a field value by field index without type conversion.
      Specified by:
      getValue in interface RowReader
      Returns:
      the raw value, or null if the field is null

    • isNull

      public boolean isNull(String name)
      Description copied from interface: StructAccessor
      Check if a field is null by name.
      Specified by:
      isNull in interface StructAccessor
      Parameters:
      name - the field name
      Returns:
      true if the field is null

    • isNull

      public boolean isNull(int columnIndex)
      Description copied from interface: RowReader
      Check if a field is null by field index.
      Specified by:
      isNull in interface RowReader

    • getFieldCount

      public int getFieldCount()
      Description copied from interface: StructAccessor
      Get the number of fields in this struct.
      Specified by:
      getFieldCount in interface StructAccessor
      Returns:
      the field count

    • getFieldName

      public String getFieldName(int index)
      Description copied from interface: StructAccessor
      Get the name of a field by index.
      Specified by:
      getFieldName in interface StructAccessor
      Parameters:
      index - the field index (0-based)
      Returns:
      the field name