Class RowWriter

java.lang.Object
dev.hardwood.writer.RowWriter
@Experimental public final class RowWriter extends Object

Writes a Parquet file record by record, over the same columnar core ColumnWriter exposes through ColumnWriter.writeBatch(Consumer).

This is the write-side mirror of the reader's rowReader(): the ergonomic API for a caller that holds records rather than columns. Fields are addressed by the name they carry in the schema, nesting is entered with a filler per level, and logical-type values are written as the Java types the reader returns for them.

try (ParquetFileWriter writer = ParquetFileWriter.create(out, schema)) {
    RowWriter rows = writer.rowWriter();
    for (Person person : people) {
        rows.writeRow(row -> row
                .setLong("id", person.id())
                .setString("name", person.name())
                .setDate("hired", person.hired())
                .setStruct("address", address -> address.setString("city", person.city()))
                .setList("phones", phones -> person.phones().forEach(phones::addString)));
    }
}

Records are staged into a batch and submitted through the columnar path, so paging, row group cadence, dictionary encoding, compression and statistics are exactly those of a file written through ColumnWriter.writeBatch(Consumer). A batch is submitted once it holds enough records, or once its variable-width payload reaches the configured row-group size, so staging never exceeds one row group regardless of how large the values are.

A RowWriter is not closeable: the ParquetFileWriter it came from owns the file, and closing it writes the records still staged here along with the footer. ParquetFileWriter.endRowGroup() ends a row group after the records written so far, staged ones included.

This API is Experimental: the shape may change in future releases.

  • Method Details

    • writeRow

      public void writeRow(Consumer<StructBuilder> filler) throws IOException

      Writes one record.

      The writer creates the builder — bound to the schema — passes it to filler to be populated, then stages the record. A field the filler leaves unset is written as null if it is OPTIONAL, and fails the record if it is REQUIRED.

      Any exception fails the writer, whether the record is rejected, its filler throws, or a batch of staged records cannot be written: the writer accepts no more records, and ParquetFileWriter.close() discards the output. To skip a rejected record without failing the writer, use tryWriteRow(Consumer).

      Parameters:
      filler - populates the record
      Throws:
      IOException - if writing a completed batch fails
      IllegalArgumentException - if the filler names a field the schema does not have, sets one twice, uses a setter that does not fit a field's declared type, or leaves a REQUIRED field unset; or if a record of the batch this call completes has more values for a column than a column chunk can hold
      IndexOutOfBoundsException - if the filler addresses a field by an index the struct it is setting does not have
      IllegalStateException - if the writer is closed, or a previous write has failed

    • tryWriteRow

      public RowWriteResult tryWriteRow(Consumer<StructBuilder> filler) throws IOException

      Stages one record, or returns the rejection without failing the writer.

      A rejection of the record itself — a value the column cannot hold, or a REQUIRED field left unset or set null — is returned as RowWriteResult.Rejected and the staged batch is left as it was. RowWriteResult.Staged means the record is in the batch, not that it has been flushed: a later batch write can still fail the writer.

      An exception from filler, a builder used incorrectly, or a failure while a staged batch is written still fails the writer.

      Parameters:
      filler - populates the record
      Returns:
      RowWriteResult.Staged if the record was staged; RowWriteResult.Rejected if it was not
      Throws:
      IOException - if writing a completed batch fails
      IllegalArgumentException - if the filler names a field the schema does not have, sets one twice, or uses a setter that does not fit a field's declared type (any value setter on an UNKNOWN column included); or if a record of the batch this call completes has more values for a column than a column chunk can hold
      IndexOutOfBoundsException - if the filler addresses a field by an index the struct it is setting does not have
      IllegalStateException - if the writer is closed, or a previous write has failed