Package dev.hardwood.writer
package dev.hardwood.writer
Parquet file writer, with a columnar and a row-oriented API over one core.
ParquetFileWriter writes to a OutputFile, banding values into
size-bounded pages and row groups. Build the target schema with
FileSchema.builder(String) and pick the API that fits the caller:
ParquetFileWriter.columnWriter()returns aColumnWriter, which takes an aligned slice of typed arrays throughColumnBatch, for a caller that already holds columns.ParquetFileWriter.rowWriter()returns aRowWriter, which takes one record at a time throughStructBuilder,ListBuilderandMapBuilder, for a caller that holds records. It stages records into batches and submits them through the same core.
A file is written through one of the two, not both. WriterConfig carries the page and
row-group targets, the codec and the dictionary settings for either.
-
ClassDescriptionOne aligned slice of a file's columns.How a column's values are encoded, set file-wide or per leaf column through
WriterConfig.Builder.encoding(ColumnEncoding).Writes a Parquet file one aligned batch of columns at a time.Appends the entries of oneLISTinstance.Appends the entries of oneMAPinstance.Writes a Parquet file through a columnar or a row-oriented API:ParquetFileWriter.columnWriter()takes an aligned slice of typed arrays,ParquetFileWriter.rowWriter()takes one record at a time, and one file is written through one of the two, not both.WhatRowWriterdoes with a value carrying more precision than the column it is written to can hold — anInstantwith microseconds into aTIMESTAMP(MILLIS)column, aBigDecimalwith three decimals into aDECIMAL(_, 2)column.Writes a Parquet file record by record, over the same columnar coreColumnWriterexposes throughColumnWriter.writeBatch(Consumer).Sets the fields of one struct — a record being written, or a struct nested inside one.Tuning knobs forParquetFileWriter.Builder forWriterConfig.