Class Hardwood

java.lang.Object

dev.hardwood.Hardwood

All Implemented Interfaces:

AutoCloseable

public class Hardwood
extends Object
implements AutoCloseable

Entry point for reading Parquet files with a shared thread pool.

Use this when reading multiple files to share the executor across readers:

try (Hardwood hardwood = Hardwood.create()) {
    ParquetFileReader file1 = hardwood.open(InputFile.of(path1));
    ParquetFileReader file2 = hardwood.open(InputFile.of(path2));
    // ...
}

To control the decode parallelism, or to reuse one context (and its thread pool) across many reads and standalone ParquetFileReaders, create the HardwoodContext yourself and pass it in:

try (HardwoodContext context = HardwoodContext.create(4)) {
    try (Hardwood hardwood = Hardwood.create(context)) {
        // ...
    }
    // context is still open here for further use
}

For single-file usage, ParquetFileReader.open(InputFile) is simpler.

Method Summary

Modifier and Type	Method	Description
void	close()
static Hardwood	create()	Create a new Hardwood instance with a thread pool sized to available processors.
static Hardwood	create(HardwoodContext context)	Create a new Hardwood instance backed by the given context, e.g. one created via HardwoodContext.create(int) to size the decode thread pool.
ParquetFileReader	open(InputFile inputFile)	Open a single Parquet file.
ParquetFileReader	openAll(List<? extends InputFile> inputFiles)	Open multiple Parquet files for reading with cross-file prefetching.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

create

public static Hardwood create()

Create a new Hardwood instance with a thread pool sized to available processors. The context is owned by this instance and closed when it is closed.

create

public static Hardwood create(HardwoodContext context)

Create a new Hardwood instance backed by the given context, e.g. one created via HardwoodContext.create(int) to size the decode thread pool.

The caller retains ownership of the context: it is not closed when this instance is closed, so the same context — and its thread pool — can be reused for later reads and shared with standalone ParquetFileReaders opened via ParquetFileReader.open(InputFile, HardwoodContext).

open

public ParquetFileReader open(InputFile inputFile)
                       throws IOException

Open a single Parquet file. The file is opened immediately and closed when the returned reader is closed, or before the exception propagates when opening fails.

Throws:

IOException

openAll

public ParquetFileReader openAll(List<? extends InputFile> inputFiles)
                          throws IOException

Open multiple Parquet files for reading with cross-file prefetching. The schema is read from the first file. Files are opened on demand by the iterator. All files are closed when the returned reader is closed, or before the exception propagates when opening fails.

Parameters:

inputFiles - the input files to read (must not be empty)

Throws:

IOException - if the first file cannot be opened or read

IllegalArgumentException - if the list is empty

close

public void close()

Specified by:

close in interface AutoCloseable