Class S3InputFile

java.lang.Object
dev.hardwood.s3.S3InputFile
All Implemented Interfaces:
InputFile, Closeable, AutoCloseable
public class S3InputFile extends Object implements InputFile

InputFile backed by an object in Amazon S3 (or an S3-compatible service).

Each readRange(long, int) call issues a signed HTTP GET request with a byte-range header, so only the requested bytes are transferred.

open() uses a suffix-range GET instead of a HEAD request. This discovers the file length from the Content-Range response header and pre-fetches the Parquet footer (which sits at the end of the file) in the same round-trip — eliminating a separate HEAD request per file.

When the owning S3Source is configured with RangeBacking.SPARSE_TEMPFILE, an internal mmap-backed range cache serves repeat reads of the same byte ranges without re-issuing HTTP requests. The cache is invisible at the API surface — the same counters (networkRequestCount, networkBytesFetched) reflect only network traffic in either mode.

Thread-safe once open() has been called.

  • Method Details

    • open

      public void open() throws IOException
      Description copied from interface: InputFile
      Performs expensive resource acquisition (e.g. memory-mapping, network connect). Must be called before InputFile.readRange(long, int) or InputFile.length().
      Specified by:
      open in interface InputFile
      Throws:
      IOException - if the resource cannot be acquired

    • readRange

      public ByteBuffer readRange(long offset, int length) throws IOException
      Description copied from interface: InputFile
      Read a range of bytes from the file.
      Specified by:
      readRange in interface InputFile
      Parameters:
      offset - the byte offset to start reading from
      length - the number of bytes to read
      Returns:
      a ByteBuffer containing the requested data
      Throws:
      IOException - if the read fails

    • length

      public long length()
      Description copied from interface: InputFile
      Returns the total size of the file in bytes.
      Specified by:
      length in interface InputFile
      Returns:
      the file size

    • name

      public String name()
      Description copied from interface: InputFile
      Returns an identifier for this file, used in log messages and JFR events.
      Specified by:
      name in interface InputFile
      Returns:
      a human-readable name or path

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException

    • networkRequestCount

      public long networkRequestCount()
      Number of HTTP requests issued against the object since open(). Counts the suffix-range tail fetch from open plus every network-fetch readRange(long, int) call. Tail-cache hits do not count.

    • networkBytesFetched

      public long networkBytesFetched()
      Number of bytes fetched from the network since open(). The tail fetch from open contributes its actual response size; each network-fetch readRange(long, int) contributes the requested length. Tail-cache hits do not count.