Skip to content

S3 Reference

Look-it-up reference for the hardwood-s3 module. For worked examples — credentials, S3-compatible endpoints, and multi-file reads — see Reading from S3.

S3Source builder options

Option Default Effect
region(String) — AWS region. Required unless endpoint is set.
endpoint(String) — Custom endpoint for S3-compatible services (R2, GCS HMAC, MinIO). Region may be omitted when set.
pathStyle(boolean) false Path-style addressing, for services that require it (MinIO, SeaweedFS).
credentials(...) — Static S3Credentials, an S3CredentialsProvider, or the AWS chain via hardwood-aws-auth. Required.
connectTimeout(Duration) 10s Maximum time to establish a TCP connection. Ignored when a custom httpClient is supplied.
requestTimeout(Duration) 30s Maximum time for an individual HTTP request to complete.
maxRetries(int) 3 GET retries on transient failures (HTTP 500, 503, network errors) with exponential backoff and jitter. 0 disables retries.
httpClient(HttpClient) internal Caller-supplied client for full transport control. The caller is responsible for closing it — S3Source.close() will not.

When the retry budget is exhausted, the last failure is re-thrown as an IOException. For HTTP errors the message has the form GET s3://<bucket>/<key> failed: HTTP <status>; for network errors the original IOException message is preserved.

Fetch-cost counters

Each S3InputFile tracks the network I/O it has performed since open():

  • networkRequestCount() — HTTP GET requests issued: the footer tail fetch plus every column-chunk range fetch. Tail-cache hits are not counted.
  • networkBytesFetched() — total bytes fetched across those requests, counted the same way.

Read the counters before the try-with-resources block exits, while the S3InputFile is still open.

Not currently supported

  • Anonymous (unsigned) requests — S3Credentials requires an access key + secret key pair; there is no built-in mode for public buckets without credentials.
  • Requester-pays buckets — Hardwood does not send the x-amz-request-payer header. Reads against requester-pays buckets will fail with 403 AccessDenied even with valid credentials.