Batch runner¶

sheaf.batch ¶

Offline batch inference — JSONL → JSONL via Ray Data.

Public surface::

from sheaf.batch import BatchRunner, BatchSpec, JsonlSource, JsonlSink

Install with: pip install 'sheaf-serve[batch]'

BatchRunner ¶

BatchRunner(spec: BatchSpec)

Offline batch inference over a BatchSpec source → sink pipeline.

Example

spec = BatchSpec( name="chronos-batch", model_type=ModelType.TIME_SERIES, backend="chronos2", backend_kwargs={"model_size": "small"}, source=JsonlSource(path="in.jsonl"), sink=JsonlSink(path="out.jsonl"), batch_size=64, ) BatchRunner(spec).run()

run ¶

run() -> int

Run the batch job end-to-end. Returns the number of output rows.

BatchSpec ¶

Bases: BaseModel

Declares an offline batch inference job.

Example

spec = BatchSpec( name="chronos-batch", model_type=ModelType.TIME_SERIES, backend="chronos2", backend_kwargs={"model_size": "small"}, source=JsonlSource(path="inputs.jsonl"), sink=JsonlSink(path="outputs.jsonl"), batch_size=64, ) BatchRunner(spec).run()

Fields backend / backend_cls / backend_kwargs follow the same semantics as on ModelSpec: backend_cls takes precedence over the registry lookup when set, letting callers pass a custom or test-only backend class directly.

JsonlSink ¶

Bases: BatchSink

JSONL file output — one response dict per line.

The file is overwritten if it already exists. Output rows are written in the same order as input rows.

JsonlSource ¶

Bases: BatchSource

JSONL file input — one request dict per line.