ArchiveLimits

Configure resource safety limits for creation, opening, validation, and unpacking.

ArchiveLimits defines the resource budget allowed when OCTX creates, opens, validates, and safely unpacks a Package. Every primary entry point accepts this immutable configuration object.

Basic usage

from octx import ArchiveLimits, validate_octx

limits = ArchiveLimits(
    max_entries=2_000,
    max_file_size=64 * 1024 * 1024,
    max_total_uncompressed=256 * 1024 * 1024,
    max_jsonl_line_size=2 * 1024 * 1024,
    max_jsonl_records=100_000,
    max_arrow_batches=10_000,
    max_arrow_rows=1_000_000,
    max_arrow_values=20_000_000,
    max_issues=100,
)

report = validate_octx("upload.octx", limits=limits)

The same object can be passed to create_octx(), open_octx(), validate_octx(), and unpack_octx().

Configurable fields

Field	Default	What it limits
`max_entries`	10,000	Total number of entries in a ZIP file or directory.
`max_file_size`	512 MiB	A single file and Arrow metadata.
`max_total_uncompressed`	4 GiB	Total uncompressed size and Arrow decoded bytes.
`max_compression_ratio`	200	Compression ratio of a single ZIP entry.
`max_jsonl_line_size`	16 MiB	Byte size of a single JSONL record.
`max_jsonl_records`	1,000,000	Number of records in each JSONL file.
`max_json_depth`	100	JSON nesting depth.
`max_yaml_depth`	100	YAML frontmatter nesting depth.
`max_arrow_dimension`	65,536	Dimension of a single vector.
`max_arrow_batches`	100,000	Total number of Arrow DictionaryBatch and RecordBatch messages.
`max_arrow_rows`	10,000,000	Total number of Arrow rows.
`max_arrow_values`	100,000,000	Total number of scalar values in Arrow vectors.
`max_issues`	1,000	Number of issues in a validation report.

Usage guidelines

Every limit must be positive. Services that process uploaded content should set smaller values according to their available memory, disk space, and request budget. Do not treat the defaults as the scale that every production system must accept.