Database#

Open (or create) a database directory.

Parameters

path (str)

Filesystem path to the database directory.

create (bool, default False)

Open-or-create: make the directory if it does not exist.

read_only (bool, default False)

Reject every write at the handle level; write calls raise PolicyError.

Raises

NotFoundError

The directory does not exist and create is False.

Release the native handle. Idempotent, and also called by __exit__. In-flight operations on other threads finish normally; later calls on this object raise H5iError with code == "closed".

Whether the handle has been closed.

The directory this handle was opened on.

Create a table from an Arrow schema.

Parameters

name (str)

Table name, unique within the database.

schema (pyarrow.Schema)

The Arrow schema. Field order is preserved.

time_column (str, optional)

The time-axis column, strongly recommended for time-series tables. It enables segment pruning, ASOF joins, range plans, and tail, and is forced non-nullable.

sort_key (Iterable[str], optional)

Columns the table is sorted by on disk. Defaults to [time_column].

Returns

dict with creation metadata (table id, schema revision).

db.create_table("trades", schema, time_column="ts", sort_key=["ts", "symbol"])

Names of all tables in the database.

Schema of a table at a read point (latest by default).

Parameters

name (str)

Table name.

version (int, optional)

Read the schema as of this exact version.

as_of (str, optional)

RFC3339 timestamp; the schema as of the latest commit at or before it.

snapshot (str, optional)

Named snapshot to resolve the version from.

Committed versions, oldest first: one dict per version with the version number, operation, commit time, and row / byte / segment counts, plus any note.

Permanently drop the table and its data.

Raises

ConflictError

A snapshot pins the table; delete the snapshot first.

Strict ordered append.

Parameters

name (str)

Table name.

data (TableLike)

A pyarrow.Table, RecordBatch, or sequence of batches. Rows must respect the table's sort order.

expected_version (int, optional)

Optimistic guard: commit only if the head is exactly this version, else ConflictError. Use it when the append depends on what you last read.

note (str, optional)

Free-text note recorded in the version manifest.

Returns

dict with commit metadata (version, rows, bytes, segments).

Raises

InvalidInputError

sort_order_violation if rows are out of order, or schema_mismatch.

ConflictError

Another writer moved the head; retryable. Pure appends are retried internally (up to 5 times) before this surfaces.

Replace the table's contents in one commit. It is a restatement rather than an overwrite: the previous state stays readable as its own version. Parameters match append.

Make a historical version current by committing a new version with its contents. History only moves forward, and nothing is erased.

Parameters

name (str)

Table name.

version (int)

The version to restore.

Run SQL: full DataFusion plus the h5i extensions. Returns a QueryResult.

Parameters

query (str)

The SQL text.

memory_limit (int, optional)

Query memory budget in bytes; enables disk spilling under pressure.

timeout (float, optional)

Deadline in seconds. On expiry, raises TimeoutError and cancels execution.

max_rows (int, optional)

Raise LimitError as soon as the result exceeds this; execution stops early rather than truncating silently.

Returns

QueryResult, with .to_arrow(), .to_pandas(), .to_polars(), len().

df = db.sql(
    "SELECT * FROM h5i('trades', 42)", timeout=30, max_rows=1_000_000
).to_pandas()

Direct scan of one table version, with no SQL layer and minimal overhead.

Parameters

name (str)

Table name.

version / as_of / snapshot

Read point (latest by default); at most one. as_of is an RFC3339 string.

columns (list[str], optional)

Project to these columns.

time_start (int, optional)

Inclusive lower time bound, in raw time units (µs for timestamp[us]). Prunes segments before I/O.

time_end (int, optional)

Exclusive upper time bound, same units.

limit (int, optional)

Stop after this many rows.

timeout (float, optional)

Deadline in seconds.

Returns

pyarrow.Table

window = db.read("trades", columns=["ts", "price"],
                 time_start=t0_us, time_end=t1_us)

Look-ahead-bias diagnostic (the Python surface of the CLI arrival-delta). Runs query twice, against the current head (leaking: every commit, including rows that only became available after the decision instant) and against a decision read point (non-leaking), and returns the delta between the two results.

Parameters

query (str)

The SQL to evaluate under both read points.

version / as_of / snapshot

The decision point; exactly one is required. as_of is an RFC3339 string matched by commit availability time.

tolerance (float, optional)

Per-cell numeric noise floor below which a difference is ignored (default 1e-9).

Returns

dict with the arrival-delta report: changed, per-column head → asof (delta), max_abs_delta, row_count_differs, and withheld_versions (per table, the head-vs-as-of version gap).

Raises

InvalidInputError

No decision point was given, or more than one.

report = db.arrival_delta(
    "SELECT symbol, vwap(price, size) AS vwap FROM trades GROUP BY symbol",
    as_of="2026-07-01T16:00:00Z",
)
if report["changed"] and not report["vacuous"]:
    print("moved by:", report["max_abs_delta"])

Pin current table versions under a name. Address it later from SQL as h5i('t', 'name') or read(snapshot=…).

Parameters

name (str)

Snapshot name.

tables (list[str], optional)

Tables to pin. Defaults to all tables.

note (str, optional)

Free-text note.

Stage a previewable replacement of the half-open time range [start, end).

Parameters

name (str)

Table name.

start (int)

Inclusive range start, in raw time units (µs for timestamp[us]).

end (int)

Exclusive range end, same units.

data (TableLike, optional)

Replacement rows. Omit (or None) to delete the range.

note (str, optional)

Free-text note carried onto the resulting version.

Returns

MutationPlan; inspect .summary / .before_sample, then .apply().

Sugar for plan_replace_range(name, start, end, None, note), staging a range deletion.

Pending (not yet applied or discarded) plans for a table.

The mutation policy as a dict of boolean flags: direct_append, direct_write, direct_replace, direct_delete, direct_restore, direct_compact.

Update the mutation policy; unspecified flags keep their value. The merge is atomic (read-modify-write under the metadata lock).

Parameters

policy (dict, optional)

Flags to set, as a dict.

**flags (bool)

Flags to set, as keyword arguments, e.g. db.set_policy(direct_delete=False).

Returns

dict holding the merged policy that was stored.

Raises

InvalidInputError

An unknown flag name.

The table's data-safety policy as a dict, or None when unset.

Install (overwrite) a table's data-safety policy. Returns the stored policy.

Parameters

table (str)

Table name.

policy (dict)

A typed policy document. Predicates compose not_null, compare, and in_set with and / or / not; each constraint's on_fail is "reject" (fail the write) or "warn".

db.set_data_policy("trades", {"constraints": [
    {"name": "positive_price",
     "predicate": {"compare": {"column": "price", "op": "gt",
                               "value": {"float": 0.0}}},
     "on_fail": "reject"}]})

Raises

InvalidInputError

A malformed policy document, or (when a later write breaks a constraint) a data_policy_violation (the write is refused before it lands).

Remove a table's data-safety policy (writes become unconstrained).

Rewrite small segments into target-sized ones as a new version. It is a query-speed tool; old segments stay pinned by history.

Remove unreachable objects (crashed-writer debris, discarded plans). Committed history is never touched.

Parameters

table (str, optional)

Restrict to one table. Defaults to the whole database.

grace_seconds (int, default 3600)

Never touch objects newer than this; keep it above your longest ingest.

apply (bool, default False)

Actually delete. The default is a dry run.

Returns

dict holding the candidate (or deleted) object list.

Structural integrity check: checksum chain and object existence.

Parameters

name (str)

Table name.

deep (bool, default False)

Also re-read every segment and verify content checksums.

Returns

dict report; problems are listed in it rather than raised.