Python API#

The h5i_db package is an ergonomic wrapper over the native Rust engine. All tabular data crosses the boundary as Arrow, so it plugs directly into pyarrow, pandas, and Polars.

$ pip install h5i-db

The only required dependency is pyarrow >= 14. to_pandas() / to_polars() activate when pandas / Polars are installed.

The five-minute tour#

import pyarrow as pa
import h5i_db

db = h5i_db.Database("market.db", create=True)

schema = pa.schema([
    pa.field("ts", pa.timestamp("us", tz="UTC"), nullable=False),
    pa.field("symbol", pa.string()),
    pa.field("price", pa.float64()),
    pa.field("size", pa.int64()),
])
db.create_table("trades", schema, time_column="ts")

db.append("trades", table)                    # pyarrow Table / RecordBatch(es)
df = db.sql("SELECT * FROM trades").to_pandas()

old = db.read("trades", version=3)            # time travel
plan = db.plan_delete_range("trades", t0_us, t1_us)   # previewable mutation
plan.apply()                                  # or plan.discard()

db.close()                                    # or use `with h5i_db.Database(...) as db:`

The pieces#

CLASS Database Open/create databases; tables, ingest, SQL, time travel, snapshots, plans, policy, maintenance. CLASSES QueryResult & MutationPlan Result conversion to Arrow/pandas/Polars, and the preview → apply mutation flow. EXCEPTIONS Error types The typed hierarchy under H5iError, with .code, .hint, and .retryable on every error. TUTORIALS Cookbook 37 executed notebooks that put the API to work: fundamentals, market data, alpha research, and risk.

Data in, data out#

Everything tabular is Arrow:

In: write() / append() accept a pyarrow.Table, a RecordBatch, or a sequence of batches (TableLike). Coming from pandas or Polars: pa.Table.from_pandas(df) / pl_df.to_arrow().
Out: read() returns a pyarrow.Table; sql() returns a QueryResult with .to_arrow(), .to_pandas(), .to_polars().

Because the interchange is Arrow IPC, there is no per-row conversion cost and no type fidelity loss.

Error handling#

Every failure raises a subclass of h5i_db.H5iError carrying the same structured envelope the CLI prints: .code (stable string), .hint (what to try next), .retryable (whether a retry can help).

try:
    db.sql("SELECT * FROM h5i('trades')", timeout=30, max_rows=1_000_000)
except h5i_db.TimeoutError:
    ...                                   # raise the timeout or narrow the query
except h5i_db.LimitError as e:
    print(e.code)                         # "limit_exceeded"
except h5i_db.ConflictError:
    ...                                   # retryable: another writer won the race

See Exceptions for the full hierarchy and code table.

Versioning note#

h5i_db.__version__ reports the installed engine version. The pure-Python wrapper (Database, QueryResult, MutationPlan) sits on the private native module h5i_db._native; treat everything not exported from h5i_db itself as internal.