Notebooks#

h5i-db nb is a notebook client for the terminal: it owns a real .ipynb file, drives a real Jupyter kernel, and renders results for two audiences at once — a human in a TUI, and a program reading a token-budgeted digest on stdout. The same commands are available on a standalone binary, h5i-nb, which drives the same sessions.

The property everything else follows from: the kernel persists between invocations. exec twice and the second cell sees what the first defined, so a forty-second load is paid once instead of once per idea. If you would run a script once and never again, run the script; this is for the case where state accumulates.

$ h5i-db nb new research.ipynb --kernel python3 --db market.db
$ h5i-db nb exec research.ipynb --code "import pandas as pd; df = pd.read_parquet('big.parquet')"
$ h5i-db nb exec research.ipynb --code "df.shape"          # df is still there

The file is nbformat v4.5 and stays readable in JupyterLab. Writes are atomic (temp file, fsync, rename) and byte-compatible with nbformat's own writer, so opening a notebook in JupyterLab does not churn the diff.

Running cells#

exec appends a cell and runs it. run re-runs cells the notebook already has.

$ h5i-db nb exec research.ipynb --code "edge(book)"
$ h5i-db nb exec research.ipynb --code - <<'PY'
def edge(book):
    return book.bid - book.ask
PY
$ h5i-db nb run research.ipynb --cells 3-7
$ h5i-db nb run research.ipynb --from-clean

--code - (the default) reads the cell from stdin, which is how code containing quotes and newlines gets in without fighting the shell.

run --from-clean is the reproducibility check: if it passes, the notebook tells the truth about what produced its outputs.

Output is budgeted, and nothing is lost#

Cell output is summarised on stdout, not dumped:

• Text is elided in the middle: head lines, tail lines, and a … 4,912 lines elided … marker between them.
• Frames are re-rendered as a compact aligned table — first and last rows plus a [10,000 rows x 12 columns] shape line. For the next decision, the shape is usually the whole information content.
• Images are never inlined as base64. They are written to <notebook>_files/cell-<id>-<n>.png (nbconvert's convention, so JupyterLab renders them too) and reported as a path.
• Tracebacks are printed in full, ANSI stripped. A tool that discards the traceback on error forces a re-run to find out what broke.

The untruncated output stays in the .ipynb. --raw prints it, and nb output --save <path> writes a binary output to a file.

$ h5i-db nb cells research.ipynb          # index, id, type, exec count, output shape
 #  id        type  exec  outputs                                           source
 0  a7822631  code  1     result(application/vnd.h5i.table+text+text/html)  %%sql
 1  5afe1a3f  code  2     -                                                 x = 41
 2  e00300f7  code  3     result                                            x + 1

$ h5i-db nb output research.ipynb 2            # index or cell id
$ h5i-db nb output research.ipynb 2 --index 1  # one output, when a cell made several
$ h5i-db nb output research.ipynb 2 --raw

Reach for --raw when you need the bytes. The digest is what makes a notebook cost less context than a script, and --raw gives that back.

%%sql: querying without an interpreter#

A cell whose first line is %%sql runs against an h5i-db database in-process. The magic is resolved by the client, not by the kernel, so the statement never crosses into Python: no interpreter, no driver round trip. Schema discovery and the first twenty exploratory queries need no kernel at all.

$ h5i-db nb exec research.ipynb --code - <<'SQL'
%%sql
SELECT symbol, count(*) AS n FROM trades GROUP BY symbol ORDER BY n DESC
SQL
[1] cell 0 · ok · 3ms
+--------+-----+
| symbol | n   |
+--------+-----+
| AAPL   | 120 |
| MSFT   | 120 |
+--------+-----+
[2 rows x 2 columns]

h5i-db nb new … --db market.db records a notebook-wide default in the file's own metadata, so it travels with the notebook and the %%sql cells need no --db.

Two options are worth reading twice:

• --into binds the full result, not the rendered one. The handoff is Arrow through a temp file, so --max-rows 1 --into frame prints one row and still binds all of them.

console $ h5i-db nb exec research.ipynb --code - <<'SQL' %%sql --into frame --max-rows 1 SELECT symbol, price FROM trades LIMIT 5 SQL [5] cell 4 · ok · 1ms stderr: note: kept the first 1 rows of a larger result (--max-rows) … frame: pandas.DataFrame 5 rows x 2 columns

• --write is required before a cell can modify anything. Without it the database is opened read-only, which also means opening it never runs transaction recovery against a database another process may be using. Add it deliberately, not by habit.

The syntax deliberately matches IPython's %%sql, so the notebook still reads correctly in JupyterLab even though JupyterLab would run the cell a different way.

Long-running cells#

--detach returns as soon as the cell is queued. The session process owns the notebook, so outputs are recorded whether or not anyone is still attached — including the failure, if it fails.

$ h5i-db nb exec research.ipynb --detach --code "backtest(years=10)"
$ h5i-db nb cells research.ipynb              # poll, about once a second
$ h5i-db nb output research.ipynb 7

There is never a state where "still running" and "died" look the same.

$ h5i-db nb kernel interrupt research.ipynb   # stop the cell, keep the state
$ h5i-db nb kernel restart research.ipynb     # throw the state away
$ h5i-db nb kernel status research.ipynb      # answers even mid-cell

Editing without running#

$ h5i-db nb edit research.ipynb set 3 --code "fixed()"
$ h5i-db nb edit research.ipynb insert --at 2 --kind markdown --code "## Findings"
$ h5i-db nb edit research.ipynb delete 4
$ h5i-db nb edit research.ipynb move 4 1
$ h5i-db nb edit research.ipynb clear-outputs

set drops the cell's now-stale outputs. insert appends when --at is omitted and takes --kind code|markdown|raw (default code). Edits are routed through the running session, so they are safe while a kernel is alive.

Showing a human what you are doing#

$ h5i-db nb view research.ipynb                # editable TUI
$ h5i-db nb watch research.ipynb --split right # live, read-only
$ h5i-db nb ls                                 # sessions running on this machine

view is the full terminal UI: Jupyter's key bindings (Esc/Enter for command and edit mode, a/b to insert, dd to delete), a completion popup from complete_request, an inspection overlay from inspect_request, and inline images through the kitty and iTerm2 graphics protocols. Ctrl-C interrupts the running cell rather than killing the UI, because the notebook is the state.

watch is a live read-only view. It writes nothing, holds no lock, and starts no kernel, so any number of them can follow a session someone else is driving; from it a human can interrupt the cell and nothing else. Both take --split right|left|down|up, which opens the view in a new pane beside the current one (tmux, zellij, WezTerm, kitty) and returns immediately.

ls reports every notebook session on the machine with its kernel, state, cell count, pid and idle time, and clears up after sessions whose supervisor died.

Exporting#

$ h5i-db nb export research.ipynb --to md      # a readable summary for a PR
$ h5i-db nb export research.ipynb --to py      # a runnable script, magics commented out
$ h5i-db nb export research.ipynb --to html    # one self-contained file, images inlined
$ h5i-db nb export research.ipynb --to md -o notes.md --without-outputs

--without-outputs leaves the outputs out, for a clean diff or a script meant to be run rather than read.

Introspection from a shell#

The two requests the TUI uses are also commands, which is what lets an editor or a script ask the kernel what it knows:

$ h5i-db nb inspect research.ipynb "pd.read_parquet"    # what `?` does in IPython
$ h5i-db nb complete research.ipynb "df.gro"            # completions at the cursor

Both take --cursor <offset> when the cursor is not at the end of the code.

Errors#

Failures print the usual structured envelope on stderr. Branch on code, not on the wording.

cell_raised is the ordinary outcome of exploring, not a tool failure. The traceback is the answer.

What does not render#

ipywidgets do not draw. Comm traffic is accepted so a widget-producing library does not crash the session, but nothing interactive appears. Markdown in exported HTML is shown as preformatted text rather than parsed. Syntax highlighting is a per-line lexer, so a multi-line string highlights only on its first line.