RFC: Dated Futures and Options Internal Reforms (widening `InstrumentId`)

Date: 2026-06-10 Revised: 2026-06-23 — widen the existing InstrumentId to u64 in place and introduce a new ProductId for the umbrella. (Prior draft renamed InstrumentId → ProductId and added a separate ContractId; this revision keeps InstrumentId as the tradable-contract identity and adds only one new type.)

Background

DB and SDK support for dated futures exists (instruments.expiration in db/postgres/1.sql), but we still cannot list a dated future without ceremony. The blocker is how instruments map to TigerBeetle:

InstrumentId (rs/sdk-internal/src/instrument_id.rs) is an odd u32 that is the TigerBeetle ledger number. Each instrument consumes a ledger pair: the odd ID is the position-quantity ledger, ID+1 (even) is the basis ledger.
The symbol↔︎ID mapping is a compile-time table in rs/sdk-internal/src/constants.rs (22 perps today). Listing anything new means editing the table, recompiling, and redeploying every consumer.
TB account IDs are derived as user_ulid | ledger_id (UserId::derive_tb_account_id), i.e. [user 64b | zero 32b | ledger 32b]. Bits 63–32 were once earmarked for subaccounts; that plan is dropped and the bits are free.

For monthly futures (e.g. COMPUTEDESK-H100-JAN-2026), minting one InstrumentId per contract month means a code change and deploy per listing, unbounded growth of dead ledger pairs (TB never deletes accounts or ledgers), and — because TigerBeetle transfers must stay within one ledger — calendar contracts of the same product that can never transact atomically with each other (no atomic rolls, no spread settlement).

Recycling a fixed set of IDs (e.g. 12 monthly slots) was considered and rejected: TB account history is immortal, so reuse aliases two different economic contracts onto the same accounts, and customer-facing symbol reuse without a year is ambiguous on statements and trade confirms.

Goal

Listing a dated contract is pure data: a Postgres row plus an EP3 instrument. No sdk-internal change, no TigerBeetle provisioning, no deploy.
Contract identity is unique forever — no recycling, no aliasing.
Existing perpetual accounts in TigerBeetle remain bit-identical — zero data migration.
All invariants in risk-engine2/INVARIANTS.md remain enforced (G1–G3, U1–U5, P1–P4), including across restart/recovery.

Non-goals

Options. This RFC only reserves the scheme-tag bit and fixes the layout rule that keeps the options design open (see Bit layout).
The expiry settlement engine (forcing contracts flat at expiration). It is required before the first dated listing and sketched in Companion work, but specified separately.

Design

Naming: keep `InstrumentId` for the contract; add `ProductId`

Today's InstrumentId collapses two concepts that dated contracts must separate: the product (the umbrella — COMPUTEDESK-H100) and the tradable contract (COMPUTEDESK-H100-JAN-2026). We keep the name InstrumentId for the tradable contract and introduce a new ProductId for the umbrella.

This matches industry convention rather than fighting it: in FIX, the Instrument component is the specific tradable contract (maturity, strike, put/call are instrument-level fields); CME's vocabulary is product (ES, CL) vs contract (ESH6); EP3 and our own per-contract instruments table follow suit. So the tradable contract keeping the name InstrumentId is correct, and the umbrella — which AX has never had a type for — becomes ProductId. (UnderlyingId was considered and rejected for the umbrella — the underlying of COMPUTEDESK-H100-JAN-2026 is the H100 rental index; COMPUTEDESK-H100 is the product.)

Mechanically the move is small: InstrumentId widens from u32 to u64 in place — the perpetual case is date = 0, so today's odd-u32 values are numerically unchanged (see below) — and the odd u32 it used to be, the TigerBeetle ledger number, is lifted out into the new ProductId. Map keys (BTreeMap<InstrumentId, _>) keep their name; only the genuinely product-level sites (ledger derivation, the products table, the odd/even parity discriminator) pick up ProductId.

EP3 already draws this line (and where ours diverged)

EP3 models exactly this split. In connamara.ep3.instruments.v1beta1, an EP3 Product is the umbrella — id, attributes, is_terminated, and a type_attributes oneof (FutureAttributes, OptionAttributes, …) — while an EP3 Instrument is the tradable contract, carrying product_id, an independent state (Open/Expired/Terminated/…), and external_symbol. So EP3's instrument is our per-contract row (our InstrumentId) and EP3's product is the umbrella (our new ProductId) — the split this RFC adopts. We already carry both halves on the spec side (rs/ep3/src/instrument_spec.rs: ProductSpec/ep3_products and Ep3Spec.product_id), but only ever materialize the instrument; the EP3 product is a sync-time artifact that is never persisted in Postgres. The new products table is the first time AX persists EP3's product level.

This also settles the table naming: the numeric ledger ID becomes ProductId (umbrella) and the per-contract Postgres table keeps the name instruments, because instrument is EP3's (and FIX's) word for the tradable contract. Renaming instruments → products would invert EP3's own vocabulary — the table holds EP3 instruments, not EP3 products. (A denormalized scheme — one products row per umbrella with an array of expiries — was considered and rejected: per-contract expiration, EP3 instrument id, and state are per-row mutable fields, and special_settlements/monthly_loss_limits/ cme_future_roll_schedule all FK to a specific symbol, which an array element cannot be. Options need a row per live series regardless, since the registry resolves series → (expiry, strike, put/call); see options sketch. The row count is never a Postgres scaling problem — the cost is management ergonomics, which a good admin surface plus the registry addresses, not denormalization.)

Widening `InstrumentId`

The odd u32 — now ProductId — identifies a product (e.g. COMPUTEDESK-H100), not a contract. InstrumentId widens to 64 bits and identifies a tradable contract:

// ax-sdk-internal

/// A tradable contract: a product plus a nominal contract date.
/// Perpetuals are the `date = 0` case — so an old odd-u32 `InstrumentId`
/// is exactly that case, and existing values widen numerically unchanged.
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, ...)]
#[serde(transparent)]
pub struct InstrumentId(u64);

impl InstrumentId {
    pub fn new(product: ProductId, date: ContractDate) -> Self {
        Self((date.yyyymmdd() as u64) << 32 | product.ledger_id() as u64)
    }
    pub fn product(&self) -> ProductId;           // low 32 bits
    pub fn date(&self) -> ContractDate;           // bits 62..32, YYYYMMDD
    pub fn is_perp(&self) -> bool;                // date == 0
}

/// A product (umbrella), e.g. COMPUTEDESK-H100. The odd value is the
/// TigerBeetle ledger number; the even twin (+1) is the basis ledger.
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, ...)]
#[serde(transparent)]
pub struct ProductId(u32);

impl ProductId {
    pub fn ledger_id(&self) -> u32;               // the value itself
}

impl From<ProductId> for InstrumentId { /* perp: date = 0 */ }

Because the perpetual case is date = 0, every existing odd-u32 InstrumentId value V widens to the numerically-identical u64 V (the high bits are zero). Nothing on the wire or at rest changes value for perps — the type widens, the bits don't.

The date is the nominal contract designator, not the expiration schedule. It is the listed date as an immutable name (20260130 for JAN-2026); the authoritative expiration TIMESTAMPTZ stays in Postgres and may be adjusted after listing (holiday rules etc.) without touching identity. Full YYYYMMDD rather than YYYYMM keeps weeklies and dailies representable later.

Ord on the packed u64 sorts by date then product, which is a sensible term-structure ordering; nothing persists a sort order today (position caches and risk state are rebuilt from TB/EP3 on startup), so the re-keying is safe.

TB account ID layout

bits 127..64   UserId (unchanged)
bit       63   scheme tag — RESERVED, always 0 in this RFC
bits  62..32   contract discriminator — interpretation depends on the tag
bits  31..0    ProductId (odd; even = basis twin) — invariant across schemes

Under tag 0 (this RFC), the discriminator is the nominal contract date as YYYYMMDD (0 for perpetuals) — self-describing and derivable from (user, product, date) with no lookup. Two layout rules are normative for any future scheme:

The low 32 bits are always the ProductId. This is what keeps ledger derivation, the odd/even parity discriminator, and the basis +1 twin uniform across perps, dated futures, and anything later.
Tag 1 is sketched for options as: discriminator = an opaque, registry-assigned series number (2³¹ ≈ 2.1B per product). Strikes are arbitrary decimals with product-specific tick rules, so they are deliberately not bit-encoded — the registry resolves series → (expiry, strike, put/call), the same lookup that resolves symbol → InstrumentId. Options thereby inherit per-series basis accounts and the whole transfer machinery for free; only futures keep a hexdump-readable discriminator.

derive_tb_account_id now takes the widened InstrumentId:

pub const fn derive_tb_account_id(self, instrument: InstrumentId) -> u128 {
    self.to_ulid().0 | instrument.0 as u128
}

Properties that make this safe and migration-free:

Existing perp accounts already conform. Every account in TB today has bits 63–32 zero, which is exactly the date = 0 (perpetual) encoding.
derive_from_tb_account_id is untouched — it masks off the entire low 64 bits, so user recovery is independent of what we encode there (rs/sdk-internal/src/user_id.rs:110).
The basis +1 trick survives. Basis account = qty account + 1 flips only the low bit of the (odd) ProductId; the date bits are unaffected. All +1 arithmetic (types.rs basis encodes, get_account_vwap_and_cum_realized_pnl, resume()'s existing_accounts.get(&(account.id + 1))) works unchanged.
Bit 63 stays free. YYYYMMDD ≤ 99991231 < 2^27, so the top bits of the date field are structurally zero. Bit 63 is reserved now as the scheme tag for the options encoding sketched above. Do not spend it.

Ledger model

	Today	Proposed
Qty ledger (odd u32)	one per contract	one per product (`ProductId`)
Basis ledger (even)	qty + 1	qty + 1 (unchanged)
USD ledger	1 (`USD_ASSET_ID`)	1 (unchanged)
Contract identity	the ledger number	account-ID bits 62..32 + ledger
Listing a contract	new ledger pair + table entry + deploy	a Postgres row

All transfers remain intra-ledger exactly as today: a trade's qty leg posts between two qty accounts of the same contract (same ledger, same date bits), the basis leg between their basis twins, fees/PnL on the USD ledger. The transfer schema — codes, discriminants, user_data_64 timestamps, user_data_128 packed decimal pairs, linked-batch atomicity — is unchanged.

Additionally, at account creation we mirror the nominal date into the account's user_data_32 (currently zero/unused on every account). TB's query_accounts can filter on user_data_32 + ledger server-side, so per-contract reconciliation sweeps stay one query per contract instead of a client-side decode over the whole ledger.

Decode and recovery changes

The study of the TB layer (main's rs/sdk-internal/tigerbeetle/ and cortex PR #1756) found every place contract identity is currently derived from a ledger number. They split into two patterns:

Transfer decoding (ax-tigerbeetle/src/types.rs, TryFrom<tb::Transfer>):

TbTrade (types.rs:227), TbBasis (types.rs:182), TbBasisReset (types.rs:162) derive instrument_id from transfer.ledger. These fields keep the name instrument_id but widen to InstrumentId(u64), decoded as: product from the ledger (as today), date from either account ID's bits 62..32 — every transfer carries both account IDs, so no information is lost. The decoder ensure!s the two accounts agree on the date bits, which doubles as a read-side cross-contract tripwire.
The PNL-vs-BasisReset disambiguation by ledger parity (types.rs:95, transfer.ledger % 2) survives unchanged — product IDs stay odd, basis ledgers stay even.
Collateral decode (types.rs:50, USD ledger) is unaffected.

Account-scan recovery (risk-engine2/src/tigerbeetle_driver.rs::resume, identical logic in cortex PR #1756):

Lines 129/137/147 currently do InstrumentId::from_u32(account.ledger) to key instrument params, mark prices, and the user-position map. These become: product from account.ledger, date from account.id bits 62..32, combined into the widened InstrumentId that keys the maps — InstrumentId::new(ProductId(account.ledger), date).
The full unfiltered query_accounts scan strategy is unchanged — recovery never needed to enumerate ledgers a priori, only to classify each account, which it can still do from (ledger, code, id).

Write-side guard. The structural firewall we lose — TB rejecting a transfer between accounts of different contracts, because they no longer sit on different ledgers — is replaced at the single choke point all postings go through: prepare_trade already asserts maker/taker fills agree on instrument; this extends to the full InstrumentId (product and date), and the account-ID derivation makes it impossible to construct a qty/basis leg spanning two dates without failing the decoder's ensure! and the reconciliation sweep. Deliberate cross-month transfers (rolls, spread settlement) become possible — a feature per-contract ledgers forecloses permanently — but require their own transfer codes and are out of scope here.

Invariants

From INVARIANTS.md, the four that key off per-contract identity:

G1 (Σ position_qty = 0 per instrument): holds per contract because qty transfers only ever pair two same-contract accounts. The TB-structural version ("each ledger nets to zero") coarsens to per-product — still true (a sum of per-contract zeros) but no longer the per-contract check itself. The per-contract check moves to: in-memory hot-path enforcement (unchanged — RiskState is keyed by InstrumentId), plus the recon sweep filtering accounts by (ledger, user_data_32).
U3 (net_position_basis = Σ position_basis): per-(user, contract) basis accounts still exist (the date bits make them distinct accounts), so unchanged.
P1/P4 (notional from qty × mark; zero qty ⇒ zero basis): per-contract account granularity is preserved, so the BasisReset mechanism — including AX_PNL's per-contract basis counterparty account, which now derives per (ProductId, date) — carries over verbatim.

check_all_invariants() and the scenario/real-world test pattern (write events → fresh resume() → exact state equality) remain the enforcement vehicle, now exercised with mixed perp + dated fixtures.

Dynamic symbol registry

The compile-time INSTRUMENT_IDS table is replaced by data:

-- db/postgres/1.sql
CREATE SEQUENCE product_id_seq START 1001 INCREMENT 2;  -- odd forever; values
                                                        -- are never reissued,
                                                        -- even on rollback

CREATE TABLE products (
    product_id INTEGER PRIMARY KEY DEFAULT nextval('product_id_seq'),
    name TEXT NOT NULL UNIQUE,           -- e.g. 'COMPUTEDESK-H100'
    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    CONSTRAINT product_id_odd CHECK (product_id % 2 = 1)
);

ALTER TABLE instruments
    ADD COLUMN product_id INTEGER REFERENCES products,
    ADD COLUMN contract_date INTEGER NOT NULL DEFAULT 0;  -- YYYYMMDD, 0 = perp

Write-once product IDs

A product ID is burned into TigerBeetle account IDs forever the moment the first account is created with it, so identity must be immutable. The load-bearing mechanism is a trigger, not role separation: a trigger is role-agnostic — it stops the app role, the migration role, and the table owner alike — and needs no dedicated database users. Defense in depth, all declarative in 1.sql:

products is append-only: a BEFORE UPDATE OR DELETE row trigger plus a BEFORE TRUNCATE statement trigger (TRUNCATE bypasses row triggers) that unconditionally RAISE EXCEPTION 'products is append-only'. products has no legitimately-mutable column, so freezing the whole row is correct. The only path through is a code-reviewed one-off migration that drops the trigger, edits, and recreates it — deliberately convoluted. This covers name too: renaming a product re-labels all history keyed through it and deserves the same ceremony.
instruments identity columns are frozen, the rest stays writable: unlike products, a per-contract row legitimately mutates (funding schedules, margins, holiday-adjusted expiration). So instead of freezing the row, a BEFORE UPDATE trigger rejects changes to the identity columns only — RAISE when NEW.product_id, NEW.contract_date, or NEW.symbol IS DISTINCT FROM OLD — leaving every spec field writable. This is the precise "a contract can never be repointed to another product (or date) after accounts exist" guard, which whole-table privilege revocation cannot express.
Sequence allocation: nextval is non-transactional — an aborted insert burns the value rather than letting the next insert reuse it — and serializes concurrent listings without max + 2 races.
Recon tripwire: TigerBeetle is the ultimate witness. resume() already fails hard on a ledger with no known product; the recon sweep additionally asserts every ledger seen in the account scan resolves to a products row matching the mapping cached at service startup. An edit that somehow survives layers 1–3 pages loudly instead of silently re-labeling history.

A one-line REVOKE UPDATE, DELETE, TRUNCATE ON products FROM <app role> is cheap belt-and-suspenders on the existing application role and worth keeping, but it is not load-bearing and does not require minting new roles — the triggers already stop every writer, owner included.

(Verify at implementation time that our Atlas setup diffs triggers/functions declaratively; if not, the triggers ship as a one-off migration file.)

symbol → InstrumentId becomes a runtime map loaded from Postgres (and kept fresh the same way instrument params already are — cortex's database_monitor replicates the instruments table via cortex_publication; order-gateway loads at startup and on instrument refresh). The 12 current instrument_id_for_symbol / symbol_for_instrument_id call sites (order intake, EP3 position resolution, margin check, formatting) switch to this registry and return/accept the widened InstrumentId.

Seeding: the 22 existing perps get products rows with their current IDs (explicit inserts; the sequence then starts above them via setval), with contract_date = 0 on their instruments rows. For one release the static table remains as a startup consistency assertion against the registry; then it is deleted. New product IDs come only from the sequence in the listing flow; USD_ASSET_ID = 1 stays reserved in code.

Blast radius

From the codebase inventory (exhaustive sweep, 2026-06-10):

Surface	Sites	Change
Map/cache keys (`OrdMap`/`BTreeMap<InstrumentId, _>`)	~33	key type unchanged textually — `InstrumentId` widens `u32`→`u64`, now per-contract (perps `date = 0`): OG `position_cache`, `check_margin`, `open_orders`; RE2 `RiskState`, `open_orders_margin`, `stress`
Ledger decode/recovery	8	`tigerbeetle_driver.rs:129,137,147`; `types.rs:50,162,182,227` (per above)
`derive_tb_account_id` callers	17	signature takes the widened `InstrumentId`; USD accounts derive via `InstrumentId::from(ProductId(USD_ASSET_ID))`
Symbol↔︎ID resolution	12	dynamic registry returning `InstrumentId`
Protocol/IPC serialization	6 fields	`protocol.rs:938`, RE2 `events.rs` — u32 → u64 (`#[serde(transparent)]`); internal-only surfaces, coordinate OG/cortex deploys
Test fixtures	~15	hardcoded odd IDs keep working via `From<ProductId>`; add dated-contract fixtures

Explicitly unaffected: the public SDK (rs/sdk — symbol-keyed throughout), Postgres and ClickHouse schemas (symbol-keyed), the Python reference engine (symbol-keyed), EP3 (string instrument IDs; per-contract symbols are EP3's normal case), and all TB data at rest.

Interaction with cortex PR #1756: the PR predates the ax-tigerbeetle micro-crate extraction and needs a rebase regardless; it modifies the same decode paths this RFC changes (and fixes a main-branch decode bug where DEPOSIT/WITHDRAWAL/FUNDING all decode to Deposit). Sequencing: rebase #1756 first, land the widened InstrumentId on top — or fold the InstrumentId decode changes into the rebase. Decide with the cortex owner.

Companion work: expiry settlement and the delist fence

Nothing in the system today ever closes a position except trading to zero — there is no expiry mechanism. Before the first dated listing we need (spec'd separately):

Final settlement: at expiration, for every nonzero (user, contract) position: a settlement-qty transfer against an AX_SETTLEMENT system account driving qty to zero (Σ qty = 0 per G1, so AX_SETTLEMENT also ends flat), plus the PnL + BasisReset pair at the settlement price. New transfer discriminants; same linked-batch machinery.
Delist fence: delist in EP3 only after query_accounts(ledger, user_data_32 = date) confirms every qty and basis account for the contract has zero posted balance and there are no resting orders (GTC included).

Per the project's lifecycle-testing rule, both must be exercised under client disconnect, server-initiated disconnect, crash/restart, and recovery before rollout.

Alternatives considered

Mint a unique ledger (ProductId) per contract (+ dynamic registry). The u32 space (2³¹−1 odd/even pairs; TB's only ledger constraint is non-zero) would never exhaust for futures. Rejected because (a) same-product contracts on different ledgers can never transact atomically in TB — rolls and calendar spread settlement are permanently structural impossibilities; (b) ledger churn: every listing provisions new ledger pairs and new AX_PNL basis accounts with no cleanup path; (c) it still requires the entire dynamic registry, so it saves only the InstrumentId widening — the smaller half of the work — while spending the larger architectural option. For options the per-series version of this alternative also burns the keyspace at scale; the proposed encoding is the same shape options will need anyway.

Recycle a fixed slot set (12 monthly IDs per product). Rejected: TB accounts are immortal, so reuse aliases two contracts' history onto one account; correctness then hinges on a provably-exactly-zero drain of every position and basis account at each recycle fence, with any dust silently poisoning the next year's contract; and the symbol itself becomes ambiguous on customer statements.

Put the contract date in user_data_32 only (account IDs unchanged beyond uniqueness salt). Rejected: account IDs must be derivable — touch_account, recovery pairing, and basis +1 all depend on computing an account ID from (user, contract) without a lookup. The ID is the natural home; user_data_32 is the query-side mirror, not the identity.

Rollout

Phase 0 (now, one line): reserve bit 63 in the derive_tb_account_id doc comment and a const; fix the stale "(e.g. subaccounts)" note.
Phase 1 (no behavior change): widen InstrumentId to u64 and introduce ProductId; widen the protocol fields and the TB encode/decode paths (map-key declarations are unchanged textually). All contracts are perps (date = 0), so every derived account ID and transfer is bit-identical to today. Scenario + real-world resume-equality tests prove it.
Phase 2: dynamic registry (Postgres products + columns, runtime map, startup assertion against the static table for one release, then delete the table).
Phase 3: expiry settlement + delist fence (separate spec), dated fixtures through the full scenario/invariant/recovery suites.
Phase 4: list COMPUTEDESK-H100 monthlies.

Open questions

Account codes: keep PERP_QTY/PERP_BASIS (8000/8001) for dated contracts — the codes describe the accounting role, and contract-ness lives in the ID — or mint FUT_QTY/FUT_BASIS so account scans can classify product type without decoding IDs? Leaning keep-and-rename (POSITION_QTY/POSITION_BASIS); resume() branches only on code + user.
Nominal date for monthlies: last trading day vs. a fixed convention (e.g. first of the named month)? Pure naming, but it appears in account IDs forever — pick once.
Who owns the products insert: allocation itself is settled (the sequence), but which admin surface drives a new-product listing — admin-cli, api-gateway admin route, or a manual reviewed migration — is open.
EP3 symbol convention for dated contracts (COMPUTEDESK-H100-JAN-2026 vs CME-style month codes) — orthogonal to this RFC but should be fixed in the listing spec.

RFC: Dated Futures and Options Internal Reforms (widening InstrumentId)