RFC: Accrual Engine — a shared library for incentive-program compute

Date: 2026-08-05

Status: Draft — deferred. No producer is being built against this design. The commercial programs specify their own producer in commercial-program-accruals, and the liquidity program shipped without this crate. Both keep the decisions settled here: the optional payee, the accrual-row shape, ledger placement, and the seam to program-payouts. Whether and when a shared engine is built is tracked on A-4647.

Author: (with Claude)

Related: the producer-side mirror of program-payouts, which owns the payer side. Hosts the accrual compute for referral-program, liquidity-program (Part 1), and the existing fee-tier, MMLP and leaderboard tasks. Follows the library-not-service pattern of ax-transaction-engine.

ax-accrual-engine is a library crate with no binary. It holds the plumbing every incentive program re-implements — windowed ClickHouse reads, ladder evaluation, eligibility resolution, and a provisional-then-frozen period upsert — and draws a line at the accrual row, where program-payouts takes over.

Each program keeps its own policy and its own accrual table. The engine owns the drive loop. The immutable accrual row is the seam between producer and payer.

Offered from the producer side as input, not as a gate. This design becomes worth building when:

1. Two liability producers run in production. Generalizing over two independently built implementations is an extraction; over one it is a guess. Liquidity is the first. The commercial producer (commercial-program-accruals) is the second.
2. A third producer is starting. Two justify the shape; the third pays for the refactor. With no new consumer it is rework of code that already works.
3. The first two converged without coordinating. If they are structurally alike, the shared shape has been discovered and is safe to lift. If they diverged, the divergence is the finding, and a common trait would hide it.
4. What is shared exceeds a trait and a drive loop. Each helper named below needs re-checking against the real producers: windowed ClickHouse reads are already shared (ChTradeRow::query_account_tier_metrics), the idempotent period upsert turned out program-specific, and eligibility resolution has one consumer. If the residue is accrue() plus upsert(), the crate boundary costs more than it returns.
5. Library or service is settled before adoption is scheduled. As a library, adoption is a wrapper in place. As a service, it re-homes every producer — a deployment, config and on-call change rather than a code one.

recon-engine runs both in one watch() loop:

1. Invariant checks, around 22 of them — read-only assertions over Postgres and ClickHouse that fire incident.io alerts when the ledger disagrees with itself. This is the crate's stated purpose.
2. Mutating cadence tasks, spawned alongside the checks in lib.rs: fees::refresh_fee_rates_task (daily) re-tiers every account against the fee schedule and writes trading_accounts.maker_fee / taker_fee; leaderboard::refresh_leaderboard_task (roughly every 15 minutes) recomputes leaderboard_entries. mm_reports::run generated MMLPv1 PDFs to S3 and was removed under A-4812.

An auditor that also mutates the state it audits mixes two responsibilities. The incoming programs add referral's rebates.rs and the liquidity scorer as a fourth and fifth writer.

ax-transaction-engine is "an embedded transaction engine that can be used as a library": no binary, I/O behind a Store trait, hosted by four service binaries (api-gateway, settlement-engine, trade-engine2, btnl-trade-engine), with balance mutations serialized under a pg_advisory_xact_lock. The programs' accrual compute wants the same treatment: a deterministic core and a thin host.

ax-accrual-engine is a lib crate with no binary. Its I/O sits behind a trait, as ax-transaction-engine hides Postgres and ClickHouse behind Store, so the accrual math is deterministic and unit-testable without a live database. A host — recon-engine today, a dedicated incentives host or ax-scheduler later — drives it on cadence.

This makes the work a move of code behind a crate boundary rather than a new service, and it lets the auditor stop being a mutator without a big-bang split.

The engine reads the raw activity (trades, quotes, fees collected), applies a program's policy, and writes a per-period accrual: account X is owed or owes $Y for period P. It does not move money.

That is the seam program-payouts draws: producers own what is owed, the payout module owns paying it. The liquidity RFC calls the same boundary its Part 1 / Part 2 seam. accrual-engine is the generalization of Part 1 across producers, as program-payouts is the generalization of Part 2.

Each program keeps its own policy: ax-policy-mmlp for liquidity scoring, the fee-share waterfall for referral, the fee ladder for fee-tiering. The engine does not unify what a dollar is worth, which is program-specific and must not be forkable. It unifies the mechanics every program re-implements:

• windowed trade and fee reads from ClickHouse (trailing-30d, per-symbol, per-account);
• ladder evaluation, including the fee engine's tier-with-hysteresis logic;
• live eligibility — resolve a fee-ladder tier from billed rates for each referral accrual run and snapshot it on the accrual row (referral D4);
• idempotent per-period upsert, one row per (program, subject, period), recomputable while open and frozen after approval.

program-payouts unifies the table because payout is uniform: every producer wants "credit $Y to account X, once." Accrual is the opposite — the columns and math differ per program. rebate_accruals carries partner and ladder-level fields; liquidity_accruals carries epoch, symbol and score fields.

So accrual-engine unifies the code path while each program keeps its own accrual table. Accrual math is program-specific, so share the library; payout is uniform, so share the table.

Programs accrue in two directions:

• expense — a trader owes the exchange (fee-tiering, dynamic fees);
• liability — the exchange owes a party (a referral rebate, a liquidity reward).

Both share a lifecycle (ACCRUED → APPROVED → PAID / FORFEITED), an idempotency discipline, and a recompute-while-open rule. The engine models that lifecycle as a trait over a per-program row with a direction discriminator, rather than forcing one physical schema (D4). A liability accrual, once APPROVED, is what a producer hands to program-payouts.

The engine ships by extracting the plumbing already in recon-engine — the hysteresis in fees.rs, the windowed-read helpers — into the crate, then re-pointing the existing tasks at it. Behavior-preserving, snapshot-guarded. Referral and liquidity are then written against the crate. No task changes host in the first cut; only the code moves behind the boundary. Re-homing the host (D1) is a later, independent step.

D4 keeps per-program accrual tables; D7 says where they live. One dedicated accrual_engine schema, following the domain-schema precedent (partner_programs, mm_liquidity_performance, treasury_engine). The engine's cross-program tables — a programs registry and a period_runs bookkeeping log — and each program's accrual ledger co-locate there, so the engine's grants and logical-replication scope are one schema rather than a table hunt.

The commercial producer does not follow this. rebate_accruals lives in partner_programs beside its sources (commercial-program-accruals D7, referral D8a). The liquidity program created accrual_engine for its own ledger. If a shared engine is built and wants the ledgers co-located, moving one table between schemas is an ALTER TABLE ... SET SCHEMA.

AccrualRow exposes payout_account() -> Option<AccountId>, not account_id() -> AccountId. None is a real and expected state: attribution binds a user and account provisioning is lazy (referral D2), so a partner can hold a referral code, earn a rebate, and own no trading account. That accrual is correct and the money is owed; there is nowhere to send it yet.

enqueue_approved therefore declines such a row rather than dropping it or inventing a payee, and returns EnqueueOutcome { enqueued, held } so the unpayable count is a reported figure and not a silent skip. The row stays APPROVED in the producer's ledger, and a later run enqueues it once a payout account exists.

The hold lives here, not in the payout module. The module's requests.account_id is NOT NULL REFERENCES trading_accounts(id), so a payee-less accrual cannot be represented in its queue even as HELD. The module's HELD covers a different case: a payee who exists but cannot yet receive, for want of KYB or because the account is restricted.

A Subject associated type on AccrualProgram was considered and dropped. The engine never computes with a subject; it only identifies a row for the log and the payout URN, which idempotency_key() already does.

• One library home for the accrual and scoring plumbing every incentive program re-implements.
• Deterministic, replay-safe, unit-testable accrual math, with I/O behind a trait.
• A single seam to program-payouts at the approved accrual row.
• Let recon-engine shed its mutating work without a service rewrite.

• Moving money, which program-payouts owns (D2).
• Unifying program policy or a single physical accrual table (D3, D4).
• Standing up a new service on day one (D1, D6).
• Re-homing the host in the first cut (D6).

A lib crate. The public surface:

The engine owns the drive loop — upsert the open period and unapproved prior periods, provisional while open, frozen on approval — and the shared helpers below. Each program supplies accrue().

The row carries what the seam needs and nothing more. The last three accessors are what a payout request row is built from (§4), which is why there is no Subject associated type (D8).

Extracted from what recon-engine and the two program RFCs need:

• Windowed ClickHouse reads — trailing-N-day trade and fee aggregates, per account and per symbol.
• Hysteretic ladder — the fee engine's tier-with-hysteresis logic.
• Live eligibility — resolve a fee-ladder tier from billed rates for each referral accrual run and snapshot it on the accrual row (referral D4).
• Idempotent period upsert — one row per (program, subject, period), recomputable while open, frozen after approval.

Mirrors ax-transaction-engine's Store: a production PgChStore over Postgres and ClickHouse, plus in-memory fakes for property tests. Accrual math never touches a live connection in a unit test.

When a liability accrual reaches APPROVED, the engine enqueues the payout request row, with the funding account and currency read from the program's payout config. Enqueue is uniform across producers and belongs with the shared lifecycle. What stays with each producer is approval: which accruals become payable, and at what threshold.

Two kinds of approved row never leave the engine, and neither is an error:

• an Expense row — fee-tiering sets a rate, so there is nothing to pay;
• a row with no payout_account() — held producer-side and counted into EnqueueOutcome.held (D8).

The idempotency_key() a producer returns is the module's primary key, so it must carry its own program: <program_kind>://<period>/<subject>. The module enforces this with CHECK (idempotency_key LIKE program_kind || '://%'), which turns a cross-program collision — silent, and therefore a missing payment rather than an error — into a write failure.

First cut: recon-engine stays the host, because it already has the Postgres and ClickHouse pools and the ax-scheduler cadence loop, but the accrual tasks call into ax-accrual-engine instead of living inline (D6). This does not resolve recon's split responsibilities; it makes them fixable, because the mutating work becomes a crate the auditor calls, and can be lifted to another host by moving three tokio::spawn lines. Coordinate with program-payouts O6 on placement: the two modules likely want the same eventual host.

One dedicated accrual_engine Postgres schema (D7), holding the engine's own tables plus each program's accrual ledger:

• accrual_engine.programs — a registry, one row per accrual program: program_kind PK, direction, cadence, window, enabled. The host reads it to know what to drive.
• accrual_engine.period_runs — append-only bookkeeping, one row per (program, period, run) with started, finished and row-count, so a re-run is idempotent and observable.
• accrual_engine.liquidity_accruals — the liquidity Part 1 ledger.

Not held here: partner_programs.rebate_accruals, which lives with its sources (D7). Unchanged and not moved: fee-tiering writes trading_accounts.maker_fee / taker_fee, a rate with no ledger row; MMLP stays in mm_liquidity_performance; the leaderboard stays in leaderboard_entries.

Schema is declared in db/postgres/1.sql (Atlas, declarative), not as ad-hoc migration files.

Cadence and window knobs move from recon-engine's WatchConfig into per-program config the host passes to the engine: fee hysteresis days, leaderboard refresh interval, accrual period. No new secrets. Funding-account and currency knobs stay in the payout program config, read at the seam.

A strangler, behavior-preserving (D6). There is no data migration.

1. Add the accrual_engine schema plus the programs and period_runs tables to db/postgres/1.sql, applied through just migrate-db <env>.
2. Extract the fees.rs hysteresis and the windowed-read helpers into ax-accrual-engine; re-point refresh_fee_rates_task at the crate. Snapshot-guard the fee output: no rate should change.
3. Write the liquidity scorer against the crate, landing its ledger in accrual_engine.
4. Decide fee-tiering, leaderboard and MMLP membership (O1, O2); move or leave per that decision.
5. Later, independent: lift the host off recon-engine (O3) if the split responsibilities still bite.

• Unit: each AccrualProgram::accrue() against in-memory Store fakes; property tests on the hysteretic ladder and on eligibility resolution, with fault-injection stores as transaction-engine does.
• Snapshot (insta, inline): fee-rate output byte-identical before and after the extraction. This gates step 2.
• Lifecycle: provisional-while-open recompute self-heals on corrected inputs; frozen-after-approval is immutable; a crash mid-upsert leaves no partial period.
• Seam: an approved liability accrual produces exactly one payout request row; an approved Expense row produces none.
• Hold (D8): an approved row with no payout_account() is neither enqueued nor dropped. It is counted into EnqueueOutcome.held, stays APPROVED, and enqueues on a later run once a payout account exists. Tested through a pure AccrualRow, without a store or a pool, so a regression cannot hide behind fixture setup.

• O1 — Fee-tiering membership. Fee-tiering sets a rate; it produces no payable accrual row. Does it belong in accrual-engine, as the canonical hysteretic-ladder consumer (D3), or is it rate-setting that shares only helpers? Leaning: move it in, with direction = Expense and no payout enqueue. Confirm and close, or move it back out before more producers assume it.
• O2 — Leaderboard and MMLP membership. Both are reporting — a ranking, a PDF — not accruals. Recommend leaving them as recon-engine tasks. Confirm.
• O3 — Host re-homing. Keep the driver in recon-engine, stand up a dedicated incentives host, or fold into ax-scheduler? Decide jointly with program-payouts O6.

• O4 — One lifecycle over per-program rows. The shape holds. Three producers implement AccrualRow over their own tables — fee-tiering (Expense, a rate and no payable), liquidity, referral — with no lowest-common-denominator schema, so D4 and D5 survive contact with all three. The one change contact forced is D8: the payee is Option<AccountId>, because referral can accrue for a partner who has no account.

• O5 — Who enqueues to program-payouts. Engine-owned. The engine writes the payout request row when a liability accrual reaches APPROVED, reading the funding account and currency from the payout program config; approval stays with the producer. Enqueue is uniform across producers, so leaving it to each one would give three copies of the URN convention and the config read. With this design deferred, each producer's host task makes the call instead (commercial-program-accruals D9).

• O6 — Ledger schema placement. Decided here in favour of seam cohesion: every accrual ledger has the same single consumer, so the engine's grants and its logical-replication publication would be one schema rather than a table hunt growing with each program. What makes such a split safe is that the row is value-typed — it carries the numbers that produced it, so it never reaches across the schema boundary for the authority behind its own number.

  The commercial producer decided otherwise for its own ledger. With the shared engine deferred and no settled payer to consolidate grants toward, there is nothing to consolidate into, so rebate_accruals stays in partner_programs beside its sources (commercial-program-accruals D7, referral D8a). Value-typing is unaffected and still required: it is a property of the row, not of the schema.

• rs/accrual-engine/src/lib.rs — AccrualRow, AccrualProgram and Store as shipped; enqueue_approved and EnqueueOutcome (D8, §4).
• rs/accrual-engine/src/fee.rs, liquidity.rs, referral.rs — the three producers implementing the trait over their own tables (O4).
• rs/recon-engine/src/incentives.rs — the host wiring: the accrual tasks and the payout sweep on cadence (§5).
• rs/recon-engine/src/lib.rs:578 — leaderboard::refresh_leaderboard_task spawned in watch().
• rs/recon-engine/src/lib.rs:587 — fees::refresh_fee_rates_task spawned in watch().
• rs/recon-engine/src/fees.rs:32 — refresh_fee_rates_task; the hysteretic-ladder logic to extract.
• rs/recon-engine/src/leaderboard.rs:15 — the reporting task (O2).
• rs/transaction-engine/src/lib.rs:24 — "Embedded transaction engine that can be used as a library", the D1 template.
• rs/transaction-engine/src/store.rs — the Store / StorePgTxn I/O-behind-a-trait pattern to mirror (§3).
• docs/rfc/program-payouts.md — the payer side; the seam this RFC hands off to.
• docs/rfc/liquidity-program.md — the Part 1 / Part 2 seam at the accrual row; the ax-policy-mmlp oracle.
• docs/rfc/referral-program.md — rebates.rs mirroring fee.rs (D6); the partner ladder (D3); live eligibility (D3); D8a places rebate_accruals in partner_programs.
• db/postgres/1.sql — the declarative schema (Atlas); the partner_programs, mm_liquidity_performance and treasury_engine domain-schema precedents.