FlatBuffers Policy Schema

The binary files under a study’s policy/ directory are FlatBuffers buffers. Novomodelo’s runtime writes and reads them through a hand-rolled, allocation-free path in Rust, but external consumers (Python, C++, TypeScript, Java, Go, …) can use the canonical schema file shipped with the source tree to generate a typed reader in any language flatc supports.

File path	Root table
`policy/manifest.bin`	`CheckpointManifest` (study-global; written last, read first)
`policy/cuts/NNN.bin`	`StageCuts`
`policy/basis/NNN.bin`	`StageBasis`
`policy/states/NNN.bin`	`StageStates` (only when `exports.states = true`)

NNN is a three-digit, zero-padded id: the pool id for cuts/, the 0-based node position for basis/ and the 0-based stage position for states/. manifest.bin is a single study-global file with no id. The reader derives identity from inside each buffer, never from the file name. The CheckpointManifest root carries the study graph (nodes/edges), the stage count, the producer provenance, and the format_version marker.

The schema lives at crates/novomodelo-io/schemas/policy.fbs under namespace Novomodelo.IO.Policy. It declares file_identifier "CBVF" and file_extension "bin" but no root_type, so --root-type is required per file to select the entry point for flatc. The reader refuses any buffer whose leading identifier is not CBVF before decoding it, so a buffer without the identifier is never misparsed.

Quick start: dumping a `.bin` to JSON

flatc ships a converter that turns any FlatBuffers buffer into JSON when given the schema. This is the closest thing to a human-readable view of a policy checkpoint:

flatc -t --strict-json \
    --root-type StageCuts \
    crates/novomodelo-io/schemas/policy.fbs \
    -- output/policy/cuts/000.bin
# writes 000.json next to the .bin

For the basis or states files, swap the --root-type argument for StageBasis or StageStates. --raw-binary forces flatc to read a buffer that carries no CBVF identifier. A buffer written by Novomodelo carries the identifier, so flatc -t verifies it on its own.

Generating a typed reader

flatc emits idiomatic source code for any of its supported target languages. Pick the one matching your toolchain.

Python

flatc --python crates/novomodelo-io/schemas/policy.fbs
# emits Novomodelo/IO/Policy/{AffinePiece,EntitySlot,EntityType,StageCuts,StageBasis,StageStates}.py

from Novomodelo.IO.Policy.StageCuts import StageCuts

with open("output/policy/cuts/000.bin", "rb") as f:
    buf = bytearray(f.read())

cuts = StageCuts.GetRootAs(buf, 0)
print("stage_id =", cuts.StageId())
for i in range(cuts.CutsLength()):
    piece = cuts.Cuts(i)
    print(piece.PieceId(), piece.Intercept(), [piece.Coefficients(j) for j in range(piece.CoefficientsLength())])

C++

flatc --cpp crates/novomodelo-io/schemas/policy.fbs
# emits policy_generated.h

TypeScript / JavaScript

flatc --ts crates/novomodelo-io/schemas/policy.fbs
# emits TypeScript modules under novomodelo/io/policy/

For other targets see flatc --help.

Field-by-field reference

The authoritative description of every field lives in policy.fbs itself — every field carries an inline doc comment. The Output Format page has a tabular summary suitable for reading on the web.

Reserved slots

The schema reuses one field id and permanently burns three others:

AffinePiece.intercept (id 4) reuses the vtable slot of the retired domination_count field. This id reuse is the exceptional case in this page’s versioning policy — ordinarily a burned id is never reused.
AffinePiece.reserved_7 (id 7) is marked deprecated. It is a retired, always-empty vector field. The vtable slot number is permanently burned so no future field can reuse it.
EntitySlot.delivery_anchor (id 4) is marked deprecated. It is a retired month-integer anchor (year * 12 + (month - 1)).
EntitySlot.delivery_date (id 5) is also marked deprecated. It is a retired single YYYYMMDD calendar date; the live date fields are per-family: reference_date (id 6) — the reference past-stage start_date (YYYYMMDD) of a HydroInflowLag slot; interval_start (id 7) / interval_end (id 8) — the half-open forward-family window (a HydroTransitBucket arrival window or an AnticipatedThermalState delivery window). Each date field reads sentinel -2147483648 (the int32 minimum) where a slot’s family carries no such date. An AnticipatedThermalState ring slot carries its delivery window only while it holds a live commitment at the pool’s stage. Both interval fields read the sentinel otherwise, and also when the delivery lands past the study calendar extended by the post-study stages. The burn rule is per-table: EntitySlot’s id 7 (interval_start) is a live field, distinct from the burned AffinePiece.reserved_7 (also id 7).

Unlike an ordinary appended field, reusing or burning an id is not wire-backward-compatible: a reader built for another layout misreads the slot. A buffer with no CBVF identifier or a format_version other than 3 is therefore rejected outright on load rather than read with graceful-absence defaults — see Versioning policy below.

Generated readers emit no accessor for a deprecated field; generated writers cannot emit them. The Novomodelo runtime’s own writer never sets them.

How drift is prevented

The schema is not consumed by Novomodelo’s own build. Two independent implementations describe the same wire format:

The schema file crates/novomodelo-io/schemas/policy.fbs, with explicit (id: N) attributes on every field.
The hand-rolled writer/reader in crates/novomodelo-io/src/output/policy/codec.rs, which encodes vtable slots via the *_FIELD_*: u16 constants. The slot offset is (field_id + 2) * 2.

A conformance test, tests/flatbuffers_schema_conformance.rs in novomodelo-io, round-trips representative buffers in both directions:

Hand-rolled writer → flatc -t → JSON: catches the writer emitting a slot the schema does not declare, or at the wrong offset.
JSON → flatc -b → hand-rolled reader: catches the schema declaring a slot the reader expects at a different offset.

The test is gated behind the flatc-conformance cargo feature so that the everyday cargo test does not depend on flatc. To run it:

cargo test -p novomodelo-io \
    --features flatc-conformance \
    --test flatbuffers_schema_conformance

If you change either the schema or the slot constants, run the conformance test before merging. The CI workflow that has flatc available runs it on every pull request that touches policy/codec.rs or the schema file.

Versioning policy

Two separate rules govern a policy checkpoint, on top of the field-level rules below.

Wire readability. A reader parses policy/manifest.bin first, before any .bin payload. Every .bin buffer, the manifest included, carries the CBVF file_identifier described above, and a reader refuses a buffer without it before decoding. The manifest’s CheckpointManifest carries a required format_version marker, which must equal 3 (FORMAT_VERSION); a missing manifest.bin or another format_version stops the read before any payload is parsed. There is no conversion between format versions; Policy-load errors lists what a refused load raises.

Load admission. A checkpoint that reads is not thereby admitted. Every load path requires the software (id 20) and software_version (id 1) recorded in manifest.bin to equal the running software’s name and version exactly; the version gate states the rule, the message, and the remedy.

FlatBuffers’ graceful-absence rule lets us add new fields to any table without breaking older readers, as long as new fields are appended at the end with the next available id. This is the only schema change that does not require an output-format version bump — the format_version gate above governs everything else:

Adding a field at the next free id → backward compatible at the wire level only. Old readers see the field as absent and use the FlatBuffers default (zero / empty vector). New readers see the value when the writer was new enough to emit it. Wire compatibility does not imply the values are safe to consume: a field can change what existing fields mean (the cost_scale_factor provenance marker marks cut coefficients as canonical currency units, which a reader that ignores it silently misinterprets — no error, wrong numbers). Whether a checkpoint loads is governed by the version gate, not by this wire-level rule.
Removing a field → mark it deprecated, never reuse the id. See AffinePiece.reserved_7, EntitySlot.delivery_anchor, and EntitySlot.delivery_date for worked examples. AffinePiece.intercept reusing id 4, the slot of the retired domination_count field, is the one exception this page documents, not a precedent for reusing a burned id under the ordinary rule.
Changing a field’s type → breaking. Bumps the major output format version.
Renaming a field → breaking for flatc-generated code (the accessor name changes). Avoid; if necessary, treat as a major bump.
Reordering fields → harmless if (id: N) attributes stay put. The wire layout is determined by the ids, not by source order.