Skip to content
Novomodelo Documentation

Metadata Files

Both training/metadata.json and simulation/metadata.json use an atomic write protocol:

  1. Serialize JSON to a temporary .json.tmp sibling file.
  2. Atomically rename the .tmp file to the target path.

This ensures consumers never observe a partial file. If a metadata file exists, it contains a complete, valid JSON document. If a run is interrupted before the final write, the .tmp sibling may remain, but the target file reflects the last successfully completed write.

status is "complete" or "partial". In training/metadata.json it is "partial" when a shutdown request ended training and neither its stopping rules nor its iteration budget ended it at that iteration: a signal to novomodelo run, or a stop from a Python on_iteration callback. In simulation/metadata.json it is "partial" for a simulation skipped after a signal stop. Every other ending reads "complete", a failed training included, so read convergence.termination_reason ("error" on a failure) and the exit code. convergence.achieved is true when the stopping rules ended training with a gap or bound_stalling rule triggered at that iteration. For simulation, read scenarios.failed.

Both files are plain JSON, so jq reads them directly. For a script or CI check, see Output format in the CLI Reference.

From Python, novomodelo.results.load_results(output_dir) returns both metadata documents in one dict, together with a per-phase complete flag — see the Python Quickstart.

training/metadata.json

Section titled “training/metadata.json”

The training metadata file is written atomically at the end of the training run. It merges run context, configuration, convergence outcome, row-pool statistics, objective bounds, LP solver statistics, and distribution information into a single file. The convergence block records how the run stopped.

Methodology: Determinism & Provenance · Stopping Rules

Example (a novomodelo run of the 1dtoy template; hostname and timestamps replaced):

{
  "software": "novomodelo",
  "software_version": "0.18.0",
  "hostname": "<hostname>",
  "solver": "highs",
  "solver_version": "1.13.1",
  "started_at": "<timestamp>",
  "completed_at": "<timestamp>",
  "duration_seconds": 0.468,
  "status": "complete",
  "configuration": {
    "seed": null,
    "max_iterations": 128,
    "forward_passes": 1,
    "stopping_mode": "any",
    "policy_mode": "fresh"
  },
  "problem_dimensions": {
    "num_stages": 4,
    "num_hydros": 1,
    "num_thermals": 2,
    "num_buses": 1,
    "num_lines": 0
  },
  "iterations": {
    "completed": 128,
    "converged_at": null
  },
  "convergence": {
    "achieved": false,
    "final_gap_percent": -96.28359773344324,
    "termination_reason": "iteration_limit"
  },
  "row_pool": {
    "total_generated": 384,
    "total_active": 384,
    "peak_active": 384,
    "cuts_active": 384,
    "rows_in_lp_total": 0,
    "rows_in_lp_solve_count": 0,
    "rows_in_lp_max": 0,
    "total_loaded": 0
  },
  "bounds": {
    "final_lower_bound": 15595518.381798636,
    "final_upper_bound": 579592.1986224409,
    "final_upper_bound_std": 0.0,
    "final_upper_bound_kind": "statistical"
  },
  "solve_stats": {
    "total_lp_solves": 5632,
    "first_try": 5632,
    "retried": 0,
    "failed": 0,
    "forward_solve_seconds": 0.05161780600000029,
    "backward_solve_seconds": 0.24256579799999914,
    "parallelism": 1
  },
  "setup": {
    "load_seconds": 0.001174509,
    "stochastic_fit_seconds": 0.000100001,
    "production_fit_seconds": 0.000026218,
    "evaporation_fit_seconds": 4.48e-7,
    "broadcast_seconds": 0.000202196
  },
  "distribution": {
    "backend": "local",
    "world_size": 1,
    "ranks_participated": 1,
    "num_hosts": 1,
    "threads_per_rank": 1,
    "hosts": [
      {
        "hostname": "<hostname>",
        "ranks": [0]
      }
    ]
  }
}

Top-level fields:

NameTypeNullableUnitsDescription
softwarestringNo—Name of the software that produced this output: "novomodelo".
software_versionstringNo—Version of that software, for example "0.18.0".
hostnamestringNo—Hostname of the machine that ran training.
solverstringNo—LP solver backend: "highs" or "clp".
solver_versionstringYes—Version string of the linked LP solver library. Omitted when not available.
started_atstringNo—ISO 8601 timestamp when training started.
completed_atstringNo—ISO 8601 timestamp when training completed.
duration_secondsnumberNosTotal training wall-clock duration in seconds.
statusstringNo—"partial" when a shutdown request alone ended training (then convergence.termination_reason is "graceful_shutdown"), "complete" otherwise, a failed training included; convergence.termination_reason records how the run ended ("error" when training stopped on a failure).
production_fit_deviationobjectYes—Run-level rollup of the per-entity production-model fit deviation (informational, never hashed). Omitted when the run fitted no model whose deviation is measured.

configuration fields:

NameTypeNullableUnitsDescription
configuration.seedintegerYes—The configured training.tree_seed; null when absent (the run then uses 42).
configuration.max_iterationsintegerYes—Maximum iterations: the limit of the first iteration_limit entry of training.stopping_rules; always set in a file novomodelo run writes, because the configuration must hold an iteration_limit rule.
configuration.forward_passesintegerYes—Number of forward-pass scenario trajectories per iteration. null under enumerated selection.
configuration.stopping_modestringNo—How multiple stopping rules combine: "any" or "all".
configuration.policy_modestringNo—Policy warm-start mode: "fresh", "warm_start", or "resume".

problem_dimensions fields:

NameTypeNullableUnitsDescription
problem_dimensions.num_stagesintegerNo—Number of stages: the study stages plus any pre-study stages.
problem_dimensions.num_hydrosintegerNo—Total number of hydro plants.
problem_dimensions.num_thermalsintegerNo—Total number of thermal plants.
problem_dimensions.num_busesintegerNo—Total number of buses.
problem_dimensions.num_linesintegerNo—Total number of transmission lines.

iterations fields:

NameTypeNullableUnitsDescription
iterations.completedintegerNo—Number of training iterations that finished.
iterations.converged_atintegerYes—Equal to completed when achieved is true; null otherwise.

convergence fields:

NameTypeNullableUnitsDescription
convergence.achievedbooleanNo—true when the configured stopping rules ended training and a gap or bound_stalling rule triggered at that iteration, even when termination_reason names another rule triggered at the same iteration; false when only iteration_limit or time_limit triggered, when a shutdown request alone ended training, and after a failure.
convergence.final_gap_percentnumberYes%Optimality gap between lower and upper bounds at termination as a percentage. null when the final lower bound is not positive.
convergence.termination_reasonstringNo—Name of the stopping rule that ended the run: "iteration_limit", "time_limit", "bound_stalling", or "gap" (under "all", the first rule other than iteration_limit, or "iteration_limit" when the largest limit capped the run); "error" when training stopped on a failure, in which case novomodelo run exits non-zero; "graceful_shutdown" when a shutdown request ended training and neither the stopping rules nor the iteration budget ended it at that iteration (a SIGTERM or SIGINT to novomodelo run, or an on_iteration callback of novomodelo.Study.train or novomodelo.run.run that returns a truthy value or raises).

row_pool fields:

NameTypeNullableUnitsDescription
row_pool.total_generatedintegerNo—Total cut rows generated over the entire run.
row_pool.total_activeintegerNo—Cut rows still active in the pool at termination.
row_pool.peak_activeintegerNo—Highest number of simultaneously active cut rows observed.
row_pool.cuts_activeintegerNo—Cut rows currently active in the LP at termination.
row_pool.rows_in_lp_totalintegerNo—Sum of resident rows-in-LP over every lazy-selection solve in the run. Zero when no lazy selection ran.
row_pool.rows_in_lp_solve_countintegerNo—Number of lazy-selection solves in the run. Zero when no lazy selection ran.
row_pool.rows_in_lp_maxintegerNo—Largest resident rows-in-LP over any single lazy-selection solve. Zero when no lazy selection ran.
row_pool.total_loadedintegerNo—Cut rows loaded at run start rather than generated by this run: the cuts a warm-start or resume run loads from policy.path, plus any policy.boundary cuts (the sum of every pool’s warm_start_count); a subset of total_generated. 0 on a fresh run without policy.boundary.

bounds fields:

NameTypeNullableUnitsDescription
bounds.final_lower_boundnumberNo—Final lower bound on the objective at termination.
bounds.final_upper_boundnumberYes—Final upper-bound value; always a number in a file novomodelo run writes.
bounds.final_upper_bound_stdnumberYes—Sample standard deviation of the last completed iteration’s forward-pass scenario costs. null when final_upper_bound_kind is "exact".
bounds.final_upper_bound_kindstringNo—Upper-bound regime of the run: "exact" under enumerated selection, "statistical" under sampled selection; the same value as upper_bound_kind on every row of training/convergence.parquet. Under "exact", the gap between the bounds is a valid optimality certificate only when the risk measure is the same at every stage; under a stage-varying measure final_upper_bound is the policy’s expected cost, which can fall below final_lower_bound (see Cut Management — when bounds and certificates hold).

solve_stats fields:

NameTypeNullableUnitsDescription
solve_stats.total_lp_solvesintegerYes—Total number of LP solves performed during training.
solve_stats.first_tryintegerYes—Number of LP solves that succeeded on the first attempt.
solve_stats.retriedintegerYes—Number of LP solves that succeeded after one or more retries.
solve_stats.failedintegerYes—Number of LP solves that failed terminally.
solve_stats.forward_solve_secondsnumberYessCumulative wall-clock seconds in forward-phase LP solves.
solve_stats.backward_solve_secondsnumberYessCumulative wall-clock seconds in backward-phase LP solves.
solve_stats.parallelismintegerYes—Degree of parallelism (worker count) used during training.

distribution fields:

NameTypeNullableUnitsDescription
distribution.backendstringNo—Communication backend: "mpi" or "local".
distribution.world_sizeintegerNo—Total number of processes in the communicator. 1 for single-process runs.
distribution.ranks_participatedintegerNo—Number of processes that participated in computation.
distribution.num_hostsintegerNo—Number of distinct physical hosts.
distribution.threads_per_rankintegerNo—Worker threads per process.
distribution.mpi_librarystringYes—MPI implementation version (e.g. "Open MPI v4.1.6"). Omitted for the local backend.
distribution.mpi_standardstringYes—MPI standard version (e.g. "MPI 4.0"). Omitted for the local backend.
distribution.thread_levelstringYes—Negotiated MPI thread safety level. Omitted for the local backend.
distribution.slurm_job_idstringYes—SLURM job ID when running under SLURM. Omitted otherwise.
distribution.hostsarrayNo—Per-host rank assignment. One entry per physical host. For local single-process runs, contains a single entry with ranks: [0].
distribution.hosts[].hostnamestringNo—Hostname for this entry.
distribution.hosts[].ranksarrayNo—Array of integers: sorted global ranks assigned to this host.

setup fields (informational):

NameTypeNullableUnitsDescription
setup.load_secondsnumberNosWall-clock seconds spent loading the input case.
setup.stochastic_fit_secondsnumberNosWall-clock seconds spent fitting the stochastic process.
setup.production_fit_secondsnumberNosWall-clock seconds spent fitting the production model (FPHA hyperplanes).
setup.evaporation_fit_secondsnumberNosWall-clock seconds spent fitting the evaporation model.
setup.broadcast_secondsnumberNosWall-clock seconds spent broadcasting setup data across MPI ranks.

These values are non-deterministic: they vary run-to-run with machine load and are excluded from any parity computation.

production_fit_deviation fields (absent when the run fitted no model whose deviation is measured):

NameTypeNullableUnitsDescription
production_fit_deviation.n_entriesintegerNo—Number of per-entity entries the rollup summarizes.
production_fit_deviation.mean_absnumberNo—Arithmetic mean of the per-entry mean absolute deviation magnitudes.
production_fit_deviation.max_absnumberNo—Maximum of the per-entry max absolute deviation magnitudes.
production_fit_deviation.worst_relativenumberNo—Largest per-entry relative (dimensionless) deviation across all entries.
production_fit_deviation.worst_entryobjectYes—The entry with the largest relative deviation. null when no entry exists.

production_fit_deviation.worst_entry fields:

NameTypeNullableUnitsDescription
production_fit_deviation.worst_entry.entity_idintegerNo—Identifier of the entity owning the worst entry.
production_fit_deviation.worst_entry.stage_idintegerNo—Declared stage id (from stages.json) of the first stage the worst entry covers.
production_fit_deviation.worst_entry.relativenumberNo—Relative (dimensionless) deviation of the worst entry.
production_fit_deviation.worst_entry.mean_absnumberNo—Mean absolute deviation magnitude of the worst entry.
production_fit_deviation.worst_entry.max_absnumberNo—Max absolute deviation magnitude of the worst entry.

simulation/metadata.json

Section titled “simulation/metadata.json”

The simulation metadata file is written atomically when simulation completes, and when a signal stop ends training before a configured simulation: the simulation is then skipped, and the file records status "partial", scenarios.completed 0 and cost null, followed by simulation/_SUCCESS. It captures run context, scenario completion counts, aggregate cost statistics, LP solver statistics, and distribution information.

Methodology: Determinism & Provenance

Example (the same novomodelo run of the 1dtoy template; hostname and timestamps replaced):

{
  "software": "novomodelo",
  "software_version": "0.18.0",
  "hostname": "<hostname>",
  "solver": "highs",
  "solver_version": "1.13.1",
  "started_at": "<timestamp>",
  "completed_at": "<timestamp>",
  "duration_seconds": 0.324,
  "status": "complete",
  "scenarios": {
    "total": 100,
    "completed": 100,
    "failed": 0
  },
  "cost": {
    "mean_cost": 9679385.922404818,
    "std_cost": 23696548.3646435
  },
  "solve_stats": {
    "total_lp_solves": 400,
    "first_try": 400,
    "retried": 0,
    "failed": 0,
    "solve_seconds": 0.053534688,
    "parallelism": 1
  },
  "distribution": {
    "backend": "local",
    "world_size": 1,
    "ranks_participated": 1,
    "num_hosts": 1,
    "threads_per_rank": 1,
    "hosts": [
      {
        "hostname": "<hostname>",
        "ranks": [0]
      }
    ]
  }
}

Top-level fields:

NameTypeNullableUnitsDescription
softwarestringNo—Name of the software that produced this output: "novomodelo".
software_versionstringNo—Version of that software, for example "0.18.0".
hostnamestringNo—Hostname of the machine that ran simulation.
solverstringNo—LP solver backend: "highs" or "clp".
solver_versionstringYes—LP solver library version string. Omitted when not available.
started_atstringNo—ISO 8601 timestamp when simulation started.
completed_atstringNo—ISO 8601 timestamp when simulation completed.
duration_secondsnumberNosTotal simulation wall-clock duration in seconds.
statusstringNo—"complete" when the simulation ran; "partial" when a signal stop skipped it (no scenario ran).

scenarios fields:

NameTypeNullableUnitsDescription
scenarios.totalintegerNo—Number of scenarios whose output partitions were written; on a skipped simulation (status "partial"), the number of scenarios the run was configured to simulate, with completed 0.
scenarios.completedintegerNo—Number of scenarios whose output partitions were written.
scenarios.failedintegerNo—Number of scenarios whose output partitions could not be written; a failed LP solve stops the run with an error instead.

cost fields (null on a skipped simulation):

NameTypeNullableUnitsDescription
cost.mean_costnumberNoUSDMean total cost across simulated scenarios.
cost.std_costnumberNoUSDStandard deviation of the total cost across simulated scenarios.

solve_stats fields:

NameTypeNullableUnitsDescription
solve_stats.total_lp_solvesintegerYes—Total number of LP solves performed during simulation.
solve_stats.first_tryintegerYes—Number of LP solves that succeeded on the first attempt.
solve_stats.retriedintegerYes—Number of LP solves that succeeded after one or more retries.
solve_stats.failedintegerYes—Number of LP solves that failed terminally.
solve_stats.solve_secondsnumberYessCumulative wall-clock seconds spent in simulation LP solves.
solve_stats.parallelismintegerYes—Degree of parallelism (worker count) used during simulation.

The distribution object has the same field structure as in training/metadata.json. See the distribution fields table above.

training/model_provenance.json

Section titled “training/model_provenance.json”

Records which data sources fed the inflow and hydro-production models. Written on every novomodelo run, before training starts; diagnostic. Top-level object with two sub-objects, inflow and hydro_production.

Methodology: Determinism & Provenance · PAR(p) Inflow Model

inflow fields:

NameTypeNullableUnitsDescription
inflow.estimation_pathstringNo—Stable label of the estimation path taken.
inflow.seasonal_stats_sourcestringNo—Origin of seasonal mean/std data: "estimated", "user_file", or "n/a".
inflow.ar_coefficients_sourcestringNo—Origin of the AR lag coefficients (same value set).
inflow.correlation_sourcestringNo—Origin of the spatial correlation decomposition (same value set).
inflow.opening_tree_sourcestringNo—Origin of the noise opening scenario tree (same value set).
inflow.n_hydrosintegerNo—Number of hydro plants in the system.
inflow.ar_methodstringYes—Order-selection method used when AR coefficients were estimated. null when AR was not estimated.
inflow.ar_max_orderintegerYes—Maximum AR order across all hydro plants. null when AR was not estimated.
inflow.white_noise_fallbacksarrayNo—Array of integers: IDs of hydro plants that fell back to white noise. Empty when none did.
inflow.historical_library_seed_digestintegerYes—Fingerprint of the inflow-lag seed for the historical library. Omitted when no library was built.

hydro_production fields:

NameTypeNullableUnitsDescription
hydro_production.n_fpha_computed_from_geometryintegerNo—Hydro plants whose FPHA hyperplanes were computed from reservoir geometry; a computed-FPHA plant with no turbine capacity is not counted.
hydro_production.n_fpha_precomputed_hyperplanesintegerNo—Hydro plants whose FPHA hyperplanes were supplied precomputed; a precomputed-FPHA plant with no turbine capacity and no hyperplane rows is not counted.
hydro_production.n_evaporation_ref_user_suppliedintegerNo—Hydro plants whose evaporation reference level was user-supplied.
hydro_production.n_evaporation_ref_default_midpointintegerNo—Hydro plants whose evaporation reference level defaulted to the curve midpoint.

Third-party notices

© 2026 Operador Nacional do Sistema Elétrico - ONS, Câmara de Comercialização de Energia Elétrica - CCEE, Empresa de Pesquisa Energética - EPE and Contributors, adapted from the Cobre Documentation © 2026 Rogerio J. M. Alves and Cobre Contributors — licensed under Apache-2.0