Skip to content

PAR(p) Inflow Model

This chapter defines the Periodic Autoregressive model of order pp (PAR(p)) used to capture temporal correlation in inflow time series. It states the model, its parameters and the quantities stored for it (§1); the form the LP stage subproblem consumes (§2); the estimation of the parameters from historical inflow data, including order selection (§3); the closure that derives the innovation scale from the AR coefficients (§4); the validation invariants (§5); the estimation and factorisation of the spatial correlation across hydros (§6); and the optional PAR(p)-A extension, which adds a single annual coefficient on top of the periodic AR structure to capture multi-year hydrological persistence (§7).

The Periodic Autoregressive model of order p (PAR(p)) captures temporal correlation in inflow time series while accounting for seasonal variation in parameters. For hydro hh at stage tt corresponding to season m(t)m(t):

ah,t=μm(t)+∑ℓ=1pψm(t),ℓ(ah,t−ℓ−μm(t−ℓ))+σm(t)⋅εta_{h,t} = \mu_{m(t)} + \sum_{\ell=1}^{p} \psi_{m(t),\ell} \left( a_{h,t-\ell} - \mu_{m(t-\ell)} \right) + \sigma_{m(t)} \cdot \varepsilon_t

where:

  • ah,ta_{h,t}: Incremental inflow at stage tt (m³/s)
  • μm(t)\mu_{m(t)}: Seasonal mean for season m(t)m(t)
  • ψm(t),ℓ\psi_{m(t),\ell}: Autoregressive coefficient for lag ℓ\ell in season m(t)m(t)
  • σm(t)\sigma_{m(t)}: Innovation standard deviation for season m(t)m(t) (derived at load — see §4)
  • εt\varepsilon_t ∼N(0,1)\sim \mathcal{N}(0, 1): Innovation (standardized noise)
  • m(t)m(t): The resolved season of stage tt; lag seasons are calendar predecessors (see Notation Conventions)

The model order pp can vary by season and by hydro plant.

For each hydro hh and each season m∈{1,…,M}m \in \{1, \ldots, M\} (e.g., M=12M = 12 for monthly, M=52M = 52 for weekly), the complete PAR(p) model requires:

ParameterSymbolDescription
Seasonal meanμm\mu_mMean inflow for season mm
AR coefficientsψm,1,…,ψm,p\psi_{m,1}, \ldots, \psi_{m,p}Autoregressive coefficients
Innovation standard deviationσm\sigma_mScale of the innovation term — derived, not independent (§4)

The data model stores seasonal sample statistics and standardized AR coefficients — nothing else. The innovation scale is not a third stored quantity: the coefficient file holds the scale-invariant ψ∗\psi^* alone; at load the dimensionless innovation scale rmr_m is derived from ψ∗\psi^* by a periodic-ACF closure (§4.1), then everything is converted to original-unit ψ\psi and σ\sigma using the seasonal stats and consumed by the LP stage subproblem.

Two planes. Every PAR(pp) parameter sits on one of two planes, and the split is deliberate:

  • Conditioning plane — the seasonal mean μm\mu_m and seasonal sample standard deviation sms_m (both m³/s): the level and magnitude of the series. A study may re-condition these — e.g. a climate scenario that shifts both the seasonal mean and the seasonal variability.
  • Dynamics plane — the standardized coefficients ψm,ℓ∗\psi^*_{m,\ell} and the standardized innovation scale rm=σm/smr_m = \sigma_m/s_m (both dimensionless): the shape of the temporal dependence, independent of magnitude. Only ψm,ℓ∗\psi^*_{m,\ell} is an independent parameter here — rmr_m is pinned by ψm,ℓ∗\psi^*_{m,\ell} itself.

Runtime re-couples the two through the coefficient conversion of §2.2 and the innovation standard deviation of §4.2, so re-conditioning the first plane rescales both the coefficients and the noise while preserving the correlation structure. The standardization basis is the seasonal std: ψ∗\psi^* is the coefficient of the process normalised by sms_m — the classical standardization of the periodic AR literature, and the same basis an externally-fitted model supplies (see the Inputs & Outputs tab).

Storage format — files on diskPeriodic-ACF closurederives rₘ from ψ* aloneIn-memory runtime formatconsumed by LP stage subproblemseasonal statisticsAR coefficientsOriginal-unit AR coeff: ψₘ,ℓ = ψ*ₘ,ℓ · sₘ / sₘ₋ℓOriginal-unit innovation std: σₘ = sₘ · rₘμₘ — seasonal meansₘ — seasonal std (sample)ψ*ₘ,ℓ — standardized AR coeffpₘ — AR order × sₘ / sₘ₋ℓrₘ, then × sₘ

Stored quantities. For each hydro and season the data model stores:

Stored quantitySymbolDescription
Seasonal sample meanμm=aˉm\mu_m = \bar{a}_mMean of historical observations for season mm
Seasonal sample stdsms_mStandard deviation of historical observations for season mm
AR coefficientsψm,ℓ∗\psi^*_{m,\ell}AR coefficient standardized by seasonal std — the direct Yule-Walker output

The AR order pmp_m is not stored explicitly. It is derived at runtime from the number of stored coefficients of each (hydro, stage).

The standardized coefficient ψm,ℓ∗\psi^*_{m,\ell} is the direct output of the Yule-Walker fitting procedure (§3.5). It is dimensionless — the coefficient of the standardized process (ah,t−μm)/sm(a_{h,t} - \mu_m) / s_m — and §2.2 converts it to the original-unit coefficient ψm,ℓ\psi_{m,\ell} used in the LP.

This section derives the explicit algebraic transformation from the canonical PAR(p) model (§1) into the form consumed by the LP subproblem. The derivation identifies three precomputable components that are cached once at initialization and reused at every forward-pass stage transition.

The PAR(p) model (§1) operates on deviations from the seasonal mean, normalised by the seasonal (marginal) standard deviation sm(t)s_{m(t)} — the same basis the stored coefficients ψ∗\psi^* use, and the same normalisation as the reference formulation (standardise by the marginal std, not the innovation std). In this standardized form:

ah,t−μm(t)sm(t)=∑ℓ=1pψm(t),ℓ∗ ah,t−ℓ−μm(t−ℓ)sm(t−ℓ)+rm(t) εt\frac{a_{h,t} - \mu_{m(t)}}{s_{m(t)}} = \sum_{\ell=1}^{p} \psi^*_{m(t),\ell}\, \frac{a_{h,t-\ell} - \mu_{m(t-\ell)}}{s_{m(t-\ell)}} + r_{m(t)}\,\varepsilon_t

where:

  • ψm(t),ℓ∗\psi^*_{m(t),\ell}: the stored AR coefficients, standardized by the seasonal std sms_m (the direct Yule-Walker output of §3.5)
  • sm(t)s_{m(t)}: the seasonal (marginal) standard deviation for season m(t)m(t)
  • rm(t)=σm(t)/sm(t)∈(0,1]r_{m(t)} = \sigma_{m(t)}/s_{m(t)} \in (0, 1]: the innovation standard deviation of the standardized process — the closure-derived innovation scale of §4.1, 1−∑ℓψm,ℓ∗ ρm(t)(ℓ)\sqrt{1 - \sum_\ell \psi^*_{m,\ell}\,\rho_{m(t)}(\ell)}, with ρm(t)\rho_{m(t)} the model’s implied periodic ACF
  • εt∼N(0,1)\varepsilon_t \sim \mathcal{N}(0, 1): unit-variance innovation noise

The innovation of the sms_m-standardized process is not unit-variance: because that process has unit marginal variance, its one-step innovation has standard deviation rm(t)≤1r_{m(t)} \le 1 — which is exactly why rm(t)r_{m(t)} appears here explicitly. The next step converts ψm,ℓ∗\psi^*_{m,\ell} to original-unit ψm,ℓ\psi_{m,\ell} for use in the LP.

The stored standardized coefficients ψm,ℓ∗\psi^*_{m,\ell} are converted to original-unit coefficients ψm,ℓ\psi_{m,\ell} at runtime using the stored seasonal standard deviations:

ψm,ℓ=ψm,ℓ∗⋅smsm−ℓ\psi_{m,\ell} = \psi^*_{m,\ell} \cdot \frac{s_m}{s_{m-\ell}}

The innovation standard deviation σm\sigma_m of §4.2 is also derived at this preprocessing step, from the closure-derived innovation scale rmr_m (§4.1).

These conversions are performed once at LP construction time. They require only the seasonal stats (sms_m), the stored ψm,ℓ∗\psi^*_{m,\ell}, and the closure-derived rmr_m — no historical data.

Multiplying both sides of the canonical form (§2.1) by sm(t)s_{m(t)} and rearranging yields the LP-ready equation (the noise term becomes sm(t) rm(t) εt=σm(t) εts_{m(t)}\,r_{m(t)}\,\varepsilon_t = \sigma_{m(t)}\,\varepsilon_t):

ah,t=∑ℓ=1pψm(t),ℓ⋅ah,t−ℓ+[μm(t)−∑ℓ=1pψm(t),ℓ⋅μm(t−ℓ)]+σm(t)⋅εta_{h,t} = \sum_{\ell=1}^{p} \psi_{m(t),\ell} \cdot a_{h,t-\ell} + \left[ \mu_{m(t)} - \sum_{\ell=1}^{p} \psi_{m(t),\ell} \cdot \mu_{m(t-\ell)} \right] + \sigma_{m(t)} \cdot \varepsilon_t

where ψm(t),ℓ\psi_{m(t),\ell} and σm(t)\sigma_{m(t)} are derived from stored quantities as described in §2.2.

This decomposes the inflow into three additive components:

  1. Lag contribution: ∑ℓ=1pψm(t),ℓ⋅ah,t−ℓ\displaystyle\sum_{\ell=1}^{p} \psi_{m(t),\ell} \cdot a_{h,t-\ell} — linear function of past inflows (state variables or known values)
  2. Deterministic base: μm(t)−∑ℓ=1pψm(t),ℓ⋅μm(t−ℓ)\displaystyle\mu_{m(t)} - \sum_{\ell=1}^{p} \psi_{m(t),\ell} \cdot \mu_{m(t-\ell)} — constant offset per (stage, hydro), precomputed once
  3. Stochastic innovation: σm(t)⋅εt\sigma_{m(t)} \cdot \varepsilon_t — noise draw scaled by the seasonal innovation standard deviation

The deterministic base is defined as:

bh,m(t)=μm(t)−∑ℓ=1pψm(t),ℓ⋅μm(t−ℓ)b_{h,m(t)} = \mu_{m(t)} - \sum_{\ell=1}^{p} \psi_{m(t),\ell} \cdot \mu_{m(t-\ell)}

This is a precomputed constant per (stage, hydro) pair. It absorbs the mean-adjustment arithmetic that would otherwise be repeated at every forward-pass stage transition. With this definition, the LP-ready form (§2.3) simplifies to:

ah,t=∑ℓ=1pψm(t),ℓ⋅ah,t−ℓ+bh,m(t)+σm(t)⋅εta_{h,t} = \sum_{\ell=1}^{p} \psi_{m(t),\ell} \cdot a_{h,t-\ell} + b_{h,m(t)} + \sigma_{m(t)} \cdot \varepsilon_t

For partial-year studies, the lag-season means μm(t−ℓ)\mu_{m(t-\ell)} for seasons preceding the study start are sourced from the pre-study lag window (§3.8); when no such statistic exists for a given lag season, that lag’s mean contribution is treated as zero.

2.5 Lagged Inflows Pinned by Column Bounds

Section titled “2.5 Lagged Inflows Pinned by Column Bounds”

The lagged inflows ah,ℓa_{h,\ell} are LP variables, not substituted values. In the LP, they appear with coefficients −ψm(t),ℓ-\psi_{m(t),\ell} in the AR dynamics constraint row, and each incoming lag column is pinned to the trajectory’s lag value a^h,ℓ\hat{a}_{h,\ell} by setting both of its column bounds to that value: there are no fixing rows, as for every state variable in State Augmentation §2. The value a^h,ℓ\hat{a}_{h,\ell} is patched per scenario from the trajectory record.

Because the lag contribution ∑ℓψm(t),ℓ⋅ah,ℓ\sum_\ell \psi_{m(t),\ell} \cdot a_{h,\ell} is carried by the constraint matrix (not the RHS), the AR dynamics constraint RHS reduces to:

RHSh,t=bh,m(t)+σm(t)⋅εt\text{RHS}_{h,t} = b_{h,m(t)} + \sigma_{m(t)} \cdot \varepsilon_t

where:

  • bh,m(t)b_{h,m(t)} is the deterministic base for (stage, hydro), precomputed once at LP construction (§2.4)
  • σm(t)\sigma_{m(t)} is the noise scale for (stage, hydro), derived from the closure-derived rmr_m (§4.1) at initialization (§2.2)
  • εt\varepsilon_t is the scenario noise draw for this (stage, hydro)

The ψm(t),ℓ\psi_{m(t),\ell} coefficients are written into the constraint matrix once at LP construction time as the coefficients on the lagged inflow variables; they are not recomputed per scenario.

No division, no mean subtraction, no repeated coefficient transformation — the three precomputed LP components eliminate all redundant arithmetic from the hot path.

Whether the lagged inflows enter a stage’s cuts is a per-stage choice; a cut that projects them out carries their contribution in its intercept at the trial lag values (see State Augmentation §7).

ComponentSymbolShape per stageLP RoleSource
Lag coefficientsψm(t),ℓ\psi_{m(t),\ell}One per (hydro, lag)Constraint matrix (AR dynamics row)Derived from stored ψ∗\psi^* and sms_m at initialization (§2.2)
Deterministic basebh,m(t)b_{h,m(t)}One per hydroAR dynamics constraint RHS (fixed term)Precomputed from μ\mu and ψ\psi
Noise scaleσm(t)\sigma_{m(t)}One per hydroAR dynamics constraint RHS (noise factor)Derived from the closure-derived rmr_m (§4.1) and sms_m at initialization (§2.2)

For multi-resolution studies (monthly→quarterly aggregation), the same fitting procedure applies after duration-weighted aggregation; see Multi-Resolution Studies.

This section documents the procedure for fitting PAR(p) parameters from historical inflow data. The fitting is performed when the system derives parameters from the inflow history. When pre-computed seasonal statistics and AR coefficients are provided directly, this procedure is not executed. The innovation scale is deliberately not part of the fitting: it is derived from the stored coefficients at load (§4.1), identically for fitted and user-supplied models (§4.2).

The figure below traces both origins of the coefficients to the runtime model. When the coefficients are fitted from the inflow history, the seasonal statistics of each (hydro, season) are estimated and the bucket classification applied (§3.2, §3.3); a season whose seasonal std is zero takes order 0 directly. Every other season selects its order from the periodic PACF (§3.4, §3.6) and solves the periodic Yule-Walker system at that order for its standardized coefficients ψ∗\psi^* (§3.5). A season is reset to order 0 when its first coefficient is negative, after the initial fit or any re-fit, or when its initial fit has a coefficient of magnitude above the optional magnitude bound. A season with a negative composed influence loses one lag of its order ceiling and is re-selected and re-fitted, until no season fails; the coefficients are then stored (§3.7), and the residual correlation is estimated when the case supplies none (§6.1). Supplied coefficients pass the stationarity gate of invariant 2 (§5) instead, and a set that fails it is rejected. When the case supplies only the coefficients, or only the seasonal statistics, alongside the history, the history provides the other part and the residual correlation is estimated as above. Every model, fitted or supplied, then passes through the closure that derives its innovation scale rmr_m (§4.1), which rejects a singular closure system or a negative implied residual variance; the runtime coefficients ψ\psi and noise scale σ\sigma follow (§4.2).

Inflow historySeasonal statisticsbucket classificationPeriodic PACForder selectionYule-Walker fitstandardized ψ*Order-0 resetsψ*ₘ,₁ < 0initial fit: magnitude boundComposed influence ≥ 0for every season?Store ψ*residual correlationSupplied ψ*Stationarity gatesolvable, |ρ| ≤ 1,r² above floor,monodromy < 1RejectedClosurederives rₘRuntime ψ, σ yesŝₘ = 0: order 0passfailsingular, or r² < 0no: ceiling − 1

Let Ym={ah,t:m(t)=m}Y_m = \{a_{h,t} : m(t) = m\} be the historical observations for season mm. Define:

SymbolDescription
NmN_mNumber of observations for season mm
aˉm\bar{a}_mSample mean for season mm
sms_mSample standard deviation for season mm
γm(ℓ)\gamma_m(\ell)Autocovariance at lag ℓ\ell for season mm
ρm(ℓ)\rho_m(\ell)Autocorrelation at lag ℓ\ell for season mm

3.2 Seasonal Means and Standard Deviations

Section titled “3.2 Seasonal Means and Standard Deviations”

Seasonal Mean:

μ^m=aˉm=1Nm∑t:m(t)=mah,t\hat{\mu}_m = \bar{a}_m = \frac{1}{N_m} \sum_{t: m(t) = m} a_{h,t}

Seasonal Standard Deviation:

s^m=1Nm∑t:m(t)=m(ah,t−aˉm)2\hat{s}_m = \sqrt{\frac{1}{N_m} \sum_{t: m(t) = m} (a_{h,t} - \bar{a}_m)^2}

The estimator uses the population divisor 1/Nm1/N_m, not the Bessel-corrected 1/(Nm−1)1/(N_m - 1). This matches the Maceira & Damázio (2006) convention and is shared by the classical PAR(p) and PAR(p)-A paths. The population divisor is required for self-consistent conditional FACP values and selected orders on the PAR(p)-A path — under a Bessel correction the sample-vs-population scale factor leaks through every Z⊗A cross-correlation. Using the same divisor for the classical path keeps the two paths’ seasonal-stats output reusable across configurations.

Before the seasonal stats and AR coefficients are used by the order-selection rules, each per-(hydro, season) historical bucket is classified by the shape of its observations. The classification can override the empirical (μ^m,s^m)(\hat{\mu}_m, \hat{s}_m) for fitting purposes, and the override propagates to both the classical PAR(p) and the PAR(p)-A paths because both paths share the seasonal-stats producer.

Four classes are defined:

ClassDetection ruleOverride applied
DefaultNone of the conditions belowNone — use empirical (μ^m,s^m)(\hat{\mu}_m, \hat{s}_m)
ConstantEvery observation equals the same value within float tolerance(μ^m,s^m)←(value,0)(\hat{\mu}_m, \hat{s}_m) \leftarrow (\text{value}, 0)
Many negativeStrictly negative observations exceed 10% of the bucketNone — diagnostic only, fit proceeds on the empirical stats
SaturatedThe modal value (rounded to m³/s) occupies more than 50% of observations(μ^m,s^m)←(cap,0)(\hat{\mu}_m, \hat{s}_m) \leftarrow (\text{cap}, 0)

The classifier runs in the priority order constant → many negative → saturated → default. Constancy takes precedence over negative-pathology detection, which in turn takes precedence over saturation.

Why a zero seasonal std short-circuits the fit

Section titled “Why a zero seasonal std short-circuits the fit”

When the override sets s^m=0\hat{s}_m = 0 for a season, the fit short-circuits that season on both paths:

  • On the classical PAR(p) path, the season takes order 0 directly; its PACF (§3.6) is not evaluated.
  • On the PAR(p)-A path, the season likewise takes order 0 directly, with annual coefficient ψmA∗=0\psi^{A*}_m = 0; its conditional FACP (§7.5) is not evaluated.

The seasonal statistics that feed the fit are estimated from the history, even when the case supplies its own, and estimating them needs at least two observations per season, so no season reaches the fit with a nonzero std and Nm<2N_m < 2. A season with a single observation (Nm=1N_m = 1) rejects the case. A season with none (Nm=0N_m = 0) has no statistics; the fit reads its std as zero and short-circuits it in the same way on both paths. On the PAR(p)-A path it also does so when the season’s annual-regressor bucket is empty (NmA=0N^A_m = 0) or has zero std (σ^mA=0\hat{\sigma}^A_m = 0, §7.3), and the season then carries no annual contribution.

The zero-std guard of §3.4 also zeroes every sample autocorrelation that involves the season, so the bucket cannot inject spurious autoregressive structure into adjacent months’ PACFs, and no spatial-correlation contribution flows from it during scenario generation.

  • Constant captures plants whose incremental inflow is structurally constant for a given month — typically regulated or transposed flows where the upstream subtraction yields the same value every year. Forcing (value,0)(\text{value}, 0) records the deterministic level without inventing autoregressive dynamics.
  • Saturated captures flow caps (turbine or reservoir capacity) and low-flow constants (transposed ecological flows). The modal value is treated as the cap. There is no magnitude threshold — a cap of 0 m³/s qualifies just as readily as a cap at installed capacity.
  • Many negative flags buckets that the upstream incremental-inflow construction has driven below zero for more than 10% of observations. The condition is recorded for operator diagnostics but does not override the fit — the cause is upstream-data quality, not a methodological signal.
  • Default is the standard path; the empirical stats and the chosen order-selection rule decide the order.

The autocorrelation at lag ℓ\ell for season mm is computed from standardized deviations.

Cross-seasonal autocovariance:

For observations at season mm with lag ℓ\ell reaching back to season m−ℓm - \ell (mod MM, where MM is the cycle length):

γ^m(ℓ)=1Nm(ℓ)∑t:m(t)=m(ah,t−aˉm)(ah,t−ℓ−aˉm−ℓ)\hat{\gamma}_m(\ell) = \frac{1}{N_m^{(\ell)}} \sum_{t: m(t) = m} \left( a_{h,t} - \bar{a}_m \right) \left( a_{h,t-\ell} - \bar{a}_{m-\ell} \right)

where Nm(ℓ)N_m^{(\ell)} is the number of year-aligned valid pairs at lag ℓ\ell for reference season mm. The estimator uses the population divisor 1/Nm(ℓ)1/N_m^{(\ell)}, matching the convention adopted in §3.2 and shared by the classical and PAR(p)-A paths.

Autocorrelation:

ρ^m(ℓ)=γ^m(ℓ)s^m⋅s^m−ℓ\hat{\rho}_m(\ell) = \frac{\hat{\gamma}_m(\ell)}{\hat{s}_m \cdot \hat{s}_{m-\ell}}

where s^m−ℓ\hat{s}_{m-\ell} is the standard deviation of season m−ℓm - \ell (cyclically, so season 0 = season MM).

Zero-std guard. ρ^m(ℓ)=0\hat{\rho}_m(\ell) = 0 when s^m\hat{s}_m or s^m−ℓ\hat{s}_{m-\ell} is zero, or when season mm has no year-aligned pair at lag ℓ\ell (Nm(ℓ)=0N_m^{(\ell)} = 0). Every estimate is clamped to [−1,1][-1, 1].

For each season mm, the PAR(p) coefficients ψm,1∗,…,ψm,p∗\psi_{m,1}^*, \ldots, \psi_{m,p}^* in standardized form are found by solving the periodic Yule-Walker system. Unlike the classical (stationary) Yule-Walker equations where all rows use the same reference season, the periodic variant shifts the reference season per row. This correctly accounts for the non-Toeplitz covariance structure of periodic autoregressive processes.

Matrix construction: Entry (i,j)(i, j), 1≤i,j≤p1 \leq i, j \leq p, is the correlation between the lagged observations a~t−i\tilde{a}_{t-i} and a~t−j\tilde{a}_{t-j}; its reference season is that of the more recent of the two lags:

[Rm]i,j=ρ^(m−min⁡(i,j)) mod M(∣j−i∣)[\mathbf{R}_m]_{i,j} = \hat{\rho}_{\left(m - \min(i,j)\right) \bmod M}\bigl(|j - i|\bigr)

where MM is the number of seasons in the periodic cycle (e.g., 12 for monthly). The diagonal entries are always 1 (since ρ^m′(0)=1\hat{\rho}_{m'}(0) = 1 for any season m′m'). The matrix is symmetric but not Toeplitz when M>1M > 1, because the reference season varies across the entries.

RHS construction: The RHS holds the correlations of the target a~t\tilde{a}_t (season mm) with each lag:

[ρ^m]i=ρ^m(i)[\hat{\boldsymbol{\rho}}_m]_i = \hat{\rho}_{m}(i)

Equivalently: form the extended (p+1)×(p+1)(p{+}1) \times (p{+}1) correlation matrix of (a~t,a~t−1,…,a~t−p)(\tilde{a}_t, \tilde{a}_{t-1}, \ldots, \tilde{a}_{t-p}); the system matrix is its lag block (rows/columns 1..p1..p) and the RHS is its first row (the target-lag correlations). Row ii of the system is the model’s second-moment recursion at lag ii.

The full system is:

(1ρ^(m−1)(1)ρ^(m−1)(2)⋯ρ^(m−1)(p−1)ρ^(m−1)(1)1ρ^(m−2)(1)⋯ρ^(m−2)(p−2)ρ^(m−1)(2)ρ^(m−2)(1)1⋯ρ^(m−3)(p−3)⋮⋮⋮⋱⋮ρ^(m−1)(p−1)ρ^(m−2)(p−2)ρ^(m−3)(p−3)⋯1)(ψm,1∗ψm,2∗ψm,3∗⋮ψm,p∗)=(ρ^m(1)ρ^m(2)ρ^m(3)⋮ρ^m(p))\begin{pmatrix} 1 & \hat{\rho}_{(m-1)}(1) & \hat{\rho}_{(m-1)}(2) & \cdots & \hat{\rho}_{(m-1)}(p{-}1) \\ \hat{\rho}_{(m-1)}(1) & 1 & \hat{\rho}_{(m-2)}(1) & \cdots & \hat{\rho}_{(m-2)}(p{-}2) \\ \hat{\rho}_{(m-1)}(2) & \hat{\rho}_{(m-2)}(1) & 1 & \cdots & \hat{\rho}_{(m-3)}(p{-}3) \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \hat{\rho}_{(m-1)}(p{-}1) & \hat{\rho}_{(m-2)}(p{-}2) & \hat{\rho}_{(m-3)}(p{-}3) & \cdots & 1 \end{pmatrix} \begin{pmatrix} \psi_{m,1}^* \\ \psi_{m,2}^* \\ \psi_{m,3}^* \\ \vdots \\ \psi_{m,p}^* \end{pmatrix} = \begin{pmatrix} \hat{\rho}_{m}(1) \\ \hat{\rho}_{m}(2) \\ \hat{\rho}_{m}(3) \\ \vdots \\ \hat{\rho}_{m}(p) \end{pmatrix}

where season indices are taken modulo MM on {1,…,M}\{1, \ldots, M\}, so season 00 is season MM (see Notation Conventions).

In matrix notation: Rmψm∗=ρ^m\mathbf{R}_m \boldsymbol{\psi}_m^* = \hat{\boldsymbol{\rho}}_m

where:

  • Rm\mathbf{R}_m is the p×pp \times p periodic correlation matrix (symmetric but not Toeplitz for M>1M > 1)
  • ρ^m\hat{\boldsymbol{\rho}}_m is the vector of target autocorrelations with per-row reference season shifting

Solution:

ψ^m∗=Rm−1ρ^m\hat{\boldsymbol{\psi}}_m^* = \mathbf{R}_m^{-1} \hat{\boldsymbol{\rho}}_m

The system is solved via Gaussian elimination with partial pivoting, which is numerically adequate for the small systems that periodic Yule-Walker estimation solves.

Singular system. When the periodic Yule-Walker system is singular at order kk, the PACF of §3.6 stops at lag k−1k-1, so only lags below kk can be selected and the fit at the selected order meets no singular system.

The PAR order pp can vary by season. Two order-selection rules are available: the classical periodic PACF rule, described below, and the order-selection rule with the annual component, which augments the same PACF rule for PAR(p)-A (§7). The fit considers lags up to a maximum lag pmaxp_{max} and applies an optional magnitude bound to the coefficients of the initial fit (below).

The classical rule computes the periodic PACF (periodic partial autocorrelation function) via progressive periodic Yule-Walker matrix solves at orders k=1,2,…,pmaxk = 1, 2, \ldots, p_{max}, then selects the order using a significance threshold.

Algorithm:

  1. For each order kk from 1 to pmaxp_{max}, build and solve the periodic Yule-Walker system (§3.5) at order kk. The last coefficient ψ^m,k∗\hat{\psi}^*_{m,k} from the order-kk solution is the periodic PACF value at lag kk.

  2. Select the order as the maximum lag with significant PACF:

    pm=max⁡{k:∣PACFm(k)∣>z0.975Nm}p_m = \max \left\{ k : |\text{PACF}_m(k)| > \frac{z_{0.975}}{\sqrt{N_m}} \right\}

    where z0.975=1.96z_{0.975} = 1.96 (95% confidence) and NmN_m is the number of observations for season mm. If no lag is significant, pm=0p_m = 0 (white noise).

  3. Estimate AR coefficients at the selected order using the periodic Yule-Walker system (§3.5).

The figure below applies the rule to a synthetic series: a twelve-season PAR(1) process with unit marginal variance in every season, simulated with a fixed random seed for Nm=60N_m = 60 years and shown for season 6. The upper panel compares the sample periodic autocorrelation ρ^6(ℓ)\hat{\rho}_6(\ell) of §3.4 with the model’s implied autocorrelation of §4.1, which for a PAR(1) is ρm(ℓ)=∏j=0ℓ−1ψm−j,1∗\rho_m(\ell) = \prod_{j=0}^{\ell-1} \psi^*_{m-j,1} with season indices taken modulo MM. The lower panel shows the periodic PACF of step 1, for the sample and for the model, against the band ±z0.975/Nm\pm z_{0.975}/\sqrt{N_m}, which is the significance threshold of step 2: a lag is significant when its sample PACF lies outside the band. The model’s PACF is ψm,1∗\psi^*_{m,1} at lag 1 and zero beyond, so the autocorrelation decays over all six lags while the PACF cuts off after lag 1. Here the sample PACF at lag 1 lies outside the band and the sample PACF at lags 2 to 6 lies inside it, so, with pmax=6p_{max} = 6, the rule selects p6=1p_6 = 1.

Post-selection validation — iterative order reduction (Maceira & Damázio, 2006): After PACF selection, the composed influence of each lag on the current inflow through the periodic chain is computed for every season, holding the inflows before t−ℓt-\ell and the innovations fixed:

∂ah,t∂ah,t−ℓ=ψm(t),ℓ+∑ℓ′=1ℓ−1ψm(t),ℓ′ ∂ah,t−ℓ′∂ah,t−ℓ,ℓ=1,…,pm(t)\frac{\partial a_{h,t}}{\partial a_{h,t-\ell}} = \psi_{m(t),\ell} + \sum_{\ell'=1}^{\ell-1} \psi_{m(t),\ell'}\, \frac{\partial a_{h,t-\ell'}}{\partial a_{h,t-\ell}}, \qquad \ell = 1, \ldots, p_{m(t)}

where ψm,ℓ\psi_{m,\ell} is the original-unit coefficient of §2.2, taken as zero when sm−ℓ=0s_{m-\ell} = 0 or ℓ>pm\ell > p_m, and each derivative on the right follows from the same recursion at stage t−ℓ′t-\ell'. The check is a sign condition on the composed lag influence, not a stationarity test: a season fails when any of its composed influences is negative. A failing season loses one lag of its order ceiling (initially pmaxp_{max}) and is re-selected and re-fitted at the new ceiling; a season whose ceiling reaches 0 takes order 0, and the loop repeats until no season fails. A season whose first coefficient satisfies ψm,1∗<0\psi^*_{m,1} < 0, after the initial fit or after any re-fit, is reset to order 0, and so is a season whose initial fit has a coefficient of magnitude above the optional magnitude bound.

For the PAR(p)-A path (§7), two additional rules extend the PACF gate:

  • Structural-zero short-circuit at lag 1. When the conditional FACP value at lag 1 is exactly zero — which happens when the standardised inflow series ZZ of §7.5 collapses in the preceding season, typically because a degenerate constant or saturated bucket has zeroed that season’s std (§3.3) — the selected order is forced to 0 (white noise). This blocks degenerate buckets from injecting spurious AR structure.
  • Minimum order 1 when lag 1 is well defined. When the lag-1 conditional FACP is non-zero but no lag exceeds the significance threshold, the model defaults to order 1 rather than order 0. Hydrological persistence makes a strict order-0 fit a poor default unless the lag-1 value is structurally absent.

The Yule-Walker solution ψm,ℓ∗\psi_{m,\ell}^* is in standardized form — the direct output of §3.5. It is stored as-is, one coefficient per lag. No conversion to original units is performed, and no other quantity is written alongside it: the innovation scale is not a fitting output. It is derived from ψ∗\psi^* afterward, uniformly for every model regardless of its origin, by the periodic-ACF closure of §4.1 — not computed here.

3.8 Partial-Year Studies and the Pre-Study Lag Window

Section titled “3.8 Partial-Year Studies and the Pre-Study Lag Window”

A study horizon may be narrower than the seasonal cycle — e.g. a monthly model (M=12M = 12) running only September–December. The per-season fitting described above must then handle seasons that have few or no in-window observations. Two rules keep it well-defined.

Lag-reachability. A season is lag-reachable only if some stage of the (extended) horizon carries it. Each historical observation is resolved to a season from the stage date ranges, falling back to the season-map calendar for dates predating the horizon; an observation whose resolved season has no stage at all is skipped — its statistics would never be consumed. Full-cycle history therefore does not perturb a partial-year fit.

Pre-study lag synthesis (for p>0p > 0). The first study stage’s autoregressive lags reach back to seasons before the study start. For each lag ℓ=1,…,min⁡(p, M−1)\ell = 1, \ldots, \min(p,\, M - 1), the season ℓ\ell calendar positions before the first study season is introduced as a pre-study season — unless that season is already covered by a study stage (an in-window wrap lag, handled by the cycle-correct lag lookup). The seasonal statistics (μ^m,s^m)(\hat{\mu}_m, \hat{s}_m) of those out-of-window seasons are estimated from history exactly as for in-window seasons, then feed the lag terms of the opening study stages — both the coefficient conversion (§2.2) and the deterministic base (§2.4).

The walk steps back through the calendar occurrences of the first study season’s own resolution, whatever the seasons’ numbering — so, e.g., a March-start study maps the lag-1 season to February, not December.

Full-cycle invariance. When the study spans the full cycle (every season already has a study stage) or carries no out-of-window history, nothing is synthesized and the fit equals the full-cycle fit.

Unlike μm\mu_m, sms_m, and ψm,ℓ∗\psi^*_{m,\ell}, the innovation scale rmr_m is not read from a file. It is pinned by ψ∗\psi^* itself under the model’s unit-marginal-variance contract — every season’s standardized process (ah,t−μm)/sm(a_{h,t}-\mu_m)/s_m has unit variance — via a periodic-ACF closure.

Extend the recursion that defines ψm,ℓ∗\psi^*_{m,\ell} (§3.5) to every lag ℓ≥1\ell \ge 1, not just ℓ≤pm\ell \le p_m:

ρm(ℓ)=∑ℓ′=1pmψm,ℓ′∗ ρ(m−min⁡(ℓ′,ℓ)) mod M(∣ℓ−ℓ′∣),ρm′(0)=1 for every season m′\rho_m(\ell) = \sum_{\ell'=1}^{p_m} \psi^*_{m,\ell'}\, \rho_{(m-\min(\ell',\ell)) \bmod M}\bigl(|\ell - \ell'|\bigr), \qquad \rho_{m'}(0) = 1 \text{ for every season } m'

Solving this system jointly across every season m=1,…,Mm = 1, \ldots, M and every lag ℓ=1,…,Ph\ell = 1, \ldots, P_h (where Ph=max⁡mpmP_h = \max_m p_m) yields the model’s own implied periodic autocorrelation function ρm(ℓ)\rho_m(\ell) — the autocorrelations the process would have if ψ∗\psi^* described an exactly stationary periodic AR process. The innovation scale follows from the same variance decomposition used in the fitting step (§3.5):

rm2=1−∑ℓ=1pmψm,ℓ∗ ρm(ℓ)r_m^2 = 1 - \sum_{\ell=1}^{p_m} \psi^*_{m,\ell}\, \rho_m(\ell)

A season with pm=0p_m = 0 contributes no terms to the sum, so rm=1r_m = 1 — a white-noise season gets no variance reduction from an AR part it does not have.

When every season of the cycle shares the same AR order, the closure value coincides exactly with the fitting step’s own Yule-Walker system:

rm=1−ψm∗⊤ρ^mr_m = \sqrt{1 - \boldsymbol{\psi}_m^{*\top} \hat{\boldsymbol{\rho}}_m}

(using the Yule-Walker solution ψm∗\boldsymbol{\psi}_m^* and the RHS vector ρ^m\hat{\boldsymbol{\rho}}_m of §3.5). When per-season orders differ across the cycle, the two need not agree: the closure solves for the model’s implied ACF jointly across every season, not from this season’s sample autocorrelations alone.

From the stored quantities and the derived rmr_m, the LP requires two additional quantities computed once at initialization: the original-unit AR coefficients ψm,ℓ\psi_{m,\ell} of §2.2 (for LP constraint matrix entries) and the innovation standard deviation (the noise scale), recovered at load from the closure-derived innovation scale rmr_m of §4.1:

σm=sm⋅rm\sigma_m = s_m \cdot r_m

No further autocorrelation values are needed beyond the closure derivation of §4.1. All required quantities — ψm,ℓ\psi_{m,\ell}, σm\sigma_m, and the rmr_m they depend on — are derived solely from the stored seasonal stats and AR coefficients, with no historical data and no separate noise-scale input.

For a model estimated from history this is σ^m=s^m rm\hat{\sigma}_m = \hat{s}_m\, r_m, and when every season of the cycle shares the same AR order, the equal-order identity of §4.1 reduces it to the fitting system’s own closed-form expression:

σ^m=s^m1−ρ^m⊤Rm−1ρ^m\hat{\sigma}_m = \hat{s}_m \sqrt{1 - \hat{\boldsymbol{\rho}}_m^\top \mathbf{R}_m^{-1} \hat{\boldsymbol{\rho}}_m}

Novomodelo enforces a small, well-defined set of invariants; they fall into three groups by where the check runs. Fitted coefficients are not tested for stationarity; supplied coefficients pass the stationarity gate of invariant 2.

Enforced when loading the AR coefficients:

  1. AR order derivation: the number of stored coefficients per (hydro, stage) determines the AR order pmp_m, and the lags must be present and contiguous {1,2,…,pm}\{1, 2, \ldots, p_m\}.
  2. Stationarity of directly-supplied coefficients: when the AR coefficients for a (hydro, season) group are supplied directly rather than produced by the internal fitting procedure (§3), the periodic-ACF closure (§4.1) must accept them — the closure system must be solvable (non-singular), every implied innovation variance rm2=1−∑ℓψm,ℓ∗ ρm(ℓ)r_m^2 = 1 - \sum_\ell \psi^*_{m,\ell}\,\rho_m(\ell) must lie above a numerical floor, every implied autocorrelation must satisfy ∣ρm(ℓ)∣≤1|\rho_m(\ell)| \le 1, and the periodic monodromy — the product, taken once around the full seasonal cycle, of each season’s AR companion matrix — must have spectral radius strictly below 1 (assessed via a conservative upper-bound estimate, so borderline sets are rejected rather than accepted). A set failing any condition is rejected outright. Independent of origin, however, the closure derivation itself (§4.1) is a final load-time guard: a singular closure system is a hard load error naming the hydro, and a non-finite derived rmr_m — a negative implied residual variance 1−∑ℓψm,ℓ∗ ρm(ℓ)1 - \sum_\ell \psi^*_{m,\ell}\,\rho_m(\ell) from a degenerate near-unit-root fit — is a hard load error naming the hydro and season.

Enforced when validating the assembled model:

  1. Positive sample std (error): a season with AR order >0> 0 must have sm>0s_m > 0 — a zero seasonal std cannot normalise the AR coefficients.

Enforced during fitting, not as a post-hoc test:

  1. Non-negative composed influence: after order selection, every season’s composed lag influences (§3.6) must be non-negative; a failing season loses one lag of its order ceiling and is re-selected and re-fitted until no season fails. A season whose first coefficient satisfies ψm,1∗<0\psi^*_{m,1} < 0, after the initial fit or after any re-fit, is reset to order 0, and so is a season whose initial fit has a coefficient of magnitude above the optional magnitude bound. On the classical path, a singular periodic Yule-Walker system bounds the selectable order below its own order (§3.5), so the classical fit never solves a singular system.

The residuals of the fitted autoregression are correlated across hydro plants: §6.1 estimates that correlation, and the rest of the section factorises it. Generating spatially correlated scenarios requires factorising the cross-hydro correlation matrix CC so that a vector of independent standard normal draws can be mapped to correlated noise. This section documents the choice of factorisation method and the rationale.

When the inflow model is fitted from the inflow history, in whole or in part, and the case supplies no correlation, the correlation matrix is estimated from the historical residuals of the autoregression. The standardized residual of hydro hh at historical period tt is

ε~h,t=a~h,t−∑ℓ=1pm(t)ψm(t),ℓ∗ a~h,t−ℓ\tilde{\varepsilon}_{h,t} = \tilde{a}_{h,t} - \sum_{\ell=1}^{p_{m(t)}} \psi^*_{m(t),\ell}\, \tilde{a}_{h,t-\ell}

where a~h,t=(ah,t−μ^m(t))/s^m(t)\tilde{a}_{h,t} = (a_{h,t} - \hat{\mu}_{m(t)}) / \hat{s}_{m(t)} is the inflow standardized by the seasonal statistics of §3.2, set to zero when s^m(t)=0\hat{s}_{m(t)} = 0, and ψm,ℓ∗\psi^*_{m,\ell} are the model’s standardized coefficients (§1.2). A period contributes a residual only when all pm(t)p_{m(t)} of its lagged inflows are observed. On the PAR(p)-A path the residual uses the lag coefficients only: the annual term of §7.1 does not enter it.

Two estimates are formed from these residuals: the pooled estimate C^\hat{C} from the residuals of every season, and the per-season estimate C^m\hat{C}_m from the residuals of the periods in season mm. Each off-diagonal entry is the Pearson correlation of the two hydros’ residuals over the periods at which both hydros have a residual (a pairwise-complete estimate), with the covariance and the standard deviations taken with the Bessel divisor, one less than the number of such periods, unlike the population divisor of §3.2. The entry is zero when fewer than two such periods exist or when either residual standard deviation is below machine epsilon, and it is clamped to [−1,1][-1, 1]; every diagonal entry is one.

The stages of season mm use C^m\hat{C}_m when every pair of hydros has at least 30 such periods in season mm; all other stages use C^\hat{C}. Because each entry is computed over its own set of periods, neither estimate need be positive semidefinite; §6.3 handles an indefinite matrix.

The classical approach applies Cholesky factorisation: given C=LL⊤C = L L^\top with LL lower-triangular, correlated noise is obtained as LzL z where z∼N(0,I)z \sim \mathcal{N}(0, I). Cholesky requires CC to be strictly positive-definite. In practice, estimated correlation matrices from hydro inflow series are frequently near-singular or rank-deficient for two reasons:

  • Short sample records: hydro inflow records often span only a few decades, yielding a historical record length NhistN^{\text{hist}} that is comparable to the number of hydro plants in some subsystems. When NhistN^{\text{hist}} is close to the matrix dimension, the sample eigenvalues of CC cluster near zero.
  • Heterogeneous series: Plants with near-identical hydrological regimes (upstream–downstream pairs, same river basin) produce columns that are nearly linearly dependent, reducing the effective rank of CC below its nominal dimension.

A near-singular CC causes Cholesky to fail or to produce numerically degenerate lower triangular factors. A separate filtering pass to remove “degenerate” hydros would be required before the factorisation, discarding information and introducing a non-transparent pre-processing decision.

6.3 Eigendecomposition with Clipped Square Root

Section titled “6.3 Eigendecomposition with Clipped Square Root”

Novomodelo uses the symmetric matrix square root via eigendecomposition. The correlation matrix is decomposed as:

C=UΛU⊤C = U \Lambda U^\top

where UU is the orthogonal matrix of eigenvectors and Λ=diag(λ1,…,λn)\Lambda = \mathrm{diag}(\lambda_1, \ldots, \lambda_n) is the diagonal matrix of eigenvalues. The symmetric square root is then:

C1/2=UΛ1/2U⊤C^{1/2} = U \Lambda^{1/2} U^\top

To handle near-singular matrices, any eigenvalue λi<0\lambda_i < 0 (which the pairwise-complete estimate of §6.1 can produce) is clipped to zero before taking the square root:

Λ~1/2=diag ⁣(max⁡(λ1,0), …, max⁡(λn,0))\tilde{\Lambda}^{1/2} = \mathrm{diag}\!\left(\sqrt{\max(\lambda_1, 0)},\, \ldots,\, \sqrt{\max(\lambda_n, 0)}\right)

Clipping negative eigenvalues to zero is the spectral projection of the sample matrix onto the positive-semidefinite cone — the nearest positive-semidefinite matrix in Frobenius norm; see Higham (2002) for the related nearest-correlation-matrix problem.

Correlated noise is then generated as C1/2zC^{1/2} z where z∼N(0,I)z \sim \mathcal{N}(0, I).

With the clipped root in C1/2C^{1/2}, the generated noise has covariance C~=Umax⁡(Λ,0) U⊤\tilde{C} = U \max(\Lambda, 0)\, U^\top, where max⁡(Λ,0)=diag(max⁡(λ1,0),…,max⁡(λn,0))\max(\Lambda, 0) = \mathrm{diag}(\max(\lambda_1, 0), \ldots, \max(\lambda_n, 0)); this is the projection above, and it equals CC when no eigenvalue is negative. Because CC has a unit diagonal, ∑iUhi2λi=1\sum_i U_{hi}^2 \lambda_i = 1 for every hydro hh, with UhiU_{hi} the component of eigenvector ii on hydro hh, so

C~hh=∑iUhi2max⁡(λi,0)=1+∑i: λi<0Uhi2∣λi∣≥1\tilde{C}_{hh} = \sum_i U_{hi}^2 \max(\lambda_i, 0) = 1 + \sum_{i:\, \lambda_i < 0} U_{hi}^2 \lvert \lambda_i \rvert \ge 1

No renormalisation follows, so clipping raises the marginal variance of each affected hydro’s noise above one.

The spectral form handles rank-deficient correlation matrices natively: eigenvectors corresponding to clipped (zero) eigenvalues contribute nothing to the factorisation, which is the correct behaviour for directions of zero variance. No prior filtering of degenerate hydro plants is needed.

The clipping rule is fixed and transparent — exactly the strictly negative eigenvalues are set to zero, nothing else is altered. The eigen-directions with non-negative eigenvalues keep their eigenvalues unchanged. When any eigenvalue is clipped, the generated noise has covariance C~\tilde{C} (§6.3) instead of CC: its diagonal exceeds one for every hydro with a component along a clipped eigenvector, and its off-diagonal entries in general differ from those of CC.

PropertyEigendecomposition (Novomodelo)Cholesky
Handles rank-deficient CCYes — clipping makes it robustNo — requires positive-definiteness
Computational costHigher (full eigendecomposition)Lower on well-conditioned matrices
Degenerate-hydro filtering passNot requiredRequired for near-singular CC
Transparency of approximationClips exactly the negative eigenvalues; the generated variance can exceed oneOpaque numerical failure or pivot

The higher computational cost is acceptable because the factorisation is performed once per study configuration and not on the hot path of the forward pass.

The classical PAR(p) of §1 captures temporal dependence at lags up to a small order pp (kept low for monthly cycles, since the periodic Yule-Walker system becomes ill-conditioned at higher orders). On long hydro inflow series this is enough to reproduce the within-year persistence but not the multi-year persistence visible in dry/wet super-periods of the historical record. The PAR(p)-A extension (Treistman et al., 2020) adds a single annual coefficient on top of the periodic AR structure to capture that longer-range persistence without inflating the AR order.

The extension is selected by the order-selection rule with the annual component (§3.6). When active, the model carries one additional triple per (hydro, season) on top of the classical parameter set.

Let Ah,t−1A_{h,t-1} denote the rolling 12-month average of incremental inflows ending one stage before tt:

Ah,t−1=112∑ℓ=112ah, t−ℓA_{h,t-1} = \frac{1}{12} \sum_{\ell=1}^{12} a_{h,\, t-\ell}

The window always averages the last twelve observations, so Ah,t−1A_{h,t-1} is a one-year average only on a monthly cycle (M=12M = 12); with the extension active on any hydro, the AR dynamics row (§2.5) spans twelve lags, or the largest classical order if that is larger.

The PAR(p)-A model augments §1 with the standardised deviation of Ah,t−1A_{h,t-1} from its own seasonal mean:

ah,t  =  μm(t)  +  ∑ℓ=1pψm(t),ℓ (ah,t−ℓ−μm(t−ℓ))  +  ψm(t)A (Ah,t−1−μm(t)A)  +  σm(t)⋅εta_{h,t} \;=\; \mu_{m(t)} \;+\; \sum_{\ell=1}^{p} \psi_{m(t),\ell}\,(a_{h,t-\ell} - \mu_{m(t-\ell)}) \;+\; \psi^A_{m(t)}\,(A_{h,t-1} - \mu^A_{m(t)}) \;+\; \sigma_{m(t)} \cdot \varepsilon_t

where:

  • μmA\mu^A_{m}, σmA\sigma^A_{m}: sample mean and population-divisor standard deviation of the season’s own annual regressor — the values of Ah,t−1A_{h,t-1} over stages tt in season mm, i.e. the rolling windows whose most recent observation falls in the preceding season (§7.3)
  • ψm(t)A\psi^A_{m(t)}: original-unit annual coefficient at season m(t)m(t) — derived at runtime from the standardised stored coefficient (§7.4)
  • All other symbols carry their classical meaning from §1

When the PAR(p)-A extension is inactive, the annual term is absent and the model reduces exactly to §1.

For each (hydro, season) the PAR(p)-A path stores three additional quantities:

QuantitySymbolDescription
Standardised annual coefficientψmA∗\psi^{A*}_mYule-Walker output for the annual term — dimensionless
Annual seasonal meanμmA\mu^A_mSample mean of season mm‘s annual regressor Ah,t−1A_{h,t-1} (m³/s)
Annual seasonal stdσmA\sigma^A_mPopulation-divisor std of season mm‘s annual regressor Ah,t−1A_{h,t-1} (m³/s, >0> 0)

The standardised coefficient ψmA∗\psi^{A*}_m is the direct output of the extended periodic Yule-Walker system below (§7.5). Storage of μmA\mu^A_m and σmA\sigma^A_m alongside the seasonal statistics of ah,⋅a_{h, \cdot} enables the runtime unit conversion of §7.4 without re-reading the historical record.

7.3 Estimating the Annual Seasonal Statistics

Section titled “7.3 Estimating the Annual Seasonal Statistics”

Form every rolling 12-month average At=112∑ℓ=112ah, t+1−ℓA_t = \frac{1}{12} \sum_{\ell=1}^{12} a_{h,\, t + 1 - \ell} the chronological history admits, and assign each window to the season following its most recent observation — the season whose stages use that window as their regressor Ah,t−1A_{h,t-1}. The statistics stored for season mm therefore describe exactly the regressor that season’s stages see. For each (hydro, season mm) bucket of values {A(w)}\{A^{(w)}\}:

μ^mA  =  1NmA∑wA(w)σ^mA  =  1NmA∑w(A(w)−μ^mA)2\hat{\mu}^A_m \;=\; \frac{1}{N^A_m} \sum_{w} A^{(w)} \qquad \hat{\sigma}^A_m \;=\; \sqrt{\frac{1}{N^A_m} \sum_{w} \bigl(A^{(w)} - \hat{\mu}^A_m\bigr)^2}

Both estimators use the population divisor 1/NmA1/N^A_m, matching the convention of §3.2 and ensuring no sample-vs-population scale factor leaks into the conditional FACP of §7.5. At least 13 chronological observations are required for a hydro to participate in PAR(p)-A — that is the minimum needed to form one rolling 12-month average.

The stored standardised coefficient ψmA∗\psi^{A*}_m is converted to the original-unit coefficient ψmA\psi^A_m at LP construction time using the seasonal stats and annual stats:

ψmA  =  ψmA∗⋅smσmA\psi^A_{m} \;=\; \psi^{A*}_m \cdot \frac{s_m}{\sigma^A_m}

The conversion mirrors §2.2 for the classical AR coefficients. The annual term ψm(t)A⋅(Ah,t−1−μm(t)A)\psi^A_{m(t)} \cdot \bigl(A_{h,t-1} - \mu^A_{m(t)}\bigr) is then expanded through the window definition at LP construction: each of the 12 lag coefficients gains ψm(t)A/12\psi^A_{m(t)}/12 on top of its classical value, and the deterministic base absorbs the corresponding mean contribution through the lag-season means — the LP carries no separate annual variable, and the rolling-window value is realized entirely by the lag state variables it already carries.

7.5 Order Selection and Coefficient Estimation

Section titled “7.5 Order Selection and Coefficient Estimation”

PAR(p)-A order selection conditions on the annual regressor. Let ZZ be the standardised inflow series, the inflow standardised by its seasonal statistics (§3.2), and let the annual regressor At−1A_{t-1} of §7.1 be standardised by its own seasonal statistics (§7.3). The order-selection input is the conditional FACP at lag kk, defined as the partial autocorrelation between the current-stage value of ZZ and its value at lag kk, conditioned on the intermediate lags of ZZ and on At−1A_{t-1}. Computing the conditional FACP requires a partitioned covariance decomposition that distinguishes Z⊗ZZ \otimes Z, Z⊗AZ \otimes A, and A⊗Z−1A \otimes Z_{-1} blocks.

The conditional FACP feeds the PACF order-selection rule of §3.6, with the two PAR(p)-A-specific extensions (structural-zero short-circuit and minimum-order-1) already described there.

The Z⊗ZZ \otimes Z block uses the same year-aligned population divisor as the classical autocovariance (§3.4). The Z⊗AZ \otimes A and A⊗Z−1A \otimes Z_{-1} blocks use a max-bucket-size divisor:

γ^Z⊗A(ℓ)  =  1max⁡(∣A∣, ∣Z∣)∑w(Z(w)−Zˉ)(A(w)−Aˉ)\hat{\gamma}_{Z \otimes A}(\ell) \;=\; \frac{1}{\max(|A|,\, |Z|)} \sum_w \bigl(Z^{(w)} - \bar{Z}\bigr)\bigl(A^{(w)} - \bar{A}\bigr)

The max-bucket convention is required because AA excludes the first year of ZZ by construction (a rolling 12-month window cannot anchor in the first 12 observations). The strict-pair count would distort the scale of the cross-correlations and bias the conditional FACP. The PAR(p) path never uses Z⊗A cross-correlations, so the divisor question is PAR(p)-A-specific.

Once the order pp is selected, the coefficients (ψm,1∗,…,ψm,p∗,ψmA∗)(\psi^*_{m,1}, \ldots, \psi^*_{m,p}, \psi^{A*}_m) are recovered by solving the extended periodic Yule-Walker system:

Rm ext(ψm,1∗⋮ψm,p∗ψmA∗)=ρ^m ext\mathbf{R}^{\,\text{ext}}_m \begin{pmatrix} \psi^*_{m,1} \\ \vdots \\ \psi^*_{m,p} \\ \psi^{A*}_m \end{pmatrix} = \hat{\boldsymbol{\rho}}^{\,\text{ext}}_m

where Rm ext\mathbf{R}^{\,\text{ext}}_m is the (p+1)×(p+1)(p+1) \times (p+1) partitioned covariance whose first pp rows replicate the classical periodic Yule-Walker rows (§3.5) and whose last row adds the Z⊗AZ \otimes A and A⊗AA \otimes A entries. The RHS ρ^m ext\hat{\boldsymbol{\rho}}^{\,\text{ext}}_m appends the A⊗Z−1A \otimes Z_{-1} target.

Singular extended system. The extended matrix can be singular even when its leading p×pp \times p block, the classical periodic Yule-Walker matrix of §3.5, is not, so the guarantee of invariant 4 (§5) covers the classical path only. A season whose extended system is singular at the selected order takes order 0 with annual coefficient ψmA∗=0\psi^{A*}_m = 0.

As in the classical case, the innovation scale is not a fitting output. For a PAR(p)-A model the closure of §4.1 runs on the effective 12-lag system: the annual regressor is a linear functional of the last 12 inflows, so expanding it per lag yields an effective periodic AR whose standardized lag-ℓ\ell coefficient is the classical ψm,ℓ∗\psi^*_{m,\ell} (zero beyond pmp_m) plus the annual contribution ψmA∗⋅sm−ℓ/(12 σmA)\psi^{A*}_m \cdot s_{m-\ell} / (12\,\sigma^A_m). The innovation scale rmr_m derives from that effective system’s implied ACF, and the runtime noise scale is σm=sm⋅rm\sigma_m = s_m \cdot r_m exactly as in §4.2.

The Maceira & Damázio iterative reduction of §3.6 is applied across the full periodic cycle on the PAR(p)-A path as well: after the initial fit, the composed influence of each AR lag through the periodic chain is evaluated, and any season with a negative composed influence has its AR ceiling reduced before refit. The annual coefficient ψmA∗\psi^{A*}_m does not enter the composed-influence check — it is anchored to the rolling annual mean and so does not propagate through the lag chain.

ConditionPath
Classical order-selection ruleClassical PAR(p) — annual triple absent (§3)
Order-selection rule with the annual componentPAR(p)-A — annual triple required for every (hydro, season)
Bucket classified constant or saturated (§3.3)Effective order 0 on either path; annual term suppressed when the seasonal std collapses
Extended periodic Yule-Walker system singular at the selected order (§7.5)PAR(p)-A season at order 0 with annual coefficient ψmA∗=0\psi^{A*}_m = 0
Hydro with fewer than 13 observations on PAR(p)-A pathHard failure during fitting (no silent fallback to classical)

The two paths share the seasonal-stats producer of §3.2; switching between them does not silently change μ^m\hat{\mu}_m or s^m\hat{s}_m. The PAR(p)-A path uses the same spatial-correlation factorisation as the classical path (§6).

The two-plane split of §1.2 shows that re-conditioning the seasonal stats (μm\mu_m, sms_m) alone — leaving ψ∗\psi^* and the closure-derived rmr_m untouched — rescales the classical dynamics while preserving the correlation structure exactly, for any per-season rescaling. The PAR(p)-A annual term does not share this property in general.

Like rmr_m, the annual seasonal std σmA\sigma^A_m is a functional of the process’s own second-moment structure — it is not an independent parameter of the annual term. Unlike rmr_m, it is carried as stored conditioning data (§7.2) rather than re-derived from ψ∗\psi^* at load. The runtime annual coefficient ψmA\psi^A_m (§7.4) is ψmA=ψmA∗⋅sm/σmA\psi^A_m = \psi^{A*}_m \cdot s_m / \sigma^A_m, so a conditioning swap preserves the annual term’s contribution to the correlation structure only when σmA\sigma^A_m rescales by the same factor as sms_m in every season — a uniform rescaling s~m=c sm\tilde s_m = c\, s_m for all mm, giving σ~mA=c σmA\tilde\sigma^A_m = c\,\sigma^A_m and leaving ψmA\psi^A_m unchanged. A season-varying conditioning swap — rescaling sms_m by a different factor in different seasons — leaves σmA\sigma^A_m at its pre-swap value while sms_m moves, shifting ψmA\psi^A_m and perturbing the preserved correlation structure.

The methodology above defines the PAR(p) inflow model; the tabs below cover how Novomodelo’s software surface configures, feeds, and reports on it.

Novomodelo’s config.json top-level estimation block, the scenarios/ PAR(p) input files, and scenarios/correlation.json configure the estimation and correlation pipeline the methodology above describes. This tab shows which PAR inputs to supply and how the estimation and correlation are configured; the equations these fields feed are in the sections above.

The estimation block is a top-level key of config.json (a sibling of training), and controls the PAR(p) fitting pipeline on the estimation paths (rows 4–6 of the table in the next section):

{
"estimation": {
"max_order": 6,
"order_selection": "pacf",
"min_observations_per_season": 30,
"max_coefficient_magnitude": null
}
}

The estimation section of the configuration reference lists every field and its default, and states the min_observations_per_season behaviour.

Setting "order_selection": "pacf_annual" activates the PAR(p)-A annual component (methodology §7): the periodic Yule-Walker system is extended with the annual cross-correlation term, per-season sample statistics of the rolling 12-month average are computed, and the fitted triple (ψmA∗\psi^{A*}_m, annual mean, annual std) is written to inflow_annual_component.parquet — see the Inputs & Outputs tab. When absent (or set to "pacf"), only the classical PAR(p) path runs and no annual triple is produced.

Inflow Source Resolution — Choosing Which Files to Provide

Section titled “Inflow Source Resolution — Choosing Which Files to Provide”

The PAR(p) inflow model is built from up to five files in scenarios/. Three of them drive path resolution — their presence or absence selects which of seven estimation paths Novomodelo executes:

SymbolFileRole
Hscenarios/inflow_history.parquetWindowed historical observations for fitting
Sscenarios/inflow_seasonal_stats.parquetUser-supplied seasonal mean/std
Rscenarios/inflow_ar_coefficients.parquetUser-supplied AR coefficients

The other two files layer orthogonally on top of the resolved path: scenarios/correlation.json wins on every path when present (identity correlation otherwise, unless a path estimates it from residuals); and scenarios/inflow_annual_component.parquet is honored only on the pass-through paths (rows 2, 3, 7 below) — the estimation paths (4, 5, 6) always overwrite it with fitted values.

#HSRPathSeasonal statsAR coefficientsCorrelation
1000Deterministicno PAR modelnoneidentity, unless correlation.json provided
2010UserStatsWhiteNoiseuser fileorder-0 (white noise)identity, unless correlation.json provided
3011UserProvidedNoHistoryuser fileuser fileidentity, unless correlation.json provided
4100FullEstimationfitted from Hfitted from H (PACF + Yule-Walker + Maceira & Damázio)estimated from H residuals, unless correlation.json provided
5101UserArHistoryStatsfitted from Huser fileestimated from H residuals using user coefficients, unless correlation.json provided
6110PartialEstimationuser file (the fit uses statistics estimated from H)fitted from Hestimated from H residuals, unless correlation.json provided
7111UserProvidedAlluser fileuser fileidentity, unless correlation.json provided (history is not re-consumed)

Cases with R = 1 but H = 0 and S = 0 collapse to row 1 — AR coefficients alone cannot drive estimation.

An estimated correlation (rows 4–6, when no correlation.json is provided) holds a pooled "default" profile. In a cycle of more than one season it also holds one season_<id> profile for each season whose every hydro pair has at least 30 residual pairs — the season id zero-padded to the width of the largest id, as in season_03 for a twelve-season cycle — and a schedule that maps each stage of such a season to its profile. Every other stage uses the pooled profile (methodology §6.1).

Practical recipes:

GoalFiles to providePath
Smoke-test the LP without stochasticity(no scenarios files)1
Deterministic seasonal levels, no autoregressioninflow_seasonal_stats.parquet2
Fully user-specified PAR(p) without a history fileinflow_seasonal_stats.parquet, inflow_ar_coefficients.parquet, and, for an AR order above 0, recent_observations covering the lag slots3
Hands-off: fit everything from raw observationsinflow_history.parquet4
Fit stats from history, override the AR structureinflow_history.parquet, inflow_ar_coefficients.parquet5
Override the levels (mean/std) but let Novomodelo fit the ARinflow_history.parquet, inflow_seasonal_stats.parquet6
Provide every parameter, including the PAR(p)-A annual termAll three of H, S, R (and optionally the annual file)7
Pin a custom spatial correlation on any pathAdd correlation.jsonany

scenarios/inflow_history.parquet — Windowed Historical Observations

Section titled “scenarios/inflow_history.parquet — Windowed Historical Observations”

The windowed record is required to drive any of the estimation paths (H = 1 above) and, whenever present, is the base layer the derived inflow-lag seed casts from (see “Lag and accumulator seeding” below).

Every hydro plant in hydros.json must have at least one observation window in this file when estimation is active. The columns, window rules and roles of the file are in scenarios/inflow_history.parquet.

Lag and accumulator seeding — recent_observations

Section titled “Lag and accumulator seeding — recent_observations”

Independently of which estimation path (if any) inflow_history.parquet drives, its windowed record is the seed for the opening study stage’s PAR lag slots and mid-period inflow accumulator. initial_conditions.json’s optional recent_observations array conditions that seed: entries carry the same hydro_id / start_date / end_date / value_m3s shape as inflow_history.parquet and shadow it day-wise wherever the two overlap (the two series are never blended). value_m3s accepts negative values in both files: the quantity is incremental inflow, so a negative window is real hydrology (see Inflow Non-Negativity Solution Methods).

When scenarios/inflow_ar_coefficients.parquet is supplied, the record must fully cover each lag slot up to the maximum AR order and, with an annual component, each slot of the twelve-period annual window that the study’s own completed periods do not replace; otherwise the case is refused at load (BusinessRuleViolation). A gap in a deeper slot, which the study’s own completed periods push out of the lag window before the study ends, only draws a warning.

When the coefficients are estimated, this coverage check does not run: a lag period the record covers only in part seeds from the mean over its covered days, and a period with no covered day seeds to zero.

On either path, a first-stage period already in progress at the study start that the record covers only in part is accepted, with a warning; with no covered day, its accumulator seeds to zero.

scenarios/correlation.json — Spatial Correlation Configuration

Section titled “scenarios/correlation.json — Spatial Correlation Configuration”

Named profiles of correlation groups, with an optional stage-to-profile schedule:

{
"method": "spectral",
"profiles": {
"default": {
"correlation_groups": [
{
"name": "basin_south",
"entities": [
{ "type": "inflow", "id": 0 },
{ "type": "inflow", "id": 1 }
],
"matrix": [
[1.0, 0.7],
[0.7, 1.0]
]
}
]
},
"wet_season": {
"correlation_groups": [
{
"name": "basin_south",
"entities": [
{ "type": "inflow", "id": 0 },
{ "type": "inflow", "id": 1 }
],
"matrix": [
[1.0, 0.85],
[0.85, 1.0]
]
}
]
}
},
"schedule": [{ "stage_id": 0, "profile_name": "wet_season" }]
}

Correlation is modelled independently per entity class — inflow, load, and NCS each carry their own spatial correlation structure, and there is no cross-class correlation term (see Scenario Generation §2.1).

Every field and load rule of the file is in scenarios/correlation.json.

Under the out_of_sample forward scheme, a stage whose sampling_method is historical_residuals or selective draws plain Monte Carlo noise, and the run logs a warning that names the entity class and the affected stage ids. For the message, see Scenario Generation — Implementation notes tab, Warnings.

  • LP Formulation — The realized-inflow rows of the stage LP
  • State Augmentation — AR inflow dynamics: state expansion, lag column pinning, reduced-cost extraction
  • Inflow Non-Negativity Solution Methods — Methods for handling negative realizations produced by the PAR(p) model
  • Scenario Generation — When external inflow scenarios are used in training, the backward pass turns the opening tree’s noise into inflows with the applied PAR model, in which hydros of AR order 0 and no annual component take the external samples’ mean and standard deviation.
  • Notation Conventions — Defines inflow symbols (ah,ta_{h,t}, μm\mu_m, ψm,ℓ\psi_{m,\ell}, σm\sigma_m) and unit conventions