The day-one bout: PYMC-Forecast

PyMC Labs released a Bayesian forecasting toolkit on 2026-07-13. This statement was filed the next morning. Four pre-registered hypotheses, including the first challenger arm ever given cross-series information.

Canonical update: the improvement claim is withdrawn

This page preserves the July pre-registration and results, which used a noncanonical protocol. The current canonical laplace@PyMC arm has median paired Δ log-likelihood against laplace of −0.0522 daily (500 series), −0.0124 weekly (499), and −0.0153 monthly (471). Positive means better than laplace. See the committed canonical summary and sidecar page. The general claim that the sandwich edges laplace is withdrawn; the historical results below are not the current canonical conclusion.

The historical July design

PYMC-Forecast supplies a Bayesian forecasting toolkit. The July experiment used a Student-t regression on 8 lagged values, mean-field ADVI and 500 posterior draws, scoring the exact posterior-draw density mixture. Its sand arm refit every 10 steps; the separate solo and hierarchical arms froze their pre-test fits through all 150 test steps. A calibrated laplace predictive would make its normal-score marginal N(0,1), motivating the priors, but the coordinate transform alone does not guarantee calibration, stationarity or negligible refit-staleness cost. The later canonical adapter refits and emits predictive distributions through the common density reconstruction and scoring protocol, so these are not identical experiments.

The four hypotheses: July results as previously reported

hypothesishistorical result on 226 seriesJuly verdict (superseded)
H1: raw model vs laplacelaplace wins 82%, +0.45 nats median (p = 5e-23)the usual story
H2: does the sandwich lift it?sandwiched beats raw on 86%, +0.63 nats median (p = 3e-30)yes, again
H3: convergence to laplacemedian absolute gap collapses 0.62 → 0.05 nats; sandwiched wins 65%, +0.04 median (p = 4e-06)converged, epsilon positive
H4: does one pooled fit beat per-series fits?pooling loses on 72% (p = 2e-11); wins the repeat-heavy stratum by +2.1 nats medianno, except where series share structure

The historical positive epsilon

The July report put the block-refit sand arm ahead on 65% of series, +0.04 nats median, with +2.1 nats median on the near-martingale repeat stratum. These observations are retained for that 226-series study; they do not establish a gain under the canonical protocol above. The July percentages use sign comparisons, not the canonical HAC/DM win/draw/loss classification.

Pooling: historical contrast, not rerun here

The parade reduces every series to the same coordinate system, which motivated testing partial pooling across heterogeneous series. The historical hier arm used one joint hierarchical fit across all 226 series' z streams, against an identical model fit per series. Same window, same freeze, only the priors differ, so the contrast isolates pooling exactly. Verdict: the shrinkage tax beats the borrowed strength on 72% of series, but pooling wins 79% of the repeat-heavy stratum with a +2.1-nat median in this July experiment. This pooling contrast was not rerun by the canonical comparison above.

Staleness: historical frozen-fit control

The separate July solo arm fit once on pre-test data, then froze the fit through all 150 test steps. It was ahead of laplace on 54% of series, median gap +0.005 nats, as previously reported. This is not the block-refit sand arm's +0.04 result and does not show that staleness is negligible under the canonical protocol.

Current conclusion

The July experiment remains part of the research record, including its distinct refit and pooling arms. The canonical PyMC sandwich has negative median gains on daily, weekly and monthly economic strata. The older whole-universe improvement claim is superseded; neither the historical pooling contrast nor the separate M4-Hourly study was rerun in that canonical comparison.

Historical figures are preserved as previously reported, not recomputed here. Study sources: benchmarks/pymc_forecast_sandwich_study.py and benchmarks/pymc_forecast_analysis.py, results CSV beside them, statement in benchmarks/preregistrations/2026-07-14-pymc-forecast-sandwich.md.