Two ways to correct a predictive
A forecast can be miscalibrated in more than one way, and the repair you reach for decides what you pay.
1. Two corrections
Both start from the same object. Push each realized value through the forecast distribution it was predicted by and you get a number that should be uniform, and through the normal quantile a number that should be standard normal. Call it z. Every claim below is a statement about z.
Recalibration observes that z is not uniform and applies a monotone map to make it so. The map is estimated from the empirical distribution of past residuals. Conformal predictive systems are the rigorous version, with finite-sample coverage guarantees under exchangeability.
Conditional reparameterisation observes that z has the wrong conditional shape and fits a small state-carrying model to it, then composes that model back onto the original predictive as a change of variables. Homogenization is the version measured here, using a two-point Gaussian scale mixture. See the homogenization page for the mechanism.
2. What recalibration costs
Recalibration works, in the sense it promises. Applied to three foundation models on economic change-series, one step ahead, 8,740 paired series with degenerate series excluded:
| base model | n | central-90% coverage | CRPS ratio | recalibrated wins |
|---|---|---|---|---|
| Chronos | 8,740 | 0.792 → 0.902 | 1.0179 | 34.1% |
| TiRex | 8,740 | 0.801 → 0.902 | 1.0344 | 16.4% |
| TimesFM | 8,740 | 0.801 → 0.902 | 1.0237 | 25.7% |
Coverage lands on 0.902 for all three, from 0.79 to 0.80 before. That is the guarantee delivered. The CRPS ratio is above one in every case, so the corrected forecast is worse on the proper score, winning only 16 to 34 percent of series. The guarantee is paid for in width.
The same pattern appears when conformal machinery is the whole method rather than a wrapper. Against laplace on the non-price FRED universe:
| method | n | laplace wins, log-likelihood | laplace wins, CRPS | mean log-likelihood |
|---|---|---|---|---|
| AutoARIMA+ACI@25 | 1,685 | 84% | 36% | +0.45 |
| AutoARIMA+conformal@25 | 1,685 | 82% | 36% | +0.28 |
| CSP | 5,402 | 98% | 98% | -0.30 |
| CSP-adaptive | 5,402 | 98% | 98% | -0.09 |
| laplace | 5,402 | — | — | +1.56 |
3. What the reparameterisation gains
A second case study, on a different base forecast: laplace's own residual stream, corrected conditionally instead of marginally. FRED, 792 paired series per horizon, one pool per horizon, scored by pinball loss with central-80% coverage (target 0.800):
| h | n | pinball ratio | wins | central-80% coverage (target 0.800) |
|---|---|---|---|---|
| 1 | 792 | 0.9909 | 61.4% | 0.854 → 0.841 |
| 3 | 791 | 0.9910 | 58.2% | 0.858 → 0.856 |
| 5 | 792 | 0.9666 | 64.4% | 0.920 → 0.869 |
| 6 | 790 | 0.9676 | 64.7% | 0.902 → 0.859 |
| 8 | 790 | 0.9764 | 62.7% | 0.894 → 0.849 |
| 12 | 786 | 0.9672 | 65.8% | 0.856 → 0.802 |
Ratios are below one throughout, so the corrected forecast is better on the proper score, by about 0.9 percent at one step and 3.3 percent at h=5. Coverage moves toward the 0.800 target at the same time. Within this case study the sign of the cost is the opposite of the one in section 2. The two sections differ in base forecast, panel, library epoch, proper score and interval level, so the sign contrast is not a controlled comparison of the two methods; section 4 offers a mechanism as a hypothesis, not a measured result.
4. A hypothesis for the difference in sign
A marginal map cannot tighten and widen at once. Recalibration fits one monotone map for the whole stream. If coverage is short on average, the map widens on average. A state-carrying correction can tighten in quiet stretches and widen in violent ones, so mean width can fall while coverage rises. That is only available to a correction that conditions on something.
Recalibration discards the tails it was given. A conformal predictive
system emits a distribution assembled from empirical residual quantiles, so the
parametric tail of the original predictive is gone. Under logarithmic loss that loss is
priced: it is the irreducible cost of a coarser retained representation, which is the
subject of the
conformal information
gap study. The reparameterisation composes as
F̃(y) = H(Φ-1(F(y))) and keeps the original density,
including its fitted tails.
One of them can decline to act. The candidate pool includes an identity frozen at unit variance holding most of the prior weight, so on a well-specified stream it selects "do nothing" and costs about 0.0002 nats. Recalibration always pays finite sample quantile noise, and always buys its guarantee with width.
They target different defects. laplace over-covers: measured 0.912 against a 0.900 target on the same rows. Its problem is sharpness, not coverage, so a method whose purpose is to secure coverage has nothing to offer it. The foundation models under-cover at 0.79 to 0.80, which is exactly the condition recalibration is built for, and there it does move coverage to nominal.
5. When to use which
- Base model under-covers and you need a stated rate more than you need sharpness: recalibrate, and expect to pay a few percent of CRPS.
- Base model over-covers, or is miscalibrated conditionally rather than marginally: reparameterise. Recalibration cannot help and may hurt.
- You need a finite-sample guarantee under exchangeability: only conformal offers one. Homogenization offers no guarantee, just a measured improvement and a safe no-op.
- The predictive's tails carry information you care about: prefer the correction that keeps them.
6. Limitations
- The recalibration arm is one specific construction, laplace predicting in the base model's CDF space. Split conformal and adaptive conformal inference would land differently in detail, though the coverage-for-sharpness trade is structural.
- Conformal guarantees hold under exchangeability. The streams here are drifting and seasonal, so the comparison is empirical rather than a statement about the theory.
- Both arms are one-step and multi-step density forecasts of economic change series. Level forecasting is not addressed.
- Sections 2 and 3 are separate experiments: different base forecasts (three foundation models; laplace), panels (8,740 series; about 790), library epochs, proper scores (CRPS; pinball) and interval levels (central-90%; central-80%). Each table is internally paired, which is what its own claims rest on. Nothing on this page runs the two corrections head-to-head on one frozen forecast stream, so the sign contrast between the sections is not attributed to the methods.
- The foundation-model coverage in section 2 is that of a density reconstructed from nine deciles, which on its own reads 0.81 for a calibrated source (#261); part of what recalibration repairs there is the reconstruction.
- Gains and costs are both single-digit percentages. The interest is in the sign and the mechanism, not the magnitude.
Reproducing this
PYTHONPATH=src:benchmarks python benchmarks/correction_page.py
correction_page.py — generates this page · correction_results.json · homogenize.py
Recalibration arm: 1329 degenerate series excluded of the week-study universe. Homogenization arm measured with skaters 0.16.0+ff9740f. Page written 2026-10-14. Every figure is generated, not transcribed.