Two ways to correct a predictive

A forecast can be miscalibrated in more than one way, and the repair you reach for decides what you pay.

1. Two corrections

Both start from the same object. Push each realized value through the forecast distribution it was predicted by and you get a number that should be uniform, and through the normal quantile a number that should be standard normal. Call it z. Every claim below is a statement about z.

Recalibration observes that z is not uniform and applies a monotone map to make it so. The map is estimated from the empirical distribution of past residuals. Conformal predictive systems are the rigorous version, with finite-sample coverage guarantees under exchangeability.

Conditional reparameterisation observes that z has the wrong conditional shape and fits a small state-carrying model to it, then composes that model back onto the original predictive as a change of variables. Homogenization is the version measured here, using a two-point Gaussian scale mixture. See the homogenization page for the mechanism.

2. What recalibration costs

Recalibration works, in the sense it promises. Applied to three foundation models on economic change-series, one step ahead, 8,740 paired series with degenerate series excluded:

Coverage lands on 0.902 for all three, from 0.79 to 0.80 before. That is the guarantee delivered. The CRPS ratio is above one in every case, so the corrected forecast is worse on the proper score, winning only 16 to 34 percent of series. The guarantee is paid for in width.

The same pattern appears when conformal machinery is the whole method rather than a wrapper. Against laplace on the non-price FRED universe:

3. What the reparameterisation gains

The same residual stream, corrected conditionally instead of marginally. FRED, 792 paired series per horizon, one pool per horizon:

Ratios are below one throughout, so the corrected forecast is better on the proper score, by about 0.9 percent at one step and 3.3 percent at h=5. Coverage moves toward nominal at the same time. The sign of the cost is reversed relative to recalibration.

4. Why the sign differs

A marginal map cannot tighten and widen at once. Recalibration fits one monotone map for the whole stream. If coverage is short on average, the map widens on average. A state-carrying correction can tighten in quiet stretches and widen in violent ones, so mean width can fall while coverage rises. That is only available to a correction that conditions on something.

Recalibration discards the tails it was given. A conformal predictive system emits a distribution assembled from empirical residual quantiles, so the parametric tail of the original predictive is gone. Under logarithmic loss that loss is priced: it is the irreducible cost of a coarser retained representation, which is the subject of the conformal information gap study. The reparameterisation composes as F̃(y) = H(Φ-1(F(y))) and keeps the original density, including its fitted tails.

One of them can decline to act. The candidate pool includes an identity frozen at unit variance holding most of the prior weight, so on a well-specified stream it selects "do nothing" and costs about 0.0002 nats. Recalibration always pays finite sample quantile noise, and always buys its guarantee with width.

They target different defects. laplace over-covers: measured 0.912 against a 0.900 target on the same rows. Its problem is sharpness, not coverage, so a method whose purpose is to secure coverage has nothing to offer it. The foundation models under-cover at 0.79 to 0.80, which is exactly the condition recalibration is built for, and there it does move coverage to nominal.

5. When to use which

• Base model under-covers and you need a stated rate more than you need sharpness: recalibrate, and expect to pay a few percent of CRPS.
• Base model over-covers, or is miscalibrated conditionally rather than marginally: reparameterise. Recalibration cannot help and may hurt.
• You need a finite-sample guarantee under exchangeability: only conformal offers one. Homogenization offers no guarantee, just a measured improvement and a safe no-op.
• The predictive's tails carry information you care about: prefer the correction that keeps them.

6. Limitations

• The recalibration arm is one specific construction, laplace predicting in the base model's CDF space. Split conformal and adaptive conformal inference would land differently in detail, though the coverage-for-sharpness trade is structural.
• Conformal guarantees hold under exchangeability. The streams here are drifting and seasonal, so the comparison is empirical rather than a statement about the theory.
• Both arms are one-step and multi-step density forecasts of economic change series. Level forecasting is not addressed.
• The conformal figures come from an earlier library epoch than the homogenization figures, so the two tables should not be differenced directly. Each is internally paired, which is what the claims rest on.
• Gains and costs are both single-digit percentages. The interest is in the sign and the mechanism, not the magnitude.

Reproducing this

PYTHONPATH=src:benchmarks python benchmarks/correction_page.py

correction_page.py — generates this page · correction_results.json · homogenize.py

Recalibration arm: 1329 degenerate series excluded of the week-study universe. Homogenization arm measured with skaters 0.16.0+ff9740f. Page written 2026-08-14. Every figure is generated, not transcribed.