Papers & benchmarks
A new practical benchmark for univariate distributional prediction — simpler and lighter than Prophet, and scored where it counts: held-out likelihood.
The paper
Transforms All the Way Down — Automatic Online Distributional Forecasting by Conjugation. P. Cotton.
Every prediction is a full predictive distribution, assembled by composition: invertible transforms chain together above a single distributional leaf, and ensembles combine such chains. Because the leaf fits its shape by optimising a proper scoring rule, its objective is a choice — fit the model by likelihood, then conform the predictive tail to a downstream score such as CRPS. The library is written twice, in pure Python and in zero-dependency JavaScript that agrees to 1e-6, so the same model runs on a server or in a browser.
PDF → · LaTeX (JSS) · Markdown draft
Benchmark 1 — the non-price universe
A fair rolling one-step-ahead comparison on the non-price FRED universe —
894 continuous daily change-series (57 correlated families); equity/fx/commodity returns are
excluded and treated separately in the price caveat below. Every method — ours and theirs — is
turned into the same Dist and scored on held-out log-likelihood and CRPS.
Per-series win-rate of laplace, reported raw and family-weighted
(one vote per correlated family):
| Baseline | LL (raw / fam) | CRPS (raw / fam) | mean LL |
|---|---|---|---|
| AutoARIMA | 82 / 90% | 53 / 65% | 2.80 |
| AutoETS | 97 / 99% | 83 / 92% | 2.79 |
| SARIMAX (statsmodels) | 87 / 86% | 53 / 64% | 2.92 |
| ETS (statsmodels) | 96 / 98% | 81 / 90% | 2.89 |
GARCH(1,1)-t (arch) | 68 / 65% | 54 / 45% | 3.03 |
| AutoARIMA + conformal | 86 / 88% | 31 / 31% | 2.27 |
| AutoARIMA + ACI | 88 / 88% | 29 / 29% | 2.51 |
| naive conformal | — (CDF) | 92 / 93% | — |
| NF-StudentT (NeuralForecast) | 100 / 100% | 78 / 76% | 0.82 |
laplace (mean log-likelihood 3.20) wins the likelihood race
against every baseline on non-price series — including GARCH-t, the heavy-tail
SOTA (68% raw / 65% family-weighted; mean LL 3.20 vs 3.03). On CRPS it beats the mean-model
baselines and loses only to the CRPS-specialists (fitted conformal ~30%, GARCH-t 45%
family-weighted) — their home turf. No free lunch on price/returns: on
equity/fx/commodity return series GARCH-t has the higher log-likelihood, and we recommend it
there rather than averaging the two regimes into one number.
The study →
Benchmark 2 — zero-shot foundation models
A different protocol: pretrained models used zero-shot (fixed 256-context, no refit) on 120
change-series (69 continuous). Per-series win-rate of laplace:
| Model | density | LL (all / cont) | CRPS (all / cont) |
|---|---|---|---|
| Moirai (1.1-R-small) | native Student-t | 98 / 97% | 94 / 93% |
| Lag-Llama | native Student-t | 67 / 99% | 65 / 94% |
| Chronos-Bolt (small) | quantile* | 76 / 100% | 67 / 88% |
| TimesFM (2.5-200M) | quantile* | 75 / 100% | 58 / 72% |
On the continuous series, laplace beats all four zero-shot. The native-density
models (Moirai, Lag-Llama) are the closest. Naive per-series fine-tuning doesn't
rescue them — it catastrophically overfits (logpdf +1.5 → −118). A zero-dependency online
ensemble holding its own against 200M-parameter transformers.
The foundation study →
Related reading
- Why skaters ranks by likelihood, by default. A scoring rule is a settlement rule — match it to the target. Why a reusable density is best judged by held-out log-likelihood (locality, the wealth a market pays, additivity under composition, the information gap) and where CRPS is the better target instead. Read it →
- On Prophet. The famous critique of Facebook Prophet — “Is Facebook’s Prophet the Time-Series Messiah, or Just a Very Naughty Boy?” (P. Cotton). Read it →
- Benchmarks repository. All studies are reproducible end-to-end: skaters/benchmarks.
Citation
@misc{cotton2026skaters,
title = {Transforms All the Way Down --- Automatic Online Distributional
Forecasting by Conjugation},
author = {Cotton, Peter},
year = {2026},
note = {https://skaters.microprediction.org}
}