Challengers
Everyone who has stepped up, and how it went.
skaters cooperates as well as it competes: sandwiched in laplace's coordinates, every challenger below improves dramatically (the sandwich), so a bare horserace can mislead. These pages exist as counterpoint to overly aggressive marketing in autonomous univariate distributional prediction.
Featured bouts
TabFM — David v Godzilla
A 12 GB tabular foundation model, pre-registered, eleven arms, every one behind laplace on likelihood, at one ten-thousandth the footprint. The bout →
Prophet
Three scales, 30 to 921 series, pre-registered at the largest. Raw: 4 wins in 921, and −11 nats median on repeat-heavy series. The tape →
The record
Numbers are per-series win-rates for laplace,
log-likelihood first, CRPS second. Everyone is converted to the same
Dist and scored by the same code on the same held-out
points. Every row links its tape. The one belt we do not hold is at
the bottom, stated rather than averaged away.
| challenger | corner | LL | CRPS | tape |
|---|---|---|---|---|
| AutoARIMA (statsforecast) | classical | 82% | 53% | study |
| auto.arima (real R) | classical | 79% | 51% | bout |
| AutoETS | classical | 97% | 83% | study |
| ETS / SARIMAX (statsmodels) | classical | 96% / 87% | 81% / 53% | study |
| Theta (the M3 winner, R) | classical | 87% | 63% | bout |
| ADAM (smooth, R) | state space | 96% | 82% | bout |
| BSTS (full Bayesian posterior, R) | state space | 97% | 80% | bout |
| nnetar (neural AR, R) | neural | 96% | 76% | bout |
| NF-StudentT (NeuralForecast) | neural | 100% | 78% | study |
| CSP (incl. adaptive) | conformal | 99% | 100% | bout |
| AutoARIMA + conformal / ACI | conformal | 86% / 88% | 31% / 29% | study |
| TimesFM 2.5, 200M (Google) | foundation | 100% cont. | 72% cont. | David v Goliath |
| Chronos-Bolt (Amazon) | foundation | 100% cont. | 88% cont. | study |
| Moirai (Salesforce) | foundation | 97% cont. | 93% cont. | study |
| Lag-Llama | foundation | 99% cont. | 94% cont. | study |
| TabFM 1.0 (Google, tabular) | foundation | 71–88% (11 arms) | 38–60% | David v Godzilla |
| Prophet (Meta) | calendar GAM | 99.6% | — | study |
| GARCH-t, non-price series | heavy-tail SOTA | 68–82% | 53–54% | bout |
| GARCH-t, price/return series | heavy-tail SOTA | the belt is theirs; we recommend it there | study | |
Featured bouts
One bout page so far. Challengers that earn a full treatment get one; the next candidates are the native-density foundation models, Moirai and Lag-Llama, which came closest. Rematch conditions for TimesFM are tracked in skaters#97.
Send a challenger
The protocol is packaged and copyable: the
benchmark-against-laplace
skill for any distributional forecaster, and the
timesfm-study skill for
foundation models specifically. Same Dist, same code,
same held-out points, both metrics, splits disclosed. Run it, and if
your method wins, open an
issue with the tape; the table takes new rows in either
direction.
New bouts are pre-registered: the protocol, parameters and analysis plan are committed to benchmarks/preregistrations before the results are read, with everything already observed at filing disclosed. The first filed statement is the TabFM bout.