TimesFM vs Laplace

TimesFM is a decoder-only, patched time-series transformer; we score version 2.5. Run zero-shot with a fixed 128-length context.

Resources: GitHub · Paper (arXiv 2310.10688)

This page measures one thing: how sharp and how well calibrated TimesFM's univariate predictive density is, one step ahead on a change series, scored against Laplace on identical windows. It leaves out the covariates, the multivariate inputs, the long horizons and the native quantile or path outputs the model was built for, and its log score passes through a reconstruction of the model's quantiles (the table note says which). A model can trail here and be the better tool for its own job.

Live snapshot  Derived from the week-long round-robin study; counts grow as coverage deepens. Everything scores against Laplace on the identical series and windows.

Standalone

The model's own predictive, run zero-shot, scored per series against Laplace. Win, draw and loss are decided one series at a time. Each series has a rolling test window of 64 one-step-ahead points, and both forecasters are scored on the same points. For each point take the difference of log predictive densities, dt = log pmodel(yt) − log pLaplace(yt), with each log density floored at −20 first. The series' mean d̄ is tested against zero by a Diebold–Mariano test: the standard error is Newey–West with a Bartlett kernel and lag round(n^1/3). The model wins the series if d̄ > 1.96·SE, loses it if d̄ < −1.96·SE, and otherwise the series is a draw; a series with fewer than eight finite differences is a draw. A draw says the 64 points cannot separate the two forecasters at the 5% level. It does not say they are equal. With a window this short both win and loss counts understate the true rates, and the median ΔLL is the better measure of size. Counts are over the series both arms cover.

Generous reconstruction

Arm TimesFM(wide): the same nine-decile forecasts, reconstructed at three times the local bandwidth (nozzles.nozzle_wide), the most generous of the reconstructions tested in the benchmark paper. The paper found that the tight reconstruction produces most of the half-nat gap, so this arm is the headline.

Median per-series Δ log-likelihood in nats (negative is worse than Laplace); CRPS ratio to Laplace (above 1 is worse); central-90% coverage (0.90 target). Coverage is that of the wide reconstruction; fed a calibrated normal's deciles it covers about 0.88, so a perfectly calibrated model reads 0.88 here.

Tight nine-decile reconstruction

Arm TimesFM: the same forecasts through quantile_dist, the reconstruction the original study used. Fed the exact deciles of a standard normal it covers 0.81, so a perfectly calibrated model reads 0.81 in this coverage column and pays for it in log score. It understates the model and is kept here for comparison with the earlier numbers.

Median per-series Δ log-likelihood in nats (negative is worse than Laplace); CRPS ratio to Laplace (above 1 is worse); central-90% coverage (0.90 target).

Star plot

TimesFM standalone against Laplace, on the same six regime axes as the site's standalone radar. Each radius is the log-likelihood win rate, (wins + ½·ties) / n, divided by one half so an even split with Laplace sits on the dashed 1.0 ring; outward beats Laplace more often, inward less. The quantity is a share of series. The solid polygon is the headline arm and the dashed grey one, where present, is the tight nine-decile reconstruction. The M4-hourly set splits into soft and hard waveforms by corpus order, matching that radar.

Collaborative use

Four sandwich arms wrap TimesFM's own predictive on this same base (TimesFM(wide)), each paired with the standalone run on identical series and windows. Each cell gives the median per-series Δ log-likelihood against Laplace in nats (positive means better than Laplace) and the fraction of series where the arm loses to Laplace by the Diebold–Mariano test.

• +lap: vincentize with Laplace.
• ~lap: model location, Laplace residual scale.
• @lap: Laplace on the model's normal scores.
• &lap: online portfolio with Laplace.

Central-90% coverage (0.90 target) of the standalone model, its @lap and &lap wraps, and Laplace on the same windows:

Across the six bases that have these arms, the portfolio is the one sandwich that cuts the loss rate on every stratum without moving the median much; the residual-scale and normal-score wraps cost 0.1 to 0.35 nats on the economic strata, and the normal-score wrap over-covers. The sidecar page has the cross-model reading.

Protocol

Fixed 128-length context, rolling one-step test window, no fitting, each model in its own environment. Strata split the cached FRED universe and the M4-hourly set by frequency and regime. Full method on the sidecar pattern page and in the methodology.

Architecture and methodology

TimesFM is a decoder-only transformer in the style of a language model. It splits the context into fixed-length patches, treats each patch as a token, and is pretrained on a large corpus of real and synthetic series. This study scores version 2.5 (200M parameters) with its continuous quantile head, which emits the predictive spread directly.