Anomaly detection: a literature map

Organized around one question: who can promise a false-alarm rate on a drifting stream?

The detector this map is about lives in its own project, timemachines (calibrated online anomaly detection, built on skaters). Start there for the method, its measured false-alarm rates, and benchmarks. The map below is kept as reference.

Streaming anomaly detection is two jobs wearing one name. Job 1 — where is the anomaly — is subsequence search and ranking, and it is well occupied. Job 2 — when to alarm — is threshold selection with a false-alarm rate you can state in advance and verify afterwards, and it is nearly empty: every formal guarantee in the 2024–26 literature assumes exchangeability or i.i.d.-ness that real drifting, seasonal streams violate. The map below groups the literature by which job it does and which assumption it leans on. Forward citations verified against primary sources 2026-07; corrections welcome via GitHub issues.

The seat everyone is circling — exact per-observation false-alarm calibration under gradual drift and seasonality — is open. The three modern lines (weighted conformal, conformal + online FDR, e-processes) all claim principled alarm control, none cites or benchmarks against SPOT/DSPOT or telemanom, and none delivers the guarantee off exchangeability. A forecaster that reduces the stream to calibrated PIT/z residuals is one candidate key to that seat: it manufactures the near-i.i.d. input every line above assumes. That is the skaters angle — see the study log.

The core: calibrated alarms on streams

• Siffer, Fouque, Termier & Largouët (KDD 2017) — Anomaly detection in streams with extreme value theory. SPOT/DSPOT: POT/GPD tails turn a target tail probability into a streaming threshold. Still the deployed default nine years on (author-maintained libspot 3.0.0, 2026-04); assumes quasi-stationarity after EWMA de-drift and has a documented masking failure under contaminated calibration.
• Hundman et al. (KDD 2018) — Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. Telemanom: forecast-then-threshold in production at NASA; the ancestor of every "residual + moving threshold" pipeline, with no calibration guarantee.
• Shin, Ramdas & Rinaldo (NEJSDS 2023) — E-detectors: a nonparametric framework for sequential change detection. Anytime-valid alarms with nonasymptotic average-run-length control for composite nonparametric classes. "Distribution-free" means membership in a stated class (bounded, sub-Gaussian), not arbitrary drift.
• Rebjock, Kuhn, Joppen & Amini (NeurIPS 2021) — Online false discovery rate control for anomaly detection in time series. The paper that treats when-to-alarm as its own problem: memory-decay LORD/SAFFRON/ADDIS variants fix alpha-death on long streams, with mFDR control under locally dependent p-values.
• Xu & Ramdas (AISTATS 2024) — Online multiple testing with e-values. e-LOND: online FDR under arbitrary dependence — the dependence-robustness the LORD family lacks, bought with power.
• Zhang et al. (ICML 2025) — e-GAI: e-value generalized alpha-investing. e-LORD/e-SAFFRON with dynamic level allocation; the most technically mature dependence-robust alarm-budget machinery. Targets FDR, not per-observation rates; never benchmarked against POT.
• IBM Research — W1-ACAS (ICLR 2026) — Adaptive conformal anomaly detection with time series foundation models. The most direct current occupant: weighted conformal p-values (Barber et al. 2023) with weights tuned online toward PIT uniformity. Its drift guarantee is α plus an unknowable total-variation slack (Prop. 3.1) — the strongest calibration claim found for drifting streams, and it is approximate. Notably the first evaluation reporting calibration error and FPR as headline metrics.
• Farzaneh & Simeone (2025) — C-PP-COAD: context-aware online conformal anomaly detection. Conformal p-values + LORD thresholds + prediction-powered synthetic calibration; smoothed-FDR control assuming i.i.d.-within-known-context — regime switching yes, gradual drift no.
• PITMonitor (2026, preprint) — anytime-valid monitoring of forecast calibration. Ville's inequality on a mixture e-process over PITs: P(ever alarm | H0) ≤ α with no clean baseline period. Validity requires i.i.d. PITs under the null — the closest published machinery to the skaters parade, and the same open edge.
• CALIBURN (2026, preprint) — conformal risk control for streaming intrusion alerts. Converts an operator alert budget into a threshold with a marginal FP guarantee under bounded drift, and candidly documents two collapse modes of conformal thresholding at small budgets. Evidence the conformal route is not yet a drop-in POT replacement.

Antecedents: the probability machinery

• Rosenblatt (1952) — Remarks on a multivariate transformation. The transform that reduces any stream, under a correct sequential model, to i.i.d. uniforms — the bridge between forecasting and every calibrated-alarm line above.
• Dawid (1984) — The prequential approach. Statistics done strictly one-step-ahead; the evaluation protocol no modern TSAD benchmark implements.
• Diebold, Gunther & Tay (1998) — Evaluating density forecasts. PIT histograms as the calibration check; the econometrics ancestor of PIT-based alarming.
• Page (1954) — Continuous inspection schemes. CUSUM; with Lorden (1971) and Shiryaev, the source of run-length false-alarm semantics that e-detectors made nonparametric.
• Adams & MacKay (2007) — Bayesian online changepoint detection. BOCPD: the one classical method that outputs an intervention probability — calibrated exactly to the extent its model is true.
• Vovk, Gammerman & Shafer (2005) — Algorithmic learning in a random world. Conformal prediction; Laxhammar & Falkman (2015) specialize it to sequential anomaly detection under exchangeability.
• Gibbs & Candès (NeurIPS 2021) — Adaptive conformal inference under distribution shift; with Barber, Candès, Ramdas & Tibshirani (2023) — Conformal prediction beyond exchangeability — the two escape hatches from exchangeability that W1-ACAS and the drift-aware conformal line build on.
• Efron (2004) — The empirical null. Estimate the null from the stream itself rather than trusting theory — the move the skaters detector makes with its Satterthwaite-matched d² null.
• Ramdas, Grünwald, Vovk & Shafer (2023) — Game-theoretic statistics and safe anytime-valid inference. The survey that frames e-values as the currency for acting on accumulating evidence — the natural formalism for intervention decisions.

Job 1: the scoring heads

• Yeh et al. (ICDM 2016) — the matrix profile; and Lu et al. (KDD 2022) — DAMP, its streaming left-discord form. The strongest classical head on UCR-style tasks (0.65–0.75 published; 0.587 in our fixed-window replication). Carries its own subsequence normalizer — which is why a forecaster front-end hurts it (raw 0.587 vs fronted 0.427).
• Guha et al. (ICML 2016) — Robust random cut forest. Streaming isolation scoring; front-end-neutral in our tests (0.244 raw vs 0.236 fronted).
• Sub-PCA / KShapeAD / POLY — the current TSB-AD-U leaderboard top (VUS-PR 0.42, 0.40, 0.39): subsequence PCA, shape clustering, polynomial fits. Simple statistical methods lead; deep and foundation models trail on univariate data.
• Ahmad, Lavin, Purdy & Agha (2017) — HTM and the NAB benchmark. Streaming-native scoring with windowed rewards; label quality since discredited, repo frozen.

Benchmarks and the metrics critique

• Wu & Keogh (TKDE 2023) — Current time series anomaly detection benchmarks are flawed. The critique that produced the UCR Anomaly Archive (250 series, one anomaly each, argmax scoring).
• Kim et al. (AAAI 2022) — Rigorous evaluation of deep AD. Point-adjust F1 lets random scores reach SOTA; the metric is dead, and a large slice of pre-2022 deep-TSAD claims died with it.
• Paparrizos et al. (PVLDB 2022) — Volume under the surface (VUS). Range-aware, threshold-free AUC — and therefore invariant to monotone score transforms: structurally blind to calibration.
• Liu & Paparrizos (NeurIPS 2024) — TSB-AD. The current standard (40 detectors, 1,070 series); crowns VUS-PR and never evaluates threshold-setters or false-alarm rates. TAB (PVLDB 2025) — unified TSAD benchmarking — sweeps ten threshold percentages and reports the best: an oracle protocol, not an alarm rule. EasyTSAD/TimeSeriesBench (ISSRE 2024) — event-based, delay-aware online metrics — the closest to deployment reality, still without calibration scoring.
• Verified gap (2026-07): no major benchmark runs a strictly prequential protocol, and none scores whether a stated false-alarm rate comes true. The only calibration-aware evaluation found anywhere is inside the W1-ACAS paper itself.

Where this sits

skaters' contribution is upstream of every line above: laplace is an online forecaster whose parade state exposes PIT/z residuals, manufacturing, on a drifting seasonal stream, the near-i.i.d. uniforms that conformal, online-FDR, and e-process machinery all assume and none can create. The detector built on that signal, with its measured false-alarm rates and its lifts to third-party heads (DSPOT, RRCF), is timemachines — that is where the alarm logic and the full-archive studies live. (skaters keeps the tail story that makes the surprise stream honest: censored-ML GPD tails are spliced into every predictive from v0.13.0, so the 99.9% interval is empirically 99.85%; see the coverage study and the tail essay.)

Assembled 2026-07-09 from a verified forward-citation sweep (25/25 claims confirmed against primary sources). Additions and corrections: open an issue.