How to cite the benchmark
Cite an immutable sweep, never the moving latest result — so a reference still resolves to the exact numbers it described.
Ready to cite
No published sweep to cite yet. Open any sweep page for its citation block once one is published.
The cite-stable URL contract
The website is a citation surface, so its URLs are part of the published record. These guarantees are the standing contract; the canonical definitions live in the data API and methodology docs.
Stable permalinks
Once published, these paths keep their meaning across redesigns — layout may change, the path and what it identifies do not:
| Route | Identifies |
|---|---|
/benchmarks/sweep/<id> | one immutable sweep |
/benchmarks/algorithm/<algo> | one algorithm |
/benchmarks/dataset/<corpus>/<dataset> | one dataset |
/benchmarks/corpus/<corpus> | one corpus |
/benchmarks · /methodology · /citation · /api · /changelog | stable section pages |
The <id> / <algo> / <corpus> / <dataset> segment is the same identifier used in the data tree (sweeps/<id>.json, by-algo/<algo>.json, …), so a page permalink and the data-file URL it renders never drift apart.
The guarantees
- Immutable. A finalized sweep manifest is never rewritten. A re-run gets a new
sweep_idand a new path; old manifests stay frozen. - Honest 404, never substitution. Retention may eventually delete an old sweep's trials, but a manifest URL either returns the exact bytes it had, or 404s past retention — it never resolves to different numbers.
- Old URLs keep working. The pre-IA
/benchmarksURL is the section landing, not a removed route; a genuine rename is kept as a redirect to the new path, never a silent removal. - Compare views are a convenience.
/benchmarks/compare/…is reconstructable from its operands' permalinks — shareable, but the canonical citation is the underlying sweep / algorithm permalinks. - Query-string UI state is never load-bearing for a citation (sort order, expanded panels).
Why not the latest result?
The latest.json pointer moves with every new sweep, so a reference to it would silently re-point at different numbers. It is deliberately not a citable surface — cite the immutable sweeps/<sweep-id>.json it currently points at instead. An individual trial path (trials/…/seed-<n>.json) is an input to a sweep, not the published result, so it is not the citation either.
A DOI for the benchmark corpus is planned. Until then, the immutable sweep URL plus its embedded citation block is the canonical reference. See methodology for what the numbers mean, and the changelog for how versions move over time.