📌 New: Evaluator-backed benchmarking, after the MLE-Bench moment MLE-Bench has been quietly contested across r/MachineLearning over the last week. The skepticism is not really about any single metric inside the benchmark; it is about whether a static benchmark structure can survive sustained adversarial attention from teams with economic incentive to game it. The standard answer (better methodology, rotating held-out sets, broader task coverage) is real and partial. None of it fixes the structural problem: the benchmark as an artefact is a fixed target. Evaluator-backed benchmarking is the structural counter. Every judgement contributing to a published benchmark statistic traces back to a stable evaluator identity (a W3C DID the evaluator controls), a signed verifiable credential carrying the rubric version and expertise attestations, longitudinal consistency credentials, and a status trail for revocations. The benchmark stops being a number the publisher asks the field to trust. It becomes an artefact any third party can audit at the judgement layer. Issue 02 Monday made this argument at the policy-and-research level with the METR teardown. MLE-Bench is the same warning shot moved one level closer to the user-facing capability claim. The first publishers to ship evaluator-backed benchmarking will be the ones whose results survive the next round of teardowns. This is Day 2 of Ontology Roundup, Issue 03. Read it 👉 https://ont.io/news/evaluator-backed-benchmarking/