Evaluation & BenchmarksFutureHouse
Humanity's Last Exam bio/chem error rate put at 29%; HLE's own review finds 18%
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
Raised a 29% error rate in the biology and chemistry questions of Humanity's Last Exam; the HLE team's own review put it at 18%, and it announced rolling revisions
Why it matters
A case that showed, through public refutation and response, that the answer keys of frontier benchmarks can themselves be wrong
Source — Primary sources first; secondary reports are labelled “Press”
Officialfuturehouse.orghttps://www.futurehouse.org/research/hle-examAdded in Week 35, 2026 · See that week
On the map — ⑯ Evaluation line
- Previous stopTechCrunch reports criticism of Epoch AI for not disclosing OpenAI funding
- This recordHumanity's Last Exam bio/chem error rate put at 29%; HLE's own review finds 18%
- Next stopGDPval