Hugging Face explains the MMLU score mismatch on the Open LLM Leaderboard
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
MMLU score mismatch on the Open LLM Leaderboard explained — the same LLaMA 65B scored 0.636/0.637/0.488 depending on the implementation
Why it matters
The event that publicly confirmed benchmark numbers are not comparable
Source — Primary sources first; secondary reports are labelled “Press”
Officialhuggingface.cohttps://huggingface.co/blog/evaluating-mmlu-leaderboardAdded in Week 35, 2026 · See that week
On the map — ⑯ Evaluation line
- Previous stopMMLU
- This recordHugging Face explains the MMLU score mismatch on the Open LLM Leaderboard
- Next stopSWE-bench