OpenAI releases SWE-bench Verified
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
SWE-bench Verified released — 93 developers triple-annotated 1,699 samples to build a 500-sample subset, excluding 68.3% of the original
Why it matters
The reference dataset for evaluating agentic coding. It revealed that a large share of the original benchmark was unfit
Source — Primary sources first; secondary reports are labelled “Press”
Officialopenai.comhttps://openai.com/index/introducing-swe-bench-verified/Added in Week 35, 2026 · See that week
On the map — ⑯ Evaluation line
- Previous stopGSM1k
- This recordSWE-bench Verified
- Next stopTechCrunch reports criticism of Epoch AI for not disclosing OpenAI funding