한국어EN
Evaluation & BenchmarksOpenAI Frontier EvalsOpenAI

Why SWE-bench Verified no longer measures frontier coding capabilities

Made by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made

Why SWE-bench Verified no longer measures frontier coding capabilities

Why it matters

At least 59.4% of the audited problems had flawed tests; the vendor acknowledges that the benchmark harness's signal is contaminated

Source — Primary sources first; secondary reports are labelled “Press”

Officialopenai.comhttps://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/

Added in Week 35, 2026 · See that week

On the map — ⑯ Evaluation line

  1. Previous stopTime Horizon 1.1
  2. This record · stopWhy SWE-bench Verified no longer measures frontier coding capabilities
  3. Next stopARC-AGI-3
See it on the map

Same week

Samsung to turn its manufacturing sites into AI-driven autonomous factories by 2030OfficialKorea announces its AI Action Plan (AI Basic Plan 2026–2028)GovernmentAnthropic fully rewrites its RSP as v3.0OfficialMETR announces a redesign of its developer productivity studyOfficial

Nearby on the same axis

METR releases Time Horizon 1.1OfficialLMArena renames itself ArenaOfficialARC Prize releases ARC-AGI-3OfficialMETR publishes an official note on the limits of its time horizon metricOfficial