Scale AI releases SWE-Bench Pro
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
SWE-Bench Pro released — 41 repositories and 1,865 tasks, in three tiers: public, held-out and commercial
Why it matters
A follow-up design that responds to contamination and saturation in the SWE-bench family. An attempt to prevent leakage structurally by separating tiers
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2509.16941Added in Week 35, 2026 · See that week
On the map — ⑯ Evaluation line
- Previous stopTechCrunch reports criticism of Epoch AI for not disclosing OpenAI funding
- This recordSWE-Bench Pro
- Next stopGDPval