Verifiable Social Reasoning
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
Verifiable Social Reasoning released — Fuse, a framework that uses multi-agent simulation to make LLMs' social reasoning checkable. Its fidelity was checked against 24,000 human annotations, and 21,000 examples were released. An evaluation of 12 models shows that situations in which information is relayed through the user add difficulty, that models are systematically swayed by biased user framing, and that they ask for more context than people do. Performance did not rise consistently as conversations grew longer.
Why it matters
A benchmark that separates out and measures the failures of agents working in between people as a problem of "mediated information" rather than of "ability".
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2609.17496Added in Week 39, 2026 · See that week
On the map — ⑯ Evaluation line
- Previous stopRealSWE
- This record · stopVerifiable Social Reasoning
- Next stopFDE-Gym