Researchers release Random Attention
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
Random Attention released — against the usual approach to KV-cache eviction at inference time, which scores cached tokens to decide which ones to keep, reports that the selection signal contributes almost nothing, and presents a method that preserves the prompt while removing tokens at random within each attention head, eliminating score computation altogether. Reports that across 4 models and 6 reasoning benchmarks it matches stronger alternatives in performance while delivering considerably higher throughput in a real vLLM setting
Why it matters
Raises a counterargument against the entire lineage of importance-score-based KV eviction: "the scores themselves are unnecessary". The authors explain that only the prompt is fragile, while reasoning sequences sustain themselves through a double redundancy — textual restatement and duplication across multiple attention heads — so as long as the prompt is protected, random selection is enough. Figures are as reported by the authors.
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2609.03430Added in Week 36, 2026 · See that week
On the map — ⑧ Inference Infrastructure line
- Previous stopGoogle Cloud previews its 8th-generation TPUs at Cloud Next
- This recordRandom Attention
- Next stopDeepSeek-V4.1-Flash