FlashAttention-2 published
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
FlashAttention-2 published — 2× FlashAttention, 50–73% of the A100's theoretical FLOPs/s, up to 225 TFLOPs/s per A100 in training
Why it matters
Halved the practical cost of long-context training. The precondition that made 32K–128K context models economically viable
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2307.08691Added in Week 35, 2026 · See that week
On the map — ⑱ AI Infrastructure Software line
- Previous stopZeRO++
- This recordFlashAttention-2
- Next stopLlama 3 Herd paper