AI Infrastructure SoftwareJay Shah et al. (Colfax, Together AI, Meta, NVIDIA, Princeton)MetaNVIDIAPrinceton University
FlashAttention-3 published
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
FlashAttention-3 published — Hopper-specific asynchrony delivers 1.5–2.0× in FP16 on H100 (up to 740 TFLOPs/s), and about 1.2 PFLOPs/s in FP8
Why it matters
A case showing that kernels must be rewritten for a specific generation of silicon. Quantitatively rebutted accuracy concerns about FP8 attention
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2407.08608Added in Week 35, 2026 · See that week
On the map — ⑱ AI Infrastructure Software line
- Previous stopZeRO++
- This recordFlashAttention-3
- Next stopLlama 3 Herd paper