AI Infrastructure SoftwareTri Dao, Daniel Y. Fu et al. (Stanford)Tri Dao Stanford UniversityMay 27, 2022

Dao et al. present FlashAttention

Made by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made

Presented FlashAttention — IO-aware tiling gave a 3x speedup on GPT-2 while keeping attention exact. The first Transformer to beat chance on Path-X (16K)

Why it matters

Effectively opened up the road to long context

Source — Primary sources first; secondary reports are labelled “Press”

Paperarxiv.orghttps://arxiv.org/abs/2205.14135

Added in Week 35, 2026 · See that week

On the map — ⑱ AI Infrastructure Software line

See it on the map

Same week

May 27Li et al. present Diffusion-LMPaper May 26Kusupati et al. present Matryoshka Representation LearningPaper May 24Kojima et al. present Large Language Models are Zero-Shot ReasonersPaper May 24Jung et al. present Maieutic PromptingPaper

Nearby on the same axis

Jun 30, 2022Microsoft presents DeepSpeed InferencePaper Mar 23, 2022Google presents PathwaysPaper Mar 22, 2022NVIDIA unveils the Hopper H100Official Sep 12, 2022NVIDIA, Arm and Intel present FP8 Formats for Deep LearningPaper