Microsoft presents ZeRO-Infinity
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
Presented ZeRO-Infinity — uses heterogeneous GPU+CPU+NVMe memory, exceeding 25 petaflops on 512 V100 GPUs
Why it matters
Got around the GPU memory wall by extending into the storage tier. Established the framing that the constraint is bandwidth, not memory
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2104.07857Added in Week 35, 2026 · See that week
On the map — ⑱ AI Infrastructure Software line
- Previous stopZeRO-Offload
- This recordZeRO-Infinity
- Next stopHopper H100