PyTorch Foundation

NVGEMM was put into NVFP4 decode on Blackwell, folding the dequantization scale into the GEMM and removing the cost of launching a kernel per layer. float8_e4m3fn and _scaled_mm were added to MPS on Apple Silicon, with p

The point at which numerical reproducibility (determinism, tensor-hashing debugging) made the release headline. A sign that the training stack is moving from a performance race to a debuggability phase

A direction that lets users write multi-GPU kernels directly at the framework API level. On top of NCCL-style collective communication, user kernels gain one more path

A release that shows the move to a compiler-centered stack, with a stabilized ABI for C++ extensions and compilable control flow

The serving engine went from research project to infrastructure public good

Compiler-based optimization became a framework default