Prehistory — From Neural Networks to the TransformerSepp Hochreiter
Hochreiter's TU Munich thesis identifies the vanishing gradient problem
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
Identified the vanishing gradient problem in a thesis at the Technical University of Munich
Why it matters
The first to show that the learning signal vanishes as depth and sequence length grow. The problem definition behind LSTM and residual connections
Source — Primary sources first; secondary reports are labelled “Press”
Pressen.wikipedia.orghttps://en.wikipedia.org/wiki/Vanishing_gradient_problemAdded in Week 35, 2026 · See that week
On the map — ⓪ Prehistory line
- Previous stopBackpropagation
- This recordHochreiter's TU Munich thesis identifies the vanishing gradient problem
- Next stopGradient-based learning applied to document recognition