Prehistory — From Neural Networks to the TransformerSepp Hochreiter, Jürgen Schmidhuber
Hochreiter and Schmidhuber publish Long Short-Term Memory
AIMade by AIWritten by AI (Claude) from the source below. Check anything important against the source.How it's made
Published Long Short-Term Memory — Neural Computation 9(8) 1735–1780
Why it matters
Used gates to mitigate vanishing gradients, making it possible to learn long sequences. The standard sequence model before the Transformer
Source — Primary sources first; secondary reports are labelled “Press”
Officialbioinf.jku.athttps://www.bioinf.jku.at/publications/older/2604.pdfAdded in Week 35, 2026 · See that week
On the map — ⓪ Prehistory line
- Previous stopBackpropagation
- This recordHochreiter and Schmidhuber publish Long Short-Term Memory
- Next stopGradient-based learning applied to document recognition