What is deep learning? Machine learning with neural networks stacked many layers deep

Machine learning with neural networks stacked many layers deep.

Deep learning is machine learning done with neural networks that have many layers. "Deep" doesn't mean wise; it means lots of layers. Early layers pick up simple things like edges and dots; later layers combine them into eyes, faces and whole objects.

The idea is old, but deep networks were hard to train for a long time. In 2012 a deep network won the ImageNet image contest by a huge margin, and the field switched over. Big datasets, fast graphics chips (GPUs) and deep designs had finally come together. Most of today's translation, speech recognition and large language models are built with deep learning.

Think of it like this

When you learn to draw, you start with lines, then shapes, then faces, then expressions. The layers in deep learning stack up the same way, from easy patterns to hard ones.

For example

Translation apps that turn a whole sentence smoothly into another language run on deep learning.

Common mix-ups

Mix-up"Deep" means it thinks more deeply, or is smarter.
ActuallyMore layers doesn't mean deeper judgement. Experiments have shown many times that changing a photo so slightly that people can't see any difference can make a deep network give a completely wrong answer.
Mix-upDeep learning is an idea from 2012.
ActuallyBack in 2006, Hinton's team showed that deep networks could be trained by teaching the layers one at a time and then fine-tuning the whole thing. 2012 is better seen as the year that long build-up finally broke through.

Try it in class

For teachers at the start of a lesson — or for students working on their own.

Question: Lots of data, fast GPUs, deep structure — if one had been missing, would 2012 have happened?
Activity: Like learning to draw (lines → shapes → faces → expressions), split "recognise a handwritten 7" into four stages and sketch them on paper. Draw arrows to show what each stage takes from the one before — that's what "layers" means.