What is computer vision? The field of AI that lets computers look at images and work out what's in them

The field of AI that lets computers look at images and work out what's in them.

Computer vision is about teaching computers to see. To a computer, a photo is just millions of dots, each stored as numbers for its colour. The goal is to find edges, shapes and objects in those dots and work out what they mean: a cat, a crosswalk, a face.

In 1989 a neural network learned to read handwritten ZIP codes. In 2012 a deep neural network won the ImageNet contest by a wide margin, and the whole field switched to deep learning. Today it powers face unlock, photo search and lane detection in cars. Studies have also found that some face systems were less accurate depending on skin colour and gender.

Think of it like this

Think of a page from Where's Waldo. At first it's a mess of lines, but you spot small clues, like a striped shirt or a bobble hat, and put them together until you find him. Computer vision works the same way: tiny edges first, then bigger shapes, then the whole object.

For example

Search "dog" in your photo app and it finds every picture with a dog in it: that's computer vision.

Common mix-ups

Mix-upA computer takes in a photo at a glance, the way people do.
ActuallyMany vision models first shrink a photo down to a small square before looking at it. So small writing in the corner of a big photo, or something far away, can get blurred in the shrinking and easily missed.
Mix-upFace recognition is equally accurate for everyone.
ActuallyEven for the same person, accuracy drops in dim light, side-on, with a mask, or after a face has changed over a few years. Face unlock can sometimes open for a twin or a look-alike relative, so it's safer to add a passcode for important apps.

Try it in class

For teachers at the start of a lesson — or for students working on their own.

Question: How is the way computer vision "sees" like doing a hidden-picture puzzle?
Activity: Open the 1989 zip-code (E0013), 2009 ImageNet (E0022) and 2012 competition (E0036) records in order and write one line on what the computer learned to recognise in each. Then look at one classroom photo together and fill three columns: lines, shapes, objects.