What is an embedding? A list of numbers that captures the meaning of a word

A list of numbers that captures the meaning of a word, a sentence or an image.

Computers only handle numbers, so AI turns words and sentences into lists of hundreds or thousands of numbers. That list is called an embedding, or a vector.

The numbers aren't random. They're learned so that words used in similar ways get similar numbers, which puts similar meanings close together. A computer can then tell that "lunch" and "midday meal" are close, even though they share no words. Google's word2vec, released in 2013, made this idea popular.

Embeddings power meaning-based search and recommendations. RAG also uses them to find the passages closest to your question.

Think of it like this

Imagine a music app placing every song on a map: upbeat dance tracks cluster in one corner, slow ballads in another. Recommending songs near the ones you play is like using embeddings, which are coordinates on a map of meaning.

For example

A famous example: "king" minus "man" plus "woman" lands near "queen".

Common mix-ups

Mix-upThe numbers in an embedding are labels someone assigned to each word.
ActuallyLabel-like ID numbers do exist, but it's the tokenizer that attaches those to each chunk. An embedding turns that ID into coordinates set by training, so a single number usually has no meaning a person could read — the meaning lies in the distances to other words.

Try it in class

For teachers at the start of a lesson — or for students working on their own.

Question: How can a computer tell that two words with no letters in common are "close"?
Activity: Draw a map on the board — left to right "exciting ↔ calm", bottom to top "things you eat ↔ things you do" — and decide together where twenty words from the class should go. Explain why neighbours are neighbours, then remember that a real embedding has hundreds or thousands of such directions. Open the 2013 word2vec record (E0037) too.