What is RAG? Having an AI look up relevant documents first

Having an AI look up relevant documents first, then answer based on them.

RAG stands for retrieval-augmented generation. Instead of answering only from what it memorized in training, the AI first fetches documents related to your question and reads them along with it. The name comes from a 2020 paper by Facebook AI Research and university partners.

Usually the documents are split into chunks and turned into embeddings; the chunks closest in meaning to your question get added to the prompt. Swapping in new documents updates the answers with no retraining, and it's easy to cite sources.

It helps reduce hallucinations. But if the retrieved text is wrong, the answer will be too.

Think of it like this

It's the difference between a closed-book and an open-book test. With the book open, you can find the page and back up your answer even if your memory is fuzzy. Open it to the wrong chapter, though, and your answer goes wrong too.

For example

Give it your school's festival flyer, and it can answer "When's the booth sign-up deadline?" from that flyer.

Common mix-ups

Mix-upWith RAG, the AI can't get things wrong.
ActuallyEven when it fetches the right material, the AI can misread it or quietly add things that aren't in it. Get into the habit of clicking the source next to an answer to check the sentence really is there.
Mix-upTo answer from new material, the model has to be retrained.
ActuallyThe original 2020 RAG paper joined two kinds of memory: the knowledge held in the model's numbers, and a separate collection of documents it can look things up in. When there's news, you only swap the document side; the model's numbers stay untouched.

Try it in class

For teachers at the start of a lesson — or for students working on their own.

Question: If the AI shows which page it opened, what more can you check?
Activity: Cut a short notice (say, a school festival flyer) into five numbered chunks. Classmates ask questions one by one; one person plays "librarian" and fetches the single most relevant chunk, and another answers from that chunk only. See what happens when the librarian fetches the wrong chunk, and talk about why writing the chunk number (the source) next to the answer helps. Open the 2020 record that named RAG (E0107) too.

Records about this

Source-checked moments from AI history.

May 22, 2020FAIR, UCL, NYU (Lewis et al.)

FAIR, UCL and NYU publish the original RAG paper

The canonical text of retrieval-augmented generation, combining parametric and non-parametric memory

RAG, Fine-Tuning & ContextPaper