How Large Language Models Work, An Animated Lesson

Lesson by Adnan, computer science lecturer  ·  aitrendblend editorial team  ·  Computer Science Education  ·  October 2026  ·  Reading time about 24 minutes

How Large Language Models Work
An Animated Lesson

Who this lesson is for and how to use it

This is the tenth lesson in our Computer Science Education series. It continues directly from How AI Learns, which showed how a network of neurons improves by gradient descent and ended with a short look at chatbots. Here we open the chatbot up. Students who have not done the AI lesson can still follow, because every idea is shown as a picture first, but the earlier lesson makes the training part feel less like magic.

The lesson suits grades nine to twelve and first year college students. Almost every student now uses chatbots for homework, which makes this one of the most relevant lessons a computing teacher can give. Understanding how these tools work is also the best protection against trusting them blindly, so the lesson ends with a frank discussion about mistakes and responsible use.

Learning objectives

After three periods, a student should be able to do the following five things without notes.

1. Explain that a language model generates text by predicting one token at a time.
2. Describe what a token is and why long or unusual words become several tokens.
3. Predict how raising or lowering temperature changes the text a model writes.
4. Explain an embedding as a position in space where similar words sit close together.
5. Describe what attention does and give one reason why chatbots produce false statements.

What a large language model actually is

Every phone keyboard suggests the next word as you type. Write the word happy and it may offer birthday. A large language model, often shortened to LLM, is the same idea grown to an enormous size. It has read a vast amount of text, books, websites, code and articles, and learned from it how language usually continues. When you send a question, the model does not search a database for an answer. It looks at everything written so far and predicts what should come next, one small piece at a time.

The word large refers to two things. The training text is huge, far more than any person could read in many lifetimes, and the model itself is a neural network with billions of adjustable numbers called parameters, the same kind of weights students adjusted by hand in the AI lesson. Language refers to what it works with, and model means it is a simplified mathematical description of how text behaves, not a mind and not a copy of the internet.

Tokens, how text becomes numbers

A neural network can only do arithmetic, so the first job is turning text into numbers. Models do not usually read whole words or single letters. They read tokens, pieces of text that are often a whole common word and sometimes a fragment of a rarer one. The word the is one token. A long word such as unbelievably may become three or four pieces. Each token in the model’s vocabulary has an ID number, and that list of numbers is all the network ever sees.

The pieces are learned from data. A popular method called byte pair encoding starts with single characters and repeatedly merges the pair that appears most often in the training text, until it has a vocabulary of tens of thousands of pieces. OpenAI’s guidance for its own models gives a handy rule of thumb, roughly four characters of English text per token. Type into the lab below, or try the presets, and see how your text is cut up. Our toy tokenizer uses hand written rules instead of learned merges, but it behaves in the same spirit.

Three surprises usually come out of this lab. Long and unusual words cost several tokens. Numbers are often split digit by digit, which is one reason models can stumble on arithmetic. And text in languages other than English, such as Urdu, can need far more tokens for the same meaning, because the tokenizers of many popular models were trained mostly on English text. That last point is also the answer to the letter counting puzzle from the start. The model never sees the individual letters of a word that arrives as one or two tokens, so counting them is genuinely hard for it.

Predicting the next word

Once the text is a list of token numbers, the model’s single task is to predict the next token. For every token in its vocabulary it outputs a score, and a function called softmax turns those scores into probabilities that add up to one. Then one token is chosen, added to the text, and the whole process runs again with the longer text. A chatbot answer of three hundred words is three hundred or more of these predictions in a row.

The idea is far older than chatbots. In 1948 Claude Shannon, in the paper that founded information theory, generated imitation English by choosing each word according to how often it followed the previous word in real text. Lab 2 does exactly what Shannon did. Its model has read only twelve short sentences about cats, dogs, students and code, and it predicts by counting which word followed which. Start with the, then build a sentence one word at a time.

Press Most likely word several times and the sentence soon loops, the cat sat on the cat sat on the, because always choosing the top word leads back to the same places. Press Roll the dice and the model samples a word in proportion to its probability, which gives more variety. Real chatbots almost always sample, which is why asking the same question twice can produce two different answers.

Temperature, the creativity dial

The temperature slider changes how the scores become probabilities. Each score is divided by the temperature before softmax. A low temperature makes the favourite word even more dominant, so the text becomes predictable and repetitive. A high temperature flattens the bars, so unlikely words get a real chance, and the text becomes surprising, sometimes creative and sometimes nonsense. Many chatbot tools let developers set this value for exactly that reason.

Embeddings, meaning as position

Counting word pairs, as Lab 2 does, has a serious weakness. The model treats cat and kitten as completely unrelated, because they are different words. Modern models fix this with embeddings. Each token is given a list of numbers that places it as a point in space, and during training these points move so that words used in similar ways end up close together. Nobody tells the model that cat and kitten are related. It discovers this because the two words appear in similar sentences.

In 2013 a team at Google led by Tomas Mikolov released word2vec, which learned such vectors from billions of words and showed something remarkable. Directions in the space carried meaning. Taking the vector for king, subtracting man and adding woman landed near queen. The map below is a tiny hand made version with only two dimensions, so students can see the idea. Click any word to find its nearest neighbours, or try an analogy.

Four neighbourhoods appear on the map, people, animals, food and computing, and within the people group the same small step separates man from woman, king from queen and prince from princess. That consistent step is why the analogy arithmetic works. Real embeddings use hundreds or thousands of dimensions rather than two, which lets them capture many kinds of similarity at once, such as meaning, grammar and tone, but the principle is identical. Closeness in space stands for similarity in use.

Attention, how words look at each other

A word’s meaning depends on the words around it. Bank means something different in river bank and bank account. Consider the sentence the animal did not cross the street because it was too tired. Every reader knows that it means the animal. Change tired to wide and it suddenly means the street. To predict text well, a model must work out these links, and the mechanism that does it is called attention.

In 2017 eight researchers at Google published a paper titled Attention Is All You Need, introducing the transformer, the design behind almost every modern language model, including the T in GPT. In a transformer every token compares itself with every other token and computes a weight for each, how much that other token matters for understanding this one. The token’s new meaning is a blend of the others, weighted by attention. Click it in the lab, then switch the last word between tired and wide.

The weights in this lab are set by hand to show the idea, so they are not read from a real model, but researchers who inspect trained transformers do find attention patterns like this one. Real models run many attention calculations side by side, called heads, each free to track a different kind of link, and stack dozens of these layers on top of each other. The underlying calculation is compact enough to write in one line.

In words, each token asks a question, written Q for query, every token offers a label, K for key, and the match between question and label decides how much of each token’s content, V for value, is blended in. Students who completed the graphs lesson can picture attention as a graph where every word is connected to every other word, with the edge weights recomputed for every new sentence.

A chatbot does not look things up. It continues the text in the way that seems most likely, and that difference explains both its power and its mistakes.From this lesson, period three

From a word predictor to a chatbot

A model trained only to predict the next token on internet text is not yet a helpful assistant. Ask it a question and it might continue with three more questions, because that is how many web pages look. Building a chatbot takes three stages. Pretraining is the long first stage, where the model predicts the next token across a huge collection of text and slowly adjusts its billions of parameters by gradient descent, the same downhill walk from the AI lesson. This is where it learns grammar, facts, styles and some reasoning patterns.

The second stage, fine tuning, trains the model further on carefully written examples of good conversations, so it learns the shape of a helpful answer. The third stage uses human feedback. People compare pairs of answers and choose the better one, and the model is trained to prefer the kinds of answers people chose. OpenAI described this approach in a 2022 paper on training models to follow instructions, and the same year it released ChatGPT to the public, which reached a huge audience within weeks. Newer models also add steps such as writing out their reasoning before answering, but the next token engine underneath is the same.

The timeline holds a useful lesson about progress. ELIZA, written by Joseph Weizenbaum at MIT in 1966, imitated a therapist using simple pattern rules, and many users still felt it understood them. Weizenbaum was disturbed by how readily people trusted it. Six decades later the same human tendency matters even more, because modern models are vastly more fluent, and fluency is easy to mistake for knowledge.

Why chatbots make things up

Because a language model produces the most likely continuation of text, it has no built in step that checks whether a statement is true. If a question resembles thousands of questions with confident answers, the model will produce a confident answer, whether or not it actually has the information. These invented answers are called hallucinations. A model may cite a book that does not exist, attribute a quote to the wrong person or give a wrong date in a perfectly calm tone.

Three other limits follow from how the models are built. Their knowledge stops at a training cutoff unless they are connected to search tools. They can repeat biases present in their training text. And as the tokenizer lab showed, tasks that depend on individual letters or long chains of exact arithmetic can trip them up. Modern systems reduce these problems with search, calculators and careful training, but none of them removes the need for a person to check what matters.

Responsible use, a conversation worth having in class

The most useful outcome of this lesson may be a class agreement on how to use chatbots for learning. Good uses include asking for an explanation in simpler words, generating practice questions and getting feedback on a draft. Poor uses include copying answers without understanding them and trusting a fact without checking it. A simple rule works well for students. Use the chatbot as a study partner, not as an authority, and check any fact you would be embarrassed to get wrong.

Misconceptions students bring to class

• The chatbot searches the internet for every answer. A plain language model generates text from patterns learned in training. Some tools add web search on top, but the core model does not look anything up.
• The model reads letters and words like we do. It reads tokens, which are often word pieces, as Lab 1 shows.
• It gives the same answer every time. Sampling means the same question can produce different answers, as Lab 2 shows.
• It stores the whole internet inside it. It stores patterns in its parameters, not a copy of its training text, which is also why it can misremember details.
• If it sounds confident, it must be right. Confidence in the wording says nothing about accuracy. Fluent but false answers are exactly what hallucinations look like.

Where this lesson simplifies the truth

The tokenizer in Lab 1 uses a short list of hand written prefixes and suffixes, while real tokenizers learn tens of thousands of pieces from data and work on bytes, so their splits will differ from ours. The predictor in Lab 2 looks only at the previous word, while real models look at thousands of previous tokens at once through attention. The meaning map has two dimensions and positions chosen by hand, and the attention weights in Lab 4 are illustrative rather than measured from a trained model.

The lesson also skips the internal layers that sit between attention steps, the positional information that tells a model the order of words, and the details of how human feedback training works. None of these simplifications changes the central picture, a network that turns tokens into vectors, lets them attend to each other, and predicts the next token, over and over.

Bringing it all together

The main achievement of this lesson is that students can now describe a chatbot honestly. Text becomes tokens, tokens become vectors, attention mixes in context, and the model outputs a probability for every possible next token. One token is chosen, and the cycle repeats until the answer is complete. Every impressive thing a chatbot does comes from this loop run at enormous scale.

The conceptual shift is from seeing the chatbot as a mind that knows things to seeing it as a very powerful pattern continuer. That shift does not make the technology less impressive. If anything it makes it more astonishing that next token prediction, trained on enough text, can produce explanations, translations and working code. It also makes the mistakes understandable rather than mysterious.

The ideas tie the whole series together. Tokens are numbers stored in the memory from How a Computer Works. Training is gradient descent from the AI lesson. Attention is a fully connected graph from the graphs lesson. The next token loop is the kind of repeated function call students met in the functions lesson, and the probability counting in Lab 2 is a dictionary of counts, the hash table from the data structures lesson.

The lesson has honest limits. It does not show how a full transformer layer is computed, how models are evaluated, or the energy and cost of training them, each of which deserves its own discussion. Teachers with stronger classes can extend period three by having students change the training corpus in the Python program and observe how the predictions change, and teachers without computers can teach the whole unit with the story chain game, paper tokens and the pronoun puzzle.

A natural next lesson is data and privacy, what information these systems are trained on and what happens to the questions we type into them. For now, the student who was puzzled by the wrong letter count knows exactly why it happened. The model never saw the letters, only the tokens, and it gave the answer that looked most likely.

Companion Python code for the classroom

The program below builds every idea from the lesson in plain Python. It includes a toy tokenizer, a next word model made from word pair counts with softmax and temperature, greedy and sampled generation, word analogies on a small hand made map, and the attention calculation with dot products and softmax.

# How Large Language Models Work, companion code for the aitrendblend lesson
# Runs on any Python 3 installation. No extra libraries needed.
import math
import random
import re
from collections import Counter, defaultdict

# Part 1. A toy tokenizer. Real models learn their pieces from data (BPE).
SUFFIXES = ["ingly", "ably", "tion", "ness", "able", "ing", "ed", "ly", "er", "s"]
PREFIXES = ["un", "re", "dis", "pre"]
COMMON = {"students", "student", "computer", "computers", "language", "languages", "model",
          "models", "school", "teacher", "curious", "because", "people", "question", "answer"}


def split_word(word):
    pieces = []
    for p in PREFIXES:
        if word.lower().startswith(p) and len(word) > len(p) + 4:
            pieces.append(word[:len(p)])
            word = word[len(p):]
            break
    tail = []
    for s in SUFFIXES:
        if word.endswith(s) and len(word) > len(s) + 3:
            tail.append(s)
            word = word[:-len(s)]
            break
    while len(word) > 7:                 # very long stems are chopped into chunks
        pieces.append(word[:4])
        word = word[4:]
    return pieces + [word] + tail


def tokenize(text):
    tokens = []
    for chunk in re.findall(r"[A-Za-z]+|\d|[^\sA-Za-z\d]", text):
        tokens += split_word(chunk) if chunk.isalpha() and len(chunk) > 5 and chunk.lower() not in COMMON else [chunk]
    return tokens


# Part 2. A next word model built by counting word pairs (a bigram model)
CORPUS = ("the cat sat on the mat . the cat ate the fish . the dog sat on the rug . "
          "the dog chased the cat . the cat chased the mouse . a bird sat on the wall . "
          "the bird ate the seed . the student read the book . the student wrote the code . "
          "the teacher read the code . the code ran on the computer . the computer ran the code .")

words = CORPUS.split()
following = defaultdict(Counter)
for a, b in zip(words, words[1:]):
    following[a][b] += 1


def next_word_probs(word, temperature=1.0):
    counts = following[word]
    scores = {w: math.log(c) / temperature for w, c in counts.items()}
    top = max(scores.values())
    exp = {w: math.exp(s - top) for w, s in scores.items()}      # softmax
    total = sum(exp.values())
    return {w: e / total for w, e in sorted(exp.items(), key=lambda kv: -kv[1])}


def generate(start, n, temperature=1.0, greedy=False, seed=1):
    rng = random.Random(seed)
    out = [start]
    for _ in range(n):
        probs = next_word_probs(out[-1], temperature)
        if greedy:
            out.append(max(probs, key=probs.get))
        else:
            out.append(rng.choices(list(probs), weights=list(probs.values()))[0])
    return " ".join(out)


# Part 3. Word meanings as points (tiny hand made 2D vectors)
VEC = {"man": (3, 3), "woman": (3, 1.6), "king": (5, 3.4), "queen": (5, 2.0),
       "prince": (4.6, 3.8), "princess": (4.6, 2.4), "cat": (2, 8), "dog": (2.6, 8.4),
       "kitten": (1.6, 7.4), "puppy": (2.3, 7.6), "apple": (8, 8.2), "mango": (8.6, 7.8)}


def analogy(a, b, c):
    """a minus b plus c, then find the closest other word."""
    x = VEC[a][0] - VEC[b][0] + VEC[c][0]
    y = VEC[a][1] - VEC[b][1] + VEC[c][1]
    options = [w for w in VEC if w not in (a, b, c)]
    return min(options, key=lambda w: math.dist(VEC[w], (x, y)))


def distance(a, b):
    return math.dist(VEC[a], VEC[b])


# Part 4. Attention, scores from dot products turned into weights by softmax
def softmax(xs):
    m = max(xs)
    e = [math.exp(x - m) for x in xs]
    return [v / sum(e) for v in e]


def attention_weights(query, keys):
    d = len(query)
    scores = [sum(q * k for q, k in zip(query, key)) / math.sqrt(d) for key in keys]
    return softmax(scores)


# Smoke test
if __name__ == "__main__":
    toks = tokenize("Students are unbelievably curious about chatbots!")
    print("Tokens:", toks, "|", len(toks), "tokens")

    print("\nAfter 'the' the model has seen:", dict(following["the"].most_common(4)), "...")
    probs = next_word_probs("the")
    print("P(next | the):", {w: round(p, 3) for w, p in list(probs.items())[:3]})
    cold = next_word_probs("the", temperature=0.3)
    print("Same with temperature 0.3:", {w: round(p, 3) for w, p in list(cold.items())[:3]})
    print("Greedy:", generate("the", 8, greedy=True))
    print("Sampled:", generate("the", 8, seed=7))

    print("\nking - man + woman =", analogy("king", "man", "woman"))
    print("puppy - dog + cat =", analogy("puppy", "dog", "cat"))
    print("distance(cat, dog) =", round(distance("cat", "dog"), 2),
          "| distance(cat, apple) =", round(distance("cat", "apple"), 2))

    w = attention_weights([1, 0], [[2, 0], [0, 2], [1, 1]])
    print("\nAttention weights:", [round(v, 3) for v in w])

    assert toks == ["Students", "are", "un", "believ", "ably", "curious", "about", "chatbot", "s", "!"]
    assert analogy("king", "man", "woman") == "queen" and analogy("puppy", "dog", "cat") == "kitten"
    assert distance("cat", "dog") < distance("cat", "apple")
    assert abs(sum(probs.values()) - 1) < 1e-9 and max(cold.values()) > max(probs.values())
    assert abs(sum(w) - 1) < 1e-9 and w[0] > w[2] > w[1]
    print("\nAll checks passed.")

When you run it, the sample sentence should split into 10 tokens with unbelievably cut into three pieces, the model should report that cat and code each followed the word the four times, lowering the temperature to 0.3 should raise the top probability from about 17 percent to about 42 percent, greedy generation should loop as the cat sat on the cat sat on the, king minus man plus woman should give queen, and the final line should report that all checks passed. A good extension task is to add five sentences of your own to the corpus and see which predictions change.

Check your understanding

Six questions cover the whole lesson. Choose an answer for each and press Check answers to see which were right, with a short explanation for each.

Frequently asked questions

Related reading on aitrendblend