How LLMs Work: A No-Math Guide to Transformers
The technology behind ChatGPT, Claude, and Gemini explained without equations. Just intuition and visuals.
AINow Take
The Transformer is the most impactful AI invention since neural networks themselves. It solved two problems that blocked earlier AI: long-range context and parallel training. Every major AI system uses it.
Large Language Models (LLMs) like ChatGPT, Claude, and Gemini are all built on the same architecture: the Transformer. Here’s how it works without the math.
The Problem Before Transformers
Before 2017, AI models struggled with:
- Long sentences — they’d forget the beginning by the time they reached the end
- Slow training — they processed words one at a time, couldn’t parallelise
- Context — they couldn’t understand how words relate to each other in complex sentences
The Transformer solved all three problems at once.
What a Transformer Does
A Transformer converts your input (text) into numbers (embeddings), processes those numbers through layers of attention, and generates a probability distribution for what comes next.
Key Components
1. Tokenisation
Before anything happens, the text is broken into tokens — chunks of text (words, parts of words, or characters).
- “ChatGPT is amazing” → [“Chat”, “GPT”, “ is”, “ amazing”]
- Each token gets a unique ID number
2. Embeddings
Each token ID is converted into a vector — a list of hundreds of numbers. These numbers represent the “meaning” of the token in a high-dimensional space.
Think of it as: Giving each word coordinates on a map. Similar words are close together. “King” - “Man” + “Woman” ≈ “Queen”.
3. Attention (The Magic)
The attention mechanism lets each token “look at” every other token in the input and decide which ones are important.
Example: In “The cat sat on the mat because it was tired,” the model needs to figure out that “it” refers to “cat,” not “mat.”
Attention allows the model to connect “it” to “cat” even though there are several words in between.
4. Feed-Forward Layers
After attention, each token goes through a series of mathematical transformations (multiplying and adding numbers). These layers learn complex patterns in the data.
5. Stacking Layers
A Transformer has many layers (GPT-4 reportedly has 120+). Each layer builds on the previous one. Early layers learn simple patterns (grammar, word pairs). Later layers learn complex patterns (reasoning, facts, style).
6. Next-Token Prediction
At the final layer, the model outputs a probability for every possible next token. It picks one (usually the most likely, with some randomness added for creativity).
Then it repeats: the new token is added to the input, and the model predicts the next one.
How Training Works
Training an LLM involves:
- Feed it a sentence with the last word hidden
- Let it predict the hidden word
- Compare prediction to the actual word
- Adjust parameters slightly to make the prediction better next time
- Repeat billions of times
Why Scale Matters
| Model | Parameters | Training Data | Cost |
|---|---|---|---|
| GPT-1 (2018) | 117M | ~5GB text | ~$10K |
| GPT-3 (2020) | 175B | ~45TB text | ~$10M |
| GPT-5 (2025) | ~1T+ | ~100TB+ | ~$100M+ |
The “scaling hypothesis” says: as models get bigger, they get better — not just at memorisation, but at reasoning, creativity, and understanding.
Limitations
Transformers are powerful but have weaknesses:
- Context window — they can only “see” a limited amount of text at once
- No persistent memory — each conversation starts fresh
- Computationally expensive — running a large model costs significant energy
- Hallucinations — they confidently generate false information
- No true understanding — they predict patterns, not meaning