Training: Data, Compute, and Algorithms

Three Ingredients, One Recipe

Building a large language model requires three things at sufficient scale:

1. Data — enormous quantities of text to learn from.
2. Compute — massive processing power to crunch through that data.
3. Algorithms — clever architectures that can learn from the data efficiently.

Think of it like baking. Data is the flour — the raw material. Compute is the oven — the energy that transforms it. The algorithm is the recipe — the instructions that determine whether you get bread or a mess. You need all three.

Data: The Raw Material

Pre-Training on Internet Text

The first phase of training is called pre-training. The model reads astronomical amounts of text — web pages, books, academic papers, code, forums, and more. Modern training datasets contain trillions of words, more than any human could read in thousands of lifetimes.

Data Quality Matters

Not all data is equal. Unfiltered internet text includes spam, misinformation, and hate speech alongside Shakespeare. AI companies invest heavily in data curation — filtering, cleaning, and balancing training data.

This is where bias concerns arise. If the training data over-represents certain perspectives, the model absorbs those biases. If most text is in English, the model will be weaker in other languages. The data shapes the model's worldview in ways that are pervasive and hard to correct.

A language model can only be as good as the data it learns from. Curating training data isn't a technical afterthought — it's one of the most important decisions in the entire process.

Compute: The Engine

Why GPUs Changed Everything

GPUs (Graphics Processing Units), originally designed for video games, turned out to be perfect for AI training. Both tasks involve millions of simple mathematical operations done simultaneously.

A CPU is like a brilliant professor solving one complex problem at a time. A GPU is like a gymnasium full of 10,000 students who can each solve a simple math problem simultaneously. For AI training, the gymnasium wins.

The Scale of Modern Training

• Training GPT-3 on a single modern GPU would take over 350 years. In practice, thousands of GPUs work in parallel for weeks or months.
• A single training run for a frontier model costs $100 million or more.
• The most advanced AI chips cost $25,000–$40,000 each, and a training cluster uses tens of thousands.
• Training a large model can consume as much electricity as a small town uses in a year.

The computational cost is so enormous that only a handful of organizations in the world can afford to train frontier models. This concentration of capability raises important questions about who shapes this technology.

Algorithms: The Recipe

The Transformer Architecture

Data and compute were available before the current AI revolution. The missing piece was the right algorithm. That breakthrough came in 2017 with a paper titled "Attention Is All You Need," which introduced the transformer architecture.

Before transformers, language models processed text sequentially — one word at a time, left to right. This was slow and made it hard to capture relationships between distant words.

The transformer's key innovation is the attention mechanism, which lets the model look at all words simultaneously and determine which ones are most relevant to each other. Instead of reading left to right, it looks at the entire page at once and draws connections.

The transformer made it possible to:

• Train much faster by processing words in parallel.
• Capture long-range relationships that earlier architectures missed.
• Scale effectively — bigger transformers consistently perform better (a property called scaling laws).

Nearly every major language model today — GPT-4, Claude, Gemini, Llama — is built on the transformer.

How Training Works (Simplified)

1. Initialize randomly. The model starts knowing nothing.
2. Make a prediction. Given text with the last word hidden, predict it.
3. Check the answer. Compare the prediction to the actual word. Measure the error.
4. Adjust parameters. Nudge billions of parameters slightly toward a better prediction.
5. Repeat billions of times on trillions of words.

It's like learning to throw darts. Throw, see where you hit, adjust, throw again. After millions of throws, you're hitting the bullseye — not because anyone taught you physics, but because you calibrated through practice.

RLHF: Teaching the Model to Be Helpful

Pre-training produces a great text predictor, but not necessarily a helpful assistant. A raw model will complete any text pattern — including harmful ones — because it's learned from the full internet.

Reinforcement Learning from Human Feedback (RLHF) bridges this gap:

1. The model generates multiple responses to the same prompt.
2. Human evaluators rank which are most helpful, honest, and harmless.
3. A reward model learns to predict human preferences.
4. The language model is fine-tuned to produce highly-rated responses.

RLHF transforms a text predictor into an assistant. It's the difference between a model that answers "How do I pick a lock?" with detailed instructions and one that thoughtfully declines while suggesting alternatives.

Pre-training gives a model knowledge and fluency. RLHF gives it values and judgment — or at least a reasonable approximation.

Key Takeaways

• Training requires three ingredients at scale: data (trillions of words), compute (thousands of GPUs for weeks), and algorithms (the transformer architecture).
• GPUs provide the parallel processing power needed, at costs reaching hundreds of millions of dollars.
• The transformer architecture (2017) was the algorithmic breakthrough enabling modern LLMs.
• RLHF fine-tunes a pre-trained model to be helpful and safe.
• Scaling laws suggest bigger models keep getting better — driving the current AI investment boom.