AI Foundations

How LLMs write

Predicting the next word, one step at a time

When an LLM answers you, it is not looking up the answer in a database. It is writing it, one small piece at a time. Knowing this explains a lot of its strengths and its mistakes.

Predicting the next word

You already know this trick. If I say "Peanut butter and...", your brain fills in "jelly". You guessed the next word from patterns you have seen many times.

An LLM does the same thing, over and over:

It looks at all the text so far.
It guesses the most likely next word.
It adds that word and guesses again.
It stops when the answer feels complete.

Your phone keyboard suggests the next word in the same way. An LLM is like a much bigger and smarter version of that.

Your prompt

"The sky is"

→

Predict

most likely next word

→

Add it

"The sky is blue"

→

Repeat

until the answer is done

An LLM writes by predicting one next piece at a time, then repeating.

LLMs actually predict small pieces of words called tokens, not whole words. You'll learn about tokens soon. For now, "next word" is close enough.

Training vs using

An LLM's life has two stages. Think of a student who studies for years, then takes a job.

Training (studying)

The company feeds the model a huge amount of text. The model practices predicting the next word billions of times and slowly gets good at it. This takes months and costs a lot. It happens once, before you ever use it.

Using (working)

When you send a prompt, the trained model uses what it learned to write a reply. It does not learn from your chat in the moment. It only uses what is in front of it.

Training stops at some date. That is why a model may not know about recent events unless it can search the web.

Why answers vary

Ask the same question twice and you may get two different answers. That's normal.

At each step, the model has several good choices for the next word. It doesn't always pick the top one. A little randomness makes the writing sound more natural and less robotic.

Try it. Paste this prompt, then click the regenerate button (or send it again in a new chat):

Prompt

Give me one creative name for a cafe that also sells plants.
Reply with just the name.

You will likely get a different name each time.

Temperature is a setting that controls this randomness: low temperature gives safer, more predictable answers, high temperature gives more creative ones.

If you get a weak answer, don't give up. Regenerate it, or rephrase your prompt. A second try is often much better.

Common mistakes

Thinking the AI looks facts up. By default it writes what sounds likely. Likely is not the same as true.
Expecting the exact same answer every time. Small differences are normal. Ask for a fixed format if you need consistency.
Assuming it learns from you instantly. The model was trained before your chat. It only "remembers" what is in the current conversation.

Practice

Try these exercises to lock in what you just learned.

Quiz

Not started

How does an LLM write its answer?

Quiz

Not started

You send the same prompt twice and get two different answers. Why?

What's next?

You now understand the basics of AI and LLMs. Next, let's learn the five key ideas you need to work with LLMs well, starting with an overview.