You open WhatsApp and type: "Happy birthday to..."

Before you even finish, your keyboard suggests one word: "you".

You never taught your phone that. It just knows, because millions of people have typed that sentence before and "you" is the most likely next word.

Hold on to that little moment. It is the seed of everything in this blog.

Now, let's ask ChatGPT something

Say you type: "Why is the sky blue?"

You hit enter and two seconds later a clean, friendly answer appears, word by word. It feels like a tiny person on the other side is thinking, typing and explaining.

There is no tiny person. And here is the surprising part: what's happening is a much, much bigger version of your keyboard's guessing game.

Let's slow things down and follow your question on its journey.

Step 1: Your sentence gets chopped up

Computers don't read words the way we do. So the first thing that happens is that your question gets chopped into small pieces called tokens.

"Why is the sky blue?" becomes something like:

Why | is | the | sky | blue | ?

A token is not always a full word. Long or unusual words get split into smaller chunks and a rough rule of thumb is that 1 token is about 3 to 4 English characters. This is also why you will hear people say "the model has a limit of X tokens". That is just the size of its working memory.

Step 2: Words become numbers

Here is the thing about computers: they only understand numbers. So every token gets converted into a long list of numbers called an embedding.

Think of it like giving every word a GPS coordinate on a giant map of meaning.

• "King" and "Queen" sit close together.

• "Dog" and "Puppy" are neighbours.

• "Dog" and "Tax return" live in different cities.

The model never sees the word "sky". It sees where "sky" lives on that map and what sits around it: clouds, blue, sun, air.

Step 3: The brain does its thing

Now comes the heart of ChatGPT, a design called the Transformer. Don't let the name scare you. The one idea you need is attention.

Imagine reading this sentence:

"The trophy didn't fit in the suitcase because it was too big."

What does "it" mean? The trophy or the suitcase? You instantly know it is the trophy, because you pay attention to the right words in the sentence.

The model does the same. For every token, it asks: "Which other words in this conversation matter most to understanding this one?" It does this over and over, across many layers (dozens, in big models) and each layer understands your question a little better.

That is the famous idea behind the 2017 research paper titled "Attention Is All You Need" and it is why modern AI suddenly got so good.

Step 4: The guessing game, just like your keyboard

After all that processing, the model does exactly what your phone keyboard does. It asks:

"Given everything so far, what is the most likely next token?"

It produces a list of candidates, each with a probability:

It picks one (with a little randomness, more on that below), adds it to the text and then does the whole thing again for the next token. And again. And again.

That's how the answer appears word by word on your screen. It is not a typing animation. The model is generating it live, one token at a time.

About that "little randomness"

There's a setting called temperature. A low temperature means the model plays it safe and always picks the top guess. A higher one means it sometimes picks a less obvious word, which feels more creative. That is why asking the same question twice can give slightly different answers.

But wait, how did it get so smart?

A fair question. Your keyboard only knows "Happy birthday to you". ChatGPT can explain physics and write poems. What's the difference?

Training. And a lot of it, in two main stages:

Stage 1: Pre-training (reading the internet). The model is shown a huge amount of text, such as books, articles, websites and code. Its only task is: hide the next word and guess it. It guesses wrong, gets corrected, adjusts its internal numbers a tiny bit and repeats this billions and billions of times. Those internal numbers are called parameters (or weights). Big models have hundreds of billions of them. Think of them as tiny dials, tuned until the guessing gets really good.

Stage 2: Fine-tuning with human feedback (learning manners). A model that only predicts text might ramble or say weird things. So humans rate its answers ("this one is helpful, this one is bad") and the model is nudged toward the helpful ones. This is often called RLHF (Reinforcement Learning from Human Feedback). It is the step that turns a text-guesser into something that feels like an assistant.

So... does it actually understand?

This is where it gets interesting and honestly debated, even among experts. What we know for sure:

• It doesn't look up answers in a database. The knowledge is baked into those billions of numbers.

• It doesn't "know" things the way you do. It predicts what a good answer would look like.

• That's why it can sometimes sound confident and be completely wrong. This is called a hallucination. The model isn't lying. It is producing the most plausible-sounding text and plausible is not always true.

So the practical lesson: use it like a brilliant, fast-reading friend. Great for explaining, drafting and brainstorming. But double-check the important facts.

Let's rewind the whole story You typed "Why is the sky blue?" and here is what happened in those two seconds:

1. Tokenization: your sentence was chopped into pieces.

2. Embeddings: each piece became a point on a map of meaning.

3. Transformer + attention: the model worked out which words matter to which.

4. Next-token prediction: it picked the most likely next piece, then repeated, one token at a time.

5. All of this was possible because of training: reading a huge amount of text, then learning manners from humans.

And it all started with the same idea as your phone suggesting "you" after "Happy birthday to".

The next time ChatGPT surprises you, remember: it's not magic and it's not a tiny person. It is the world's most well-read autocomplete and that is still pretty amazing.

Enjoyed this? I'm just getting started with this blog. Next up: what happens when an AI "remembers" your conversation. Follow along!