The Statistical Nature of LLMs: an interactive logprobs explorer

I'm putting together a small "Introduction to AI" presentation that I'm giving on Wednesday morning. The target audience is roughly a 50/50 mix of engineers and non-engineers, so I'm trying to not make too many assumptions about my listener's technical depth.

I've only got 15 minutes, which makes it tricky to know what to cover (and maybe that's a lost cause), but I think I'm going to spend most of that time on the statistical nature of LLMs. It's one of the first things I use to try and demystify AI (and de-hype them a little) for people who mostly know them through marketing claims and chat interfaces.

I think a lot of people -- engineer or not -- have a general idea that an LLM predicts the next token. But very few people have actually seen that happen, much less played with it in a sandbox where the alternatives stay visible.

I looked around a bit for a logprob explorer that did what I wanted -- didn't find one, so asked Copilot to build me one. This is what we made:

The Statistical Nature of LLMs: an interactive logprobs explorer

The source is also available on GitHub.

What are logprobs?

"Logprobs" is short for "log probabilities." They're the numbers an LLM API can expose to show how strongly the model scored the possible tokens that could come next.

The logarithmic form is useful for computation, but not especially intuitive to look at, so the explorer shows both the raw log p values and ordinary probability bars. You can see that the model might give one token 12%, another 11%, another 8%, and so on.

Then something still has to make a pick.

That distinction matters: the model offers chances; the sampler makes the pick. The longest probability bar does not always win, and changing the temperature reshapes the odds before sampling.

Seeing the branches

The explorer starts with a small story prompt and shows the possible next tokens. You can select one, generate another token, and keep going.

Just like a good chess engine does, there's a tree view in the side panel. You can generate text down one pathway, back up, choose another token, and continue down that path instead. The abandoned alternatives don't disappear.

This is especially useful for seeing how much one early choice can matter. Given the prompt:

Once upon a time, a small robot discovered a large bear. He felt very...

...the model considers continuations such as sc, excited, happy, and sad.

The explorer can force each of those as the first token, start over from the same point, and then keep going for 35 more tokens with the same seed and temperature. Very quickly, they become four different stories.

That's something ordinary chat interfaces hide. You see the one path that happened to be selected, not the nearby paths that could have been selected instead, or how likely each of them were.

It is still one token at a time

Another thing I wanted to make visible is that asking for 20 or 35 tokens does not make the model produce a paragraph all at once.

The first lesson starts by generating one token. Then one more. Then five more. Then twenty more.

Those larger requests are still the same operation repeated: score the next possibilities, select one token, add it to the context, and do it again. Every token changes the input used to predict the next token.

A paragraph may feel like one generated object when it arrives in a chat window, but underneath, it was assembled one token at a time.

Seven things to try

I've built seven guided lessons into the explorer:

1. A language model predicts what could come next, one piece at a time.
2. Models build text from tokens, not always whole words.
3. One early choice can reshape the whole continuation.
4. The model offers chances; the sampler makes the pick.
5. The model predicts language patterns, not physical randomness.
6. Models can disagree while using the same basic process.
7. Likely is not the same as true.

Each lesson loads an actual scenario into the same explorer rather than just displaying an explanation.

The temperature lesson, for example, generates three 35-token continuations from the same prompt and seed at temperatures 0, 1.5, and 3.0. At temperature 0, the sampler always takes the highest-scoring token. As the temperature rises, lower-ranked choices have more influence, and the odd choices can compound as generation continues.

The physical-randomness lesson contrasts a fair coin and a uniform number picker with the model’s uneven language predictions. It makes the distinction concrete: the model predicts what a likely writer of similar text would write next, based on patterns learned during training. It is not running a physics simulation or a random-number generator.

The final lesson is my favorite (and perhaps the most important): a high probability means "this token fits patterns the model learned." It does not mean "this is true."

The demo asks a small base model to complete a sentence about the capital of Illinois. It strongly prefers Chicago over Springfield. The probability bars are doing exactly what they're supposed to do -- showing what text the model expects -- but they are not a fact checker. Garbage in, garbage out.

One of my favorite Andre Karpathy tweets on this subject:

Everything runs locally

The explorer runs actual GGUF models directly in the browser using wllama. It currently includes TinyStories 15M and SmolLM2 135M.

The first visit downloads the selected model, and later visits reuse the browser cache. Prompts and generated text are not sent to an inference service. Everything in the demo runs locally, and none of your prompt data leaves your computer.

These are deliberately tiny base models. They are fast enough to make the mechanics interactive, but they are not going to produce frontier-model-quality prose. Sometimes the continuations get strange. That's okay -- arguably, that's useful. The point is not to impress people with polished output; the point is to make the machinery visible.

What I'm trying to offer

I don't think understanding next-token prediction explains everything interesting about modern LLMs. It doesn't settle questions about reasoning, representation, tool use, or what larger models learn internally.

But I do think it's an important foundation.

I want people to come away with a mental model that is more concrete than "AI magic," without replacing that with "it's just autocomplete" and pretending nothing interesting is happening.

The model scores possibilities. A sampler chooses. The choice becomes context. Then it happens again.

Once you've actually watched that process branch, compound, get weird, recover, and occasionally sound much smarter than it has any right to, a lot of the larger conversation about LLMs (such as hallucinations, or reliability, or intelligence / reasoning) becomes easier to think about.

That's what I'm hoping this explorer helps people see.