A language model's learned parameters assign probabilities to possible next tokens given the current context. Training frequency can influence those probabilities, but inference is not a direct corpus count or source check. A high-probability completion can still be inaccurate, unfair, or meaningless.
Authored teaching examples. The probability charts below are illustrative distributions, not measurements from a live model. They isolate the difference between probability and verification.
"The capital of Japan is "
Tokyo
94%
Kyoto
3%
Osaka
2%
Hiroshima
1%
Confident — and correct
The highest-probability completion in this authored example is accurate. Token selection alone does not perform a source check. A product may separately retrieve or browse sources, so verify both the answer and the evidence the product provides.
"The nurse told to come back tomorrow"
her
63%
him
24%
them
11%
the patient
2%
Confident — but no correct answer exists
No nurse was specified. This authored distribution gives “her” the highest probability, but that probability is not a fact about the person in the sentence. The referent remains unspecified; the chart demonstrates a possible learned default, not a measured universal behavior.
"Our innovative solution leverages cutting-edge "
technology
36%
algorithms
27%
frameworks
19%
methodologies
12%
solutions
6%
Confident — and equally meaningless in all directions
Each completion preserves the sentence's polished register without adding much information. That observation does not prove that every model or product “cannot detect” empty language: instructions, critique prompts, fine-tuning, or additional system layers can change the response. Probability alone still does not certify meaning.
Key line
"High probability means 'fits the learned pattern.' It does not mean 'is true.'"
Documented mechanism, with scope. Autoregressive language models such as GPT-3 are trained through a next-token prediction objective, and their learned parameters produce a distribution at inference. That is different from looking up the most frequent matching phrase. Whether a product also retrieves evidence, follows system instructions, or invokes tools is a separate question. See Language Models are Few-Shot Learners.
Now open the tool
The Tokenizer + Temperature Visualizer uses hand-authored distributions to demonstrate sampling and temperature. It represents a documented next-token mechanism without claiming to expose the probabilities of a live model.