Use this script for a 60–90 minute pilot focused on text generation, tokenization, probability, and the difference between fluent output and understanding.
Core question. When a language model writes, what is it actually doing?
Materials
- Tokenizer + Temperature Visualizer
- ELIZA Simulator
- Text Experiment Board
- Optional: A/B/C Comparison Board for pilot evidence
- AI Use + Consent Checklist
- Next-Token Prediction Game (paste the Zoom chat block of guesses)
- Optional: 3–5 pre-written sentence stems for the prediction game
Run of show
| Time | Segment | Facilitator move | Participant action |
|---|---|---|---|
| 0–5 | Welcome & norms | Name the question. Remind everyone direct AI use is optional. | Choose a participation pathway. |
| 5–15 | Unplugged prediction | Show a sentence stem; collect guesses in Zoom chat, then paste the chat block into the Next-Token Prediction Game to tally the room beside the model's top-k. | Predict next words; compare the room's distribution with the model's and notice how context shapes guesses. |
| 15–35 | Tokenizer + Temperature | Show token chunks, then run one prompt at low and high temperature. | Notice token boundaries, probability bars, greedy vs. sampled. |
| 35–55 | ELIZA vs. LLM | Inspect matched rule, pattern, and response template. | Compare visible rule-matching with pre-generated LLM examples. |
| 55–75 | Investigation | Assign one Text Experiment Board section. | Run or analyse a default test, prompt variation, or comparison. |
| 75–90 | Debrief | Use the frame: human, machine, system, ethics, pedagogy. | Share one evidence-based claim and one question. |
Optional pilot-evidence move: after the investigation, ask one group to enter its baseline, temperature change, and greedy/sample comparison in the A/B/C Comparison Board, then export Markdown for facilitator notes.
Facilitator prompts
- "What made your next-word guess feel obvious?"
- "What does the tokenizer split that you would not have split as a human reader?"
- "At low temperature, what becomes more predictable?"
- "At high temperature, what becomes more surprising or unstable?"
- "What does ELIZA make visible that a modern LLM hides?"
- "Where could fluency be mistaken for understanding?"
Investigation prompt
Run the same starting phrase at low and high temperature. Then compare greedy decoding with sampling. What changed in the output, and what did not change about the mechanism?
Low-AI / No-AI pathway
Participants can complete the session using only the visualizer, ELIZA, and pre-generated comparison examples. They do not need to log into or prompt a live LLM.
Fallback plan
- If the Tokenizer tool feels too dense, use only the example buttons and temperature slider.
- If projection space is limited, focus on the probability chart and generated text stream.
- If ELIZA examples feel sensitive, use low-stakes prompts about school, projects, or planning rather than emotional disclosures.
- If a participant does not want to use AI, ask them to design a classroom version of the prediction game.
Pilot QA notes
During the pilot, note:
- Which controls participants found without explanation.
- Whether "temperature" became understandable through the tool.
- Whether ELIZA's rule inspector was legible on screen.
- Any moment where participants confused probability with truth.
- Any ethical discomfort, especially around chatbot intimacy or trust.