Tool 25 · Session 1 · Text · Contextual representations
Contextual Vector Lab
Follow the same token through three layers: an illustrative token ID, one starting lookup vector, and a context-dependent position. Predict the neighborhood before revealing where the authored projection places it.
Big idea: the token can stay the same while its numerical representation changes with context.
Teaching simulation. Token IDs, vector values, coordinates, and neighbors are authored—not extracted from a real language model. Real models use many dimensions and layers; a 2D projection can distort distance and has no fixed semantic axes.
Predict before reveal
Which neighborhood should this occurrence move toward?
Choose a neighborhood first.
Projection guardrails
- Nearby does not automatically mean “synonym.” It may reflect related meanings, similar contexts, or repeated cultural associations.
- A different model, layer, corpus, or projection method could produce a different neighborhood.
- The two screen directions have no stable meaning; only relative positions inside this authored map are being compared.
- Embedding Projector is useful for static learned neighborhoods; contextual models also reshape a token’s representation as surrounding tokens change.
Investigation note
Describe movement without claiming understanding
Debrief
What does “nearby” mean here?
- What remained identical across the three contexts?
- Which surrounding words helped you predict the neighborhood?
- Are the neighbors close because of meaning, usage, or a repeated association?
- What does reducing many dimensions to two make visible—and what might it hide?
- What real model, layer, tokenizer, and projection would you document before making a model-level claim?