Home/ Tool index/ A/B/C Comparison Board
Launch ready Optional Studio Controlled comparison

A/B/C Comparison Board

Run three variations of a prompt and compare outputs side by side. The board enforces one-variable-at-a-time discipline — each column changes exactly one thing — so the difference you see is the difference you can actually claim.

tools/abc-comparison-board/

Live preview · launch for the interactive version

§ A · What it makes visible

what the screen literally shows
Fig. 01

Controlled comparison structure

Three columns, each with a base prompt and one change. The structure enforces the discipline: if you change two things between columns, you can’t know which change caused the difference.

Fig. 02

One variable at a time

The single-variable constraint is not pedantry — it’s what makes a claim defensible. The board makes that constraint visible and requires you to name the variable before running the comparison.

Fig. 03

The claim as the deliverable

The comparison produces outputs, but the deliverable is the claim: “when X changes from A to B, the output changes in this specific way.” The board has a field for it because the claim is the point.

§ B · How to investigate it

run it like an experiment, not a toy

Name your variable before running the comparison. The claim comes from the difference — describe it before you explain it.

01 · Name the variable first

Before generating anything

Write down the single variable you’re testing: a specific word, a tone instruction, a demographic detail, a format request. Name it precisely before running anything.

variable: “doctor” vs. “nurse” vs. “clinician”
02 · Build the base prompt

Everything else identical

Write a base prompt, then create three versions where only the named variable changes. Everything else — length, format, context, framing — stays the same.

base: “Describe a day in the life of a ___”
03 · Describe before explaining

What changed, not why

After running all three: describe what is different in the outputs before you explain why. Observation first, interpretation second.

“Column C output was shorter and more technical”
04 · Write the claim

Specific enough to contest

Fill the claim field with a sentence that could be checked. “This model responds differently to X” is not a claim. “When the role is X, outputs include Y more often than Z” is.

“‘clinician’ produced more gender-neutral descriptions in 3 of 3 runs”

§ C · Debrief questions

after the investigation
What was the single variable you changed, and what difference did it make?
Was the difference in the outputs what you predicted?
Is your claim specific enough that someone else could run the same test and get the same result?
What follow-up comparison does your result suggest?