Can the group use this pilot comparison?
Check the denominator before a polished claim reaches the final slide.
The question
You are one participant in a scripted Harbour Projects exercise comparing two support pilots. The preliminary report gives A 8 unique complainants from 80 customers, and B 12 unique complainants from a reported base of 200. The periods are the same; the group has not yet checked the source behind B’s base. All data and dialogue are invented.
| A | B | |
|---|---|---|
| Unique complainants | 8 | 12 |
| Reported base | 80 | 200 |
A’s base is unique customers. B’s reported base still needs its unit checked.
A is better because it has fewer complaints. Use A on the recommendation slide.
- BethThe summary is clear and we have a slide ready.
- ChenI would like to know what the 200 means.
- AmirDo we really need to reopen this?
“Can the group use this pilot comparison?”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Which part of the claim needs checking?
What would you do here? Open the move you’d make.
Ask what B’s 200 counts and compare like-for-like rates.
The strong move. This checks the evidence that determines the comparison.
Accept A because eight is less than twelve.
Tempting, but it misses something. That ignores the different reported bases.
Decide whether the paragraph sounds AI-written.
Tempting, but it misses something. Writing style cannot establish which pilot has a lower comparable rate.
See how a strong candidate says it
“Ask whether both denominators mean unique customers before comparing the complaint rates.”
Why it works: The group needs the unit and source, not a judgment about who wrote the summary.
Step 2: What do the reported bases imply?
What would you do here? Open the move you’d make.
A 10%, B 6%, provisionally on the reported bases.
The strong move. The arithmetic is right and its unit assumption stays visible.
A 8%, B 12%, because the complainant counts are percentages.
Tempting, but it misses something. Counts become rates only when divided by the relevant base.
B caused 40% fewer complaints.
Tempting, but it misses something. This was not a controlled causal comparison, and B has more complainants in absolute count.
See how a strong candidate says it
“A is 8/80=10%; B is provisionally 12/200=6%, four percentage points lower. That comparison depends on the bases having the same unit.”
Why it works: A lower count is not necessarily a lower rate. The preliminary calculation does not remove the source question.
Step 3: What should the group say for now?
What would you do here? Open the move you’d make.
State the provisional rates and request the base definition before ranking.
The strong move. This preserves the useful calculation while resolving the missing evidence.
Recommend B for every service because 6% is lower.
Tempting, but it misses something. The base is unverified and the exercise does not establish wider causal performance.
Hide both counts to prevent disagreement.
Tempting, but it misses something. That removes the evidence the group needs to make a sound comparison.
See how a strong candidate says it
“Hold the winner claim until B’s base is checked. Describe the preliminary rates and their limitation if they must be discussed.”
Why it works: A useful contribution can stop an unsupported conclusion without dismissing the work already done.
Step 4: What must change on the slide?
The data owner now confirms B’s 200 was visits from 100 unique customers. Its 12 complainants were unique customers. A’s 80-customer base and eight complainants are unchanged.
What would you do here? Open the move you’d make.
Keep the 6% figure because it is already on the slide.
Tempting, but it misses something. Its denominator is visits, not the comparable customer count.
Show A 10% and B 12%, and explain the corrected unit.
The strong move. This makes the material source correction visible.
Change the number without telling the group why.
Tempting, but it misses something. That leaves the group with an unexplained reversal and an unreliable rationale.
See how a strong candidate says it
“B’s comparable rate is 12/100=12%, versus A’s 10%. Withdraw the old comparison and show the corrected bases and rates.”
Why it works: The direction changes when B is compared on unique customers. This is descriptive evidence, not proof of a causal or statistically reliable difference.
The answer, in short
- Decision
- Correct the rates and withdraw the original winner claim.
- Because
- The verified unit changes B from a provisional 6% to 12%, above A’s 10%.
- What would change it
- A representative comparison with agreed definitions and suitable evidence of effects.
Common traps
- Ranking complaint counts without denominators.
- Treating a fluent summary as verified evidence.
Check your answer
- What does each base count?
- Did I state the unit assumption?
- What does the comparison actually show?
- Did we explain the correction?
Last reviewed Nov 10, 2026