Print the whole workshop

Part of a series written for one product team. The companion series it refers to, From Concern to Evidence, is not online yet; where this page points at that packet, the team has it in hand.

Reuse from From Concern to Evidence (From Concern to Evidence)

If the team has done From Concern to Evidence Session 4, Session 3 here can be cut to the decision record, about 40 minutes.

Number sources

Keys

Keys: open after the judging is done
Item Key
Brad and Bruno, first version D
Brad and Bruno, second version the file does not mark one; Mason's point is that only the story decides
Sprocket wrench B (their fingers brushed)
Pip C (the once-golden forest had become a maze)
Nadia B (explore the forbidden throne room)

Intended reasoning

Session 1

Brad and Bruno: in the first version the key is the only warm, lesson-shaped choice, so a reader picks it without the story. In the second, all four are lesson-shaped and about the same length, so only the story can decide. Rules 2 and 3 are the ones it illustrates. Someone will say the second version is harder. It is harder to guess. Whether it is harder for a child who read the story is unmeasured, since similar choices can also raise the reading demand, and there is no child data on this item.

Sprocket wrench item: the key, "Their fingers brushed as the exchange happened between them", is a detail from the sentence, not an answer to "what did Imani do". Wrong answer C is a reversal of a real event, which is good practice; wrong answer A is a filler. Rule 2 is kept on length; the fault is that the question and key do not match, which is a validity fault the five rules do not cover. Say it outright: the rules are about the choices, and a question can keep all five and still be a bad question.

Pip item: all four choices begin "Her parents had vanished, and", which is rule 2 done well; each wrong answer names a real thing from the story at the wrong time. The key is right. This is the item to hold up as what the rules produce when they work.

Session 2

Nadia: the story's Lexile is 315, low in the app's range, and the four choices are 11 to 13 words each. An adult reading aloud at a guessed pace is a rough check, not a measure of what children can do, so say that when you time it. Rule 1 says K-3 should prefer full sentences; the literacy reviewer's ceiling is 15 words. Both are met. The question is whether "under 15 words" is the right anchor at K, or whether the anchor at K is something else (sentence shape, vocabulary, one clause). Do not settle it; write the candidates down for the literacy reviewer's criterion.

Q11: this is a definition gap, and it is in the rubric wording. The fix is to state which reading the line means.

Rivet items: the tightened-results file lists them as young-grade motivation items "with two sensible reasons", and does not say whether the story states one reason. The answer is "cannot tell without the story", and the next action is to open it.

Session 3

The sort to aim for, and the packet's three piles do not hold all of it: Q5 and Q11 are definition gaps (rubric wording, Mason owns the change, the literacy reviewer approves the wording). Q1 is a judgment call that goes to the literacy reviewer, and the record should say that a teacher's reading of "collection" stands over the judge's word match. Q2 is a judge fault, not a disagreement about the quiz: a judge that passes an item because it labelled its own answer a guess is grading itself; the record is "the code counts a blind pick as a pick, labelled or not". Q12 is a definition gap about which hint is read.

End on how these were found. A person scored the same story the judge scored without seeing the judge's answers. The review sheet's collapsed columns exist for that, which is why the rule says not to open them until you are done.

Giveaway tally (Session 1)

Tally the giveaways for the first Brad and Bruno version before showing the rules. Expect "Only kind, brave or moral-sounding choice" and "Sounds like a lesson", with some "Longest choice". That tally makes the argument for rules 2 and 3 before Mason states them. On the second version, expect "Nothing" and a scatter. Tally the Sprocket and Pip items before the rules are applied too. "Nothing" after a wrong guess is a recording convention from the sheet. A wrong guess does not prove the item has no cue, and a right one does not prove it has one. Pip should draw mostly "Nothing", and it is held up for that reason. The list comes from the READ ME tab of the 2026-11-09 review sheet.

Running the live activity

Send the link to the session set: https://masonborchard.com/workshops/whose-good-enough/activity.html?set=session. Everyone works alone and nobody talks until the tally is up. Each person copies their results at the end and sends them to Mason, who pastes them into the facilitator box at the bottom of the results screen. Nothing leaves anyone's browser until they send it, and the page collects no names. The box gives a per-question table to put on the screen.

cd ~/code/masonborchard.com && python3 Scripts/build_quiz_activity.py

Appendix numbers

The item counts (156 on the 39 frozen stories, 96 on the 24 new ones) assume four items per quiz, which is the gate in the One Change One Claim pipeline table; say so if the group asks, and confirm from the harness output whether the second-answer figures (3.8 and 5.3) are counted over items before using the per-item point values; 5.3 percent is not a whole number of items out of 96, so the measure may be weighted or defined differently. The K-2 figure (4 of 36) is the one already in this packet. The 3-over-n rule is arithmetic, and the fuller working is in From Concern to Evidence's appendix.