Print the whole workshop

Part of a series written for one product team. The companion series it refers to, From Concern to Evidence, is not online yet; where this page points at that packet, the team has it in hand.

Three 75-minute sessions for the moment when the literacy reviewer, the engineers and the manager mean different things by "a good quiz question". The parent design's sequence was: surface assumptions, build literacy-quality criteria, examine language and representation, negotiate disagreements, record an accountable decision. This version compresses the first two and the last two, because the team already has written rules and written bars, and the live work is agreeing what they mean on a real item. It overlaps Sessions 1 and 4 of From Concern to Evidence (From Concern to Evidence) and reuses their worksheets. The literacy reviewer leads more of this series than any other.

What the team should be able to do afterward

The three sessions

Same shape as From Concern to Evidence: write first, short explanation, supported practice, individual exit task, the five anonymous questions (the From Concern to Evidence packet, closing section).

Session 1: Say what good looks like, then compare

Cases (packet Part 1): Mason's good and bad versions of a lesson question about Brad and Bruno; the five answer-choice rules; a current QA item whose key is a detail rather than an answer.

Time Activity
0 to 8 Purpose, norms. Each person writes their own one-sentence definition of a good quiz item and hands it in.
8 to 18 Alone: the bad and good Brad and Bruno items, side by side. Which is better, and what exactly is different?
18 to 30 The literacy reviewer reads the definitions without names and groups them. Where do the engineers' and the reviewer's definitions part?
30 to 42 Mason presents the five rules, two minutes, then the group maps each rule to the thing it fixes in the bad item.
42 to 47 Break.
47 to 62 Pairs apply the rules to the Sprocket wrench item and one other current QA item. Name the rule broken and the words that break it.
62 to 75 Individual: rewrite one wrong answer so it carries as much detail as the key. Five questions.

Session 2: Language, reading load and who the question is for

Cases (packet Part 2): a Lexile 315 story with answer choices of 11 to 13 words; the K-2 hold-out items where young-grade why questions had two sensible reasons; the literacy reviewer's 15-word ceiling and the judge's different reading of the same rubric line.

Time Activity
0 to 8 Report back. Retrieve the five rules.
8 to 18 Alone: read the Lexile 315 item aloud at a child's pace and time it. Would a kindergartener finish the choices? Write yes, no or depends, and on what.
18 to 30 The literacy reviewer leads: what reading load means at K-2, 3-5 and 6-8, and why full sentences at grade 4 and up and shorter choices at K-3 are both rules.
30 to 45 Pairs: the two young-grade why items where a teacher accepted a second answer. Is the fault in the question, the key, or the story?
45 to 50 Break.
50 to 65 Groups write a reading-load criterion with an observable anchor per grade band, and test it on three items.
65 to 75 Individual: one place where "right at this grade" and "right" come apart. Five questions.

Session 3: When the rater and the judge disagree, who decides?

Cases (packet Part 3): the Zara's Quiet Garden comparison, 51 of 72 cells agreed; the five disagreements with their reasons; the open question of which one is the reference.

Time Activity
0 to 8 Report back. Retrieve the Zara disagreements.
8 to 18 Alone: for each of the five disagreements, write who you think is right and why.
18 to 30 Reveal the split. Sort the five into: definition gap (fix the rubric wording), judgment call (the literacy reviewer decides), and open (needs more cases).
30 to 45 Groups draft the decision record for one disagreement: the ruling, the owner, the rubric line it changes, the date it is reviewed.
45 to 50 Break.
50 to 65 Groups trade records and ask the two questions that would stop them acting on it.
65 to 75 Individual: the one line in the rubric you would change first. Five questions.

What the series keeps from the research

Three points, where they change what happens in a session:

Reuse from From Concern to Evidence

Appendix

The appendix is a short optional note on reading a bar as a rate with a sample. It adds three fields to the decision record: the accepted rate and its sample, the per-child reading of that rate, and the slice the bar was not checked on.

Citation audit

All three sources were verified for From Concern to Evidence on 2026-11-07 (Crossref match or publisher record, read in the copy on disk). Peters and Armstrong (1998), doi:10.1002/ace.7908, p. 79: verified. Mezirow (1997), doi:10.1002/ace.7401, p. 10: verified. Brookfield (2006) workshop packet, author's site, p. 22: verified, practitioner account. Kirschner, Sweller and Clark (2006), doi:10.1207/s15326985ep4102_1, pp. 75 and 80: verified. No other sources are cited in this folder.