Beer Study Buddy
← Back to syllabus topics

BJCP Beer Judge Entrance Exam · Judging Procedure and Scoresheets

Panel Consensus and Calibration

Syllabus Path

After each judge completes a scoresheet, the panel compares observations and agrees on one assigned score. That consensus should come from discussing the evidence against the style, not from averaging first totals or pressuring a quieter judge. Published procedure asks for scores within seven points, preferably five or fewer. A candidate should know how the head judge runs that discussion and why consensus need not be the arithmetic mean.

  1. 2. Judging Procedure and Scoresheets
  2. G. Panel Consensus and Calibration

Exam Focus

  • Distinguish sensory agreement from forced uniformity of descriptors or initial scores.
  • Use sample evidence and guideline criteria to bring panel totals within the published procedural range.
  • Describe the head judge's responsibility to invite observations, prevent coercion, and verify revised paperwork.

How consensus works

Once every judge has finished a scoresheet, the panel discusses the entry and its scores. Judges explain what they perceived and how it relates to the style guidelines, then adjust their scores as needed. Scores should end within seven points of one another and preferably within five or fewer, and some competitions set their own maximum spread or minimum score. The head judge works with the panel to assign the consensus score, which is written on the cover sheet and flight sheet. It does not have to be the arithmetic average of the individual scores. When a judge changes a score, the comments and section scores on that sheet must be revised to support the new total, and the head judge checks the revised sheets, cover sheet, and flight sheet before they are turned in.

Disagreeing well

When scores differ, try to understand how the other judges reached theirs. People differ in sensitivity to compounds such as diacetyl and DMS, so a judge who missed a note may simply not perceive it at that level. Discussing how heavily a fault should be weighted is fair, as is general calibration. Telling another judge they are wrong, harassing them, or forcing them to change their perceptions is not. Present opinions as opinions, and accept that you could be the one who is mistaken. The head judge may end the discussion when it stops making progress.

When the table is stuck

Judges should not ask judges at other tables for help unless the panel cannot reach consensus and every judge agrees to the consultation. Before asking, check that the person consulted has no entry in that category. In the rare case of a true impasse, the head judge asks the organizer for help. A head judge who simply imposes their own number has not built consensus.

Calibration

Calibration means judges using scoring ranges and vocabulary the same way. Some competitions serve a calibration beer before a flight so the panel can narrow its scoring range. The calibration beer is not a standard that entries are compared against; entries are still judged against the style guidelines. Judges should also be careful not to underrate the less aggressive styles because a more intense beer seems more impressive.

Why It Matters

  • Averaging initial totals without discussion can preserve contradictory reasoning instead of producing a defensible panel result.
  • Dominant voices can erase valid sensitivity differences unless the head judge structures an evidence-based conversation.

Practical Sensory Exercises

Reconcile a split panel

Write a short consensus script for judges who scored the same beer 28 and 37. Identify two observations to compare, one guideline passage to revisit, and the paperwork each judge must update after agreement.

Key Terms

Consensus score
The single score a panel agrees on and records, which need not be the average.
Score spread
The difference between judges' totals; published procedure asks for 7 or fewer, preferably 5 or fewer.
Calibration
Bringing judges' use of scores and descriptors into alignment.
Calibration beer
A beer tasted before a flight to align scoring ranges; not a standard for judging entries.
Sensitivity difference
Variation between people in how strongly they perceive a given compound.

References

4 available