Nine Judges, Two Votes

Nine robot judges sit behind a bench, all holding up answer B, under the heading Nine AI judges vote. Below, an approximately-equals sign and two robot judges: worth about 2 independent judges, because of the same internet, same homework, same mistakes. At the bottom, a question, When all nine agree, how often are they wrong?, answered by two bars: these nine, 9.1%, and nine independent judges, 0.02%, a sliver.

ai, llm-as-a-judge, reliability

Nine robot judges sit behind a bench, all holding up answer B, under the heading Nine AI judges vote. Below, an approximately-equals sign and two robot judges: worth about 2 independent judges, because of the same internet, same homework, same mistakes. At the bottom, a question, When all nine agree, how often are they wrong?, answered by two bars: these nine, 9.1%, and nine independent judges, 0.02%, a sliver.

Comic