This experiment tested whether the conformism observed among panels of judges in aesthetic sports also occurs among judges in judo. Similar to aesthetic sports, judo judging relies upon a form of open feedback. However, in judo, this system is reactive (i.e. two judges have to publicly ‘correct’ the score given by the higher-status referee), whereas it is active in aesthetic sports (i.e. judges with equal status report their score simultaneously and can use the feedback about the scores of their colleagues for evaluating later performances). In order to test whether such reactive open-feedback system leads to conformism among judges in judo, we designed an experiment in which this feedback was manipulated. Participants were 20 certified Flemish judges, who had to score two sets of 11 ambiguous video sequences that are used during formation and training of judo judges: one set with feedback about the referee's score and one set without feedback. The results revealed that when participants knew the referee's score, their scores were significantly more in line with this score than when they did not know this score. More specifically, for both sets of sequences at least 10% less deviations from the referee were observed when participants were given feedback about the score of the referee. These results suggest that preventable conformism can occur in typical judo judging, that is with reactive open feedback.
(Reference: Boen F, Ginis P, Smits T. Judges in judo conform to the referee because of the reactive feedback system. European Journal of Sport Science 13, 6: 599-604, 2013. DOI: 10.1080/17461391.2012.756070)
Judges in judo conform to referee decisions
Note: Our friend wdax has started another thread in the kata section on kata judging. Kata judging differs from the above, but has the same problems such as the judging of aesthetic sports (gymnastics & iceskating). I am not going to talk about this in this thread because from a scientific point of view it is simply a different animal.