Home
According to a new scientific study published in the European Journal of Sport Science by Boena and colleagues, judges in judo conform to the referee because of the reactive feedback system.
The study is based on an experiment. The abstract [with some edits] is as follows:
"This experiment tested whether the conformism observed among panels of judges in aesthetic sports also occurs among judges in judo. Similar to aesthetic sports, judo judging relies upon a form of open feedback. However, in judo, this system is reactive (i.e. two judges have to publicly ‘correct’ the score given by the higher-status referee), whereas it is active in aesthetic sports (i.e. judges with equal status report their score simultaneously and can use the feedback about the scores of their colleagues for evaluating later performances). In order to test whether such reactive open-feedback system leads to conformism among judges in judo, we designed an experiment in which this feedback was manipulated. Participants were 20 certified judges, who had to score two sets of 11 ambiguous video sequences that are used during formation and training of judo judges: one set with feedback about the referee's score and one set without feedback. The results revealed that when participants knew the referee's score, their scores were significantly more in line with this score than when they did not know this score. More specifically, for both sets of sequences at least 10% less deviations from the referee were observed when participants were given feedback about the score of the referee. These results suggest that preventable conformism can occur in typical judo judging, that is with reactive open feedback." (...)
My personal thoughts without going into the pure science of methodology, stats, etc., is that these findings are most likely in line with what we have all suspected for a long time, and with that what those among us who are referees have felt and perceived many times before.
Another thought would be that those findings might be extrapolated to judging kata in a sense, that the having judges score "aesthetic performance" without knowledge of each others scores avoids the problems highlighted by this research, (one of the many) problems which cause bias in scores.
Please, share comments, questions or caveats.
The study is based on an experiment. The abstract [with some edits] is as follows:
"This experiment tested whether the conformism observed among panels of judges in aesthetic sports also occurs among judges in judo. Similar to aesthetic sports, judo judging relies upon a form of open feedback. However, in judo, this system is reactive (i.e. two judges have to publicly ‘correct’ the score given by the higher-status referee), whereas it is active in aesthetic sports (i.e. judges with equal status report their score simultaneously and can use the feedback about the scores of their colleagues for evaluating later performances). In order to test whether such reactive open-feedback system leads to conformism among judges in judo, we designed an experiment in which this feedback was manipulated. Participants were 20 certified judges, who had to score two sets of 11 ambiguous video sequences that are used during formation and training of judo judges: one set with feedback about the referee's score and one set without feedback. The results revealed that when participants knew the referee's score, their scores were significantly more in line with this score than when they did not know this score. More specifically, for both sets of sequences at least 10% less deviations from the referee were observed when participants were given feedback about the score of the referee. These results suggest that preventable conformism can occur in typical judo judging, that is with reactive open feedback." (...)
My personal thoughts without going into the pure science of methodology, stats, etc., is that these findings are most likely in line with what we have all suspected for a long time, and with that what those among us who are referees have felt and perceived many times before.
Another thought would be that those findings might be extrapolated to judging kata in a sense, that the having judges score "aesthetic performance" without knowledge of each others scores avoids the problems highlighted by this research, (one of the many) problems which cause bias in scores.
Please, share comments, questions or caveats.
