Author: Tran Cathy
Institution: Biopsychology/Professional Writing
Date: October 2004
Judges will never get a perfect 10 because they have brains. Things that should not matter do matter.
(c) Associated Press Paul Hamm's 2004 Olympics all-around gold medal was disputed when judges' scores were called into question.
During the Athens Olympic games, after a high score flashed for a routine riddled with errors, television analyst Elfi Schlegel commented: "I know she has the reputation but that's not right." Maybe it isn't right, but it happens - and cognitive researcher Diane Ste-Marie, of the University of Ottawa, may have the reason why.
Ste-Marie suggests that the issue at hand is memory bias. In a laboratory setting, she showed fifteen judges a video of gymnastics clips. The judges were asked to report whether or not a form error was made. A second video was shown in which some of the moves from the previous tape also appeared, but with contrasting form. Ste-Marie found that judges were more likely to see non-existent errors in the second tape if the moves were shown with an error in the first tape. This suggests that the judges' memories were biased from what was seen beforehand.
"The memory bias is a systematic, unconscious bias that cannot be controlled," adds Ste-Marie. The judges do not recall judging the same move with a form error on the previous tape, yet it still affects their new perceptions.
These findings suggest that a gymnast that errors in warm-ups may not only be hurting his or her own confidence but also may bias the judges' perceptions at the same time.
"This particular bias seems to be quite long-lasting still evident a day later and even a week later," says Ste-Marie. In subsequent studies, she brought judges back into the lab several days after the first video and found the same results. These findings are pertinent to events such as the recent Olympics. "[Performance] during qualifying [rounds] may bias a judge's decisions during finals," explains Ste-Marie.
Order of competition is another variable that plays a role.
"The best possible place [in a lineup] is last," says Don Peters, 1984 Olympic head coach. He explains that it is difficult for a first-up gymnast to get a high score because points need to be reserved for gymnasts who may do better later on.
John Scheer, a professor at the University of Nebraska, found that a gymnast that is the fourth competitor on a team has a significant scoring advantage over the gymnast that is the first competitor. Scheer managed the 1973 Big Eight Gymnastics Meet, creating two different version of the meet with the first and fourth competitor switched. The order effect in scoring was seen in nationally certified judges.
The judging biases that are influenced by cognitive processes are very minor, but so is the difference between a gold and silver medal. Paul Hamm, the all-around champion in Athens, won his title by 0.012 points the smallest margin in Olympic history. However, gymnasts who are heads and heels above the rest of the field may not need to worry about these subtle effects. Carole Ides, president of the National Women's Gymnastics Judging Association, comments:
"There are first up gymnasts whose scores have held, who have won the competition."
Judging gymnastics is a tough discipline, and as with all sports, it has its kinks. However, Ste-Marie praises the objectivity in gymnastics judging. "The gymnastics judging system is set up much better than figure skating because of the Code [of Points]. The amount of points awarded for each skill is clearly stated and the deductions are systematic." She does, however, offer a suggestion. To reduce effects of memory biases, Ste-Marie suggests,"have judges rotate from event to event for a multi-day competition."
To reduce order effects, Peters would like to see a limit on the first score. "For example, the highest that the first person up can get is a 9.5," says Peters. This way, the judges have more room to rank the remaining gymnasts relative to one another.
And with these changes, perhaps judges will be a little closer to that elusive 10.