Background/Purpose Intraclass correlation coefficients and a modified Bland-Altman analysis provide incomplete information about judge behavior. They cannot identify anomalies in judges' individual scoring patterns. A many-facet Rasch analysis has been recommended as a way to identify these anomalies (Looney, 2004). The purpose of this study was to use a many-facet Rasch analysis to determine if there were anomalies in judges' scoring patterns for men's figure skating at the 2010 Olympics.
Method Nine judges' scores, retrieved from the Internet, were analyzed after receiving approval from the Office of Research Compliance. Judges' free skate scores for each of five program components (choreography, interpretation, performance, skills, and transitions) for 24 skaters were analyzed using a three-facet Rasch rating scale analysis (skater by program component by judge).
Analysis/Results Plushenko was the only one of the top three skaters overall to have unexpected scores (standardized residual > |2|). The scores given to Plushenko did not fit the model well (Outfit MNSQ statistic = 2.67; Estimated Discrimination Index = -0.69). Judge 1 scored Plushenko much higher than Lysacek (1st place) when the model expected these skaters to receive the exact same scores (p=.05; differential judge functioning). Judge 2 gave Plushenko about 1.25 points higher than expected scores for Interpretation and Performance while scoring Transitions lower than expected by 2 points.
Conclusions Plushenko's overall free skate score represents a different construct than the scores do for the other skaters. Skating officials should investigate the unexpected scores given to Plushenko by Judges 1 and 2 for possible reasons (e.g., nationalistic bias).