Performance Assessment for Korean Dance Competitors of ‘Dong-A Concours'

Background/Purpose: Many different judges are often considered to assess dancers’ performances because of the variability in the judges’ severity. Dancers’ performances rated by different judges, however, cannot be compared directly due to some limitations including sample and judge dependence. This issue can be addressed using the equating method to set two or more judges’ rating scores on the same scale. The present study was using the Rasch rating scale model to examine the severity and inconsistency of judges’ rating for Korean dance competitors advanced to the final round of ‘Dong-A Concours,’ which is a very popular art contest and to identify the best dance performer since 2002. 

Method: Dancers’ performance ranks at the final round of competition each year were determined by seven judges selected by Dong-A Concours Selection Committee. At least two judges among seven were automatically selected for the following year. The performance ranks of 60 Korean dance competitors, who had advanced to final round, were assessed by the total of 47 judges. Using the Rasch rating scale model, the parameters of judge’s severity and dance performance were estimated. The common-item nonequivalent group equating design (Kolen, 1988) was applied to equate different judges’ rating scores. The data were calculated by WINSTEPS version 3.690 (Linacre, 2009).

Analysis/Results: The judges’ severity level ranged from -0.46 (less severe: SE=0.06) to 0.61 (more severe: SE=0.71) logits. Most judges exhibited a high degree of internal consistency (Outfit: 0.5 - 1.5). Five judges were less variable in their ratings compared to other judges (Outfit < 0.5) while four judges had too much variation in their ratings compared to other judges (Outfit > 1.5). The dancer, ‘Jin-Hee Choi,’ performed from 2002 to 2013 showed the highest logit score (1.18) among 60 dance competitors.

Conclusions: Judges’ severity level has a substantial impact on a raw score. Correcting for judges’ severity level improves the fairness of dancer rating scores and of the subsequent winner decisions because the uncorrected raw scores favor dancer performances rated by lenient judges. The results analyzed by using the Rasch rating scale model make it possible for not only ‘Dong-A Concours,’ but also various contests to compare the ranks of the competitors performed through different years more objectively.

Keywords: judges’ severity, dance performance, Rasch model, Dong-A Concours