Scheduled for Measurement Symposium - New Perspective and Practice in Setting Performance Standards, Tuesday, April 1, 2003, 3:00 PM - 5:00 PM, Convention Center: 113C


Generalized Examinee-Centered Method for Determining Cut-Off Scores

Miyoung Lee and Weimo Zhu, University of Illinois at Urbana-Champaign, Urbana, IL

Standard setting methods can be classified into two categories: Test- and examinee-centered. Decided by judges¢ subjective judgments, the test-centered method focuses on item-by-item judgments and determines the cut-off score for each individual item. Because of the impact of subjectivity, the method can generate decision-accuracy problems about cut-off scores, due to judges¢ errors of performance standards. The examinee-centered method, in contrast, focuses on the differences between groups. The borderline and contrasting-groups methods are two good examples. While the limitation of subjectivity is greatly eliminated using the examinee-centered method, it sometimes suffers from the problem of sample bias, and a large number of examinees is usually needed to make a more objective decision (Jaeger & Mills, 1997). Kane et al. (1997) recently developed a new examinee-centered method called the generalized examinee-centered method, which includes the following steps: (a) Take a sample from the population of test-takers; (b) Judge each examinee¢s performance using a set of integrated criteria, including performance standard, external criterion assessment, or experienced raters; (c) Develop a functional relationship between the rating scale generated from the raters and the test-score scale generated from examines; and (d) Determine cut-off scores for both the rating scale and score scale. The uniqueness of this method is that it combines the strengths of both test- and examinee-centered methods. By using some practical examples, this presentation will introduce the generalized examinee-centered method in detail.

Back to the 2003 AAHPERD National Convention and Exposition