Commonly Used Statistical Indexes and Software for Sensitivity Determination

Sensitivity, although cannot be directed measured, can be estimated by collecting the evidence of precision, which can be further classified as “generic precision” and “context-specific precision” (Kane, 1996). Standard errors in the classical test theory (CTT) and generalizability theory (GT) are commonly used generic precision measures in practice. The one used in CTT can be estimated from reliability coefficient and the one in GT focuses on the estimation of variance components. While they are convenient to use, both of them do not provide any standard, or benchmarks, for deciding if the errors should be considered large or small. Further, they only provide a global estimation of precision although the prevision varies along a measurement scale. The limitations of global precision can be eliminated by employing conditional standard errors and the local standard errors provided through item response theory (IRT) models. Because the consequences of errors and precision of different sizes depend on the context and intended use,” tolerance for error” should be considered and determined. Kane (1996) called for employing “context-specific precision” and proposed several “tolerance for error” measures. This presentation will first provide a brief review on commonly used variability and reliability measures. Generic and context-specific precision measures under CTT, GT and IRT will then be described in details. Finally, how to compute these measures using common statistical (e.g., SPSS and SAS) and specialized measurement software (e.g., GENOVA and FACET) will be illustrated.