Background/Purpose Lee and Lee (2004) developed a multimedia test program for basketball unit using audio, video, and text to evaluate the 8th graders' knowledge such as rules, regulations, and traditions. The preliminary test contained live video clips of 10 questions. Although Lee and Lee (2004) provided the content validity and test-retest reliability, specific information about the goodness of the test has not been reported. Thus, the purpose of this study was to examine goodness of multimedia test program.
Method Participants were eighth graders (N=453) in middle schools. Students completed 10 item multimedia test. Using students' test scores, validity, reliability, item and test characteristics were examined. Also, differentially functioning items were explored to examine item bias. Three parameter logistic model in item response theory (IRT) as well as classical test theory (CTT) was used.
Analysis/Results Results showed that concurrent validity was 0.48, reliability coefficient 0.543, and IRT's marginal reliability 0.558. Test items were analyzed using the CTT and the IRT. The CTT identified unusual items (i.e., item number 1, 3, 7, and 8) showing lack of discrimination power. The IRT identified item 1, 6, 7 and 8 were not appropriate in terms of the discrimination power. Mantel-Haenszel method identified differentially functioning items (i.e., item number 2 and 9) indicating two items are in favor of male students. Test information indicated that examinees with ability level between 0 and 2 showed best scores.
Conclusions Discussions were provided in terms of the degree of goodness, item characteristics, and discrimination power of the multimedia test program.