Scheduled for Sports Medicine Symposium: What and How on “Evidence-Based”?, Tuesday, April 25, 2006, 1:30 PM - 3:15 PM, Convention Center: 150DEF


Evidence-Centered Assessment Design: Principle and Practice

Weimo Zhu, University of Illinois at Urbana-Champaign, Urbana, IL

Collecting accurate and reliable information is essential to making any evidence-based decision. While great efforts have been made in developing frameworks and procedures for evaluating evidence, how to design an effective assessment and/or study to collect evidence has often been overlooked. This presentation is to introduce Evidence-Centered Assessment Design (ECD), a method to construct assessment in terms of evidentiary arguments. Developed by Mislevy (1994) and his colleagues at ETS, ECD was developed based on advances in evidentiary reasoning (Schum, 1994) and statistical modeling (Gelman et al., 1995). ECD includes six models: Student, evidence, task, assembly, presentation and delivery, with the first three acting as the core. The student model focuses on what knowledge, skills, or other attributes should be assessed; the evidence model focuses on what behaviors or performances should reveal those constructs and what are the connections; and finally, the task model focuses on what tasks or situations should elicit those behaviors. Very recently, a four-part framework has been proposed to enhance the design and delivery of assessment systems based ECD, including Activity Selection, Presentation, Response Processing, and Summary Scoring (Almond et al., 2002). Two “actors” interacting with the framework are test administrators and takers. ECD is based on three premises: (a) An assessment must build around important knowledge/attribute in the domain of interest, and an understanding of how that knowledge/attribute is acquired and put to use; (b) The chain of reasoning from what test-takers say and do in assessments to inference about what they know, can do, or should do next, must be based on the principles of evidentiary reasoning; and (c) Purpose must be the driving force behind design decisions, which reflect constrains, resources and conditions of use (Mislevy et al., 2003). The connections between ECD and conventional test/assessment construction, as well as to traditional psychometric evidence (e.g., validity and reliability), will be described in detail. Examples on how to construct low- and high-stakes ECD assessments and apply ECD for test accommodations will be illustrated.
Keyword(s): measurement/evaluation, safety/injury prevention, standards and ethics

Back to the 2006 AAHPERD National Convention and Exposition