Scheduled for Motor Behavior and Measurement Posters, Friday, April 2, 2004, 1:00 PM - 2:00 PM, Convention Center: Exhibit Hall Poster Session


Successful Program Management Through Quality Evaluation: Development of the Scale for Program Facilitators (SPF) to Assess the Effectiveness of After-School Achievement Programs

James J. Zhang1, Eddie T. C. Lam2, Dennis W. Smith3, Jennifer Brimer3, John Todorovich1 and Daniel P. Connaughton1, (1)University of Florida, Gainesville, FL, (2)Cleveland State University, Cleveland, OH, (3)University of Houston, Houston, TX

Children with inadequate care are more susceptible to the influence of others than those who attend structured and supervised care. Violent crimes committed by youth occur most frequently during after-school hours. In recent years, there has been considerable growth in public attention to the potential benefits of after-school programs. In order to attain program objectives, it is critical to have a quality curriculum, implementation, supervision, facility, and evaluation. Because program facilitators are the front-line staff members, they profoundly influence children’s motivation levels as they engage in activities, and their opinions about the program would help to ensure program quality. The purpose of this study was to develop the Scale for Program Facilitators (SPF) to assess the effectiveness of after-school achievement programs through four steps: (a) identification of a theoretical framework, (b) formulation of the initial scale, (c) test of content validity, and (d) conducting confirmatory factor analyses (CFA). A comprehensive review of literature revealed that the nature and curriculum of after-school programs are often heterogeneous and may include enrichment activities (e.g., sport and recreational activities), academic enhancement (e.g., homework tutorial), personal skill development (e.g., problem solving), community involvement (e.g., volunteerism), and/or delinquency prevention (e.g., good citizenship). However, consensus usually exists among program administrators in terms of four key program objectives: Professional Skills, Social Behaviors, Caring Environment, and Personal Inspiration. Guided by these four program objectives, 18 items in a Likert 5-point scale were derived from a review of literature, on-site observations, and interviews with program facilitators. A panel of five experts participated in the test of content validity. Based upon 80% agreement, all items were retained after minor wording improvements were made. Following the SPF, after-school program participants (N=2,995) were evaluated by facilitators at 353 program sites implemented in two large metropolitan areas. Because the basic assumption of multivariate normality of data was not met, the weighted least square estimation model was used in the CFA, which revealed that the four-factor model provided a good fit to the data. The goodness-of-fit indexes were in the upper range (e.g., GFI=.99 and RMSEA=.04). Although the factors were highly correlated, second-order CFA revealed that other hypothesized models were significantly (p < .01) inferior to the four-factor solution. Composite reliability coefficients were all above .90 and variances extracted were all above .80 for the factors. Discussion is focused on the application of the scale, as well as implications to curriculum construction and program operation.
Keyword(s): community-based programs, measurement/evaluation, violence/prevention

Back to the 2004 AAHPERD National Convention and Exposition