In past decades, coaching evaluation has drawn increased attention of researchers in performance appraisal. Much effort has been given to identify specific criteria and develop proper instruments to assess coaching performance (e.g., Leland, 1988; Docheff, 1989). MacLean and Chelladurai (1995) developed a Scale of Coaching Performance (SCP) to assist coaching evaluation by using the input of Canadian athletics administrators and coaches and contained such factors as: 'Team Products'; 'Personal Products'; 'Direct Task Behaviors'; 'Indirect Task Behaviors'; 'Administrative Maintenance Behaviors'; and 'Public Relations Behaviors'. However, a reservation of using the SCP in the coaching evaluation of NCAA member institutions was held due to the concerns of difference existing between the collegiate athletics programs of Canada and the United States (e.g., Barber & Eckrich, 1998; Chen, 1998). Could the SCP be generalized to the evaluative systems of NCAA because of the shared similarities? If not, what would be an appropriately alternative model? An examination of the SCP applied in different populations has become necessary. The study was designed to examine the reliability and factorial validity of SCP using the sample of NCAA. The participants (n=448, out of 820) were randomly selected from three NCAA divisions including 201 administrators and 247 head coaches. The 35-item SCP with an informed consent form, demographic data sheet, and self-addressed envelope was sent to each targeted participant. Confirmatory Factor Analysis and associated statistics (e.g., EFA, Alpha coefficients, composite reliability, and variance-extracted measures) were employed to test the reliability and factorial validity of SCP by inputting the NCAA data. The findings indicated that the SCP had moderate reliabilities (internal consistency measured by Alpha coefficient) among the factors except for 'Public Relations Behaviors' (r=.61). However, the 6-factor model failed to fit the NCAA data (chi-square [545, n=448]=2913.59, p < .0001, RMSR=.084, GFI=.73, NNFI=.75, IFI=.75) and no alternative model (e.g., 8-factor, 4-factor, 2-factor, and single-factor model) compensated a better fit to the NCAA data. Even though the SCP is valuable in coaching evaluation literature based on the constructs derived from a plausibly theoretical framework and produced through a soundly scientific procedure, the results implicated that the SCP may be population specific for Canadian setting without fully satisfying the needs of NCAA programs due to the differences existing between two systems regarding the coaching responsibilities, financial resources, program philosophies, and the perceptions of evaluators and evaluatees.Keyword(s): administration/mgmt, measurement/evaluation, performance