Direct observation has been considered one of the valid methods to assess physical activity participations. Yet, in order for this method to be truly recognized as a “criterion” measure, it is necessary to satisfy the basic assumption of direct observation, that is, accurate reports from observers and their agreements. This study intended to fulfill two purposes by measuring the different intensity/speeds of walking using direct observation method: (a) to examine whether observers sense the same pace correctly in terms of validity and (b) to determine observers' agreements for the measurement of walking speeds. A total of 105 college students (61 male and 44 female aged 23.28±1.94 and 21.95±2.61, respectively) were shown video clips of walking on a treadmill at 5 different speeds (6.4, 5.6, 4.8, 4.0, and 3.2 km/h) and asked to answer on a five-point rating scale (5: very fast, 4: fast, 3: normal, 2: slow, 1: very slow). To test the assumption of direct observation, data were analyzed using Pearson product-moment correlation and one-sample t-tests for validity evidence and using intra-class correlation (ICC) for intra- and inter-observer reliability. The results indicated that Pearson correlation coefficient between criterion score and observers' response was .744, which implied a moderate to high positive relationship. Statistically significant differences, however, were found on most of speeds (ps < .001) except for 4.0 km/h, t = -0.14, p = .889. The intra-and inter-observer reliability were .639 and .735, respectively. It was concluded that, overall, the paces of walking recognized by non-trained observers were neither consistent nor accurate. The results from this study suggested that standardized guidelines and extensive training for observers prior to the data collection are required to ensure valid and reliable data using direct observation in walking activity.Keyword(s): assessment, exercise/fitness/physical activity, measurement/evaluation