“Practical Significance:” That Is What We Should Care About

Interpreting literature correctly is the one of the most important aspects to transfer research into practice. To do so, a researcher needs to understand the difference between statistical and practical significance and to report statistics correctly, especially the “significant” ones of their research, so that the readers will not misinterpret the results. In contrast, researchers often present findings of “statistical significance” in the literature that leads many to believe that a certain intervention will be effective in practice. This is an incorrect way of using the results of a statistical significance test because the “significance” in this case could be greatly influenced by sample size; a very large sample size can make any study result statistically significant even with small mean differences. The focus of the reporting, therefore, should focus on practical significance, i.e., what is the true meaning of a study's finding? To do so, effect size or magnitude of the intervention or treatment must be first determined; then, true meaning of change or difference must be defined and interpreted based on the nature or construct the studied variable represents; finally, 95% confidence limit of the true value must be reported. In addition, related assumptions of a specific statistical test should be examined and reported. If these assumptions are violated, making inferences from the sample data to the population is risky. Using some real-life examples, this session will distinguish “statistical” and “practical” significance and provide a strong argument why the latter is more important than the former. Finally, how to determine “practical significance” based on the information of statistical significance tests and related information derived from significance tests (e.g., effect size and meta-analysis results) will be introduced step by step using a real-life example.