So what? Beyond statistical significance to substantive significance in strategy research

Abstract

Purpose: The purpose of the paper is twofold: first, to examine whether the progress of strategic management research has been damaged by an excessive focus on statistical significance to the exclusion of substantive significance and second, to provide recommendations for improving research practice toward establishing the substantive significance of empirical findings.

Methodology/Approach: We conduct the same survey described in McCloskey and Ziliak (1996) on a sample of all 41 papers published in Strategic Management Journal during 2007 that use regression methodology. We use the criteria for good science represented by these survey questions as the foundation for our discussion. We present our arguments for the relevance of each of these criteria in strategy research with examples of best practice and provide a detailed analysis of areas of research practice that can be improved with associated recommendations.

Findings: Our survey suggests that there is indeed cause for concern, since 90% of our surveyed papers make no distinction between statistical and economic/substantive significance of their results. At the same time, many of the surveyed papers make some attempt to interpret their results in a substantively meaningful fashion.

Originality/Value of Paper: Our paper addresses a critical set of issues that influence progress in strategic management research. We provide a roadmap for how we can address these issues for progress in our field.

Citation

Seth, A., Carlson, K.D., Hatfield, D.E. and Lan, H.-W. (2009), "So what? Beyond statistical significance to substantive significance in strategy research", Bergh, D.D. and Ketchen, D.J. (Ed.) Research Methodology in Strategy and Management (Research Methodology in Strategy and Management, Vol. 5), Emerald Group Publishing Limited, Leeds, pp. 3-27. https://doi.org/10.1108/S1479-8387(2009)0000005001