Reconsidering the default p value threshold for statistical significance test

Posted by K.WANG on September 22, 2017

Source of P value - Great contribution of R.A. Fisher

Zabell, S. L., 2001: Ronald Aylmer Fisher. Statisticians of the Centuries, C. C. Heyde, E. Seneta, P. Crépel, S. E. Fienberg, and J. Gani, Eds., Springer New York, 389-397.

R.A Fisher suggested that:

  • “The value for which P=0.05, or 1 in 20, is 1.96 or nearly 2 ; it is convenient to take this point as a limit in judging whether a deviation is to be considered significant or not.” [Fisher, Ronald, 1925]

Then you should feel encourage to pursue the finding furthermore.

Fisher, Ronald (1925). Statistical Methods for Research Workers. Edinburgh: Oliver and Boyd. p. 47. ISBN 0-05-002170-2.

What are new things after almost one century

73 statisticians want to change the threshold, as Benjamin et al. (2017) said:

  • “We propose to change the default P-value threshold for statistical significance from 0.05 to 0.005 for claims of new discoveries.”

Benjamin, Daniel J, James Berger, Magnus Johannesson, Brian A Nosek, Eric-Jan Wagenmakers, Richard Berk, Kenneth Bollen, et al. 2017. “Redefine Statistical Significance”. PsyArXiv. July 22. psyarxiv.com/mky9j.

It has received a lot of attention.

Actually, 30 years ago, Berger and Delampady (1987) has indicated:

  • “when testing precise hypotheses, formal use of P-values should be abandoned. Almost anything will give a better indication of the evidence provided by the data against H0.” (p. 330)”

Berger, J. O., & Delampady, M. (1987). Testing precise hypotheses. Statistical Science, 2, 317-352