The importance of predefined rules and pre-specified statistical analyses: do not abandon significance


4 April 2019 - For decades, statisticians and clinicians have debated the meaning of statistical and clinical significance. 

In general, most journals remain married to the frequentist approach to statistical testing and using the term statistical significance. 

A recent proposal to ban statistical significance gained campaign-level momentum in a commentary with 854 recruited signatories. The petition proposes retaining P values but abandoning dichotomous statements (significant/non-significant), suggests discussing “compatible” effect sizes, denounces “proofs of the null,” and points out that “crucial effects” are dismissed on discovery or refuted on replication because of non-significance. 

The proposal also indicates that “we should never conclude there is ‘no difference’ or ‘no association’ just because a P value is larger than a threshold such as 0.05 or, equivalently, because a confidence interval includes zero,” and that categorisation based on other statistical measures (eg, Bayes factors) should be discouraged.

Read JAMA Viewpoint

Michael Wonder

Posted by:

Michael Wonder