The Increasing and Disappointing P Value Save

In the March 15th issue of JAMA, Chavalarias and coworkers describe the evolution of the “P value” and its use in biomedical research in the last 25 years.  (CItation Source http://buff.ly/22L0pG7)

Based on extensive review of 12 million MEDLINE abstracts and more than 800,000 abstracts and full-text articles in PubMed Central, the authors find that P values have become increasingly used over time, increasing from 7.3% in 1990 to 15.6% in 2014. In 2014, P values were reported in 33.0% of abstracts from the 151 core clinical journals (n = 29,725 abstracts) and 54.8% of randomized controlled trials.

Nearly 96% of the P values reported at least 1 “statistically significant” result, with strong clustering for P values.05 and .001. Dr. J. Ioannidis (one of the authors) stated, "The fact that you have so many significant results is completely unrealistic." 

The authors call for more use of effect size (including odds ratios and risk differences), confidence intervals (indicates the degree of certainty) and for the use of false-discovery rates (estimates how likely a result is to be true or false).

Of the 796 papers reviewed, only 13.9% reported effect sizes, 2.3% reported confidence intervals, and 2% reported both an effect size and a confidence interval. None reported Bayes factors or false-discovery rates.

Although clinicians lean on P values as empiric evidence of significance, many researchers and biostatisticians, believe that the P values should be eliminated altogether. Changing the way significance is represented will require better training of clinicians and researchers and changing policies with those who rely on such results (journal editors, regulatory agencies, etc.).

P values may indicate statistical significance, but P value cannot tell you whether a result is true, or how likely it is that something has no effect.

In an accompanying editorial Dr. Demetrios Kyriacou, reviews the history of the P value and offers his own suggestions on changes needed. He points out that the Chavalarias article demonstrates the lack of progress in the value and portrayal of “significance” in the medical literature and how confidence intervals are really more important than P values. (Citation source http://buff.ly/22L3CFF)