Statistically significant difference simply means there is statistical evidence that there is a difference. Statistically significant difference does not mean the difference is large, important or significant in the usual sense of the word.
When researchers study within groups or between group differences, they need a technique to determine if this difference would have occurred by chance. Various statistical techniques can determine this and if it is unlikely the differences could have occurred by chance it is called a statistically significant difference.
Small and non-notable differences can be found to be a statistically significant difference. Statistically significant result is not always of practical significance.
Usually a .05 level of significance is used (there are 5 chances of one hundred trials that this difference would occur by chance) but other levels can be used. When a researcher finds no statistically significant difference and writes as follows:
"we found no statistically significant difference," he could be misquoted as "they found that there was no statistically significant difference."
What is the probability of
replicating a statistically significant difference effect?
Jeff Miller, University of Otago, Dunedin, New Zealand
Abstract: If an initial experiment produces a statistically significant effect, what is the probability that this effect will be replicated in a follow-up experiment? Although the data from an initial experiment can be used to estimate one type of replication probability, this estimate will rarely be precise enough to be of any use. Although it would be nice to know the probability of replicating a statistically significant effect, researchers must accept the fact that they generally cannot determine this information, whichever type of replication probability they seek.
The Orthopaedic Trauma Literature: An Evaluation
of Statistically Significant Findings in Orthopaedic Trauma Randomized Trials -
Jinsil Sung; Judith Siegel; Paul Tornetta; Mohit Bhandari
Abstract: Statistically significant differences between two treatments may not necessarily reflect a clinically important difference. We aimed to quantify the sample sizes and magnitude of treatment effects in a review of orthopaedic randomized trials with statistically significant findings.
Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials - Mohit Bhandari, Jason W. Busse, Dianne Jackowski, Victor M. Montori, Holger S, Sheila Sprague, Derek Mears, Emil H. Schemitsch, Dianne Heels-Ansdell and P.J. Devereaux
In this study, we examine the association between industry funding and the statistical significance difference in results of recently published medical and surgical trials.
Gender Has a Small but Statistically Significant Difference Effect on Clearance of CYP3A Substrate Drugs - David J. Greenblatt, MD and Lisa L. von Moltke, MD
The role of gender on the disposition of drugs metabolized by cytochrome P4503A (CYP3A) remains controversial. Some sources suggest that CYP3A activity in women exceeds that in men, but evidence to support this position is inconsistent at best. It was found that gender has a small and statistically significant difference, although most likely clinically unimportant, influence on CYP3A phenotype for substrates not transported by P-gp.
ELISA: Structure-Function Inferences based on statistically significant and evolutionarily inspired observations
Boris E Shakhnovich, John M Harvey, Steve Comeau, David Lorenz, Charles DeLisi
BioInformatics Program, Boston University, Boston, MA, 02215, USA
Eugene Shakhnovich, Department of Chemistry and Chemical Biology, Harvard University.