Statistical Significance
Definition
A judgement that a result is unlikely to have arisen by chance under the null hypothesis, conventionally at p below 0.05. It says nothing whatever about whether the difference is big enough to care about.
What this means for research sourcing
Understanding evidence quality helps researchers prioritize compounds. Volta's research library summarizes the literature behind stocked compounds.
Related Terms in Evidence & Research
p-Value
The probability of seeing a result at least as extreme as the one observed if the null hypothesis were true. Below 0.05 is the convention for significance, and it is not the probability that the hypothesis is wrong.
Confidence Interval
The range of values compatible with the data at a stated level, usually 95%. A wide interval means an imprecise estimate however striking the central figure looks, and it is the more honest number to read first.
Hazard Ratio
The ratio of the rate at which an event occurs in the treatment arm to the rate in the control arm over time. Below 1 favours treatment, and an interval crossing 1 has not demonstrated a difference.
Sample Size
How many participants a study enrolled, written as n. Small samples give unstable estimates and are the commonest reason an early positive result fails to replicate.
Pilot Study
A small preliminary study run to test feasibility and gather estimates for designing a real one. Pilot results are routinely over-read as findings in their own right, which they are not.
Case Report and Case Series
A published description of one patient, or a handful, with no control group. Useful for flagging a rare harm or an unexpected observation, and not evidence that something works.
Observational Study
Research that records what happens without assigning the intervention, in cohort or case-control form. It shows association at a scale trials cannot reach and can never rule out that the groups differed to begin with.
Systematic Review
A review that follows a search and appraisal protocol written in advance, so which studies were included is reproducible rather than left to the author's discretion.