Marketing Measurement Terms
Statistical Significance
Definition
The likelihood that the difference in conversion rates between two versions in an A/B test is not due to random chance.
Contextual Example
An A/B test result with 95% statistical significance means you can be 95% confident that the results are real and not just a random fluctuation.
Common Misunderstanding
It is crucial to run tests until they reach statistical significance. Ending a test too early can lead to making decisions based on faulty data.
Related Terms