Marketing Measurement Terms

Statistical Significance

Definition

The likelihood that the difference in conversion rates between two versions in an A/B test is not due to random chance.

Contextual Example

An A/B test result with 95% statistical significance means you can be 95% confident that the results are real and not just a random fluctuation.

Common Misunderstanding

It is crucial to run tests until they reach statistical significance. Ending a test too early can lead to making decisions based on faulty data.