Definition

In A/B testing, this is the probability that a winning variation is actually better than the original. It is another way of expressing statistical significance.

Contextual Example

A 95% confidence level means there is a 5% chance that you are wrong about the winner (a false positive).

Common Misunderstanding

Most businesses use a confidence level of 95% as the standard for declaring a winner in an A/B test.