← Statistics for Data Science learning path

Statistics for Data Science · Module 9 of 10

A/B testing

Run experiments to make data-driven product decisions.

📖 Key ideas

A/B test: randomly split users between version A (control) and B (variant).
Pick one primary metric (conversion rate) before starting.
Decide sample size in advance using power analysis.
Run long enough to cover weekly patterns.
Do not stop early just because results look good.

💡 Real-life example: An A/B test is a fair race: two runners, same track, same time, decided by a stopwatch.

ControlOriginal version A

VariantChanged version B

Conversion rateShare of users completing a goal

RandomisationAssigning users by chance