A/B-test winners are often communicated like “12% uplift in sales (p<0.05), while the hypothesis that is being tested only tests if the variant would outperform control. Due to variance in both variant as control, you can only communicate the probability that the variant outperforms control. The observed uplift is just a coincidence and can only […]

