Stop Guessing. Start Testing.
Running an A/B test is the easy part. Knowing if the result is real is the hard part.
Human brains are wired to find patterns where none exist. If Version B gets 5 more sales than Version A, you might pop the champagne. But often, that is just random statistical noise.
This calculator uses Frequentist Statistics (Z-Test) to tell you mathematically if you should deploy the winner or keep the test running.
Interpreting the Results
Significant
The calculator is 95% (or more) confident that the difference is real. Action: It is safe to declare a winner and roll out the changes.
Not Significant (Yet)
There is not enough data to be sure. The difference could be due to luck. Action: Keep the test running to collect more sample size.