🧪A/B Test Calculator

About this tool

Two variants in, a verdict out: significant or not, the relative lift, the 95% confidence interval on it, and a planner that says how many visitors each variant needs to detect a given lift. Computed in your browser.

Reading a test honestly

The calculator runs the standard two-proportion z-test: it asks how likely a difference this large would be if the variants were actually identical, and calls it significant when that probability is under 5%. Three habits turn that number into a good decision. First, look at the interval, not just the p-value — “B is 12% better, somewhere between +1% and +23%” is a real but uncertain win, and an interval that crosses zero means B might be worse. Second, decide the sample size before the test and run to it: the planner shows that detecting a 10% lift on a 4% conversion rate needs tens of thousands of visitors per variant, which is why most small-site tests are inconclusive rather than “no effect”. Third, don't stop the first time p drops below 0.05 — checking daily and stopping on a good day inflates false positives several-fold; that is the most common way A/B tests lie. Nothing here is stored. The ad cost calculator tells you what the lift is worth; the percentage calculator handles the relative-change arithmetic.

Frequently asked questions