Revenue Uplift Calculator

How it is calculated

Revenue uplift from a change is (new revenue − baseline revenue), and expressed as a rate it is the percentage difference between the two.

For a test, uplift = (variant revenue per visitor − control revenue per visitor) ÷ control revenue per visitor × 100.

Annualising a tested uplift requires assuming it persists, which is the assumption most often wrong in this calculation.

Numbers on it

A test lifting revenue per visitor from $1.45 to $1.62 is an 11.7% uplift. Across 40,000 monthly sessions that is $6,800 of revenue and $3,740 of contribution a month.

Annualised, $81,600 and $44,880, which is the figure that gets reported and the one that requires the most caution.

Uplifts decay. Novelty effects fade, seasonality shifts, and competitors respond, so a tested 11.7% frequently settles at 6% or 7% over a year.

Reporting the annualised figure with a decay assumption applied is both more honest and more defensible when the results are reviewed later.

What it does not tell you

Winning tests are systematically overstated, because a result has to clear the significance threshold to be declared a winner and the ones that do are disproportionately those where noise happened to favour the variant.

Stopping a test early when it looks positive amplifies this substantially, and it is the most common methodological error in commercial testing.

What follows from it

Fix the sample size before starting and run to it regardless of what the interim results look like. Peeking and stopping on a favourable reading is what produces uplifts that never materialise.

Then re-measure a few months later. A change that still shows the effect after a quarter is real; one that has faded was noise or novelty.

Why claimed uplifts rarely aggregate

A business that implemented twelve tests each claiming a 10% uplift would expect revenue to triple. It does not, and the gap between the claimed and the realised is the standing embarrassment of the discipline.

The causes are overstated individual results, effects that overlap rather than compound, and decay over time. All three are predictable and none is usually accounted for.

The honest approach is to track total revenue per visitor over time as the check on the sum of the claims. Where the aggregate has not moved as the individual tests implied, the testing programme is producing reports rather than results, and that is worth knowing before another year of it.

Reporting uplift as a range rather than a point estimate reflects the actual uncertainty, since every test result carries a confidence interval that the headline figure discards.

A result quoted as between 4% and 19% is less satisfying and considerably more honest than the 11.7% midpoint, and it sets expectations that survive the following quarter.

Documenting the assumptions alongside the result makes a later review possible, and testing programmes that keep that record improve considerably faster than those that only keep the wins.

Holding back a small permanent control group is the most reliable way to verify that accumulated changes have produced the aggregate improvement claimed for them.