Subscription pricing tests fail more often from operational sloppiness than from bad prices. A test can prove the "right" price and still lose money if it corrupts cohort analysis, breaks the dunning cadence, or triggers chargeback spikes. Here is the discipline we use.
# Rule 1: Grandfather aggressively
Every user who signed up at a given price stays at that price forever, unless they voluntarily change plans. This is not a courtesy — it is an operational necessity:
- Grandfathering keeps cohort analysis clean. If prices change under a cohort, you cannot compare LTV across time.
- It eliminates the largest single cause of subscription chargebacks: "the price went up and nobody told me."
- It protects your fair-billing story with regulators. In several jurisdictions, changing recurring prices without explicit consent is unlawful.
Store the customer's price at subscription creation. Reference that value for every renewal charge. Never read the current price from the plans table at charge time.
# Rule 2: Test on new signups only
Pricing experiments happen at signup, never at renewal. Split incoming traffic, not the installed base. Concretely:
- New user in test bucket A → sees $29/month.
- New user in test bucket B → sees $39/month.
- Existing users at either price continue to be charged whatever they signed up at.
If you must test on the installed base — for example, an upgrade CTA to a new plan — treat it as an opt-in flow, not a price change. The user actively clicks "switch to the new plan," reads the new terms, and consents.
# Rule 3: Isolate the pricing variable
A price test that also changes the plan name, the feature list, and the CTA copy tells you nothing about price. Change one variable per experiment. If you need to test packaging and price together, factorial the experiment (2×2) and pay for the extra sample size.
# Rule 4: Sample size matters more than significance
Pricing tests are famously underpowered. A 5% conversion lift on a 3% baseline needs tens of thousands of visitors per cell to detect at 80% power. Most subscription businesses call winners on 200 signups per cell. Those "winners" are noise 60% of the time.
Before starting a test:
- Compute the minimum detectable effect you care about.
- Calculate required sample size for 80% power at α=0.05.
- Commit to running until that sample size is reached, whatever the interim results say.
Peeking at experiments and stopping early is how you accumulate a graveyard of "wins" that never replicate in production.
# Rule 5: Include LTV in the readout
Signup conversion is a leading indicator, not the target metric. A price test that raises signups 15% by dropping the price 30% loses money at every LTV horizon.
The right readout compares expected revenue per visitor across cells:
expected_revenue = conversion_rate * average_price * expected_lifetime_monthsEstimate expected_lifetime_months from cohort retention curves for the same price point. If you do not have historical retention data at the tested price, extrapolate conservatively — do not assume the churn curve of your $29 tier applies to a $49 tier. It usually doesn't.
# Rule 6: Dunning cadences must be price-agnostic
Do not hard-code recovery cadences to specific price tiers. Your recovery system should read the customer's price at failure time and adjust:
- Higher-price customers may warrant a phone call, not just email.
- Lower-price customers should not receive an aggressive multi-channel cadence — the recovery cost may exceed the recoverable revenue.
The RRLabs engine picks cadence by expected recovery value, not by tier name. That way, adding a new price tier does not require new dunning code — just a new row in the pricing table.
# Rule 7: Watch chargebacks per cohort
New pricing cohorts sometimes generate elevated chargebacks — new descriptor, new price, new customer expectations. Monitor chargebacks per cohort, not globally. A price test with 0.8% chargebacks in the test cohort but 0.3% overall is not "acceptable"; it is a leading indicator that will show up as a merchant-account review a quarter later.
# Rule 8: Kill switches, always
Every live pricing test needs:
- A single toggle to disable the test and revert all new traffic to the control price.
- A revert plan for anyone who signed up under the test — either grandfather them (default) or actively contact and refund.
- A logging trail that captures which user saw which price and when.
Pricing tests fail in weird ways. A CDN caching a variant price to the wrong bucket, a mobile app version pinned to an old price, a bug in the price display component — all of these have happened to real teams. The kill switch is not paranoia; it is professional operations.
# Rule 9: Regional pricing is not a test
Charging different prices in different regions is a business decision, not an experiment. Run it as a permanent price policy with clear geolocation rules, disclosed on the pricing page. Do not A/B test regional pricing on the same country — regulators consider this deceptive in several jurisdictions, and it produces support tickets you cannot answer honestly.
# The RRLabs default
The Revenue Recovery Labs engine reads each customer's signup price from an immutable field, calculates recovery economics per customer (not per plan), and integrates with feature-flag systems so pricing experiments do not leak into the recovery cadence. Grandfathering is enforced at the data model level, not by convention. Test pricing hard — but test it carefully.