Glossary · ConversionTOFU
A/B testing
The short answer
A/B testing is an experiment where visitors are randomly split between two versions of something — a page, an ad, an email subject line — to measure which performs better on a chosen metric.
Version A is usually the current control; version B changes one thing. Because visitors are assigned randomly, differences in results can be attributed to the change rather than to who happened to see each version.
What's worth testing
- Offers and pricing presentation
- Headlines and value propositions
- Proof placement — reviews, guarantees, logos
- Form length
- Checkout flow and payment options
- Ad hooks and creative angles
Good test hygiene
- A written hypothesis before launch
- Sample size calculated in advance
- One primary metric
- Full weekly cycles
- A log of every test, including losers
A quick example
A D2C brand tests two product-page headlines: "Cold-pressed oil for everyday cooking" vs "Cold-pressed oil, no refining — taste the difference". Both run for two full weeks to equal traffic. The primary metric is add-to-cart rate; the result is read only after the planned sample size is reached, with checks for mobile vs desktop differences. See A/B testing services and statistical significance.
A/B testing in practice
An eCommerce brand tests two product-page versions: one shows the delivery date above the buy button, the other doesn't. Traffic is split evenly; after two full weeks and enough orders for a reliable result, the delivery-date version converts better, so it becomes the default — and the next test begins.
How to run better A/B tests
- Start with a clear hypothesis based on research
- Test one meaningful change at a time
- Decide the sample size and duration before starting
- Run through full weekly cycles
- Record results, including losing tests
A/B testing for Indian businesses
Many Indian sites have enough traffic to test landing pages, offers and checkout steps — especially D2C brands during sale seasons. Festive weeks distort behaviour, so avoid concluding tests that span Diwali or big marketplace sales unless both versions saw the same period.
Common A/B testing mistakes
- Ending tests as soon as one version pulls ahead
- Testing tiny changes with too little traffic
- Running overlapping tests on the same page
- Ignoring mobile and desktop differences
Frequently asked questions
How long should an A/B test run?
Until it reaches its planned sample size, and for at least one to two full weeks.
What's the difference between A/B and multivariate testing?
A/B tests compare whole versions; multivariate tests combinations of several elements at once and need far more traffic.
Can you A/B test ads on Meta and Google?
Yes. Both platforms offer experiment tools that split audiences fairly. Test one meaningful variable at a time.
What should we A/B test first?
High-impact elements — offer, headline, page structure, form length — before small details like button colours.
What is the difference between A/B and multivariate testing?
A/B tests compare versions of one change; multivariate tests examine several elements and their combinations, needing far more traffic.
Can ads be A/B tested?
Yes — platforms such as Google and Meta offer experiment tools for ads, audiences and bidding.
What if a test shows no difference?
That's a valid result: the change didn't matter enough to detect. Move on to a bolder test.
What tools can run A/B tests?
Dedicated testing platforms, ad-platform experiments, and many eCommerce and landing-page builders have built-in testing.
How much traffic does an A/B test need?
Enough conversions in each variant to detect the difference you care about — often hundreds per variant.
Related terms
Full glossaryTalk to a strategist
Thirty minutes. Your numbers. A straight answer.
Book a strategy call with the people who'd actually run your account. We'll tell you what we'd do — or that you don't need us yet.