A/B Tests That Actually Moved Conversion Rates — and the Ones That Didn't

I've been running structured A/B tests since my early Shopify dropshipping days, and later across service business accounts at agencies. Most tests lose. That's normal — the discipline is in running them anyway and keeping honest score. Here's my scorecard.

Tests that consistently won

• Headline-to-keyword match. Making the landing page headline mirror the ad group's core keyword ("House Cleaning in Edmond" for "house cleaning edmond" traffic). Boring, repeatable, effective.
• Form length cuts. Dropping fields from 7 to 3 for first-contact forms. More leads, and sales teams confirmed quality held.
• Phone-first mobile layouts. For service businesses, putting a tappable call button above the fold on mobile beat form-first layouts nearly every time.
• Concrete proof near the CTA. Review count, star rating, "500+ jobs completed" placed next to the button — small lifts, almost always positive.
• Offer reframing. "Free quote in 30 minutes" vs "Contact us" — matching the promise to what the visitor actually wants next.

Tests that were mostly noise

• Button colors. The internet's favorite test almost never reached significance on real traffic volumes.
• Minor copy edits below the fold. If they don't see it, it can't convert them.
• Slider vs static hero images. No consistent winner; sliders often lost slightly.

The rules that make testing real

• One variable at a time, or you learn nothing.
• Pre-commit to a sample size; don't stop at the first exciting day.
• Judge on leads or sales, never on time-on-page or other comfort metrics.
• Log every test — the archive becomes your playbook and stops you re-testing losers.

The meta-lesson: tests that reduce friction or sharpen the promise win; cosmetic tests don't. Spend your traffic on experiments with a hypothesis about visitor behavior, not decoration.