Inbound Marketing for Manufacturers Blog

    A/B Testing Basics (and 4 Things You Should Never Do)

    3 minute read

    In simple terms, an A/B test compares two versions of a web page to see which one performs better. (Let’s call them version A and version B.) The tester shows the experiment to users at random. The tester then uses statistical analysis to determine which version performed better.

    Why You Should A/B Test

    ab testing.jpg

    Odds are you want your website visitors to convert from visitors to leads, then to customers.

    A/B testing allows you to make the most out of your existing website traffic by minimizing guessing when it comes to your optimization efforts. Instead of guessing, you can make data-driven decisions to convert your website visitors into leads more effectively.


    So, what can you test? The short answer is: Anything on your website that affects your visitors' behaviors, including:

    • Headlines
    • Sub-headlines
    • Paragraph copy
    • Testimonials
    • Call-to-actions
    • Images
    • Content near the fold

    But, you can’t just test things to test them. You have to make sure the things you’re testing tie back to your goals and revenue. You also have to make sure your tests are statistically significant.  Here are 4 things to avoid when A/B testing:

    4 Things You Should Never Do While A/B Testing

    1. Pull the Plug Early

    Running a test too long or not long enough can have consequences, including limited information or a sample size too small to actually learn anything.

    To get a big enough sample size to see an impact, you need to run a website test for at least a week.

    2. Test Too Many Small Things

    Testing small elements, like buttons and forms, won’t produce the impact you need or, more importantly, result in any learning opportunities.

    There are an unlimited amount of variables you can test on any of your web pages. It’s crucial you develop a hypothesis before you start testing. Before testing anything, ask yourself:

    • Why do I want to test this variable?
    • What is the expected outcome from testing this variable?

    3. Ignore Positive Results

    This one probably speaks for itself. If you have a test that’s statistically significant (one version performs better than the other), you need to implement your changes.

    Don’t be the person who runs a test then does nothing with the results. It’ll just be a waste of time and potential revenue.

    4. Not Test at All

    Every test you run isn’t going to be a slam dunk, but every test will teach you something -- including what works and what doesn’t work. So fail ... then fail some more. It’ll only make you better at A/B testing.

    I’ll leave you with this FunnelEnvy quote: “The truth is, nothing else offers the ROI potential of a good conversion rate optimization campaign.”

    Happy testing! 

    business growth 8 steps to improve marketing performance

    Topics: Analytics

    Related posts