7 reasons your SEO tests fail and how to fix them

Incrementality testing successful SEO sounds simple: make a change, measurement the impact, comparison it to the control, vessel the winner, and iterate. But tests often autumn isolated earlier they moreover commencement because of flaws successful the testing framework.

After a decade of starring SEO testing programs pinch a 70% mean occurrence rate, I’ve learned that reliable, actionable results dangle connected a methodology that tin show whether your trial really drove value. Avoid these communal mistakes that tin undermine your SEO experiments.

1. You’re utilizing the incorrect testing methodology

A/B testing, aliases divided testing, sends users to different versions of the aforesaid page. Some information of your postulation sees Version A, and immoderate sees Version B. You’ll measurement differences successful behaviour aliases conversion. This is awesome for UX and CRO testing, but it doesn’t thief you isolate ranking impact.

Pre/post testing compares capacity earlier and aft a alteration connected the aforesaid page aliases group of pages. It’s elemental and fast, but it’s besides the slightest reliable for SEO because it can’t automatically power for outer variables for illustration seasonality, algorithm updates, aliases competitor changes. It’s adjuvant for directional insight, but you’ll request to return other steps to corroborate your results pinch confidence.

Incrementality testing compares a group of pages pinch a circumstantial alteration against a power group of akin pages without that alteration complete the aforesaid period. This is the golden modular for SEO because it isolates the effect of a azygous variable. The quality successful capacity betwixt the 2 groups shows whether the alteration really influenced rankings, visibility, aliases traffic. You tin besides usage before-and-after information to support your findings.

When to usage each trial method

Use an A/B trial when:

• You’re testing a caller creation aliases characteristic earlier you build it.
• You don’t want to trial connected each your postulation connected a fixed page because it’s precocious consequence aliases debased confidence.
• You’re testing engagement, interaction, aliases conversion.
• You tin reliably way your trial versions.

Use a pre/post aliases incremental trial when:

• You don’t person an easy measurement to divided trial pages, channels, aliases audiences.
• You want to way the afloat acquisition and conversion funnel.
• You want to measurement semipermanent results.
• You want to measurement SERP and ranking impact.
• You want to recognize the imaginable for gains connected unrecorded pages now.
• You don’t person a divided testing instrumentality aliases plugin connected your website.

2. Your presumption is flawed

Developing a good hypothesis and an due trial scheme is the cardinal to effective SEO testing. The much you study from each test, the amended you’ll get astatine processing each hypothesis.

To create a much rigorous hypothesis that’s little prone to error, make judge your trial meets these criteria:

• Actionable: A large capable alteration to matter, pinch capable sessions, clicks, etc., to corroborate your hypothesis.
• Consistent: The aforesaid alteration connected aggregate pages to corroborate it’s not a fluke.
• Measurable: Has search and capacity information available.
• Extensive: Long capable to cheque for ranking and visibility impact.

For example, changing 1 connection successful the mediate of a fewer pages pinch small postulation is astir apt excessively mini to beryllium worthy a general test. But changing 1 connection successful the H1 connected 30 pages pinch 100+ monthly sessions complete 4 weeks mightiness person a measurable effect connected postulation and rank.

3. You haven’t done risk/reward analysis

Consider the champion versus worst imaginable outcomes erstwhile you’re developing your hypothesis. 

Some of the worst-case scenarios mightiness see the page not loading, search not working, aliases conversions and gross dropping. Consider really apt each script is. Then hole strategies and processes to mitigate those issues earlier they go a problem.

• QA: Include rigorous validation connected aggregate browsers and devices.
• Small scale: Go unrecorded connected 1 page, cheque search aft 3 days, past rotation retired to much pages.
• No Friday tests: Don’t spell unrecorded earlier a play aliases erstwhile nary one’s disposable to show the site.
• Lower value: Avoid testing first connected pages that thrust the astir leads aliases revenue.
• Frequent checks: Check trial results weekly, aliases much often if it’s a super-high-risk test.
• Plan B: Set up a scheme earlier the trial starts successful lawsuit you request to quickly revert if the early results look bad.
• Timing: Give yourself capable clip to trial and rotation retired changes earlier the engaged season.

If the consequence and effort to spell unrecorded aliases revert are low, see testing much wide aliases rolling retired everyplace you tin and measuring results afterward.

---

4. You don’t person a power group

Incremental testing includes a clear group of trial pages that person a circumstantial alteration made astatine the aforesaid time, compared pinch power pages that don’t person that change. Then you tin compare:

• Before and after: Did the trial consequence successful a change?
• Control: Did different akin contented person the aforesaid consequence complete the aforesaid period?
• Site overall: Was the tract impacted by an algorithm update, hunt inclination change, aliases moreover an incorrectly tracked SEM campaign?
• This twelvemonth vs. past year: Could seasonality beryllium a factor?
• Clean data: Did you make immoderate different awesome changes to your trial aliases power pages recently?

If you don’t person a applicable power group, you tin still tally a pre/post trial because you’re comparing before-and-after results. If you’re capable to rotation retired the trial to much pages, you’ll still beryllium capable to corroborate the results are accordant during the rollout.

5. You’re not reference your results correctly

This is 1 of the trickiest parts of testing that’s reliable to get correct without a bully consciousness of your site’s emblematic capacity and really to publication your capacity reports.

• Check each your data: What if your sessions accrued arsenic you predicted, but conversion complaint dropped?
• Validate your data: If your numbers are surprising, outsized successful either direction, aliases don’t lucifer up, tin you corroborate them pinch different study aliases double-check the underlying information source?
• Go deeper than the surface: What if your postulation increased, but it was for the incorrect keywords? What if you’re sending lower-quality leads because you pulled successful the incorrect traffic?
• Filter your results: What if desktop improved, but mobile sewage worse? What if definite users converted better, but others converted worse?
• Check for outliers skewing results: What if 8 retired of 10 pages sewage worse, but the 2 pages pinch much postulation improved? What if half the pages were surgery during the test?

6. You’re not mounting up to rotation retired pinch precocious confidence

With immoderate tests, you’ll astir apt person immoderate follow-up tests, specified arsenic applying the aforesaid changes to related contented aliases expanding connected your presumption based connected what you learned.

You mightiness find that definite transcript useful good for your audience, truthful that whitethorn springiness you immoderate insights into akin connection you could test.

If you’re first testing connected a mini batch of pages, make judge you see capable trial pages that you’re assured successful the results earlier rolling retired fully.

If you deliberation your presumption could use to merchandise class and merchandise reappraisal pages, see immoderate pages from some types successful your first test.

Your rollout will besides corroborate your results, truthful dainty the rollout for illustration different trial and measurement each the aforesaid things you did successful your first information of testing. Then comparison that to your first round’s results.

But what do you do if your trial underperformed? Expect that not each trial will spell precisely arsenic planned. ou tin revert the changes, return what you learned from that test, and use it to a early hypothesis.

7. You’re not pursuing up connected your results

The past step: After you’ve finalized your capacity results, follow-up connection will thief you get the connection retired astir your findings, promote others to effort retired caller ideas, and admit immoderate thief you received for your test.

• Document each of your results and stock them truthful everyone tin find them.
• Be clear astir what you tested and why you deliberation it worked.
• Include immoderate applicable discourse astir what you excluded aliases didn’t trial yet.
• Explain adjacent steps and rollout plans.
• Include imaginable and realized effect for trial pages and the full trial scope.
• Apply what you learned to thing you can.

Build assurance successful what really works

A successful SEO testing programme isn’t astir making each trial a winner. It’s astir knowing why capacity changed and having capable assurance successful the results to determine what to do next.

A stronger methodology helps you isolate the effect of your changes and nutrient much reliable, actionable results. Even an underperforming trial tin supply useful insights, sharpen your adjacent hypothesis, and thief you determine what to trial aliases rotation retired next.