Increased Google Crawl & Doubled SEO Traffic in 3 Months: A Case Study

2nd February 2017Elizabeth

How an online Auto Marketplace* increased Google crawl by 19x (and doubled organic traffic) in less than 3 months.

^{*Shared anonymously to protect this customer’s recipe for success}

A Nationwide Marketplace Hidden From View

Based in the U.S., this online vehicle marketplace helps users find the right car or truck that fits their exact search criteria in their own neighborhood. A nationwide service, the site uses industry-leading data and has thousands of participating dealerships for the best listing discovery. The only thing stopping them? Getting their site found and indexed by Google.

Uncovering Invisibility

With a site based on an industry-leading dataset, it was their very own resources that were preventing them from being seen. Making such a huge volume of data available to their users was having negative SEO consequences: it was too much for Google to crawl.

Analyzing Google Crawl

In order for website content to drive traffic, it has to show up in search results. It can only show up in search results if Google’s crawling robot ‚Äì Googlebot ‚Äì finds and explores it.

Sure enough, in their first Botify analysis, reports showed that there were plenty of pages of content, but very few were being crawled and indexed.

Over 99% of the website had not been crawled by Googlebot.

That’s 9.9 million pages out of 10 million crawled that are unlikely to ever receive organic traffic from Google searches.

Just 2% of the pages Google had seen were actually in the site structure. The rest of the 2.66 million URLs Google had crawled were wasting the entirety of Google’s Crawl Budget on pages not attached to the site and not capable of driving organic traffic to the rest of the website.

In their very first SEO analysis, this online marketplace discovered:

• Just 1% of the website had ever been crawled by Google
• Only 2% of Google’s Crawl Budget was going to pages linked in the site structure
• Nearly 80% of the website was non-indexable
• More than 1 million pages had a single inlink, limiting accessibility

With millions of strategic pages going unseen by search engines, it was time to make changes that would show the true value of the site’s data.

Data-Driven SEO Results

Through a series of changes to the site structure and the way it is explored by search engines, the Auto Marketplace’s SEO Project Manager drove real, visible results in less than 3 months.

With an immediate goal of stopping wasted crawl, initial implementations included updates to the robots.txt file, putting an end to infinite refinements generating 10s of millions of useless URLs, and increased internal linking to boost content accessibility.

So what happened?

Significant results became clear just 6 weeks later. They saw improvements in Google’s crawl and increased SEO traffic as a result.

• Cut the number of known URLs in half to focus Crawl Budget on valuable content
• Increased internal linking to aid search engine robots’ discovery of more pages
• Increased Google’s crawl by 19X – up to 9.5% from just 0.5% before Botify:
  
  and
• Doubled organic traffic from 40,000 to 80,000 weekly visits:
  

And that’s the story of how one Auto Marketplace went from practically invisible online to increasing Google crawl by 19X and counting‚Ķ plus doubling their organic traffic all in less than 3 months!

Surprised? Read the full case study to see exactly what they did to boost their SEO thanks to unmatched insights in Botify.