This is part one in a multi-part series about enterprise SEO. Check out the other three installments in the series: An Enterprise SEO Methodology: From Crawling to Conversions, Getting Your Pages Crawled, Rendered, and Indexed, Content That Ranks for Real Searcher Queries, and Using SEO to Drive Conversions & Revenue.
What does it take to perform well in search engines like Google? That question has been at the heart of search engine optimization (SEO) since its inception, and with some studies indicating that search engines contribute ~35% of all website traffic, it’s a question that deserves an answer.
For many years, people have viewed Google as a “black box” — completely mysterious and unknowable. As a result, SEO specialists have historically spent much of their time trying to reverse engineer the algorithm in an attempt to leverage the traffic and revenue-driving potential of the organic search channel.
But just as quickly as we think we understand Google, it changes and matures, causing many exasperated SEOs and marketers to ask, “Can this channel ever be mastered?”
At the intersection of search engine maturity and web complexity lies the future of SEO. The question is no longer “What can I do to rank?” but “How can I make sure every piece of content I publish is immediately discovered and indexed by search engines so that my audience can find it?” This is arguably the most difficult for enterprises that have to account for potentially millions of pages.
To better prepare for the future of enterprise SEO, we need to explore the complexities of the web today, and how they change the way we think about our websites.
Complexities presented by the modern web
When we say the web is more complex than ever before, we mean on just about every level.
Mobile-first indexing and understanding mobile/desktop parity
Google recently began moving websites over to mobile-first indexing. While Google used to crawl, index, and rank web pages based on their desktop version, Google now does this based on the website’s mobile version.
Based on Botify’s analysis, smaller websites are more likely to be among the first to enter the mobile-first index compared to large websites. Google is giving the largest sites longer to prepare, is moving them over at a slower pace to ensure successful transitions, or both.
From a risk assessment point of view, this makes sense, because smaller websites should be easier to move for both Google and the site stakeholders than enterprise-grade websites.
Botify’s research also indicates that responsive websites are much more likely to enter the mobile-first index first. Google appears to feel that it’s less risky to transition responsive websites, since the same page is served regardless of device/user agent. By contrast, Mobile URL and Dynamic Serving websites serve different pages depending on the user agent, meaning content lacks parity when accessed on mobile versus desktop.
Mobile-first indexing makes it critical for enterprises to ensure that there’s parity between the mobile and desktop versions of their websites. If the mobile version of a web page didn’t contain all the elements the desktop version had, its organic performance could suffer upon being moved over to the mobile-first index.
For many enterprises, the move over to mobile-first indexing has been challenging. Due to the size of their websites, achieving parity has become a long project tackled in phases. Some have opted to fix parity issues on small sections of their site first, monitor the results, then take their learnings and apply them toward fixing larger sections of their site.
Here we find another critical activity that’s nearly impossible to scale -- how can enterprises hope to have the time to compare the mobile and desktop versions of all their pages? Botify has solved for that as well.
With reports that help you ensure mobile compliance, enforce content parity, and optimize load time, enterprise websites can move into the mobile-first index with confidence.
Google indexes trillions of pages, and more are being published every day
Both the web and Google’s index of the web are growing daily. According to Google, “Search starts with the web. It’s made up of over 130 trillion individual pages and it’s constantly growing.”
This places a heavy demand on Google, who has to crawl and render all this content. As we discussed in our blog post on render budget, Google doesn’t have the time and the resources to be able to crawl and render everything. Their answer to this is a budget, or limit of how many URLs Google’s crawler will crawl before leaving your website.
Because of these budgets, Google may be missing some of a website’s important content. In fact, our data reveals that Google is ignoring about half of the content on enterprise websites. Google may also be spending time on “non-compliant” pages — pages that don’t create any meaningful experiences for searchers such as duplicate content or pages responding with errors. When Google has limited time to explore your content, you can’t afford to have them waste time on these types of pages.
If Google isn’t crawling your important content, it won’t be indexing your content either, and content that doesn’t exist in the index has no chance of earning any traffic from Google organic search. In order to correct course, enterprises need to be able to see their website how Google is seeing it. This is made possible by the Botify Log Analyzer.
This adds transparency to the search process for enterprises who traditionally have little insight into how Google is treating their pages. Once you know which of your important pages are invisible to Google, Botify makes it easy to diagnose the cause and correct course.
Search engines and enterprise websites: creating a mutually beneficial relationship
With statistics like Google ignoring half of enterprise website content, it might seem like search engines simply don’t like large or technologically advanced websites. The reality is, search engines and enterprise websites don’t have to be at odds.
Google’s stated goal is to “organize the world’s information and make it universally accessible and useful.” That includes the information on enterprise websites. If enterprise websites want to benefit from the traffic that search engines can send them though, they need to adopt a new methodology.
Large websites simply cannot start with rankings and keywords or they will miss enormous opportunity. They need to go deeper, focusing on the entire search process from crawling to conversion.
We’re going to explore that unique methodology in the next post in this series, following it up with a deep dive into each facet of that methodology.
When adopted, this methodology creates a symbiotic relationship between search engines and enterprise websites -- help Google find and understand your pages and you’ll be rewarded.