Rank position is one of the most well-known and commonly-used KPIs in SEO, but it can pose some difficulties for large organizations with huge websites. In this article, we’re taking a deep dive into keyword tracking for enterprise SEO: how to scale it, manage it, and get to the bottom of why your rankings are what they are.
But first, a refresher.
If you’re a little rusty on the details of keyword tracking, aren’t an SEO practitioner, or are new to the concept of rank tracking, we encourage you to read the next three sections on “What is keyword tracking?”, “Why do you need to track keywords for SEO?”, and “What factors impact keyword ranking?”
If you’re already familiar with these topics, feel free to skip down to the section titled “What makes keyword tracking difficult for enterprises?”
Keyword tracking, also known as “rank tracking,” is the process of identifying what position your URLs occupy on the search results pages for certain keywords.
Most companies, at least the ones that have an SEO program, use software to keep track of their rankings. A popular free option for rank tracking is Google’s own report in Google Search Console called “Performance.”
It looks like this:
Anyone who manages a website can set up a free Google Search Console account. Learn how by reading through Google’s documentation. Google also has something called a “Search Console API,” which we’ll talk about in more detail later.
Keyword rank tracking is one of the most popular SEO KPIs for a reason: it helps you understand how visible your web pages are in the organic section of Google (and other search engines!) search results. While other SEO KPIs like organic traffic are important, rank tracking helps you understand how you got that traffic.
When choosing a rank tracking solution, make sure you’ll have the ability to not only see what position you’re ranking in, but which of your URLs rank for each keyword. Pictured below is Botify’s rank tracking solution, Botify Keywords, which shows you the URL, impressions, clicks, CTR, and average position of all your keywords.
The reason you even need to track keywords is that rank position changes often. You may be ranking in position 4 one day and in position 7 the next. These fluctuations in rank position can happen for a number of different reasons, including:
Additionally, even if you maintain the same rank position, your traffic could still fluctuate for reasons such as:
Rank tracking may not pose too many difficulties for a small-to-medium sized business, but it can quickly turn into a headache for enterprise organizations. Why? Because enterprises often have larger websites, sometimes multiple websites, and usually more complex websites (e.g. multiple country versions). With more pages and complexity, it becomes even more important to have a scalable, organized rank tracking solution.
Let’s take a look at some specifics — exactly what kinds of issues do enterprise organizations run into with rank tracking?
Many enterprise websites are huge. Think about an e-commerce site with 8 million indexed URLs or a news/media site that publishes 100 new articles every day. When you consider the fact that a single URL can rank for multiple keywords (sometimes hundreds!), this can quickly get out of hand.
Such a high volume of potential keywords causes three major problems:
Because many rank tracking solutions require you to pick which keywords you want to track, you won’t be tracking any new keywords your site ends up ranking for, unless you add them manually over time (and we know that “manually” is a very ugly word in the enterprise world!).
Rank tracking typically comes as the result of keyword research, a process in which you determine what phrases your audience is using to search for the information or products you offer. This process can be long and cumbersome, especially for enterprises that want visibility for millions of different keywords. Because of this, keyword research is usually treated as a “set it and forget it” project, or at best, something that’s done quarterly or annually.
If you only perform keyword research once, or very infrequently, you’ll miss out on tracking new keywords that your site is gaining impressions and clicks for.
Enterprise organizations often want more granularity than most rank tracking solutions are able to give them. For example, they want to be able to answer questions such as:
Many rank tracking solutions will tell you what position you’re ranking in for your selected keywords, but can’t give you many of the additional details like how many clicks you’re getting for those rankings. This is because most rank tracking solutions scrape Google search result pages to calculate rank position, rather than use real searcher / real website data. You can learn more about that in our article How To Check What Keywords Your Site Is Ranking For.
Similar to the problem of lacking details is the problem of missing context. Enterprise organizations need to be able to know not only what positions they’re ranking in, but why they’re ranking there. Without that context, you’ll have little to go off of when it comes to taking action.
Ideally, enterprise organizations need a keyword tracking solution that allows them to:
Botify wanted to deliver exactly that, which is why we launched Botify Keywords. Botify Keywords pulls your keyword performance data from the Google Search Console API and pairs it with the rest of your data in the Botify platform, such as crawl data, log file data, and analytics data.
Let’s unpack that a bit.
The Google Search Console API allows us to pull all your website’s keyword data into Botify, and “all” really means all. In Google Search Console itself, you only get access to 1,000 rows of keyword data. If you use the API, on the other hand, you get all of it.
To learn more about Google Search Console’s keyword capabilities, check out “The Ultimate Guide to GSC Keyword Data.”
The benefit of using Botify Keywords to view your Google Search Console keyword data is that you get to layer it with your other SEO data for added context.
At Botify, we like to say that search is a process. As a process, it has a beginning and an end.
Most rank tracking solutions only show you Step 3 — where your indexed pages show up for different queries. But to get to that point, your page first needs to be crawled, rendered, and indexed. Without seeing your pages as Google sees them, you may never understand why your pages are ranking in the positions that they are.
Because search is a process, technical issues can absolutely impact keyword performance. For example, take a look at all these technical issues that can cause poor ranking performance and notice that none of them have to do with on-page optimization like content and keywords:
…and the list goes on. That’s why it’s critical to be able to pair your keyword data with your technical data.
You may also like our Botify Keywords announcement post where we unpack key features and use cases.
It’s one thing to arm yourself with all the data. It’s quite another to incorporate that data into meaningful workflows you can use in your day-to-day.
That’s why we’ve put together some of the top use cases for Botify Keywords.
Do you know which of your keywords increased or decreased in position, clicks, or impressions? In Botify Keywords, you can set any date range and see whether these important keyword KPIs improved or got worse over that period.
This is great for everyday monitoring, but it can be especially helpful when you’re going through a site migration — just set the date range to cover the time before and immediately following the transition, and see what changes result.
We also recommend looking at your keyword data through the lens of segments. Segments are meaningful groups of your URLs that help you keep better track of your data. For example, you can segment by pagetype, template, content type, product type, and categories. You can even create a segment for your most important pages, that way, your most critical keyword information doesn’t get lost.
Here’s what segmentation looks like in pie chart form:
Without segmentation, it can feel like an uphill battle identifying keyword performance trends and causes in organic search. It’s like searching for a needle in a haystack.
With segments though, you can identify great insights like how your rank position is impacting the number of clicks you get across key segments.
Or how your CTR changes by device across different segments.
Another key use case of Botify Keywords is identifying “striking distance” keywords — in other words, keywords that could result in substantial traffic wins for your website with just a little love.
In the picture below, you can see that we’ve created a filter that is going to show us all keywords ranking between position 6 and 9 — the bottom of page 1. Keywords in this position are close to positions that tend to get a lot of the clicks, but far enough away that they’re missing most of them.
We’ve also set the filter to show us only keywords that we’re getting more than 1,000 impressions for. This ensures that we’re only looking at the highest-impact opportunities (rather than keywords that would only bring us in a handful of new impressions per month).
Finally, you need to set a CTR threshold. For us 3% made sense (we want to ensure that we’re only going to be working on the keywords that actually need improvement) but keep in mind that each site is different. You can select whichever CTR makes sense for your unique site.
Once you apply those filters and have your list of “striking distance” keywords, you can start to layer on crawl data to help answer questions such as:
Like the idea of identifying keywords you’re close to ranking for? You may also like our article on Keyword Research for SEO: How to Create a Killer Keyword Strategy.
Similar to segmentation, identifying the locations you rank in is important for global brands. Botify countries data can help pinpoint any regional cannibalization that is occurring.
For example, the data could reveal that your Canadian visitors are being shown U.S. URLs. Knowing where your searchers are located is a great way to identify whether they’re being served the appropriate content.
For most enterprise websites, particularly publishers, it’s incredibly important to ensure that recently published content gets crawled and indexed quickly so it can start attracting traffic.
In the example below, you can see now only when Google has crawled your recently published pages‚Ä¶
…you can also see how that’s translating into impressions and visits.
The use cases definitely extend beyond publishers as well. If you’re launching a new section of your website, for example, this would be a great way to keep track of how quickly Google is finding and indexing those new pages.
Another interesting insight you can glean by layering your keyword data with your other SEO metrics is how factors like internal linking might be impacting your rank position.
In Botify, you can visit the “Impact of Inlinks on Ranking” report to see how many rankings you have by the total number of internal links. In the example below, the URLs that are contributing to the site’s performance the most are those that have more than 100 links to them. The chart pictured is showing clicks on the y axis, but you could also change that to rankings or impressions as well.
Sometimes it can be difficult to articulate exactly why spending time on a particular SEO initiative is a good idea, but reports like this make it much easier. Instead of saying, “We should add more internal links because it’s SEO best practice” you could say “We should add more internal links because pages with more internal links rank better and capture more clicks, on average.”
Most keyword tracking software is just that — keyword tracking. But as you can see, you’re missing critical context if you’re not:
If you want to see Botify Keywords for yourself, book a demo with us! We’ll walk you through the platform so you can get a feel for how our keyword tracking solution could work for you. Have questions before you commit to a demo? No sweat! You can always reach us on Twitter — just tag your questions @Botify and we’ll get back to you shortly.