Is Content Freshness a Ranking Factor?

26th September 2019Kyle Blanchette

Does updating your content more frequently help it get crawled more frequently and therefore rank better?

Conventional SEO wisdom tells us that fresh content is better, but as per usual at Botify, we wanted the data to back that claim up.

There was a lot to unpack here, including:

• What is content freshness?
• Does freshness matter more in some industries than others?
• Does fresh content get crawled more frequently?
• Does crawl frequency impact rankings?

…but we were committed to get to the bottom of it.

If you’re at a large enterprise organization, it’s not enough to know that “fresh content is SEO best practice.” Putting time and money into updating the millions of pages on your website is no small investment — you want assurances that your investment will be worthwhile. We get it, and we think we have the data that will help.

Let’s dive in.

What is content freshness?

Fresh content is content that has been updated recently, is updated frequently, or was published recently. Fresh content is the opposite of stale content, which is content that hasn’t been updated for a long period of time.

In this article, we’re going to use the metric “% of content change” as a proxy for content freshness. This metric in Botify shows you how much your content has changed since the previous crawl. So, if you rewrote half of your page and then re-crawled your site, the content change of that page would be 50%. Since updating existing content is one of the main ways to keep existing content fresh, “% of content change” seemed like a viable metric to use.

You can learn more about content change monitoring in our post on Content Quality Analysis: How to Evaluate the Quality of Your Content.

Why is content freshness important?

Content freshness is important because searchers want to see results that are relevant and recent, so that’s the type of content Google will serve them. In 2011, Google announced an improvement to their ranking algorithm that was “designed to better understand how to differentiate the level of freshness you need” for each unique query. So, for example, when you search for a current event, you might see pages ranking at the top that are just minutes old. As an answer to these queries, Google determined that recency was more important than age.

What is “query deserves freshness”?

Query Deserves Freshness (QDF) is a mathematical model developed by Google’s Amit Singhal that tries to determine when users want new information and when they don’t. There are three main categories of queries that deserve freshness: recent events and hot topics, regularly recurring events, and information that changes often (e.g. “best shows on Netflix”)

How does Google rank fresh content so fast?

Since we know that Google favors fresh content in certain situations, the next logical question is how are they able to rank brand-new content so quickly? The answer is the Caffeine web indexing system, which they released in 2010. In order for Google to rank fresh content, they needed to be able to find and index new content fast, and Caffeine helped them do just that.

Our assumption then is that Google will crawl fresh content more frequently than stale content, but we wanted the data to back that up. Does updating your content get Google to crawl the page more frequently?

Is there a correlation between updating content and crawl frequency?

Turns out, there is a correlation!

Take this site for example. It’s super clear from the data that the ones that changed the most since the previous crawl (2 weeks prior) are definitely crawled more frequently — they have a higher average number of days with crawls from Google.

That’s great to know, but what we really want to know is does this higher crawl frequency translate into higher rankings?

Is there a correlation between updating content, crawl frequency and ranking?

So we’ve seen that there’s a clear correlation between content change and crawl frequency, but does that increased crawl lead to better rankings?

Back to the data!

In the chart below, you can see that not only do pages where the majority of content changed get crawled more often, but those pages also rank for more keywords.

Pages with no content change are not only getting crawled the least frequently and rank for the least amount of keywords. The more content on those pages changes though, the more they’re crawled by Google and the more average keywords they’re ranking for.

Another way to look at the data is to see if there’s a correlation between content change, crawl frequency, and average rank position (as opposed to average number of keywords ranking, like the chart above depicts).

Again, we can see a clear connection. Pages that underwent the most change got crawled more often and rank higher than pages that changed less and were crawled less often.

Content freshness on publisher vs. e-commerce websites

In the world of SEO, we know that no single change always has the same effect on every website (or even on different pages within a single website!). So, we wanted to know if the correlation between freshness, crawl frequency, and ranking looks the same on publisher websites (that are primarily comprised of article URLs) as it does on e-commerce websites (that are primarily comprised of product and category URLs).

Pictured below is data from a large publisher website. From this, we can see that fresh content gets crawled evenly, but ranks for more keywords. As with any correlation, we have to fill in the gaps with our own hypothesis to explain what’s causing this to happen. So what do we think is happening here?

From our perspective, it makes perfect sense that Google is crawling this publisher site evenly. A trademark of publisher websites is that they’re always publishing new content — the biggest ones publish hundreds of articles every day. The content on publisher websites, particularly news publishers, is also timely. Google knows that they have to crawl these sites often in order to keep their index up-to-date with this constant flow of new content. So while content freshness may not do much for crawl frequency on a publisher website (it’s already crawled frequently as-is), it still has a clear correlation with ranking.

Now, what about e-commerce pages?

In the example below, content updates actually correlated with worse rankings!

We wanted to see if this was just an anomaly, or whether another set of e-commerce pages would reveal the same negative correlation.

When we looked at a second group of e-commerce pages, we found a similar pattern. Pages with higher amounts of content changes had fewer average number of keywords ranking.

So, is content freshness a ranking factor?

While we did find plenty of instances where updating content correlated with more keywords ranking and higher ranking positions, we also found instances where changing the content actually correlated with worse rankings.

There are a few things we can learn from this.

1. It’s not always about ranking factors

Sometimes we get so focused on ranking better in search engines that we forget what we’re doing it for — our audience! Even if updating your content doesn’t always correlate with better keyword rankings, shouldn’t you do it anyway to supply your audience with the most accurate, up-to-date information? We think so.

2. There are always multiple factors at play

Why would updating content produce worse rankings on one site and better rankings on another site? There are countless possible reasons for this. In this article, we only looked at the correlation between updating content and crawl/rankings — we didn’t even broach the subject of on-page factors. These results show that it’s likely not just about the content change itself, but about how you change the content.

If you notice a portion of your pages are ranking exceptionally low, it’s a good idea to look at on-page factors like content uniqueness, HTML tags, and more to see if any of those might correlate with your low rankings.

3. Always look at your own site’s data

The best way to determine what changes will produce improved rankings (and other SEO KPIs) on your site is to look at your own data. In this article, we showed you one way you can do that, but the options really are endless when you unify all your data. By combining information from your log files, site crawl, Google Search Console, and website analytics, you can start seeing correlations that help you make the smartest decisions about what changes you should make on your site.

If you want to learn more about how Botify can help you find your site’s ranking factors, request a demo with us! We’d love to show you around.