Historical HTML: Giving SEOs the Full Picture

4th October 2017Kevin Bartley

As an SEO, have you ever wanted to get a complete picture of the past versions of your pages? Our new Botify feature enables just that!

Historical HTML allows our customers to view and download the HTML source code of each page that Botify crawls. This means SEOs can engage in closer examinations of technical SEO properties and page evolutions over time. The feature also acts as a historical footprint of a website’s HTML code.

Source Code Never Lies

Historical HTML captures webpage source code when the page was crawled, and pairs the code to the URL for that specific crawl. As time goes on, these collections of source code become a living archive of a website, documenting all the changes, issues, and permutations that have happened over time.

With this feature, customers can now monitor the modifications of a page’s content alongside metrics such as ranking, traffic, and other key SEO indicators. This means you can directly compare changes on the page to performance impact. Pages that returned status code errors (3xx/4xx), redirect loops, bad canonicals, and other technical problems are easier to examine in-depth with historical HTML.

The feature gives SEOs of enterprise websites a way to track all the innumerable changes made to the site’s web pages, some of which happen without the consultation of SEO best practices. For instance, if a marketer takes a keyword out of a title tag, and the ranking for that term drops for the page, the SEO can reference earlier crawls to figure out what changed in the HTML, and when the change occurred.

Use Case: Identifying a Site-Wide Programming Error

Within a week of the release of this new Botify feature, it was used to identify a site-wide programming error for a multi-million page website.

When crawling the customer’s site, Botify’s crawl revealed an error in the structure of the site. By using historical HTML, the customer was able to see the code of these webpages as they appeared during the crawl and identify that error.

The customer used Botify Analytics in tandem with historical HTML to identify a programing error that was creating an internal linking loop, preventing further crawling of the site. This meant that Google wasn’t naturally discovering and crawling the majority of the site, and neither was Botify, leading to a poorly optimized crawl budget.

By using historical HTML, our client was able to isolate a programming error that was affecting millions of pages, and fixed the issue so that these strategic pages could be crawled by search engines. With this find, the customer was able to optimize hundreds of thousands of pages with a single programming correction.

Take Advantage of Historical HTML for Your Site

Take advantage of historical HTML for your site. Navigate to URL Explorer. Find the URL you want to examine. On the far right of a URL, click the gear, and select “View HTML source”.

The feature is also available in the URL Details page. Click “View URL Details” for a page in URL Explorer. The historical HTML feature is in the top right corner.

A window with the HTML source code will pop up. The source can be downloaded as an .html file. Response headers can also be viewed in isolation.

Historical HTML: The Complete Picture

Historical HTML can bring new insight to Botify crawls, helping to contextualize and understand errors identified by Botify.

Historical HTML is now available for all of our customers! Check it out today. If you’re not a happy Botify user yet, schedule a demo now to see what Botify can do for you.