When used effectively, sitemaps can play an active role in your SEO strategy, especially for enterprise websites. Sitemaps can increase the visibility of the pages that you want search engines to find and crawl.
This enables more of your pages to get indexed, enter organic search, and generate traffic. But our customers have some of the largest websites in the world. Compiling “strategic” XML sitemaps of a targeted set of URLs, at that scale, can be time consuming and difficult.
We’re happy to announce today, with the release of Botify’s XML Sitemap Generator, that the challenges and complexities of creating optimized sitemaps for enterprise websites have now been reduced to the click of a button.
One the best functions about sitemaps is that they can make invisible pages visible. Sitemaps are not an alternative to a great site structure. But they can help to direct search engine crawlers to uncrawled or low visibility strategic pages.
The Botify XML Sitemap Generator eliminates the grunt work in the sitemap creation process by automatically generating a proper XML sitemap from your own customized list of URLs. An instant, downloadable sitemap can be created at the click of a button.
SEOs can also organize sitemaps by segment of the site or prioritize high-value content in sitemaps. With Botify, SEOs can use any of the 500+ SEO KPIs provided by the Botify suite to filter out strategic pages that could benefit from inclusion in a sitemap, such as:
Non-active, but compliant pages
Pages uncrawled by Google
URLs with a low number of inlinks
Pages with decent word count
URLs that have high content uniqueness
Pages with low content duplication
And any scenario that will be valuable for your SEO strategy.
Sitemap generation is automated, but the process does allow for customization, with elements and tags such as:
The Botify XML Sitemap Generator can process up to 25 million URLs per export, but it’s unlikely you’ll encounter this size for a targeted section of your website.
If you have a very large website and need to build XML sitemaps over 50,000 URLs, the URLs will automatically be broken into multiple sitemaps.
All of these custom fields offer information that allows search engines to better understand the pages in your sitemap.
Once you’ve added uncrawled or low visibility pages to your sitemaps, Botify can verify that these pages have, in fact, been crawled. In the graph below, we are monitoring log files to see how Googlebot discovers then continues to crawl newly discovered pages.
This example is taken from an an e-commerce website. The site wanted to get these not crawled product pages crawled. The site filtered out these product pages thanks to Botify’s URL Explorer, built the XML sitemap with Botify. Then the site used Botify Log Analyzer to monitor the performance.
Here you can see the pages entering the crawl for the first time. But crawling itself is not the ultimate goal. What you want is for these pages to enter organic search and generate traffic. With Botify, you can place crawl and organic visits data side-by-side and see if the pages just added to the sitemap have entered search and gained visitors.
With Botify, you can identify uncrawled strategic pages, automatically generate a sitemap, chart the crawl to see if these pages have benefited from the new exposure, and finally confirm the success of the optimization by seeing if the new crawl activity has translated into new visits.
The Botify XML Sitemap Generator reduces reams of website data in a single click – all while maintaining a wide degree of customization.
Bolster your SEO strategy with sitemaps that align with your team’s objectives. Build off Botify’s unmatched depth of analysis to target the pages that need or deserve visibility. Validate your sitemap strategy with unprecedented insights into the organic search process.
And the best part? You can save your time for more pressing optimizations! Don’t worry – we’ve got you covered.