Crawl & Render Budget E-Commerce SEO JavaScript SEO Product Features Sitemaps Technical SEO

Driving More Visibility: Technical SEO Essentials for Mid-Size E-commerce Brands

A man in a green sweater holding a laptop and smiling into the camera against a blue background.

If you’re managing SEO for a mid-sized e-commerce company, you know earning visibility with consumers is more complex than ever. Not only are you competing against larger brands that may have stronger brand recognition and more resources, but you’re also facing the uncertainty of optimizing a large website in the age of AI-powered search.

These are real challenges. Larger competitors may have been in the industry longer, with outsized SEO teams and budgets. Plus, SEOs are still testing and learning what moves the needle with AI-powered search, and are getting used to the idea that visibility may expand beyond showing up in the Google SERPs

There’s good news, though: you can still compete at the same level as your larger, more well-resourced competitors, even in the shifting landscape of modern SEO. The rapid growth of AI in search has acted as a great equalizer. Right now, SEOs across industries and companies are competing at the same level of knowledge on optimizing for search and answer engines. A solid technical foundation for your website is the best way to make sure your site is found, crawled, understood, and served in responses by search and answer engines to gain top visibility with consumers. 

Let’s discuss four technical SEO challenges that mid-sized e-commerce brands face and how you can address them, even with a leaner team.

1. Optimizing faceted navigation and crawl budget

Category page faceted navigation is an easy and engaging way for consumers to filter by one or more attributes to find exactly the product they’re looking for. On the other hand, they can also create a nearly endless list of URL variations whenever a new facet is selected. This can result in duplicate content issues, crawl budget concerns, and link equity dilution.

Common ways to address faceted navigation issues

There are a few ways to mitigate these issues:

  • Canonical tags: Add canonical tags to all faceted pages to indicate the preferred version (typically the main category page without any facets selected). This helps search engines understand the relationship between different URL variations and consolidates the ranking signals to the canonical URL. 
  • Noindex tags:  Use the “NoIndex” meta tag on faceted pages to prevent search engines from indexing them. This helps prevent duplicate content issues. This method will not consolidate link equity to the canonical URL. 
  • Robots.txt: Disallowing unnecessary facets or parameter variations in the robots.txt file will keep these pages from being crawled by search engines. This will not necessarily prevent them from being indexed if the page is linked to from a different site. 

While the canonical tag method is the most common way of preventing duplicate content issues from faceted navigation, all three methods should be evaluated according to your overall SEO strategy. 

Evaluate which facets should be canonicalized (or not)

That said, not all facets are created equal. It’s crucial to understand which facets can be left on their own and which should be canonicalized back to the original page. Taking the time to analyze and test the best strategy is important. When doing so, keep these principles in mind:

  • High search demand: Is there adequate search demand for the page’s most important keywords? Let’s take the example of an online shoe store with one of the main category pages being for “Men’s Adidas Sneakers.” Adding a “Color” facet would create a page that will likely have many real user queries. 

    However, adding additional facets, like “Size” and “Price Range,” will end up creating pages with very low demand, such as “Men’s Size 10.5 White Adidas Sneakers Under $100.” This is when canonicalizing becomes important.
  • High-quality content: Prioritize faceted pages that lead to high-quality, unique content that understands and answers a consumer’s likely questions for the page, links to highly relevant pages as necessary, and is natural and readable. Avoid indexing pages with facets that generate thin or duplicate content.
  • Competitive research: Analyze how your competitors handle their faceted pages. Do they successfully index a certain type or number of facets? This could indicate how search engines think about these facets and if there’s real user demand. Of course, don’t just copy your competitor — consider your own marketing strategy and product offering first.

2. JavaScript that keeps you from ranking

Most e-commerce websites use a sizable amount of JavaScript to populate some of the links and content on a page. While it creates a good user experience and is convenient for developers, heavy JavaScript usage can be time-intensive for search engines to render

Product pages, for instance, often have third-party tools adding user-generated content, such as comments and Q&A. These modules often display great content that helps improve long-tail keyword rankings. Yet, they typically use JavaScript to render it, limiting the ability of search engines to see all the links and content, which in turn impacts crawlability and ultimately diminishes ranking and traffic performance. 

If you’re not sure whether a certain module on your site requires JavaScript to render important content, try one of the following methods:

Remember: If you can’t see your content with JavaScript disabled, search engines will have a hard time seeing it, too. 

If you’re using JavaScript to populate important links and content on your pages, work with your engineering team to ensure search engines easily see them. Our search engine bot management solution, SpeedWorkers, can make it much easier for search engines to crawl and index this content.

3. XML sitemap updates

As the old saying goes, “You can’t win if you don’t play.” And it’s no different for SEO: “You can’t win if search engines can’t find your content.”

Search engines crawl your website and find new content in two primary ways:

  • Crawling internal links on your website
  • Crawling pages listed in your XML sitemap(s)

As an e-commerce website, you’re likely creating new category and/or product pages (or retiring them) at a pretty regular pace. By having a complete and up-to-date XML sitemap, you improve the likelihood that all of your great content will be discovered quickly.

Most mid-to-large-sized e-commerce websites use some form of an automated XML sitemap generator. While the sitemaps are created automatically and regularly, the logic used to create them is usually out-of-date and isn’t capturing all the pages it should. Work with your engineering team to correct this by ensuring that your XML sitemaps follow these guidelines:

  • Only include canonical versions of URLs that return a 200/OK
  • Do not include pages that redirect, deliver a 4XX error, contain a “NoIndex” tag, and/or are the non-canonical version of a page
  • Include all key page types/subdirectories. In the case of e-commerce websites, this would certainly include all of your category/subcategory and product pages

If you’re a Botify client and you have out-of-date XML sitemaps, reach out to your Customer Success Manager about our automated XML sitemap creation tool.

When it comes to maximizing a search engine’s ability to crawl your website, it’s important to make your site’s crawl paths as seamless as possible. You want search engines to find the next page of your website quickly and easily.

Two great ways to improve this are by:

  • Fixing broken links: Internal links that link to non-existent/error pages should be fixed to link to the intended, working page.
  • Fixing long redirect chains: Does your site contain a large amount of redirects that take more than one “hop” to get to the final destination URL? If so, these redirects should be corrected to redirect straight to the final destination URL.

We address both of these issues for the same reason: having these problems at scale effectively sends search engines down a rabbit hole when trying to discover the correct pages to crawl and index. This wastes search engines’ valuable crawl budget (aka time), preventing them from crawling your new category and product content as quickly as you’d like.

If you’re a Botify customer, you can quickly correct broken internal links and long redirect chains (and more!) — without solely relying on your engineering team — with our SEO optimization deployment solution.


Competing for visibility across search and answer engines begins with a strong technical foundation. The better you enable search engines to find and understand your website, the greater your chances of beating out even large competitors with many resources.

Want to learn more about how Botify can help you build, maintain, and grow your technical SEO? Reach out for a demo — we’d love to help.

Nov 11, 2019 - 7 mins

Google Webmaster Conference: Key SEO Learnings

Crawl & Render Budget E-Commerce SEO JavaScript SEO Product Features Sitemaps Technical SEO
Nov 11, 2019 - 3 mins

An SEO’s Guide To The Botify API

Crawl & Render Budget E-Commerce SEO JavaScript SEO Product Features Sitemaps Technical SEO
Jun 6, 2015 - 2 mins

The Botify Analytics 2015 Spring Release Is Live!

Crawl & Render Budget E-Commerce SEO JavaScript SEO Product Features Sitemaps Technical SEO