Trying to find the right SEO software or tools can be overwhelming. Plus, having too many tools can be a problem in itself. That’s where Botify comes in.
If you’re new to Botify, we’re here to help! We wrote this article to introduce you to who Botify is as a company, what problems we’re solving for enterprise brands, and how our platform works.
Click below to jump to a section:
Founded in 2011 by Adrien Menard, Thomas Grange, and Stan Chauvin, the Botify team has been dedicated to anticipating what’s next in the SEO industry and creating a scalable, solutions-oriented platform to maximize organic search traffic and revenue for enterprise brands. As we’ve grown, we’ve been fortunate enough to do things like gain $20M in Series B Funding, open an office in Seattle (our fourth location!), and expand our team.
As a company, we believe in the power of SEO for driving growth and revenue when it’s properly prioritized and resourced. In fact, we recently commissioned Forrester Consulting to conduct a study on just that! To learn more, you can read the research or watch the video.
But let’s take a step back. One of the most important things you should know about Botify is that we take a “full funnel” approach to SEO, meaning that you need to look at every phase of the search process (crawling, rendering, indexing, ranking, conversions) in order to really understand what improvements you need to make to perform better in search.
After all, search is a process with a clear beginning and end — if search engines don’t crawl your site, your pages can’t rank in the index where they can earn traffic and revenue for your business. We look at everything through that lens, and it’s the foundation upon which we built our platform. We call it the Botify Methodology.
With that foundation in mind, let’s take a look at the problems we’re solving.
The fact is, Google crawls less than half of an enterprise website’s pages – that’s significant!
From the Botify study “How does Google crawl the web?“
If search engines aren’t crawling your website, your pages can’t be indexed (and they definitely won’t rank). For enterprise websites, that means there’s a ton of potential for improving traffic and revenue simply by getting search engines to find your important content.
This is where Botify solves a huge problem for large, enterprise websites.
Botify works with organizations that count on their website(s) for a substantial portion of their revenue. In many cases, that translates into working with e-commerce, publishers, classifieds, and travel organizations.
Whether you’re a technical SEO responsible for identifying indexing issues, a content marketer who needs to analyze content for quality, or an executive who’s responsible for driving traffic to and revenue from your organization’s website, Botify can help!
The Botify platform began as a comprehensive search analytics platform, and at its core was our cloud-based crawler that can get through even the biggest sites at a rate of 250 URLs per second. While knowing virtually every detail about your site is critical (from how many links to redirected URLs you have, to the average size and uniqueness of your content), we knew the most actionable SEO insights come when you unify your data.
That’s why, on top of our crawler, we layer on your log file data, your query data, and your website analytics data.
Botify allows you to see your website at every phase of the search process — from crawling to conversions — and how each part impacts another (ex: how crawl impacts ranking, how content quality impacts crawl frequency, etc.). You can layer 1,000+ data points for any URL on your website to identify exactly what’s impacting your pages.
With so much data at their disposal, our customers are able to use Botify for things like analyzing their website architecture for SEO, finding and fixing indexing issues, identifying trending keywords, and more — the use cases really are endless!
But with great quantities of data comes great responsibility — you have to take the time to peruse your data to pull out the highest-priority insights. That’s why the next iteration of Botify is all about intelligent, prescriptive actions and automation.
Botify Intelligence and Botify Activation, which we announced in October 2019, take your search data to the next level, solving for pain points like “With all this data, I’m not sure where to start,” “Critical pages were blocked by robots.txt and I wasn’t aware until our traffic suffered,” and “I can never seem to get my SEO projects implemented because my dev team doesn’t have the resources.”
Intelligent alerts and automation is only possible because of the completeness and trustworthiness of the data that’s made possible through Botify Analytics, so let’s take a quick dive into what makes Botify Analytics tick.
With Botify Analytics, it all starts with a crawl. The data we retrieve from each crawl helps our customers evaluate the structure and content of every page on their website — from mobile to desktop — just as a search engine would. Since all of your SEO data lives in Botify, you can easily keep track of changes in your crawl data over time.
Not only does Botify crawl HTML at a rate of 250 URLs per second, it can also fully crawl and render your JavaScript (using the same rendering engine as Googlebot), at a rate of 100 URLs per second, critical for JavaScript SEO audits.
The log file analysis component of Botify Analytics enables you see how search engine bots are crawling your site. You can feed your server log files into a dashboard built specifically for SEOs to overlay them with your crawl data. This allows you to see which pages Google is ignoring and which pages they’re visiting, enabling you to detect crawl budget issues. It’s essential to use this data in order to direct bots to your most critical pages — your top converters and top revenue generators.
Through the Google Search Console API on our platform, you’ll be able to see even more data than in the GSC performance dashboard. Our customers generally use this feature to get all of their query data, such as what queries are driving impressions and clicks. The keywords dashboard also displays other keyword insights, such as where your audience is searching from, what devices they’re using, missed clicks (i.e. where you’re getting impressions but not clicks), and keywords that are trending up or down with your audience. Additionally, we don’t scrape the SERPs to get the data; we pull directly from GSC.
Botify Analytics works hand in hand with Google Analytics, Adobe Analytics, and also AT Internet. We can connect with your site’s analytics account to bring in valuable visits and conversion data to overlay with your Botify Analytics data (crawl data, log file data, and keyword data). This helps you see real trends in how your site’s technical issues are impacting traffic and engagement.
When utilizing all of Botify together, you can uncover big insights that you’ve never been able to see before. Here are some examples:
And with Botify Intelligence and Activation to boot, you’ll be able to make even better use of all that data, surfacing high-impact opportunities faster and automating actions to save you resources.
Now that we’ve covered the main sections of the Botify platform, we think it’s worthwhile to dig a little deeper and talk about some of the features within the platform that make Botify unique.
One of the things that makes Botify stand out is that we take a URL-first approach to your SEO data, rather than a keyword-first approach. We start by giving you a complete picture of every URL on your website (not just desktop, but encompassing things like AMP versions, alternate language versions, mobile versions, etc.) and then layer onto that everything else you’d want to know about each individual URL, giving you a complete picture.
This makes it easy to identify what could be done at each stage of the search process to increase your SEO performance. And, in January 2020, we’re taking this concept even further by launching technologies like ActionBoard which draw from this data and surface the highest-impact tasks for you.
Data layering is key to uncovering the “why” behind your data. For example, when your top-performing URLs (a.k.a. pages that get the most organic search impressions and traffic and rank for the most queries), top strategic URLs (those that make your business the most money or support strategic initiatives), or other critical URLs aren’t ranking, it’s time to investigate.
To do so, you’ll need to layer your log file analysis data on top of a given URL to see if and how Googlebot is crawling it. If the page isn’t getting crawled, you can layer the data from your Botify crawl (in Botify Analytics) to see if there’s a correlation between low crawl rate and low content uniqueness (for example).
You can also pair log file analysis data with traffic data, rank data with crawl data, and the list goes on.
Additionally, you can use segmentation to isolate parts of your site for analysis. For example, you can look at pages by author, pages by topic, or all pages on the blog. Looking at specific sections of your site in isolation allows you to identify issues that might be affecting only those pages and to make adjustments accordingly.
Using segments and data layering opens up opportunities to see correlations you would have had difficulty seeing otherwise.
With so many tools and resources available, it’s not uncommon for SEOs to spend much of their time exporting data and translating it into reports. But when your data lives in one place, everything gets a whole lot better!
With Botify, there’s no longer a need to export your data. You can pull data from Botify Analytics, Botify Keywords, and Botify Log Analyzer into our Data Station, where you can view your report directly or download it if needed. We also have APIs that tie into Google Sheets, Google Data Studio, or your CMS, and we have a Botify Chrome Extension. Plus, if your business uses Looker as its business intelligence dashboard, we have good news. We recently partnered with Looker so you can view your SEO data alongside key business metrics through our custom Botify Looker Block®.
Because we build Botify directly in response to the problems our enterprise SEO customers are facing, Botify also offers solutions such as:
Our customers have websites with millions (or billions) of pages, which makes their SEO needs rather unique. Due to our fast-changing industry and appetite for satisfying our clients’ needs, we’re always releasing new updates. Whether this means developing backend improvements, like crawl speed or JavaScript rendering capabilities, or UI/UX improvements, we typically introduce new functionalities every 3-4 weeks. Take a look at some of the product updates we’re launching in early 2020 — from urgency alerts to instantaneous indexing!
Our customers can tell that we’ve designed everything for their needs (for scale, speed, granularity of data, security, API features, and more), and we’re excited to bring them the best solutions for their unique challenges.
If you’re curious to know more about how our customers find value in Botify, you can take a look at our customer reviews on G2!
Using Botify will help you identify where that potential lies. Expect to be able to save a lot of time and energy uncovering opportunities you may not have seen previously — and fast!
