Inside the Botify MCP: The Agents, the Data, and What You Can Do With Them

min read

September 24, 2026

Thomas Grange

Chief Innovation Officer

Three years ago, we connected Botify's database to ChatGPT for the first time. The results were both promising and frustrating. The model could see our data, but it had been trained to answer queries as fast as possible. If you asked it something difficult, it would quickly review the question, retrieve the most accessible potential answer, and then stop. Not very useful, but even then, we were convinced that agents would become the future of search and marketing work. What we were missing was a model that was willing to keep going to find the best possible answer.

Now here we are, just a few years later, with today's frontier models that are trained to match their reasoning to the complexity of the prompt. We can send them on long investigations across a site's entire collection of data, including crawl, server logs, AI visibility, Search Console, visits, sitemaps, products feeds, or your own uploaded data, and let them reconcile it all — work even a veteran SEO manager would struggle to do manually.

In our announcement last week about the Botify MCP becoming generally available, our CEO, Adrien Menard, shared what this milestone means for your organization. Today, we’ll go under the hood, and show you what the MCP actually gives your agents, why the full data chain makes it work, and what you can do with it, agent by agent.

What the Botify MCP gives your assistants

MCP is the open standard that lets an AI assistant plug into outside software. You can connect Claude, ChatGPT, Cursor, Dust, or your own MCP-compatible agent to Botify. Your assistant then gets a set of agents that do three things:

Query your Botify data model: This covers everything from crawl data, server logs, Search Console and Google Analytics, Adobe Analytics, AI Visibility and GEO, sitemaps, product feeds, and PageWorkers and SpeedWorkers delivery data.
Read HTML and JavaScript: The agents can read pages on your site or outside it, including the JavaScript-rendered version, so the assistant sees what's actually on the page and not just what a report summarizes.
Draft optimizations and push them live in the platform: This turns findings into on-page changes ready to deploy.

You can use this in two ways. The first is interactive: audits and investigations you run in conversation. The second is scheduled: AI workflows that analyze your data and draft page updates that you can publish on your website without submitting a ticket and waiting on engineering teams.

Why your data makes the difference

For 15 years, we've offered a unique benefit to our customers across our suite of solutions: a unified data model. Your crawl data, site logs, visits, Search Console data, and now AI visibility metrics all live in one place rather than fragmented between multiple dashboards and tools.

In practice, though, the difference between simply having the data and using that data still took some expertise. You had to know which table held what, how to join log data to crawl segments, and which Search Console dimensions lined up with which templates.

The Botify MCP removes that bottleneck, letting an LLM navigate every source on its own. It can discover the schema and write the queries, and follow the evidence from one dataset to the next, without anyone needing to know the internal table names.

This matters because correlation and root cause analysis depend on your full chain of data. Botify is the only AI discoverability and agentic commerce platform that can join a site's entire HTML inventory with log, visit, Search Console, and AI visibility data. Other tools cover a slice of that chain, which can show you a symptom of a bigger problem, but it can't show you the cause.

Take, for example, experiencing a drop in organic visits to a product category. With just a slice of the data, you can see that the drop is happening, but you can’t see why, and you have to start making guesses. With the full chain of data, your assistant can trace the issue step by step:

Search Console shows impressions falling for the category's core queries.
Your logs show Googlebot visiting those URLs less often, starting two weeks earlier.
The crawled HTML shows that product listings stopped appearing in the JavaScript-rendered page after a template change.

Without every piece of the puzzle, the diagnosis will be incomplete at best — and wrong at worst. With a complete view of your first-party data and supplemental third-party data, your assistant can identify the issue and determine the right next step in a single conversation.

What you can do, agent by agent

Below is a video showing you how you can use it to analyze and fix internal links and some of the agents available through the Botify MCP today, organized by the jobs they perform. You don’t need to reference them by name — simply ask your question in plain language, and your assistant will select the right agents, often combining several to deliver the answer.

‍

Discover & diagnose

What it does	Example question a user types	Data it draws on
tables_query and their other companions (tables_list_condensed, tables_schema…)
Runs a SQL query against the customer’s Botify data model and returns rows. Customers can slice crawls, logs, Search Console, AI visibility, sitemaps, product feeds, and PageWorkers/SpeedWorkers delivery logs — without knowing internal BigQuery names.	"Which of my product pages returned a 4xx or 5xx in the latest full crawl, and how many Googlebot hits did they get last month?"	Project data model: SiteCrawler (and AlertPanel) crawl tables, server logs (Google, Bing, AI bots…), Search Console, AI Visibility, sitemaps, Agentic Catalog product feeds, PageWorkers / SpeedWorkers logs
html_grep, html_structured_data
Fetches the HTML of a URL so the assistant can read the actual page. Prefers the latest Botify crawl copy; can fall back to a live fetch or a JS-rendered snapshot. Can return text-only or a cleaned body.	"Show me the crawled HTML of https://www.example.com/p/5866-toothbrush — title, H1, and main content only."	SiteCrawler cached HTML; optional live fetch; optional JS-rendered snapshot (rendering farm). Restricted to the project’s allowed domains.
html_code_executor
Runs a small JavaScript function on crawled (or uploaded) HTML to extract structured fields at scale — title, price, FAQ, JSON-LD, etc. Can also check the result against quality-control rules.	"On these 50 product URLs, extract product name, price, and availability, and flag any page missing a price."	SiteCrawler cached HTML (latest successful crawl by default), or a temporary uploaded HTML file

Analyze & benchmark

What it does	Example question a user types	Data it draws on
google_trends
Compares search interest for up to 5 keywords over the last 12 months: trend line, related topics, and related queries. Useful for seasonality, launch timing, and “is this topic growing?”	"Compare search interest for ‘electric toothbrush’ vs ‘water flosser’ over the last year — is either seasonal?"	Google Trends (interest over time, related topics, related queries)
nearest_page_for_topic
Finds the customer’s own pages that Google already associates with a topic, then scores how well each one matches. Helps spot the right internal-link target or a content gap (“we have nothing close to this query”).	"Which of my pages is closest to ‘electric toothbrush’, and how relevant is it?"	Google Search restricted to the project’s domain (site:…) + an LLM relevance score on title / description / URL

Act & optimize

What it does	Example question a user types	Data it draws on
pageworkers_generic_urls_actions
Pushes draft optimizations into PageWorkers: new title, meta description, H1, or up to 10 custom DOM edits (prepend / append / replace a CSS-selected element). Creates the shared “[locked] Global Metadata” optimization if it doesn’t exist yet.	"Update https://www.example.com/p/5866-toothbrush: set the title to ‘The Best Electric Toothbrush’, the description to a new snippet, and prepend ‘NEW! ’ to the product name."	Botify Activation / PageWorkers (existing module values + the URLs the customer names)
pageworkers_get_or_create_optimization
Finds a PageWorkers optimization by name, or creates a new optimization (actions, data schema, optional seed URLs, optional split test). Used to set up a reusable “recipe” (FAQ block, structured data, etc.) before filling it with URLs.	"Create a PageWorkers optimization named ‘FAQ Section’ that inserts FAQ HTML after #description."	Botify Activation / PageWorkers modules for the project website
pageworkers_push_items_to_optimization
Takes a SQL query, turns the result into a CSV, and uploads it as the dataset of an existing PageWorkers optimization — so hundreds or thousands of URLs get the right	"Take every in-stock product URL from the latest crawl and push title + FAQ fields into the ‘FAQ Section’ optimization."	Botify data model + PageWorkers Activation upload
html_pageworkers_preview
Lets customers try a selector, extract, or Nunjucks/Liquid/HTML template on a real page before deploying it. Shows which elements match, what would be extracted, and what the rendered snippet would look like.	"On https://www.example.com/p/5866-toothbrush, does h1.product-title match, and what would this FAQ template render to?"	Cached crawl HTML (SiteCrawler), with a live-page fallback

‍

The last group of agents is what separates the Botify MCP from other connectors that can only report, not act.

For example, here's a realistic workflow that you can have all in one conversation using Botify MCP: You ask your assistant to find in-stock product pages that are missing FAQ content, and then add FAQs to them. It then works through six steps:

It queries the data model to build the URL list.
It extracts the relevant product details from the crawled HTML.
It runs an analysis of your keyword and GEO performance to find content opportunities.
It tests an FAQ template on a real page with the preview agent.
It creates a PageWorkers optimization.
It pushes the full dataset into that optimization.

What would have taken at least one ticket, multiple spreadsheets with matching errors, and likely a sprint can happen in a single session.

How teams are using it today

Teams are already putting the Botify MCP to work in a few ways:

Full audits at the end of every crawl: When a crawl completes, the assistant generates a complete audit, so findings are ready as soon as the data is.
Weak-signal detection on Google Search Console data: The assistant spots pages starting to trend up, pages losing momentum, and product seasonality beginning to shift.
Full product feed audits: The assistant runs a complete audit of the product feed, drawing on the AgenticCatalog product feed data in the Botify data model.
Root cause analysis: The assistant works out why load times have increased, why content has dropped out of the JavaScript rendering, or why visits spiked or fell on one specific day.

Each of these use cases is the kind of long, multi-source investigation we had in mind when we first connected our data to an AI model three years ago, and today teams can actively run them without piecing the data together themselves.

Get connected with the Botify MCP

Connecting the Botify MCP takes minutes, and an admin has to add it only once. They can search for Botify in the AI tool's connector catalog, or add it as a custom connector at https://mcp.botify.com/.

Each person can then sign in with their own Botify account, and existing permissions carry over, so everyone can see only the projects they could already access in Botify.

Once connected, it works with the assistants you already use, including Claude, ChatGPT, Cursor, Dust, and any custom agent built on the MCP spec. To get answers, you need at least one crawl, and log questions need log data integrated with your project.

The full setup guide walks you through the process. If you joined our tech preview, follow these steps to migrate to the new connection.

Ready to test out everything the Botify MCP can do, or have questions? Request a demo or talk to your account manager today.

‍

Want to learn more? Connect with our team for a Botify demo!

Get in touch