Alerting & Monitoring Search Engine Crawl

Reacting to Crawl Errors

When Google crawls errors, you need to act fast. Get alerts to malformed pages or critical errors before they impact organic traffic.

Decreases in Crawl

Sudden drop in crawl on important page is a sign that something is wrong. Catch errors and quickly make changes to bring back crawl.

Monitoring Migration

Keep a close eye on search engine behavior throughout the lifecycle of your site, from small changes to full migrations.

Monitor changes in search engine behavior as it happens

Segments

Segments

HTTP Status Code

HTTP Status Code

Alerting Crawl

Alerting Crawl

New Pages Found

New Pages Found

Alerts when time is of the essence

When search engines are crawling pages that return errors, alerts help you catch problems and address them before they impact organic traffic. Set up customized alerting for specific pages you want to monitor crawl on.

  • Lost pages
  • New errors happening suddenly
  • Pages found you don't want found
  • Pages that aren’t being found that should be

Crawl volume by segment

Is crawl dropping suddenly in certain areas of your site? Has just the mobile bot suddenly stopped crawling? Has Google discovered new pages because of a change made on site, new errors, or malformed URLs?

Find out and identify the source of the problem so your pages can be crawled and start (or continue) receiving organic traffic.

Alert to changes in search engine crawl

Flag a segment or subsection of your website to be alerted to significant changes in how search engines behave on those pages. Get notified immediately when something goes wrong on strategic pages.

Monitor crawl volume, number of pages crawled, and URL-level details like HTTP status codes, combined with data on Lost and Recovered pages to identify and correct issues affecting crawl.

Monitor website migrations

Keep an eye on search engine bot activity on your site through every stage of your migration. Use log analysis in pre-planning, and then watch for the expected pattern of redirects as Google discovers your linking and crawl increases on new pages.

Monitor website migrations