Top 5 Demystifying Tips for Using Segments in Botify

10th December 2019Timothy Zhang

Segmentation is a top reason why our customers choose Botify. Launched in 2014, it’s the key to unlocking greater insights for your SEO. When you set up your segmentation, you’re essentially grouping your URLs into lists you want to pay attention to. For example, you might want to keep an eye on pages written by a specific author or compare your year-over-year trends for one of your news verticals.

Segmentation can be used throughout Botify Analytics, Keywords, and Log File Analyzer, making it easy for you to layer key data onto the pages you care about most. It’s the core of what makes our platform tick, driving results through granular, actionable insights.

You can uncover a huge amount of information with segmentation, and there are many different rules you can set in place. As you can imagine, it might seem a bit overwhelming at first. But trust us, the outcome is worth it!

Here’s how to make the most of segmentation and come out on top!

1. First thing’s first: Set up your segments!

The first step you should take when you start using Botify is to set up your unique segments. Essentially, a “segment” means “a set of rules.” In Botify, you can isolate sections of your website according to “rules” (or segments) that you create.

For example, if you have a sports news site and want to take a look at all of the “basketball” pages on your site, you can set up a segment to select all URLs with the word “basketball” located in the path of the URL. This allows you to isolate all of your basketball pages (according to a category segment) to see how they’re performing.

You can also use segment types like:

• Static URLs
• Pagetype (which can include segments for category pages, product pages, blog pages, navigation pages, etc.)
• Content type (like basketball, football, etc.)
• Category type (like electronics URLs vs. furniture URLs), and so on.

You can even create your own segment types to satisfy your custom use cases.

Bonus tip: Because each URL can only be in a set of segment rules once, you need to have multiple types of segment rules if you want to look at a specific URL in different ways.

Since your website comprises millions (or billions) of pages with a highly unique structure, it’s integral to get your segments set up as soon as possible. While one section of your site might be performing successfully, another part might be suffering. It can be extremely difficult to identify an issue that exists in one section of your site when you’re looking at your pages as a whole. That’s where segmentation comes in, enabling you to split up your site into digestible chunks for a more granular view.

Here’s how to get started with creating segmentation for your site. In Botify, find the Segment Editor (it’s pie-shaped!) on the left-hand side. You can set up segments to test by protocol (HTTP vs. HTTPS), by domain (or subdomain), by path (the line of text following the domain), query string (a.k.a. Parameters, or the line of text following the “?” in the URL string), and more. Your Client Delivery Manager can help you get started.

2. Get granular with logical operators, Regex, and warnings

Using logical operators

When setting up your segments, you’ll likely want to implement logical operators (like AND, OR, or NOT). These enable you to combine or exclude rules when creating a segment. For example, you can find all of your site’s URLs that have “basketball” in the URL path AND “2019.” Or, you can find URLs that have “basketball” but NOT “2017” (meaning you’re looking at all other years excluding 2017).

Bonus tip: Keep in mind that when layering segments, the order matters. URLs will be captured by the first segment rule that they match. Having more general rules above more specific rules will prevent those more specific rules from capturing a lot of URLs.

About Regex

You can also use Regex (Regular Expressions) to isolate URLs with a specific set of characters in a URL string, like capturing all URLs with “-id” at the end or to capture URLs with a letter occuring a particular number of times in a URL string. Regex helps you to hone in on more specific sets of URLs that might not fit into a larger, pre-existing category.

Bonus tip: If the URLs you want to look at don’t follow a unique URL pattern (as in no query strings/parameters), you can download the URLs as a CSV and paste them into Botify. Then, you can create a blanket rule for those URLs to group them into a particular segment. Use the # symbol at the beginning of the line to add comments about the URL group (ex: #These are no query rules).

Using warnings

If your site has issues with crawl budget, you can leverage warning segments to help you keep an eye on bot crawls on specific URLs that could pose a problem. For example, warning segments can highlight URLs with capitalization or strong characters. Malformed URLs can take away from your crawl budget and impact page duplication.

3. Start simple and experiment

The key to segments is to start simple. It’s best to create segments, test them, and then add on more rules as needed. You can then iterate on your segments by adding sub-segments, implementing logic rules, or assigning specific URLs to be included in the segment (if they weren’t automatically selected).

Bonus tip: You have the capability of testing if a particular URL would be captured by a segment rule. You can do so by using URL filtering or by pasting a list of URLs to test in the segments editor. You can also test your segments against different URL samples.

4. Consider your business goals

With all of your data, it’s integral to consider your KPIs and business goals. What are the most critical pages on your site? Which pages are your top revenue generators and your top converters? You can uncover this information in Botify Analytics and set up segments to isolate and more easily track these pages.

Leverage different kinds of segments with different goals in mind — like isolating your top performing pages, identifying technical issues with a specific pagetype that could be impacting conversion rate/organic search performance, or dividing up a site by category to boost e-commerce performance.

Keywords by segments

Additionally, it’s useful to layer your segments, like pagetype or content type, onto your winning and losing data dimensions (found in Keyword Explorer). It can be very helpful to hone in on a given dimension value (ex: a given keyword where there is a significant loss in traffic) across your winning and losing keywords to understand current and past trends. You can do exactly that by clicking on the trend icon (squared in red), in the Winning Data Dimensions or Losing Data Dimensions charts. You can also use the graphs to filter by content size (thin content in losing dimensions, for example) or inlinks by segment.

Log Analyzer by segments

While you’re in Log Analyzer, you can click on the segments tab and filter by crawl and visits volume by segment. This will help you analyze your crawl volume for specific sections of your site.

Utilizing and building upon segmentation of your data will help your team uncover issues that may only be affecting some parts of your website. Here’s a real example. One of our publisher customers experienced a short-term boost in traffic followed by a steep decline in traffic. A recent news event was responsible for the boost, and when they isolated their pages by content, they were able to see that their evergreen content was suffering. The only way to identify the problem was by splitting both types of content through segmentation.

5. Use segmentation to power your reports

One of the best parts of Botify is that you have all of your data in one place. This makes it super easy to create reports that pull data according to your predefined segments. Showing your team the URLs crawled by pagetype, for example, will make it easier to communicate which pages need to be prioritized to improve your crawl budget. Identifying search bot crawl activity on low value segments can make it easier to identify which pages need to be disallowed via robots.txt in order to improve crawl budget.

Leveraging category segments in conjunction with Botify Keywords can be a great way to dig into organic search performance on a category level – which categories are performing better and should you build content around those high performers?

Bonus tip: Use the filter button at the top of the “insights” section or the filter section of URL explorer to filter by pagetypes, categories, etc. and produce reports that follow the designated segment. This will make the data on your reports follow that particular segment.

Segmentation creates opportunities

Segmentation may be complex, but it’s the key to unlocking true potential for enterprise sites. When you’re able to look at your data more granularly, it’s easier to navigate through issues affecting your site (or parts of your site) and take action only on the pages that need it.

Plus, we’re here to help! If you’re a customer, your Client Delivery Manager can walk you through the process and help you set up segments of your own. Not a customer, but interested in learning more? Click here to reserve a demo.