Update: this blog post shows a previous version of the Botify interface. See the current version of the URL Explorer in the Q&As or in this video.
The URL Explorer was named modestly. It is so much more than just an explorer. It could have been named URL Metadata Miner, for instance.
What do we mean when we say that you can easily dig into your data? Here is an illustrated answer.
With the URL Explorer, you can explore urls using a multifaceted approach, and you get exactly what you need about those urls. It’s a two-step approach :
This defines how to fill your results table (i.e., the data you can export): the first step selects the lines and the second step selects the columns.
One great feature of the URL Explorer is that you don’t have to enter all this from scratch (although you can if you want to). The initial work is done for you. You can reach the URL Explorer from anywhere in the report, to see data related to an indicator.
Then, you can refine, add, remove information until you get exactly what you want.
And just click to export.
Wherever. When you see something you want to investigate or locate precisely, click on it. For instance:
From a graph, a panel will open up on the right, with details on what you clicked on (a sample, in most cases). As all lists, it includes an ‚Äòexplore all urls’ button.
A click on ‘Explore all urls’ bring you to the URL Explorer, pre-filled with the same data. All of it, this time:
This is only the beginning.
Once in the URL Explorer, you can adjust the selection to your the urls you want to focus on, and get all the information you need about these urls.
Let’s take another example. Imagine that you see this in the ‚Äòinlinks’ section:
And you go to the URL explorer to see these 18K urls that are linked only once. You not only get the url list, but also their http status code and depth distribution, in separate tabs:
This can help narrow down your selection, by providing hints on filters to add. Only deep urls, for instance.
Let’s continue with our example with urls linked only once, and take a look at the http status code distribution tab:
Hover your mouse over the blue area, and you’ll learn that among these 18K urls with a single incoming link, 739 (4%) are redirected.
Of course, you won’t have the same approach for redirected and non-redirected urls:
Let’s work on the 301s. As a next step, you can zoom in on these redirected pages, by adding a filter (http status code = 301). To do so, click on the green ‚Äò&’ to add a filter, select http code in the drop down list, and enter ‚Äò301′ in the value field.
And you can check how many more there are, if you include pages with more than one incoming link: change the filter for incoming links to ‚Äògreater than one’. As it happens, in this example, there are few : only 49.
So it makes sense to start working on the 700+ redirected urls with a single incoming link: it’s probably easier to change (in all likelihood, a small change in one template), and has a higher impact – once you have validated that the pages they redirect to are important pages.
Now that you have narrowed down the set of urls you want to work on, you need to get additional information about these pages:
‚Üí add a column for ‚Äònumber of incoming links’ (all of them this time, not the ‚Äòfollow’ type only, as you’ll want to remove this url completely from the website),
‚Üí add a column for ‚Äòredirects to’.
To do so, click in the empty area in the ‚ÄòFields to display’ and select the new column in the drop down list, or start typing to narrow down the choice.
Then click on ‚ÄòApply’ and you’ll get this:
All listed urls are cliquable, if you want to see indicators on any of them. So you’ll find out where the redirected urls are linked from.
This was a simple example, but you can go very far in the selection of urls, as you can combine filters with ‚Äò&’ and ‚Äòor’.
Once you have the table with all the information you need, click on ‚ÄòExport as CSV’. The size limit for exports is currently set to 100 000 lines. But it’s easy to reduce the size of the table with an additional filter. That way you can divide the larger table into subsets that make some sense – instead of just getting the first 100 000. Anyway, this export size limit should be removed soon!
After downloading the flat file, import it in your favorite spreadsheet software and choose a coma-separated format in the import wizard. The first line will be a header.
You’re all set to start removing problems on your website!