Anyone who has managed a website migration knows what a great source of stress it can be. If not properly anticipated, a migration can put existing traffic at great risk. Not only organic traffic, but traffic from all sources!
For a pain-free, successful migration, don’t leave anything to chance:
Botify’s Logs Analyzer provides actionable data and detailed monitoring along the process.
From an SEO perspective, migration management ideally starts with the new site’s design stage.
First, check that all your website’s important pages will exist on the new website, from a robot’s perspective. Organic traffic can only be preserved if valuable SEO target pages still exist. That may sound trivial, but in some cases it’s not. For instance, some new pages may not exist through their own URL in the new website because they are now generated through Ajax refreshes.
Second, carefully review pages dismissed as ‚Äònot important”. Website owners may be eager to get rid of old content they do not want to promote for understandable reasons, such as information about discontinued products. However, a part of this old content could generate a non-negligible, stable volume of visits (organic as well as referrals, through old links).
These pages could be worth keeping, as time-proven entry points to the website – with easy navigation to new, sexier pages. In many cases, business managers don’t suspect a business choice needs to be made. But SEO managers or traffic managers know better and can bring this to their attention.
The next steps will bring to light such unsuspected valuable pages, if any.
Existing organic traffic will be transferred to the new website, in a durable way, if and only if :
Example of a bulk redirection:
All pages from the old news section are redirected to the new news home.
www.mywebsite.com/news/* ‚Üí www.mywebsite.com/companynews/
Example of one-to-one redirections (which does not mean there has to be as many rules, there can be a single rewriting rule covering many URLs):
www.mywebsite.com/products/productA_num1234.htm ‚Üí www.mywebsite.com/catalog/productA-id1234.html www.mywebsite.com/products/productB_num4567.htm ‚Üí www.mywebsite.com/catalog/productB-id4567.html
A page-to-page permanent redirect will transfer most of the old page’s popularity to the new page, while bulk redirects will have at best a short-lived effect mainly based on redirected crawl volume.
It make sense: a 301 HTTP status code means that the content is not here any more, but it’s not gone, it can now be found there. If a very large number of URLs (URLs of pages that did not have the same content) are redirected to the same URL, it clearly means that the redirects don’t direct to similar content. That’s exactly why Google can consider a large number of 301 redirects to the same page as “soft 404s”: pages that are considered as 404s (page not found) although they do not return a 404 status code.
Ideally, the vast majority of URLs from the old website should be redirected one-to-one, provided that there is content to redirect to. But that probably won’t happen in the real world because of cost and resource constraints – implementation resources have limits, urgent matters always come up, and the result is always a compromise. In all likelihood, only high priority URLs will get one-to-one redirections, and we’ll have to settle for bulk redirections for the rest. This means we have to list valuable URLs (from an SEO perspective) that we will focus on for one-to-one redirections.
How do we make sure no valuable URL falls through the cracks?
Web server logs are a convenient data source: they trace every user visit, every crawl from every robot, every redirection‚Ä¶ Any request the web server ever replies to is saved in these log files.
With the Botify Logs Analyzer, you can easily export SEO active URLs over a given period, for instance the last month or the last 60 days. The SEO active URLs are URLs that generated a least one organic visit over the period. They are listed with the number of visits they each generated, ordered by number of visits.
Any page that brings either traffic or popularity is worth redirecting. External links potentially bring both, and are sometimes overlooked.
Google Webmaster Tools (GWT) provides information on external linking: the ‚ÄòSearch Traffic / Links to Your Site section’ lists top referers (the top 1000 domains with links to your website) , and sample target pages on your website (top pages with counts).
That should be plenty enough to decide which pages need one-to-one redirections to similar content on the new website. Tools specialized in external linking can provide additional insight such as domain value for top referrers, etc.
With top active pages from recent logs, and top external links targets, we cover the most important URLs that need one-to-one redirections from an SEO perspective.
Once all planned redirections are in place in a pre-production environment, it’s time to test them on real data. Urls recently crawled by Google are an ideal test set, as they represent Google’s current view of your website.
Botify’s Logs Analyzer provides the list of URLs crawled by Google.
Select the last 30 or 60 days for instance, and depending on the website’s size, we can use all crawled URLs, or top crawled URLs. Clearly, in the example above, we’ll take 30 days or less, and the 50K to 100K top crawled URLs for instance).
The next step is to crawl these URLs, just like Google would, using the crawler which comes with the Logs Analyzer.
To do so, we will need to :
Then, check the URLs status codes – this is one of the standard outputs provided by Botify.
There should be:
There might still be a bit of work to do after this redirection check, so ideally, this is done a little bit in advance, and not right before going live.
After the big day, it is important to monitor Google’s crawl and organic traffic.
(days with no data correspond to missing logs)
Right after the migration, expected results are :
Over time, we will monitor how fast Google ‚Äòdigests’ the redirections and starts going mainly directly to new pages (which are also crawled directly from the new website). This will result in lower volume of redirected crawl. Redirected URLs with a significant traffic history will still be crawled for a significant length of time, that’s to be expected. Those that receive external links will continue to be crawled as longs as they are linked.
Note: in the Logs Analyzer, we can also zoom in to get the same view by type of page.
The post-migration volume of active pages greatly depends on the number of pages on the new website and the number of one-to-one redirections.
A downward trend is not necessarily a bad thing – provided that visits don’t follow a similar trend.
In this example’s case, the purple pages were voluntarily removed, and the green pages voluntarily closed to robots as they were generating an extremely small number of organic visits, while consuming a significant amount of Google crawl. Visits, as shown below, are going up.
In this case, the migration went smoothly!