How To Use Google Analytics to Look for 404-Errors on Your Site

Just recently I was slightly involved in the migration of a site. The migration was somewhat successful, although a lot of planning and check lists were lagging. Although I have not been involved in a lot of migrations of websites, I know what to look out for. Unfortunately I was only consulted in a late stage of the plan, and with my current responsibilities, I did not have more time to invest in the project. I guess the team had to learn it the hard way…

One new learning for myself is how to use Google Analytics Advanced segments to setup alerts for 404-error messages. The 404 ‘Page not found’ error message is the nightmare for any project lead in a site migration. The 404 means you did a particular bad job in making sure the user experience was not impacted by the migration.

So if you see something like this in your web analytics, it means you either fix the issue real fast, or start updating your resume:

Page not found 404 error #FAIL

And although it does not look that bad, only 2.6% of page impression throw in a 404 in this report, the details speak louder than words. The report above here is on a full 30 days time frame. If you isolate the days after the site migration, the error rate is on average a whooping 12%!!! WOW, talking about a bad user experience here!

Setting up Advanced Segments to track the 404 Page not Found error message

The Google Advanced segments can come in handy here to track on a daily basis how many page impressions resulted in a 404-error. This can help identifying the problem, alert you when something is going terribly wrong, and maybe most important: collect enough data and reports to hold everybody accountable for their responsibilities. Data doesn’t lie :)

In Google Analytics simply open up a new custom advanced segment to track specific pages with the title of the 404-error page you have set up on your site. This will allow you to track each functionality within Google Analytics against this specific segment. And it’s so easy to set up:

  1. Go to advanced segments
  2. Create a new segment
  3. Under Dimensions, open the Content drop down
  4. Drag Page Title to the box: dimension or metric
  5. Now type in the title of your 404-error page in the Value box
  6. Give the segment a name

image

In all the dashboards of the reports within Google Analytics you can simply choose the advanced segment as a reporting layer. In the image below you can see the standard dashboard with the traffic on the 404-error page reported as part of the total. The red –arrow and highlighted box points out where to select which advanced segment you can choose.

image

It’s fairly easy to screen which URL’s are throwing a 404 error if you know your way around in Google Analytics. Just select the Content by Title under the Content tab in the left navigation menu (Red box). Then select the 404-error page for a detailed report. The icon in front of every page which is being reported is the URL you would need to fix (here highlighted in the Orange Box).

image

These are all the URL’s on your site which throw a bad user experience for the people that try to access it. All these people will see the following message. And although their experience has already been ruined, just make sure you learn from your mistakes and fix the issues!

image

The possibilities are endless. You can slice and dice the date to see which referral sources have caused the 404-error. So if the referral is coming from a non-search engine, you even are able to harvest link juice which would have be gone forever if you didn’t fix the issue.

Or you can look for referral keywords which get a 404-error. I bet you will see high rankings on  these keywords. If you don’t take action, you will loose these rankings, and any potential traffic associated with it.

Hack It, Break It, but always Fix It

Using Analytics to Improve Your Writing

Using Google Analytics can be a real time saver if you would like to do some advanced research on your website traffic or how you can expand the number of visitors. I will explain a number ways how I use Google Analytics to research next steps on how to grow the traffic to my websites by writing better content.

For the research cases below and in later posts, I’m using Google Analytics with specific advanced segments set up. In the examples I’m not only explaining how to use the advanced segments to do the analysis, but I will also show you how to set these up.

Old posts can drive a lot of traffic

Do you have some great ranking posts which drive most of your traffic. You either spent a lot of time researching a specific subject and wrote a valuable post on it, or you just got lucky hitting an interesting point which resonated with a lot of people. Now, think about when did you write this great piece or lucky shot of content? Wouldn’t you want to know which posts over the year are driving the most people to your site, and more importantly, are you improving in the quality you publish?

The advanced segment setup I’m about to show you is only possible when you have date based URL structure. On my Dutch blog TheNextCorner.com I have such a URL structure, which I at first thought was a big pain in the ass, now I wished I had it setup like that on this site! 

Below you see a 3 year graph of the visitors to my Dutch blog on a monthly basis. As you can see, I have a steady growth of visits to my site, with here and there some peaks or decline.

image

From the image above I can not see which posts of which year are driving the most traffic. I would like to know specifically what the posts from 2007, 2008 and 2009 contributed to the overall traffic on the site.

After setting up some advanced segments that will apply certain filters based on the year in the URL, I can look at the contribution of the individual posts from each year as part of the total traffic.

image

Setting Up Year Specific Post Reporting

Having the year and month as part of your URL structure is, like I said earlier, a must have or the set up of the advanced segments for this reporting. Currently my Dutch blog has this URL structure set up for a long period of time already.

i.e. http://www.thenextcorner.com/2010/01/el-nino-klopt-aan-de-deur.html

For some tips on how to set up custom advanced segments in Google Analytics, I posted a number of relevant resources at the bottom of this post.

If you would like to set up an advanced segment which tracks all the traffic for posts which were submitted in the year 2007, I would go into the Advanced segment editor, and click on the link, create new segment. Here I choose from the menu on the left side: Dimensions> Content> Page. I drag the Page box to the dimensions or metric box in the middle.

image

After this, all you have to do is make sure the condition mapping is on Matches exactly. To determine your new advanced segment will only report the metrics for the posts that have the year 2007 in the URL, you input the following into the value box: /2007/

image

Give your segment a name, and your done.

Pulling reports on year of post

Now that you have set up a couple of advanced segments, you can actually drill down in every report you have in the standard view of Analytics on those advanced segments you have set up; in this case the posts written in a certain year.

image

Just use your pull down menu above the dates to check the segment you would like to include in the report.  In a next posts I will go deeper into the neat things you can do using these yearly posts segments setup.

Resources

If you don’t have experience setting up advanced segments in Google Analytics, I can recommend you read the following posts:

Some real advanced set up with Advanced segments by André Scholten (sorry in Dutch):

Analytics 101 – Best Post Per Year

The Permalink structure from your blog does matter as it can help you making your life easier pulling the right stats out of Google Analytics using the Advanced segments functionality. Let me explain with the example of my Dutch blog: The Next Corner.

The URL structure of my blog, which is still running on typepad where I have plans for a long time already to migrate to WordPress, has the year and month in the URL structure as directories. Just check out one or two of the URL’s of my site:

  1. http://www.thenextcorner.com/2007/01/trojan_horse_ve.html
  2. http://www.thenextcorner.com/2009/09/gratis-virus-verwijderen.html

The two URL’s displayed above give the year and month in which these were published.

Not coincidence I use these two, as these will be used in further analysis in some later posts. Just make sure you either sign up for the RSS feed to follow the posts here, or follow me on @Thenextcorner.

The year which is displayed in the URL can be used for advanced reporting. If you have this kind of URL setup, I will show you in this post how to set up custom segments to report on exactly the traffic which came from which year/month using advanced segments in Google Analytics.

More insights from Analytics

To track which posts are delivering the most traffic to your site, you can use the advanced segments function to drill down even further down. By splitting up the total traffic reported into the years it was created, you can gain insightful knowledge and learning's how your writing and choice of subjects have evolved.

All we be more clear as soon as i show you the results of the advanced segments I have set up. First, let me show you the total visitors numbers graphs on my Dutch blog sine Jan 1st 2007, until Dec 31st 2009, which is three years. Graph is by month:

image

Wouldn’t you like to see in one graph which posts are delivering the most traffic? Wouldn’t you like to know if your writing is improving through engaging your readers more? Wouldn’t you like to know how much traffic content can drive if it matures? Just keep on reading and you will see…

Setting up Advanced Segments

First step is to set up the advanced segments in Google Analytics to report on. This relies off course on the right URL structure for your site, as you will need to have the year/month displayed in your URL!

Navigate to the Advanced Segments link in the left side navigation menu in your Google Analytics account. Click on the link at the right top, +Create new custom segment.

Here you drag the dimension Page into the first box. After that you choose from the drop down the  Matches regular expression choice. The last set up is to type the year you would like to report on in the value box, as shown here in the picture below /2007/, which will report all posts from the year 2007, as these have the /2007/ in the URL defined as a directory.

image

The longer your blog is in business, the more years you would have to set up as user custom segments.

Reporting on Year posts

The reporting on posts per year becomes very easy after you have set up all the custom segments. All you have to do is click the drop down of the segments, and check the boxes of those years you would like the posts to be reporting on:

image

After you have clicked the years, you can now see which year you have been cranking out the most interesting blog posts, or which content has matured in a nice SEO friendly way that drives traffic. Here is my graph for the last three years:

image

Cleary, my best posts are from 2007.

Now, I just have one post from 2007 that is bringing in most of the traffic on my Dutch blog, but I’m sure you can see the value of playing around with the Advanced Segments in Google Analytics to find new ways of slicing and dicing the data which is available.

It makes me wonder how my 2008 & 2009 content will do in a couple of years, as it matures the way the 2007 content has matured. I still doubt that the 2008 nor the 2009 will bring in the same amount of traffic as 2007 posts are doing at the moment. I just had some lucky shots in 2007 which are still ranking #1 for reasonable popular keywords.

More on Analytics later!