Count on me: an introduction to web analytics

Internet Works, Autumn 2004. Dead links removed February 2019.

The days of boasting about the number of hits a website gets are over. The more pictures on a webpage, the more hits it takes to download it. The site will be slower and users will be annoyed. What's to boast about?

Today, the web metrics industry is about gathering information that can support business decisions. It's about using analytics tools to work out the return on investment of a particular pay-per-click campaign, or working out where the most profitable visitors can be found. It's about understanding how the website is used and what might be stopping people from buying.

Big businesses hire statistics boffins to pore over their figures and invest heavily in developing applications in-house. MSN.co.uk has worked with analytics provider Touch Clarity on a number of projects and has developed its own in-house tool to deliver charts that can support day to day decision making. Some of the company's key metrics are the number of unique users in each area of the site, the amount of time spent on the site and the clickthrough rate on headlines.

"The key thing for us is to measure how engaged users are," says MSN.co.uk portal manager Louise Brown. "We compare percentage clickthrough rates on headlines in similar positions in different content sections. Fifteen minutes after publishing we can see if we're getting the clickthrough we should be and can tweak headlines."

"Our understanding [as an industry] of metrics on the web is getting much more sophisticated," she says. "Previously we relied too much on page views or visitors. It took a long time for people to start looking at patterns of use on the site."

It's not just big companies that need to measure website performance. "Web analytics is important to any business of any size who spends any amount of money on marketing on the Internet," says Eric Peterson, author of the book Web Analytics Demystified. "Measuring the effect of your marketing spend is just as important online as it is offline. Fortunately, while the processes for measuring online are slightly less well defined than offline, there is much more data and the data is better in general."

Using server logs

The web analytics industry has grown out of server log analysis and it's where most businesses start, if only because log analysis tools are often bundled free with hosting. You've got ready access to the data, the tools are easy to implement and you can measure traffic from spiders as well as people.

A lot of webmasters try to read more into the stats than might really be there

Bradford Barrett is the author of Webalizer, a free log analysis program. "It appears a lot of webmasters are also marketing types, so they try to read more into the stats than might really be there," he says. "There are two problems concerning log file analysis. The first is that the http protocol is stateless. You connect, make a request, get a response, then disconnect. The second is that there is no way to determine individual users, only unique IP addresses that make the requests."

Companies often connect their computers to the internet through proxy servers, so two employees visiting your site at once would share an IP address and look like one person. AOL has a proxy server system, which means a single user could appear as multiple IP addresses or multiple visitors could appear to be just one.

Although most web log programs include metrics for the number of visits, visit length and entry and exit pages, they're only guesstimates. If someone comes to your site, goes elsewhere and then comes back to your site your log analyser might report one long visit instead of two shorter ones. The reporting software can be configured with a maximum visit length so it doesn't look like regular visitors are always there, but a single visit that lasts longer than the maximum visit length will look like two visits. Repeat visits within the maximum visit time only count as one.

ClickTracks is a commercial provider of web analytics, including log file analysis software. "IP alone cannot identify users," says ClickTracks CEO John Marshall. "We use IP, useragent [browser], time, referrer and persistent cookie (if available). This goes into a heuristics bucket that yields about 97-99% accuracy. It's variable because if the site doesn't set a cookie and has lots of AOL visitors, accuracy drops."

Fancy a cookie?

Cookies are small text files a website can leave on a user's computer so that the site can identify visitors as they travel through the site and return on other days. Unless you can force visitors to register and log in, they're the most accurate way of tracking visitors we have today. But a study by analytics provider Redeye found that cookies could still overestimate unique visitors by 2.3 times. This compares favourably with IP addresses overestimating unique visitors by 7.6 times.

"People talk about cookies as if they're individuals," says Conrad Bennett, technical services manager at veteran web analytics vendor Webtrends. "A user running multiple browsers or accessing the site from multiple machines will have different cookies. In general, most people don't do that so the metrics are relatively accurate. Another risk is that there is more software out there deletes cookies regularly. Usually it's pop-up blockers or personal firewalls that are there to do something else."

Using Javascript page tags

Web logs can't accurately count people viewing pages stored in their browser cache or the ISP's cache. A solution now commonly used is to embed some Javascript code into the webpage. This enables information about the browser to be sent back to the server and a cookie to be served that tracks the visitor. Each time the webpage is accessed by a real person - not a search engine spider - the script is executed and the traffic logs are updated.

Although the Javascript needs to be added to every webpage that you plan to track, it's a low maintenance solution after implementation. The technology is usually provided as part of a hosted solution and website owners can access real time reports online without needing any additional hardware or software in-house.

Picking the right technology

This table compares the pros and cons of tag solutions, compared to using web log analysis.

Avoiding analysis paralysis

One drawback of working with hosted solutions is that everyone gets the same interface, and that means a bewildering array of reports so there's something to suit every business. "When you first see the top paths through the site that seems to be useful," says John Woods, CTO at analytics provider Site Intelligence. "By the time you're down to the 50th path, you've got people who clicked on home, then help, then home again. You're swamped in data."

There's a risk of 'analysis paralysis' setting in if you start from the reports and then try to find information you can use. The secret is to work out what you need to know, find the reports that deliver that, and ignore the rest.

Every page on your site should have an identified purpose and a means of measuring whether it's working or not

"Every page on your site should have an identified purpose and a means of measuring whether it's working or not," says Jim Sterne, author of the book 'Web Metrics' and organiser of the Emetrics summit. "Once you have clearly identified and prioritised your goals, you can start to determine which metrics can help you discern whether you are heading in the right direction or not."

John Harrison is CEO at analytics vendor EvisitAnalyst. He advises information sites to use page views and unique visitors as key performance indicators (KPIs). Ecommerce sites should use sales and profit and websites driving enquiries to call centres should measure the number of enquiry emails and calls, he says.

"One can then look at the drivers for these indicators," he says. "For example, profit depends on cost of advertising. So one may want to measure how much profit comes from a particular advertising campaign and work out the return on investment. We also recommend keeping an eye on what customers don't do on your website and why. Which is of more importance to a marketer - a list of items which visitors have bought on your website or a list of items which visitors could not find on your website?"

"ROI is the current hot topic, but we find this a somewhat blunt instrument," says Marshall. "It only shows conversion data, and ignores those visitors that nearly bought but didn't. The single number we like best is 'average time on site' and we break this number out by [search engine advertising] campaign and keyword. Average time on site (ATOS) indicates that people like your site/products. They may then find your return policy is bad during checkout. ROI alone would never indicate this, but ATOS would be high for that particular campaign (perhaps compared to others) so there's nothing wrong with the campaign, even though ROI is poor. Identify the true source of the problem, in other words."

Holiday operator TUI, owners of Thomson among other brands, receives 15% of its business online. "Ultimately, our measure of success is bookings," says head of new media Graham Donoghue. "But you can track that back and find you got an increase in bookings by increasing your marketing, so all you've done is throw more traffic at the site. We used to measure look to book [the proportion of site visitors booking]. Now we measure search to book. It's easy to drive customers to a website, but as we've got towards 300,000 unique visitors a week, you get a lot of dregs coming as well. If 60% of our site visitors do a search, they've shown interest and commitment. Then we need to know what percentage go on to book."

TUI uses Intellitracker and Webtrends for its traffic monitoring. "We believe that none of the web monitoring tools can be 100% accurate, so we use two to benchmark against each other," says Donoghue. "We see about 20% difference in reported unique users to take one measure."

Donoghue monitors reports that show the number of people at each stage of the booking process funnel, from arrival at the homepage, through search and subsequent pages until the final booking confirmation. "A key measure is to keep increasing the funnels in certain areas," he says. "We measure the funnel each week and look at week on week variance."

Soliciting feedback

The technical analysis tools can tell you what visitors did at your site or what they didn't do, but they can't give you any insight into what they were trying to do or how they felt while they were there. Opinionlab has a feedback mechanism which floats in the corner of webpages and invites visitors to grade each page on a five point scale. Visitors can also provide text comments and separate ratings for design and content. There's a free version available at www.dialogcentral.com [Service no longer available].

TUI is an Opinionlab customer. "We'll use the scoring to prioritise pages, but the most important part is the raw customer comment," says Donoghue. "We'll shape the questions in individual pages depending on the type of information we want to find out at that stage."

"Many subscribers integrate traffic, performance and our customer voice metrics," says Opinionlab VP marketing Jay Rudman. "The ultimate vision is to have an executive dashboard that appears for each and every page and would say 'this page is a hotspot' - it's proving challenging to users and it's important because it's high-trafficked. That's not something that exists now unless a client makes it for themselves. Most of the vendors in the web analytics world realise that's the end game and are working towards that."

Free for all

You can get a basic traffic reporting tool or log analysis package for free (for example Sitemeter and Webalizer), but the consultants we spoke to were agreed: you get what you pay for. Jim Sterne says: "The free tools will reveal some very basic data. If you're running a small site on a small budget, these tools will shine a bit of light on your efforts. But if you want true, robust reporting - if you want serious path analysis and conversion ratios - you'll have to open your wallet. Fortunately there are some very reasonable low-cost tools that do quite a lot."

Analytics vendor Webtraffiq recommends businesses allocate about 10% of their marketing budget to web analytics and marketing analysis and suggests SMEs who are new to web analytics start with an off-the-shelf solution costing about £70 a month.

Getting the software is only half the battle though - you'll need a team in place to create actionable changes and turn the headline statistics into actionable website improvements. "Most organisations underestimate the effort they need to put in," says Bennett at Webtrends. "Not in terms of producing or understanding reports, but in terms of using the reports. Whether you spend $500 on a small business version or several thousand dollars for an enterprise version, that will be a small portion of the budget needed to interpret the report and do something with it.

Picking a vendor

The large number of web analytics vendors can be daunting. Setting your budget and asking a few simple questions (see our checklist below) can quickly cut the list down to a manageable size. You can then approach vendors and ask for a trial installation or example reports based on your existing log files.

Lund doesn't believe the technology makes a big difference. "When you read some vendor papers, you get the impression that their approach to analytics (specifically the data collection technique) is the one that produces spectacular results, and that alternative approaches are seriously flawed," says Lund. "The fact of the matter is that one can get credible results with any of the major vendors."

Some of the ASP vendors, including Webtrends, contractually own the data they supply to you. Even in cases where the data can be exported or is owned in-house, site owners can feel locked in to their analytics provider. "For a variety of reasons, the absolute numbers will not be the same between vendors, and even between new versions from the same vendor," says Lund. "This means that historical data will be hard to use for comparison. Also, there is a bit of a learning curve for many folks to 'unlearn' the previous terminology and learn the new package terminology and user interface capabilities."

John Woods at Site Intelligence doesn't agree that migration is hard. "Most of our clients are on their second or third generation system," he says. "If you're not sure what you need, it's better to start with something small and low risk and then trade up to something more capable, than it is to dive straight in and spend megabucks on something you're not ready for."

He estimates that differences in reported traffic of about 20% are possible when solutions are switched between analytics vendors. Most of the discrepancies come from differing ways of calculating the metrics. Working with vendors who comply with standards set down by ABC Electronic can reduce the problems, by consistently eliminating spider traffic from page view totals for example. "The closer you get to business metrics, the less scope there is for interpretation," says Woods. "Robots don't buy things."

To benchmark the difference in traffic registered by the new and old solutions, Woods recommends running the two in parallel. "You don't need a ten year history," he says. "You usually only need three months. The biggest problem is with clients in the travel industry where a long-term history is useful to them. Even repeat buyers might only buy once a year. It's seasonal and a two to three year history can be useful."

Keeping it legal

If your measurement technologies could identify an individual person, you'll need to comply with the Data Protection Act.

There are two principles of the act that are particularly relevant, numbers one and eight. The first concerns the obligation to process data fairly and lawfully, and the eighth says you need permission from people before data about them is transferred to countries that don't have an equivalent data protection regime.

"You need a privacy policy or similar method for telling the user what you intend to do with its data," says Simon Halberstam, partner at specialist IT and ecommerce law firm Weblaw.

The two data protection principles have different requirements for consent. "Whereas in relation to the first principle, it may be sufficient just to notify the individual, with the eighth, a positive indication of consent is required prior to the transfer," says Halberstam.

The Privacy and Electronics Communications (EC Directive) Regulations 2003 prohibits the use of cookies and other tracking devices on websites unless the site visitor is given 'clear and comprehensive information' about their purpose and a chance to opt out. "The government anticipates that a clearly signposted privacy or cookies statement, coupled with either an express opt-out facility or an explanation of how to disable cookies through the web browser, should suffice," says Peter Manches, partner in the technology and media department at Manches Solicitors. "The Interactive Advertising Bureau has produced a series of web pages which explain how cookies work and how they can be managed. The Information Commissioner suggests it will be acceptable for a privacy policy to link directly to these pages, which may be found at www.allaboutcookies.org".

Conclusion

It's important to remember that web analytics is not the end in itself and the idea is to use the information to drive a programme of continuous website improvement. While you can't make a change that will increase sales by 100%, you might be able to make 100 small changes that increase sales by 1%.

Motoring organisation The AA looks at clickstream analysis, sales tracking, customer satisfaction and campaign performance among other things. "Through the analysis of these sources of information in isolation it is possible to make huge improvements in performance, but it is through bringing them together in a holistic view that it becomes possible to make real leaps forward in understanding how customers interact with the site," says The AA's head of ecommerce Susan Brooks. "It is key to focus on transforming simple data into clear, actionable plans for improvement. With this application in mind, it is easier to concentrate on analysis that is useful as opposed to simply interesting. The cycle then begins again as these improvements are measured."

Executive summary

1. It's not 100% accurate. Vendors quote figures as high as 98-99% accuracy, but in some markets a significant portion of your audience can't be accurately tracked. Privacy-conscious users will delete cookies more often and AOL visitors can't be accurately tracked using IP. Understand what you're really measuring and understand its limitations. Don't generalise: Analyse performance at the page level where possible.
2. Use key performance indicators. Work out how best to measure your website's overall performance and use those measurements as a gauge. Investigate underperformance by looking at the metrics of the underlying drivers. If sales drop, for example, check first whether your adverts have been underperforming (fewer visitors) or your site has become less effective (lower conversion rate).
3. Turn the stats into actionable improvements. There's no point in analysing reports for their own sake. Combine feedback from visitors with hard stats on individual pages to help identify problems. Use metrics to drive a programme of continuous website improvement, with each change checked for its effect on the website's performance.