How unique is a unique visitor?

It is possible that your unique visitor counts are two to four times too high

I access the Web each day from at least seven browsers; Chrome on my Macbook pro, Firefox on my Macbook pro, Chrome on my windows desktop in the office, Firefox on our "kitchen laptop", Safari on our "kitchen laptop", The Android browser on my Google phone, The Blackberry browser on my Blackberry 9700.

I know that I am not your typical Web user. I keep both Firefox and Chrome open at the same time on my main machine (my Macbook pro). I use three machines most every day, my Macbook, my office desktop, and our "kitchen computer". And I use two mobile devices every day.

But I am illustrative of something important. Many people access the Web from multiple devices and browsers on any given day.

Each of those browsers I use every day drops a cookie identifying me as a "unique visitor" and the Web analytics software the website is using counts me as up to seven unique visitors when I am only one.

I read some research done by a startup called Scout Analytics that reveals some interesting data on this trend.

Scout Analytics is a "behavioral analytics" provider. For this research, they tracked web users using some interesting techniques:

    Scout Analytics used tracking techniques of device and biometric signatures to follow the behaviors of hundreds of thousands of named users accessing paid content products. The biometric signature identified unique users through an individual’s typing pattern to eliminate errors in user counting such as account-sharing. The device signature identified unique devices through data elements collected from the browser to eliminate errors in device counting such as cleared cookies. By correlating the named user account, biometric signature, and device signature, an accurate mapping of individual to devices could be produced.

And what they found was that a "reliable cookie" overstates user counts 2 to 4 times. That's right, if your analytics software uses cookies, it is possible that your unique visitor counts are 2 to 4 times too high.

This is just one study by one company and I'd love to see more research on this topic, but this is not the first time this issue has been brought up on this blog. In April 2007, I wrote a post on this blog, citing some comScore research, that suggested that cookies over-count unique visitors by 2.5x.

Our portfolio companies are asking me all the time why panel based third party measurement services, like Nielsen and comScore, always generate unique visitor counts that are way lower than their internal numbers. I like to explain to them that there are many ways to count unique visitors and none of them is perfect. The panel based approach, used by Nielsen and comScore, is certainly "old school" and seems out of place in a world where you can just look at server logs. But server logs themselves are subject to inaccuracies.

I advise everyone to triangulate between the various approaches to get some idea of the real numbers. I don't think any service out there today can give you an entirely accurate read. And that is why I am so enthusiastic about comScore's hybrid approach of marrying panel data and tracking pixel data.

I was a founding investor in comScore over a decade ago, was on their board for ten years, and a fund I help manage still owns a very small amount of comScore stock. So I am not unbiased in this discussion. And this discussion is not just about comScore. It's about Web measurement and why, 15 years after the advent of the commercial web, we still aren't measuring it well enough.

Check out Fred Wilson's blog