COOKIE-BASED COUNTING OVERSTATES SIZE OF WEB SITE AUDIENCES

 

Frequent Cookie Deletion by 3 out of 10 U.S. Internet Users Leads to Overstatements in

Audience Sizes by a Factor as High as 2.5

 

Implications for Advertising and Audience Measurement Deemed Significant by ARF and Industry Experts

RESTON, Va. – comScore, a leader in measuring the digital world, today released the results of a study analyzing the validity of using cookie-based data to measure the number of unique visitors to individual Web sites or to gauge the number of unique users that were served an ad by an ad server. The study, based on an analysis of 400,000 home PC’s included in comScore’s U.S. sample during December 2006, examined both first-party and third-party cookies.

 

A cookie is a very small text file inserted on a user’s computer by a Web server and is unique to that computer’s browser. Cookies are often used by web servers to identify users and for authenticating, tracking and maintaining specific information about users. First-party cookies are those left on a computer by a Web site that has been visited, while third-party cookies are those left by a domain different than the site being visited, such as an advertising server that has just delivered an ad to a computer, or certain third-party tools used to measure site traffic. For the purposes of this study, comScore analyzed the cookies from one prominent Web property and one third party ad server, each representative of the total U.S. Internet audience and each reaching well in excess of 100 million Internet users every month. The study examined the degree to which users cleared cookies from their computers, thereby causing servers to deposit new cookies and potentially leading to overstated estimates of unique users when relying on cookie-based server data.

 

Average Computer Receives 2.5 First-Party Cookies per Site Each Month

comScore observed that 31 percent of U.S. Internet users cleared their first-party cookies during the month. Within this user segment, the study found an average of 4.7 different cookies for the site. Among the 7-percent of computers with at least 4 cookie resets, comScore counted an average of 12.5 distinct first-party cookies per computer, accounting for 35 percent of all cookies observed in the analysis.

 

Using the total comScore sample as a basis, an average of 2.5 distinct first-party cookies were observed per computer for the site being examined. This indicates that Web site server logs that count unique cookies to measure unique visitors are likely to be exaggerating the size of the site’s audience by a factor as high as 2.5, or an overstatement of 150 percent.

comScore Cookie Deletion Analysis – 1st Party Cookies*

December 2006

Source: comScore, Inc.

 

Number of Cookie                                   Avg. No. of       

Deletions/Resets                Percent of         Cookies per         Percent of

                                 Computers            Computer            Cookies

 

Total Sample                          100%                 2.5               100%

1 or more                              31%                 4.7                58%

4 or more                               7%                12.5                35%

_________________________________________________________________________________________________________

“While past studies from other research companies have shown a similar proportion of computers that clear their cookies, the comScore study is the first to highlight the disproportionately high percentage of cookies represented by those computers,” commented Dr. Magid Abraham, President and CEO of comScore. “For example, with just 7 percent of computers accounting for 35 percent of all cookies, it’s clear that a certain segment of Internet users clears its cookies very frequently. These ‘serial resetters’ have the potential to wildly inflate a site’s internal unique visitor tally, because just one set of ‘eyeballs’ at the site may be counted as 10 or more unique visitors over the course of a month. The result is a highly inflated estimate of unique visitors for sites that rely on cookies to count their audience.”

 

Third-Party Cookies Deleted at Nearly Same Rate as First-Party Cookies

comScore’s analysis of third-party cookies revealed an average of 2.6 distinct cookies per computer in December, indicating a similar rate of overstatement as the first-party cookies. For those computers where at least one cookie reset occurred, the number of third-party cookies observed was slightly higher than first-party cookies at 5.5.

 

______________________________________________________________________________________________________

comScore Cookie Deletion Analysis – 3rd Party Cookies

December 2006

Source: comScore, Inc.

 

Number of Cookie                                   Avg. No. of       

Deletions/Resets                Percent of         Cookies per         Percent of

                                 Computers            Computer            Cookies

Total Sample                          100%                 2.6               100%

1 or more                              27%                 5.5                57%

4 or more                               7%                14.2                38%

_______________________________________________________________________________________________________

 

“There is a common perception that third-party cookie deletion rates should be significantly higher than first-party cookie deletion rates,” continued Dr. Abraham. “Because many PC users reset or delete their cookies using security protection programs, conventional wisdom dictates that people are more likely to selectively expunge third-party cookies – which are generally deemed more invasive – while maintaining their first-party cookies. But these findings suggest that selective cookie management is not prevalent, a fact that comScore confirmed via a survey, with only 4 percent of Internet users indicating that they delete third-party but not first-party cookies.”

 

Industry Experts Weigh in on Results of comScore Study

Industry experts representing site operators, the academic community, trade associations, industry consultants, advertising and media agencies and ad serving networks commented on the comScore study:

 

“Panel-based measurement is needed to accurately measure the size and characteristics of a site's online audiences,” said Richard Castellini, Vice President of Consumer Marketing at CareerBuilder.com. “The comScore study quantifies the magnitude of the discrepancy one can encounter using log files, and highlights the significance of this issue to our industry.”

 

“The comScore study clearly demonstrates the limitations of measuring Internet audiences based on cookies,” commented Mohanbir Sawhney, McCormick Tribune Professor of Technology at the Kellogg School of Management, Northwestern University . “To measure audiences more accurately, it is important to link visits to unique individuals, not unique cookies. As privacy programs become more entrenched, cookie-based audience counts will get even more unreliable.”

 

“In 2005 while at JupiterResearch, I reported the results of a survey showing that 39 percent of Internet claimed to delete their browser cookies on a monthly basis,” said Eric T. Peterson, well-known web analytics consultant and author of “Web Analytics Demystified.” “The data from the comScore study, especially the findings regarding first-party cookies, clearly highlight the risk to cookie-based measurement. The comScore study emphasizes that site operators need to be extremely careful when calculating and reporting unique visitor counts from server log data, questioning both the technology they use and their underlying assumptions about cookie deletion rates among their site visitors.”

 

“This is truly a crucial study from comScore,” said Bill Cook, PhD, Senior Vice President for Research and Standards at the Advertising Research Foundation “Reach and frequency metrics are the cornerstone of any media plan, and given the size of the discrepancies that can occur when counting cookies instead of people, the study underscores the importance of panel-based measurement. For the advertising community, an accurate understanding of reach and frequency within a given target audience is vital.”

 

“This important comScore study confirms what many of us have always suspected,” commented David Verklin, CEO of Carat Americas. “Cookies are just not an accurate enough method to calculate site visits and upon which to base audience metrics.”

 

“The integration of Internet advertising into multimedia marketing plans requires accurate reach and frequency data,” remarked Karen Francis, Chairman and CEO of Publicis & Hal Riney. “It’s clear from the comScore study that cookie-based counts from site server logs are not necessarily providing the level of accuracy that is required.”

 

“Cookie-based data are still a valuable resource, but this important study certainly underscores the fact that an accurate, multidimensional picture of consumer behaviors must be compiled from multiple sources,” said Jeff Marshall, Senior Vice President, Digital Managing Director at media agency Starcom USA. “Digital media has a well-deserved reputation for enhanced accountability for media plans, but advertisers and their agencies should always examine all media from a variety of angles and with different tools and panel-based sources to enhance the value and validity of accountability reports.”

 

“The comScore study confirms that relying on cookie counting alone for audience measurement can result in inflated unique visitor counts,” said Curt Viebranz, CEO of TACODA. “That’s why TACODA is working with comScore to enhance and validate our cookie data with comScore’s panel data.”

 

About comScore

comScore, Inc. is a global leader in measuring the digital world. This capability is based on a massive, global cross-section of more than 2 million consumers who have given comScore permission to confidentially capture their browsing and transaction behavior, including online and offline purchasing. comScore panelists also participate in survey research that captures and integrates their attitudes and intentions. Through its proprietary technology, comScore measures what matters across a broad spectrum of behavior and attitudes. comScore analysts apply this deep knowledge of customers and competitors to help clients design powerful marketing strategies and tactics that deliver superior ROI. comScore services are used by global leaders such as AOL, Microsoft, Yahoo!, Verizon, Best Buy, The Newspaper Association of America, Tribune Interactive, ESPN, Fox Sports, Nestlé, MBNA, Universal McCann, the United States Postal Service, Merck and Expedia. For more information, please visit www.comscore.com .