home   |   archives   |   about
 
 
SEARCH THIS SITE

Why Third-Party Click Fraud Estimates Don't Add Up

By Shuman Ghosemajumder | Wednesday, January 31, 2007

I want to thank everyone who has written to me with questions since I started blogging about the work we do at Google to protect advertisers against click fraud. I'll be catching up on some of those questions in the next week, but today I want to address some of the more recent items in the media on click fraud rates.

There was a press release yesterday from ClickForensics stating that their quarterly measure of click fraud for Q4 was 14.2%. They also stated that this was the year's "highest level" (up from 14.1% in Q2) and that the click fraud rate for search engine content networks was 19.2%. This morning there was a competing press release from Incremental Advantage and several other click fraud firms, stating that "Click Fraud Cost Internet Advertisers $666 Million in 2006".

On a basic level, these numbers are much higher than what we see at Google, and are not at all representative of the actual statistics of our network. Most savvy advertisers and industry pundits are already aware of this (see "Why We Can't Trust Click Fraud Numbers" in yesterday's WebProNews), and generally haven't paid much attention to these estimates for a while.

However, these stats are still out there and there are some things everyone should keep in mind when reviewing them. Specifically:

  1. Many third-parties have not even counted clicks properly
    We did an analysis of Click Forensics and other click fraud consultants back in August 2006 to see why their numbers were so inflated (see "How Fictitious Clicks Occur in Third-Party Click Fraud Audit Reports" on the Google AdWords Blog). We found serious flaws in their counting of clicks - a more fundamental issue than their counting of click fraud. They were making basic counting mistakes and inflating the number of clicks by an average of 40%. The source of this problem is incorrectly counting page views – from users browsing through an advertiser's site – as clicks.
  2. Inflated click counts result in even more inflated "click fraud" estimates
    This over-counting problem results in an even more dramatic inflation of click fraud estimates, in fact consistently classifying an advertiser's best users (the ones spending time browsing their site) as fraudulent. As a result, conclusions based on this data are flawed and the small differences in overall percentages they report are not meaningful. And instead of protecting their businesses against click fraud, advertisers can actually harm their businesses by acting on recommendations from these reports.
  3. Even if they fixed those problems, they're not actually measuring click fraud
    Even if they were counting clicks correctly, they are still trying to measure only activity (attempted click fraud) and not advertiser impact (actual click fraud). That is, even if they corrected the basic engineering and accounting problems contributing to the above problems, they would still be counting clicks we filter (and do not charge to advertisers) in their click fraud estimates. They admit this.
  4. Industry metrics (in any area of our business) are not necessarily the same as Google's metrics
    The advertisers in their sample are part of many different networks and not all of these networks have invested as heavily as Google in click fraud protection.
  5. ROI on the content network is the same as it is on search
    We know there is a more direct incentive for fraud on the content network and we do much more to protect advertisers, ban bad publishers, and improve ROI through SmartPricing discounts. As a result, average ROI on our content network is nearly the same as on Google.com. Yes, you read that right. ROI is the same on average - and not by accident, but because we automatically provide discounts to advertisers to make it so.

The key point here is not that their numbers are "too high". The point is that their data collection methods are inherently flawed and any resemblance their numbers could have to reality would be coincidental. Even so, given that they are not measuring click fraud (see point #3), they apparently don't intend their numbers to reflect reality.

Click fraud protection is something we take very seriously at Google, and it requires a high level of scientific rigor to do well. It's frustrating to see basic mistakes being made by firms selling "additional protection" to AdWords advertisers - in essence, charging them money for advice which can actually hurt their businesses. I've spoken with many firms and a number of academics interested in this area, and the ones who are investing in serious R&D efforts recognize the limitations of their data and analysis and have not been focusing on publicizing unsupportable and flawed numbers such as the above. We're very supportive of those efforts (and in scientific research in this area in general) and we'll continue to work closely with them.

For more information about Google's actual metrics, you can see my previous posts here and here.

Update: I've posted a second part to this post, with more technical details on points #1 and #2.

   

Comments

Shuman,

Your points are accurate, but I don't believe attacking Click Forensics' methodology is productive. Per Andy Beal's comment in the article you reference above, "[we are] desperate for a third-party audit of click fraud, completed by a group that has no skin in the game." I would love for you to comment on Google's plan here.

-Matt

Matt Greitzer
January 31, 2007, 7:42PM


Great point Matt.

Here's our plan: we are working on just such a solution with the rest of the industry through the IAB Click Measurement Working Group. The goal is to create an independent, scientific audit of click fraud protection systems to verify that PPC advertising providers do what they claim they do. The Media Ratings Council, which audits Nielsen, is an integral part of the IAB effort.

By the way, Google is comfortable with the idea of submitting our systems to such an audit because of the level of investment we've made in our technologies and processes. We're confident that an audit will show that we lead the industry in this area, and we would obviously vastly exceed the level of click fraud protection that would be the agreed-upon minimum industry standard.

But the other benefit is that it will put pressure on other PPC advertising providers to step up their investments and protect their advertisers better. Some firms may not have the necessary expertise to do this, and this may in turn create an opportunity for serious, scientific third-party click fraud consulting firms to service their advertisers.

I've personally spoken with many of those third-party firms, and a number of them have assured me that they are putting considerable thought into how to properly conduct analysis in this area, especially as a third-party (a considerable challenge, given the limited data third-parties can see). Unfortunately those are the firms that you don't read about very much, since they would never promote flawed industry estimates in order to attract business.

Shuman, January 31, 2007, 10:52PM


I'd like to know what Google is doing to protect Adsense publishers from fraudulent clicks made by others. There are too many horror stories of people's accounts being closed when they did nothing wrong. A publisher has no control over what others do on his site.

Chris
February 02, 2007, 10:58AM


Thanks so much for this article. Click fraud consultancies are obviously just trying to create paranoia around click fraud, and have been doing a pretty successful job of that. Getting clear information from Google such as this helps remind us that these firms are more focused on their own self-interest than in the truth - just like the companies who install spyware and then popup ads for spyware removal.

Jeff Fitzgerald
February 03, 2007, 2:47AM


If you were so certain click fraud was not a real thing to worry about you would not use the clickthrough ratio as a measure of quality in adwords. I would think you would instead choose ROI as the quality variable. However, I see how that can be a problem since your cash cow is the click.

Al del Castillo
February 03, 2007, 1:16PM


Jeff,

I have some problems with the findings of the click fraud consultancies, but this does not mean the problem is being overblown. Please take some time to study what some Internet technical experts such as Ben Edelman, Bruce Schneier, and Lauren Weinstein have said on the subject. Please also familiarize yourself with the basic principles of Internet architecture, protocols, and security, so you understand how easily click fraud can be generated and how difficult (and even impossible) it can be to detect.

The true danger here lies in squabbling amongst those with understandable self-interest, while the true criminals (the click fraudsters) rob us blind.

CPCcurmudgeon
February 04, 2007, 1:02PM


Shuman,

It was good to meet you in person the other day.

I agree with pretty much everything you say here, except I think that #3 is a bit unfair. I would say that click fraud detection tools (if working correctly) measure *actual* click fraud but not *charged* click fraud. And the reason for that is that Google won't tell advertisers which specific clicks were charged for and which weren't, only the total number of each, so it's impossible for a third-party tool to ignore the uncharged clicks.

Stephen Turner
February 15, 2007, 2:52AM


As Chris stated on 02/02/07, "I'd like to know what Google is doing to protect Adsense publishers from fraudulent clicks made by others. There are too many horror stories of people's accounts being closed when they did nothing wrong. A publisher has no control over what others do on his site." I just had my account closed on 02/15/07 because of fraudulent click activity, which I as a single individual am just learning the ins and outs of website building/management. I do not have the knowledge, resources, or for that matter the time to track these individuals down. However, as a publisher I would be willing to pay a small fee each month from any revenue generated from my Adsense account to combat fraudulent clicks. Has Google ever thought about helping us little fish in the big pond out?

Steve
February 16, 2007, 5:27PM



Links: Links to this site
My speaking schedule for early 2008
ClickFraud Might Be Up
Blogs By Googlers
Google & Yahoo lanceren Ad Traffic Quality Center
Click Fraud, Google AdWords and gclid
Long Overdue Blogroll Updates
Google Dishonors War Dead
 
Copyright © 2003-2008 Shuman Ghosemajumder. All contents available under a Creative Commons License. Opinions on this web site are the author's own. Generated Friday, May 16th, 08:16:31 PM EST.