Facebook Says It Found More Miscalculated Metrics(wsj.com)
wsj.com
Facebook Says It Found More Miscalculated Metrics
http://www.wsj.com/articles/facebook-says-it-found-more-miscalculated-metrics-1479303984
103 comments
This is exactly why so much suspect ad inventory continues to be sold online. Back in the old days an advertiser could watch TV and say definitively if their ad played in the purchased slot or not.
The larger problem here is, depending on what is being optimized for - such as CPM, a big advertiser can begin to allocate an increasing chunk of their ad spend towards the worst performing inventory. I've noticed even Google has more and more very questionable ad products being sold, complete with nearly useless "conversions" being mixed together with real on-site conversions.
For all the heat Snapchat got in their IPO story, the sponsored face/AR filters are the best thing I've seen for brand advertising online in a really long time.
The larger problem here is, depending on what is being optimized for - such as CPM, a big advertiser can begin to allocate an increasing chunk of their ad spend towards the worst performing inventory. I've noticed even Google has more and more very questionable ad products being sold, complete with nearly useless "conversions" being mixed together with real on-site conversions.
For all the heat Snapchat got in their IPO story, the sponsored face/AR filters are the best thing I've seen for brand advertising online in a really long time.
As someone whose clients are the companies you listed and a lead on data and insights, I disagree with a part of this.
While it is true that many agencies plan based off share-of-voice, this is simply done because of laziness, and more prevalent, the lack of clear data that matters (we have X dollars, we can reach X number of people, post-campaign, we reached X people, campaign was a success!). Clicks don't matter, I agree, but conversions do. Specifically sales. That is what is paying for the advertising. As ad-tech advances, planning for sales will replace planning for reach. Think of an upside down triangle where digital and audience targeting fill the smallest part. This will be the first place where ad investment will go because you are able to see the direct results on sales. After you saturate this bucket, you can start filling things like lookalikes, contextual, behavioral, to TV and OOH at the top.
While it is true that many agencies plan based off share-of-voice, this is simply done because of laziness, and more prevalent, the lack of clear data that matters (we have X dollars, we can reach X number of people, post-campaign, we reached X people, campaign was a success!). Clicks don't matter, I agree, but conversions do. Specifically sales. That is what is paying for the advertising. As ad-tech advances, planning for sales will replace planning for reach. Think of an upside down triangle where digital and audience targeting fill the smallest part. This will be the first place where ad investment will go because you are able to see the direct results on sales. After you saturate this bucket, you can start filling things like lookalikes, contextual, behavioral, to TV and OOH at the top.
How do you measure the brand lift? I ask because my company, Survata, runs brand lift studies for display ad buys of the size it sounds like your company would be running using retargeted surveys, but as there is no place to self serve dropping a view tag on Facebook unless it's from a whitelisted provider (which we'd like to become, but haven't been able to find the right contact at Facebook yet), so we can't run similar studies on Facebook ads at the time of this writing. Interested in how you're measuring brand lift from these ads, outside of reach metrics.
Agreed on your point. The total dollars spent by brand advertisers simply dwarfs direct response ads, and the industry standards for "success" often seem useless to direct response advertisers. Remember, these are brands who had been buying magazine and TV ads prior to the internet, so any quantifiable numbers is a major step forward for the industry.
Agreed on your point. The total dollars spent by brand advertisers simply dwarfs direct response ads, and the industry standards for "success" often seem useless to direct response advertisers. Remember, these are brands who had been buying magazine and TV ads prior to the internet, so any quantifiable numbers is a major step forward for the industry.
Thanks I'll look up your firm.
We typically these days prefer market mix models and the like for building definite guidelines on absolute money allocation.
We do run lift studies for operational comparisons and optimizations, but find these techniques unreliable and generally overestimates for any absolute conclusions. We do technically evaluate most new techniques that come to market.
We typically these days prefer market mix models and the like for building definite guidelines on absolute money allocation.
We do run lift studies for operational comparisons and optimizations, but find these techniques unreliable and generally overestimates for any absolute conclusions. We do technically evaluate most new techniques that come to market.
Getting white-listed for view tags on FB usually requires a FB rep (certain spend and other variables). Curious, why does Survata run brand lift studies vs optimizing for lower cost per leads/sales? Since this is a digital business, I would imagine the data is there to measure this?
Sorry, I should have been clear. We don't run brand lift studies on Facebook. We run direct response and lead generation ads and optimize for conversions. We run brand lift studies for clients who spend millions on display advertising. As these advertisers run these ads to raise brand awareness, they're similar to an advertiser running Facebook ads to spread awareness in terms of goals.
And thanks, I knew about that, but without any kind of information about the spend level required to get a Facebook rep, it's hard to justify increasing spend for the purpose of getting our view tags whitelisted.
And thanks, I knew about that, but without any kind of information about the spend level required to get a Facebook rep, it's hard to justify increasing spend for the purpose of getting our view tags whitelisted.
Ah apologies, I misread! Good luck, hopefully this article cracked the wall a bit at Facebook and you can get your tags whitelisted. I'm not sure spend would be a factor anymore- I would imagine they would want to see enough advertiser requests for your tags, where they will feel justified to dedicate engineers to vet your tags.
You are using Facebook ads for exposure, the same way consumer companies have been treating TV. You are looking for mindshare. How is that working out so far? Are you able to measure exposure to impact on future purchases?
Actually for digital ads, there is a way to measure offline sales lift (albeit a few shortcomings). It requires a lot of money (i.e. ~50K for a study, and media spend 250K+). We are able to tie a digital impression to traceable tender (credit card, loyalty card, etc). There will be an exposed group and unexposed group who match each other audience-wise. Lift is then calculated and vetted for significance.
For more info, I recently did an interview on an ad-tech site about this: http://www.thalamus.co/blog/lenny-chase-interview/ Happy to talk more separately if interested.
For more info, I recently did an interview on an ad-tech site about this: http://www.thalamus.co/blog/lenny-chase-interview/ Happy to talk more separately if interested.
Any insight into options for advertisers without that sort of budget to put towards just a single brand lift test?
Brand lift or Sales Lift? Brand lift is a bit different because there are a lot more companies offering this due to more simplicity. The shortcoming of Brand Lift is every vendor has their own methodology when it comes to deciding what exactly is a marginal lift in brand awareness. I think the easiest option for a lower budget is using a FB/IG brand awareness campaign and looking at ad recall numbers (https://www.facebook.com/business/help/1029827880390718). However, we don't really know how they calculate this number and it would only be useful to benchmark between campaigns (if you trust the number). The best option right now is to find a vendor that polls the user who saw the ad, like Vizu (Nielsen). Unfortunately, to get a stat. significant read, you probably need a lot of results, and thus would need higher media spends.
Both actually.
I don't trust the black box of the ad recall stats they provide, so that's not really an option. Is Nielsen's data any less of a black box for this sort of data? And is it actually a decent sample? The older way Nielsen did things was biased because people had to sign up to be a part of it.
And yes, the challenge is that to get a statistically significant read, we need to throw a lot into a specific test, and that is hard to justify without benchmarks. Kind of a chicken and egg problem if your brand is large enough where it takes considerable display dollars to make a blip against your other traffic.
I've been curious if things like Adobe's Attribution econometric modeling tools help much here, but guessing that lack of data will be an issue there as well.
I don't trust the black box of the ad recall stats they provide, so that's not really an option. Is Nielsen's data any less of a black box for this sort of data? And is it actually a decent sample? The older way Nielsen did things was biased because people had to sign up to be a part of it.
And yes, the challenge is that to get a statistically significant read, we need to throw a lot into a specific test, and that is hard to justify without benchmarks. Kind of a chicken and egg problem if your brand is large enough where it takes considerable display dollars to make a blip against your other traffic.
I've been curious if things like Adobe's Attribution econometric modeling tools help much here, but guessing that lack of data will be an issue there as well.
I agree with you 100%. No options are ideal and statistically correct which sucks... a lot. As you mentioned there will always be selection bias and a black box, just hopefully less, with Nielsen for example. Not many people in our space even understand these problems exist, so I think you already lead the pack in this regard. For the sales lift studies, the methodology seems less "iffy" since its one to one and measurable results. One can argue there's selection bias due to no cash data. I haven't seen any cheaper options to do this while minimizing error also.
My client is in the same chicken-egg problem and the way we approached it is: Going on blind is worse than the risk of a bad study. It's completely up to us to do due diligence and ask the right questions to find holes in the vendor's methodology and minimize the risk. We felt that the data gained will be accurate directionally, albeit skewed.
I've never had first hand experience with Adobe's attribution models but from what I gather, lack of data and similar to media mix modeling.
IMO, advancements in this space will be battling privacy concerns.
My client is in the same chicken-egg problem and the way we approached it is: Going on blind is worse than the risk of a bad study. It's completely up to us to do due diligence and ask the right questions to find holes in the vendor's methodology and minimize the risk. We felt that the data gained will be accurate directionally, albeit skewed.
I've never had first hand experience with Adobe's attribution models but from what I gather, lack of data and similar to media mix modeling.
IMO, advancements in this space will be battling privacy concerns.
Yeah, privacy concerns are what will decide a lot of this. I'm very torn because as an end-user I definitely have gotten more nervous with the current state of tracking, but as an advertiser, these are arguably the toughest problems in the industry and what keep me up at night.
Have you found any good off-the-shelf tools to help with the incremental lift analysis that might be better suited for advertisers who are not as massive? Our current plan is to get DCM in place, pipe everything we can through there, and then let our Data Science team go nuts at trying to model it. Seems like that's our best bet since tools like Adometry or other dedicated attributions platforms don't make sense for us yet.
Have you found any good off-the-shelf tools to help with the incremental lift analysis that might be better suited for advertisers who are not as massive? Our current plan is to get DCM in place, pipe everything we can through there, and then let our Data Science team go nuts at trying to model it. Seems like that's our best bet since tools like Adometry or other dedicated attributions platforms don't make sense for us yet.
Of course there's no way to measure it. Rather, there was, and it was called clicks. When it turns out click numbers weren't as impressive as they should've been, now suddenly everyone is back to being comfortable with their ads being tantamount to digital billboards and pitching their service as "brand lift."
Not operating anywhere near the scale of grandparent, but spending a decent amount on FB ads results in a noticeable lift in search volume. It's hard to back this into value but it's at least independent verification that your money isn't just being dumped in a hole.
On your first question, we don't really have a choice but to begin moving money from TV to Digital because TV viewership IS declining and people increasingly spend even their TV time fiddling with their phones :-)
Yes there are a bunch of regression modeling methods and some complex setup lift and attribution studies that let you do this. Some of the techniques are under experimentation. They are generally expensive.
Yes there are a bunch of regression modeling methods and some complex setup lift and attribution studies that let you do this. Some of the techniques are under experimentation. They are generally expensive.
> The primary business objective of advertising for companies such as mine is to build brand awareness, product benefit awareness and brand identity (eg. are you partial to Coke or Pepsi? Nike or Addidas? etc). Doing this will influence your purchase when you next shop for our categories (eg. your next shopping trip to Walmart or Amazon). Note that we are looking to influence a future shopping trip, any "digital conversion" we may get (eg. click thorough to our product page or an online retailer) really is pure gravy from our point of view.
How do you calculate ROI for the ad spend? How do you figure out if you targeted correctly? For instance, I'm highly unlikely to buy either nike or Adidas, so I'd be a poor target.
How do you calculate ROI for the ad spend? How do you figure out if you targeted correctly? For instance, I'm highly unlikely to buy either nike or Adidas, so I'd be a poor target.
For absolute spend ROIs and the like, we typically prefer building market mix models these days though we keep experimenting with others.
On our scale, quality of targeting is something we trade off vs reach. The relationship is often non linear - say you get 10 million reach at an expensive CPM but very high accuracy vs. a 100 million reach that is cheap CPM but has half the accuracy, we'd might be better off putting money in the second bucket.
On our scale, quality of targeting is something we trade off vs reach. The relationship is often non linear - say you get 10 million reach at an expensive CPM but very high accuracy vs. a 100 million reach that is cheap CPM but has half the accuracy, we'd might be better off putting money in the second bucket.
I don't operate at the levels you do. Which also means we are intensely concerned about wasting ad dollars. And, I have to say, Facebook is a total waste of money. Building brand awareness is extremely expensive. Most small to mid size advertisers need conversions. They need real people interacting with their site or product, signing-up, purchasing. Facebook, over the years, has proven to be a great place to set fire to a big pile of cash when it comes to this. Sure, it can work here and there, but, for the most part, nah, waste of time and money.
Any good sources to read and learn more about this form of advertising? Thanks
Go work at a company with a $100M+ online advertising budget :)
Sorry for the trite answer, but this kind of knowledge is so new and ever changing, and there is so little incentive for it to be documented externally, that if you're really interested in it, the best way is to become one of them.
Sorry for the trite answer, but this kind of knowledge is so new and ever changing, and there is so little incentive for it to be documented externally, that if you're really interested in it, the best way is to become one of them.
I think this is rather old-school, actually. They are using FB in the same way they use TV and newspapers, etc. Positioning and maintaining status at top of mind in the consumers' consciousness.
I believe reading ad-tech blogs would be a good start as well as industry sites such AdAge, Digiday, and the blog on Thalamus.co. (I do not work for any of these sites, but have contributed thoughts)
>Hope this helps explain how big advertisers look at media.
And that is the real crux of the matter. FB is a media company. They are currently denying it in the wake of the political discussion (please don't go there) but the real reason is valuation of media vs. tech companies.
And that is the real crux of the matter. FB is a media company. They are currently denying it in the wake of the political discussion (please don't go there) but the real reason is valuation of media vs. tech companies.
It sounds like you have an extremely vague and unreliable means of measuring a return on your hundreds of millions of dollars investment.
I work in a mid-size company where "the computer guy"
manages the Facebook page. One day my boss asked me "WTH does reach mean, you mean with 50 bucks we reached 100 000 people ?" Before replying, I read fb docs, a lot of blogs and for me it was gobbledygook. I said it was marketing bullshit and that we should focus on conversion rates. I felt guilty. Impostor syndrome kind of guilty. Until now.
I'm part of a small grant writing consulting firm, and we tried advertising through Constant Contact's Ad Launcher program. We only want to reach nonprofit executive directors and some heads of public agencies (for example, housing authorities or cities managers). Facebook advertising was complete garbage for us, or, alternately, none of us could target the ads properly—including me. We got a fair number of "leads" but virtually none were qualified clients.
Adwords used to work really well, but its effectiveness has declined in recent years.
We've tried finding a LinkedIn consultant without any luck.
Adwords used to work really well, but its effectiveness has declined in recent years.
We've tried finding a LinkedIn consultant without any luck.
Have you considered highly targeted value based cold emailing?
The HN community may be weary of sending / receiving unsolicited email but if you provide massive value upfront and only contact prospects who would get 2x+ ROI from your product/services, you will not be seen as spam and will convert a good % of the prospects.
I've run a lot of these type of campaigns and the key is to:
A) Only target prospects who have a real need for your product/service and have the budget for it
In your case, I would go on different grant websites and add their grant recipients to your prospect list. This way, you only target organizations who actively use grants.
B) Do a bit of work for them to provide upfront value in your cold email
In your case, research a list of grants they're eligible for (you can share the same list to nonprofits in the same space) and provide unique insights about these grant givers are looking for based on your experience. Your value upfront + value ideas will open the conversation and naturally lead to prospects asking about about how you can help them.
You can get an idea of what the initial email could look like here: http://www.artofemails.com/sales-freelancer#writer
The HN community may be weary of sending / receiving unsolicited email but if you provide massive value upfront and only contact prospects who would get 2x+ ROI from your product/services, you will not be seen as spam and will convert a good % of the prospects.
I've run a lot of these type of campaigns and the key is to:
A) Only target prospects who have a real need for your product/service and have the budget for it
In your case, I would go on different grant websites and add their grant recipients to your prospect list. This way, you only target organizations who actively use grants.
B) Do a bit of work for them to provide upfront value in your cold email
In your case, research a list of grants they're eligible for (you can share the same list to nonprofits in the same space) and provide unique insights about these grant givers are looking for based on your experience. Your value upfront + value ideas will open the conversation and naturally lead to prospects asking about about how you can help them.
You can get an idea of what the initial email could look like here: http://www.artofemails.com/sales-freelancer#writer
I'm an executive at a nonprofit and I delete all unsolicited email unopened. I just don't have time for it. I don't know how typical I am (small organization and it's part time for everyone involved)
[deleted]
Ex-agency advertiser and consultant here. Wanted to provide some thoughts. I agree, Facebook targeting alone wouldn't work quite well. In this case, I would probably utilize FB/GDN as retargeting platforms which would then probably yield some results. Problem here is you would need to get first-party data. LinkedIn as you mentioned would be a decent avenue since you can target specific careers, however I would actually rely on the sales side to start building the list. Of which, you can start generating lookalikes (I always question lookalikes because the algos are "proprietary", but everything warrants a test to see results). Google Search will be good here too with a good set of keywords.
Unless you only want to do nonprofit/public agencies there is also a market for grant writing for small businesses. They are SBIR/STTR available from most (all?) grant funding agencies and are often how scientific startups get initial funding. Maybe these are considered not qualified candidates for your approach, but something to consider.
>> I said it was marketing bullshit and that we should focus on conversion rates.
I always say the same thing when people are buying FB ads. Doesn't matter how people you reach, how people are you converting?
Reminds me of an article that came out in 2010 comparing their add platform to a Ponzi scheme:
http://www.jperla.com/blog/post/facebook-is-a-ponzi-scheme
It also references PG's rant about how Yahoo! was doing the same thing in his 2010 article:
http://www.paulgraham.com/yahoo.html
I always say the same thing when people are buying FB ads. Doesn't matter how people you reach, how people are you converting?
Reminds me of an article that came out in 2010 comparing their add platform to a Ponzi scheme:
http://www.jperla.com/blog/post/facebook-is-a-ponzi-scheme
It also references PG's rant about how Yahoo! was doing the same thing in his 2010 article:
http://www.paulgraham.com/yahoo.html
PG's article is looking back at the state of the internet in the late 90's. His claim is that back then most internet advertisers were themselves VC-funded, which made the whole thing a ponzi scheme.
The first two ads I'm seeing right now in my news feed are Walmart and Banana Republic. I don't think 2016 is anything like 1998.
The first two ads I'm seeing right now in my news feed are Walmart and Banana Republic. I don't think 2016 is anything like 1998.
Yea, the only reason I ever pay attention to reach is when I have ads that are converting and I want to try and expand them. Reach tells me if I can just increase the budget/bid or if I'll need to find a new audience.
Even if the impression data is 100% accurate -- how does an impression in the FB sidebar compare to an impression in a commercial during a TV show, or a product placement in a TV show?
How would you even begin to measure?
Dont affiliates get paid purely on click throughs? No conversion required? So reach is a useful metric.
Depends on the affiliate program; Amazon pays affiliates per conversion (at various rates).
To echo the other explanation, you have 2 main types of payouts. Lead Generation and Performance (conversion).
In some contexts a click through might count as a lead, but most of the time you need something called a "partial", which some sort of customer information. Be it an email, phone number, address, etc, that the user willing submits.
Most affiliate programs are based off conversion because unqualified leads could quite literally be worth nothing to the business.
The amount the affiliate earns from each also varies greatly. Clicks would be a few cents, partials potentially a few $, conversions tens of $.
In some contexts a click through might count as a lead, but most of the time you need something called a "partial", which some sort of customer information. Be it an email, phone number, address, etc, that the user willing submits.
Most affiliate programs are based off conversion because unqualified leads could quite literally be worth nothing to the business.
The amount the affiliate earns from each also varies greatly. Clicks would be a few cents, partials potentially a few $, conversions tens of $.
>> Dont affiliates get paid purely on click throughs? No conversion required?
No, not all. I'd be a billionaire by now if this was true.
I've been in the affiliate business for 11 years and so far I've yet to come across any decent program that paid out just for clicks. Everything is conversion-based. For good reason - generating clicks is trivial and dirt-cheap, if you don't have to worry about conversions.
I found a few click-based programs a few years back. Turns out they were just monitoring conversions on the backend anyway, and dropping all non-performing affiliates.
No, not all. I'd be a billionaire by now if this was true.
I've been in the affiliate business for 11 years and so far I've yet to come across any decent program that paid out just for clicks. Everything is conversion-based. For good reason - generating clicks is trivial and dirt-cheap, if you don't have to worry about conversions.
I found a few click-based programs a few years back. Turns out they were just monitoring conversions on the backend anyway, and dropping all non-performing affiliates.
Interesting I thought it was the affiliates job to drive traffic towards the business and the business job to do the conversion. i.e. with sales and whatnot.
So, I see, the affiliate is responsible for driving 'quality' traffic towards a business.
It seems like industries like gambling and smoking and other vice industries would be the best place to be then?
So, I see, the affiliate is responsible for driving 'quality' traffic towards a business.
It seems like industries like gambling and smoking and other vice industries would be the best place to be then?
I didnt mean that as dengrative. I hope it didnt seem that way. I was actually pretty interested because I was looking to possibly try out affiliate marketing myself.
> None of the metrics in question impact Facebook’s billing, said Mark Rabkin, vice president of Facebook’s core ads team.
This is the point. Might not impact the billing, but might lead announcers to miscalculate which campaign to use, for how long, for which audience, in the ending resulting in bad contracts with Facebook. That will be billed correctly.
This is the point. Might not impact the billing, but might lead announcers to miscalculate which campaign to use, for how long, for which audience, in the ending resulting in bad contracts with Facebook. That will be billed correctly.
Of course it will impact billing!
Customers will be billed the same. Ok, I agree on that; but in better metrics means more accurate ROI calculations and in the medium/long term this means more business.
Customers will be billed the same. Ok, I agree on that; but in better metrics means more accurate ROI calculations and in the medium/long term this means more business.
I was kind of being sarcastic. It's interesting to say that some miscalculation won't impact billing - if you consider that they will charge you for displaying your video x times, but give you the wrong estimates on how long your video was really watched by the users. That will impact your decisions and you will adjust your campaign adequately, and will have impact on how much you spend. So, will impact your money, even if it's billed correctly
Kind of like how service providers constantly have issues with their infrastructure and support, but the billing website and auto-billing system? That's always working.
Shows you where their priorities are.
Shows you where their priorities are.
To be fair, those systems have drastically fewer users connected and it's comparatively easy to scope usage and subsequently resources for them.
Whereas with production systems usage and troubleshooting is far more unstable.
Whereas with production systems usage and troubleshooting is far more unstable.
>To be fair, those systems have drastically fewer users connected and it's comparatively easy to scope usage and subsequently resources for them.
Something that one voluntarily chooses do do being hard is not an excuse to do it poorly or in a way that places inappropriate burdens on others.
Something that one voluntarily chooses do do being hard is not an excuse to do it poorly or in a way that places inappropriate burdens on others.
I was thinking more along the lines of teleco companies but I see your point.
A working billing system is a prerequisite - if you can't bill your customers, you can't pay your developers.
Accountants adagium: Billing is a fact, profit an opinion.
What does it matter if your infrastructure is available if you can't account for it correctly?
Facebook ads aren't purchased on a contract basis.
That reminds me, did they ever address the Click Farm problem that was essentially rendering likes and views worthless for many advertisers? https://www.youtube.com/watch?v=oVfHeWTKjag
Or the rampant Freebooting? https://www.youtube.com/watch?v=t7tA3NNKF0Q
Or the rampant Freebooting? https://www.youtube.com/watch?v=t7tA3NNKF0Q
It's hard to imagine them doing so, because up until a certain point (advertisers leaving, which we ain't there yet) it's not in their interest.
It brings me back to something I complained about on these boards years ago. I was working with a fairly small social gaming company about 10 years ago, during which time Facebook was boasting about "300m, 400m, 500m users!". Meanwhile, we were learning that almost half the users in our game were "fake" accounts. It was a major issue, but wasn't in the interests of Facebook to disclose during their hyper-growth. In fact, they made a point to state that fake accounts was not an problem.
Clearly Facebook will use the metrics when they are convenient, despite their invalidity.
It brings me back to something I complained about on these boards years ago. I was working with a fairly small social gaming company about 10 years ago, during which time Facebook was boasting about "300m, 400m, 500m users!". Meanwhile, we were learning that almost half the users in our game were "fake" accounts. It was a major issue, but wasn't in the interests of Facebook to disclose during their hyper-growth. In fact, they made a point to state that fake accounts was not an problem.
Clearly Facebook will use the metrics when they are convenient, despite their invalidity.
Thanks for mentioning this. It bothers me immensely that they report 1.79 billion MAU. Is that discounted for social game mining accounts / spam bots? If not, that seems like a material issue in their reporting.
I had to learn the hard way that any number the development team gave the marketing group would be used in a way we would come to regret. It's a big part of my cynicism about the industry.
People remember numbers but lose the context. Especially people whose bonuses depend on misunderstanding them.
People remember numbers but lose the context. Especially people whose bonuses depend on misunderstanding them.
So you're telling me that they don't have a test which provisions a Facebook with N users, calls the AJAX handlers for K clicks, from various users and then checks that the metrics show that K clicks were counted, from the correct users? That they don't do this for video watches, friend display-to-click conversions, and every other metric they use to make decisions about what happens on the site? Really?
I don't believe in a conscious conspiracy within Facebook to defraud advertisers. I absolutely believe in a pervasive, semi-conscious but unspoken feeling that asking deep questions about the effectiveness of Facebook advertising or the validity of ad metrics would raise conclusions that would be definitively bad for Facebook, so it doesn't happen.
Any internal team at least slightly aware of the problem would not have a good performance review for the year if their biggest project was to reduce revenue by x% due to efforts to address this.
I think this is a fairly common pervasive, semi-conscious but unspoken feeling throughout many marketing and communications measurement organizations.
I interned on Atlas (FB's enterprise advertising platform purchased from MS a while back) and this was the impression I got. Everything has a dollop of feel-good verbiage piled on top. We're not selling ads, we're "building more meaningful connections between businesses and their customers."
The initial value proposition of online ads was click = confirmed intent. When not enough people click, they construct increasingly convoluted notions of purchasing seconds of mindshare at a time. Of course, at a certain point these purchases become as untestable as a billboard on a highway, a big difference being that highway billboards don't prop up industries, ecosystems, and even geographical regions like online advertising does.
If someone calls shenanigans on online ads, it will be far more catastrophic than if they did the same on billboards/radio/TV ads.
The initial value proposition of online ads was click = confirmed intent. When not enough people click, they construct increasingly convoluted notions of purchasing seconds of mindshare at a time. Of course, at a certain point these purchases become as untestable as a billboard on a highway, a big difference being that highway billboards don't prop up industries, ecosystems, and even geographical regions like online advertising does.
If someone calls shenanigans on online ads, it will be far more catastrophic than if they did the same on billboards/radio/TV ads.
+1 "construct increasingly convoluted notions..." to rationalize or support anything is a huge red flag in this case and, imo, life in general. Understanding the tax system requires trudging through increasingly obscure rules? It's probably a broken, inefficient system. Health care billing system rules are convoluted, vague, and complex? Yeah, it's probably pretty broken. Need a convoluted, complex explanation to rationalize your support for Peter Thiel despite his endorsement of certain political figures? Again, a huge red flag.
Best analogized with computational astrology... Oh you used numbers? Then astrology must be great source of advice.
Sort of like that YouTube auto-play-by-default bullshit. I know that as a user I've accidentally had it plow through dozens of videos -- and ads -- while I was AFK, but it probably looks good on somebody's metrics...
Yes really! And the reason they can get away with is that they have enormous number of users which means that many advertisers are still making a fortune.
Welcome to advertising. Shady as fuck.
Welcome to advertising. Shady as fuck.
This isn't surprising if you've advertised on FB. I tried, and the results I got were dismal (to be fair, it was for B2B which I think generally does poor there).
I run B2B Facebook ads, and they work just fine.
How much of the money they made from these bugs (the new ones and the older ones) will they be returning to their advertisers?
Do they have a contractual obligation to do so? Im curious.
it might get tied into some SLA?
Yes, if proven. If someone promises 1000 clicks for your dollar and delivers only 999, they are at fault, have not fulfilled the contract and either you should be reimbursed or they have to deliver the missing click.
That Facebook talk publicly about this issue (naming it a software bug) either means some lawyer business is already happening in the background, or the difference between what they promised and delivered is so obvious that it is just a question of time. This article is bad for Facebook's image as an advertisement platform and there is no reason to publish something like this, other than liability.
That said, Facebook has good lawyers and will obviously make it exceptionally hard for customers to prove Facebook is at fault. Additionally, their terms of service for the self-serve ads[1] is filled with blurry statements such as "we are not responsible for [...] technological issues [...] that may affect the cost of running ads", which will make legal battles lengthy and boring.
[1] https://www.facebook.com/legal/self_service_ads_terms
That Facebook talk publicly about this issue (naming it a software bug) either means some lawyer business is already happening in the background, or the difference between what they promised and delivered is so obvious that it is just a question of time. This article is bad for Facebook's image as an advertisement platform and there is no reason to publish something like this, other than liability.
That said, Facebook has good lawyers and will obviously make it exceptionally hard for customers to prove Facebook is at fault. Additionally, their terms of service for the self-serve ads[1] is filled with blurry statements such as "we are not responsible for [...] technological issues [...] that may affect the cost of running ads", which will make legal battles lengthy and boring.
[1] https://www.facebook.com/legal/self_service_ads_terms
> None of the metrics in question impact Facebook’s billing, said Mark Rabkin, vice president of Facebook’s core ads team.
I guess they moved fast and broke things.
98% of all statistics are made up.
--
Earlier discussion: https://news.ycombinator.com/item?id=12561302
I bet "miscalculated metrics" is a really good replacement word for "ad click fraud".
Can I ask what has prompted this "investigation" of sorts by Facebook? Just the election? I wonder who is putting pressure on them to straighten this out.
Seeing a dangerously unqualified demagogue become president is a singularly dramatic event that calls a lot of attention to various flaws in various systems, this is just one.
It demands a lot of introspection and you couldn't stop that if you tried. The pressure is coming from the elephant in the room.
It demands a lot of introspection and you couldn't stop that if you tried. The pressure is coming from the elephant in the room.
Could be this:
> Facebook’s efforts to fix its reputation with Madison Avenue may fall short of the demands from some ad buyers.
> ....
> One way Facebook could calm frustrated ad buyers would be to undergo an audit via the Media Rating Council, the media industry’s independent research watchdog.
The ad buyers might have suspected something fishy and started pushing back.
> Facebook’s efforts to fix its reputation with Madison Avenue may fall short of the demands from some ad buyers.
> ....
> One way Facebook could calm frustrated ad buyers would be to undergo an audit via the Media Rating Council, the media industry’s independent research watchdog.
The ad buyers might have suspected something fishy and started pushing back.
As an anecdote, I know that the ad industry lately has been talking about transparency and I'd like to think we're getting more advanced in tech that people start asking the right questions. Or maybe I'm naive to believe that.
The ad industry will never be "transparent" (i.e. clear and colorless), because it has falsifiability issues. Transparency would mean that they admit that most of the industry is based on guessing, that there's no science behind huge commercial transactions.
Publicly traded, billions in revenue, I'm sure LOTS of audits are done, constantly.
I think Facebook has proven by now that even if they aren't doing this on purpose to trick advertisers into thinking their ads are much more effective than they really are, at least the product doesn't seem ready for primetime with so many "bugs" in measuring popping up.
If you're a small business, Facebook may screw you over big time, so it's not worth the risk. I assume big companies just don't care.
If you're a small business, Facebook may screw you over big time, so it's not worth the risk. I assume big companies just don't care.
Is it weird or normal for a company like Facebook to share this kind of information with the public like this?
Feels oddly transparent. What is prompting this level of transparency?
Feels oddly transparent. What is prompting this level of transparency?
"Lastly, Facebook said it plans to form a Measurement Council made up of marketers and ad agency executives, and will also roll out a blog to more regularly communicate updates about measurement."
I read that as "we have no idea on how to measure, so please tell us what is important for you guys".
I read that as "we have no idea on how to measure, so please tell us what is important for you guys".
I instead read it as "We'll give you buy-in so that next time you'll have a vested interest in not being blamed." This is a PR move and should not be interpreted sincerely.
Wasn't Facebook's whole value proposition is that they knew more than the traditional advertiser/publisher partnerships?
Such a sham.
Such a sham.
I think the original proposition was: dumb fucks.
> I read that as "we have no idea on how to measure, so please tell us what is important for you guys".
Which is a great way to improve their product.
Which is a great way to improve their product.
Facebook which supported Clinton finds those bugs just after elections that Clinton lost.
What do advertisement metrics have to do with the election? As far as I know, the story about "Facebook + US Election" was related to people posting/sharing/therefore make trending fake new stories, which is unrelated to the Facebook ads/ad metrics side of the business.
The take away from all this is you should use Twitter, it worked for President Trump and it could work for you too. :)
The primary business objective of advertising for companies such as mine is to build brand awareness, product benefit awareness and brand identity (eg. are you partial to Coke or Pepsi? Nike or Addidas? etc). Doing this will influence your purchase when you next shop for our categories (eg. your next shopping trip to Walmart or Amazon). Note that we are looking to influence a future shopping trip, any "digital conversion" we may get (eg. click thorough to our product page or an online retailer) really is pure gravy from our point of view.
The primary metrics of interest to us therefore are how many people did our impressions reach (people - not cookies or devices), of the ones we did, how many were in our desired target audience and what was our impression frequency to them (at some point, high frequency becomes excessive). We don't even really look at clicks or conversions unless there is a specific interaction based marketing program.
Hope this helps explain how big advertisers look at media.