I almost called this post “Nobody Knows Anything” but that might have been overkill. I’ll say what I have to say and let you be the judge. Let’s say that you buy a friend’s newborn a gift. You have it shipped to your house. The data says, correctly, that you bought an infant gift. That might also lead to an inferred piece of data that places your household into the “presence of infant” bin, leading to you seeing lots of ads for diapers. If you’re the one placing the ads for those diapers, you’re wasting money.
Lots of the data marketers routinely use is of that sort. It’s inferred. You can see that some thinking at work if you’re a Netflix user: the recommendation engine infers what you might like based on your past viewing. Of course, if your kids or someone else in the house watch something in which you have no interest, the accuracy of those recommendations is diminished (which is part of why there are separate profiles available when you log in). Inaccurate data is, sadly, more the norm than an aberration. Since this data is really what’s behind personalization and targeting, that inaccuracy is a big problem. Any business that buys data from third parties – and an awful lot do so – may be putting garbage into their system. Unfortunately, most don’t know that because there is little transparency in the data business and it’s impossible to verify what’s good and what’s not.
What should you do? Invest in collecting your own, first-person data. You can also demand transparency in any other data you use (good luck with that) with respect to how it was gathered and what it really represents. Is it inferred or does it come directly from consumers (did someone tell you they had a baby in the house or did you guess they did because they bought one infant item?). Who owns the data and was it gathered with the consumer’s permission?
When Facebook tells its customers (marketers) that they have data on 41 million adults aged 18-49 in the US and there are only 31 million of those adults living in the US, you know much of the data is inferred and also that we have a problem. A recent study that found that 70% of marketers believe that the customer data their organizations are using for marketing is low quality or inconsistent. Why bother to market at all when you’re just flying blind?
Filed under Consulting, Huh?
The NFL is getting ready for the annual combine. This is where players get tested both physically and mentally to see if they’re NFL material. There is psychological testing to test intelligence. They run the 40-yard dash. It’s a 4-day job interview, much of which plays out on TV.
Teams use the data to make decisions about which players to select in the annual draft. They can stack the reams of information from the combine with the data generated over the course of a player’s college career and choose someone who will, hopefully, fit into a team’s depth chart as well as its philosophy.
Anyone who follows the NFL will tell you that all of this data has its place but it’s far from infallible. Kurt Warner, a 2-time NFL MVP went undrafted. So did Warren Moon, a Hall Of Fame quarterback too. Put Tony Romo on that list as well. No team looked at the data and thought any of these men were worthy of a draft pick. Oops.
You just might be guilty of the same thing in your business. The data isn’t infallible and the data only measures what it’s designed to measure. Tom Brady (selected 199th in his draft year) recently told NFL prospects that they can’t measure heart. He’s right, and it’s because there isn’t a solid way to capture that data.
How are you making this mistake? You might be using one data point to draw a conclusion that isn’t right. Correlation isn’t causation, as we hear so often. Grateful Dead fans don’t all smoke pot and have long hair. Identifying a target as those fans doesn’t mean you should be promoting to the stereotype.
Another faulty conclusion might be due to an error in the data itself. I had an advertiser on a site I ran complain that they weren’t getting great results. They had neglected to respond to a question from their salesperson about turning on frequency capping to extend their reach and limit the number of times a day someone saw their ad. They were reading the data correctly but the data itself was faulty due to an underlying issue.
One of my favorite data error is the foundation of the entire TV business, the Nielsen Ratings. The TV and ad industries have attached an accuracy level to Nielsen ratings that even Nielsen says is unreasonable. A study of a few years back found in analyzing 11 years of data that the margin of error for reported results was often more than 10%. That might not sound like much but it can represent hundreds of thousands or even millions of impressions. The issue here is that buyers are too focused on the (inaccurate) numbers rather than on precise metrics such as sales.
Measure what you can measure. Don’t extend that measurement to other things that aren’t measured as well. I bet your results will improve. Let me know?