The NFL is getting ready for the annual combine. This is where players get tested both physically and mentally to see if they’re NFL material. There is psychological testing to test intelligence. They run the 40-yard dash. It’s a 4-day job interview, much of which plays out on TV.
Teams use the data to make decisions about which players to select in the annual draft. They can stack the reams of information from the combine with the data generated over the course of a player’s college career and choose someone who will, hopefully, fit into a team’s depth chart as well as its philosophy.
Anyone who follows the NFL will tell you that all of this data has its place but it’s far from infallible. Kurt Warner, a 2-time NFL MVP went undrafted. So did Warren Moon, a Hall Of Fame quarterback too. Put Tony Romo on that list as well. No team looked at the data and thought any of these men were worthy of a draft pick. Oops.
You just might be guilty of the same thing in your business. The data isn’t infallible and the data only measures what it’s designed to measure. Tom Brady (selected 199th in his draft year) recently told NFL prospects that they can’t measure heart. He’s right, and it’s because there isn’t a solid way to capture that data.
How are you making this mistake? You might be using one data point to draw a conclusion that isn’t right. Correlation isn’t causation, as we hear so often. Grateful Dead fans don’t all smoke pot and have long hair. Identifying a target as those fans doesn’t mean you should be promoting to the stereotype.
Another faulty conclusion might be due to an error in the data itself. I had an advertiser on a site I ran complain that they weren’t getting great results. They had neglected to respond to a question from their salesperson about turning on frequency capping to extend their reach and limit the number of times a day someone saw their ad. They were reading the data correctly but the data itself was faulty due to an underlying issue.
One of my favorite data error is the foundation of the entire TV business, the Nielsen Ratings. The TV and ad industries have attached an accuracy level to Nielsen ratings that even Nielsen says is unreasonable. A study of a few years back found in analyzing 11 years of data that the margin of error for reported results was often more than 10%. That might not sound like much but it can represent hundreds of thousands or even millions of impressions. The issue here is that buyers are too focused on the (inaccurate) numbers rather than on precise metrics such as sales.
Measure what you can measure. Don’t extend that measurement to other things that aren’t measured as well. I bet your results will improve. Let me know?
It’s Foodie Friday! I’ve written before that I’m not much of a baker and only do so when a guest is counting on some sort of baked dessert. It’s not because I don’t have a sweet tooth though. One weakness I do have with respect to baked goods is cookies. The blue guy on Sesame Street has nothing on me and I suspect if I didn’t exercise some sort of self-control I’d weigh 300+ pounds.
I love me some cookies and take a vicarious thrill in looking at various cookie recipes even though I will only consume them through my eyes and not my mouth. One thing that I noticed popping up in a number of recipes was caster sugar, and an article on Food52 yesterday helped me understand what it is and why it’s used in baking. This is their very fine explanation:
Caster sugar goes by a variety of names, including castor sugar, baker’s sugar, and superfine sugar, the last of which alludes to what exactly it is: a finer granulated sugar. If a grain of granulated sugar is big and a grain of powdered sugar is tiny, caster sugar would be somewhere in between.
Which of course got me thinking about business, and about data in particular. Just as the more granular nature of caster sugar makes cookies a better product (they’re softer and lighter), so too can refining your data yield much better results. You’ve probably heard about the need to segment your data but if you’ve never done so or have never gone beyond basic age/sex or other large groups, you’re really missing out. Refining your data makes it possible to address each segment in a way that’s meaningful to them. The more personalized you can make your messaging, the more effective it will be. Getting beyond “first name” and into where in a purchase cycle a customer might be as a data segment will make for a better outcome. Special offers by segment only yield great results when the specificity of those segments make the offer truly special.
Caster sugar is more refined but not overly so. That’s a great thing to keep in mind as you analyze and use all the raw data you collect every day. The fact that the data isn’t fattening is a big plus!
There has never been a time when it’s been easier to get information. If you don’t believe me, pick up that computer you keep by your side most of the time (that would be your smartphone), push whichever button activates either Siri, Google Assistant, or whatever flavor of virtual assistant you have installed, and ask what the weather will be tomorrow. Ask who the Prime Minister of Denmark is or a few ways you can cook a turnip. We have the world at our fingertips.
That can be true with business information too. Traffic to your media properties, interactions with your content, results of your ad and social media campaigns, and feedback on how your company or brand is interacting with the world at large are all readily available for analysis and action. So is customer data, market predictions, and just about anything else you’d need to know. Pretty awesome, right?
The problem is that not everyone wants to know the truth about these things. Take the manager whose staff is leaving in droves. They “hear” it’s because of a better offer but they don’t take the time to sit down and dig into if there is an underlying problem in their operation. They couldn’t handle it if the problem was really them and their management style so they avoid the question.
Then there is the web person who is under pressure to keep growing traffic and doesn’t bother to exclude the kinds of traffic that inflate the numbers. You know: your own internal use of your website, traffic from places where you don’t do business, referrer spam or other obviously fake traffic. They know the truth but their bosses can’t handle it.
The problem with having information is that it compels you to act. We can always deny there is a problem if we don’t know about it or if we think the information we have is inaccurate. As with the law, ignorance is no excuse in my mind. I’ve been in meetings where some excellent forecasting predicts a downturn in a company’s business but several members of the management team want to expand their spending. The forecasts are subordinated to the feeling that more spending will yield more revenue despite the fact that the company’s share of the market has been steady for years and probably won’t increase in a downturn (which is basically what the managers are predicting). They couldn’t handle the truth: they need to tighten their belts and ride out the next few quarters. They’re no longer in business, by the way.
We hear an awful lot about fake news and there certainly is some out there to be ignored. Your business analytics don’t fall into that category and you ignore them at your own peril. If you can’t handle the truth, you can assume that reality will handle you one way or another. OK?
I think everyone knows that a lot of data is collected as we conduct our daily digital activities. Google and the other search engines know what we’re looking for, Amazon and other commerce sites know what we’re shopping for, Facebook knows what we like, LinkedIn knows who we know, etc., etc., etc. These data footprints are collected and in many cases sold to marketers and their agents to allow them to serve ads to you. If any of that comes as a shock to you, I’m not sure where you’ve been for the last decade or more.
What you might not have thought about, however, is that the ads themselves collect data. How many times has someone seen it? What kind of person (that pesky data that the aforementioned guys have) has responded to an ad, and how well do the ads translate to sales (lovingly called the conversion rate as if someone is changing religions…). As it turns out, there is a bit of a controversy about who actually owns that data: the advertiser or the agency. The marketers believe that they are the rightful owners while the agency folks believe just as strongly that they are. Neither side feels that the publishers who serve the ads and, therefore make data collection possible, have much of a claim to it. Of course, even publishers came out ahead of one other group as the rightful owners in the survey: consumers.
As you can see in the chart, only 10% of advertisers and 15% of agency respondents believed that consumers had a claim to their own information. That’s tragic. Why? Because it represents a mindset that is ultimately self-defeating. It can lead to legal problems at worst and consumers opting out (if they can figure out how) at best. What have the advertiser or the agency done to give the consumer value for the data? Nothing, in my mind. One could argue that the ads they serve make possible the content the consumer enjoys, but those very ads make that enjoyment nearly impossible given the state of ad-serving today, particular in mobile.
Unless and until we on the marketing side see the consumer as at least an equal partner in our business and not as a bunch of rubes or just as “data”, the problems with ad blocking, anti-spam rules, and other protective measures aren’t going to go away. What will go away are the people represented by the very data over which the agencies and marketers are fighting. You agree?
I was discussing some numbers with someone the other day. It was clear from the conversation that she was taking every bit of data as gospel. I tried to explain a few important things to keep in mind when working with data and as I thought about it perhaps my thinking could be helpful to some of you out there in screed-land.
We all want as much certainty in our business lives as we can get. Part of that is wanting all of our numbers to be facts. They’re not. You may be familiar with the term “sampling error.” Basically, it means that the data is off because the sample from which the data is drawn is not representative of whatever it is you’re trying to measure. While you might think that, for example, your analytics measure everyone, they don’t. Most of the data we read uses some sampling. Sometimes it’s a timing issue – financial data, in particular, can be skewed based on where we might be in a business calendar or where those who pay us are in theirs.
The point is that there are error rates involved with many of these “facts” because these facts are really just estimates. TV ratings, for example, are probably the most widely known estimates and multi-billion dollar businesses involving networks, agencies, and marketers revolve around numbers everyone knows are not particularly accurate. There are error rates.
Here is the advice I give people. Figure out what questions you’re trying to answer and then find as many different sources of data as you can. If possible, see if you can get multiple people to interpret those data sets. In theory, they should all come up with the same answers. It’s critically important that you NOT tell them what position you’re trying to support (can you find me some information that says we should do XYZ). That is a recipe for disaster because it encourages people only to look at data or interpretations of data that supports what you or they already think is true. That is turning “facts”, which are already often on shaky ground, into a larger fiction, and that’s not what we’re after, is it?
I don’t think there has been a baseball movie made that didn’t feature some weathered old guy seated in the bleachers somewhere. He usually utters undecipherable baseball jargon while taking copious notes. This, dear reader, is the baseball scout, who used to be how talent was discovered. If you’ve seen or read Moneyball, you know that the scout is an endangered species. This article from USA Today last week talks about how many pro scouts are still unemployed one month before the start of spring training. The reason? Data.
(Photo credit: Wikipedia)
Baseball is in the throes of the Moneyball movement. Teams have been laying off scouts and turning to sabermetrics, which Wikipedia defines as the empirical analysis of baseball, especially baseball statistics that measure in-game activity. Baseball has fallen in love with data. Maybe your business has too.
Here is the problem, both for you and for baseball. There are certain things that don’t show up in data. A player’s leadership qualities in the dugout aren’t quantifiable. Potential can often be visible but not measurable. That’s true in your office as well. The data may show you what it happening but it’s hard for it to show you what could be happening. That requires humans: scouts.
We all need scouts. We need people who use the data as a tool but who also have the experience and wisdom to know when the data is missing something. That doesn’t mean projecting one’s wishes into the numbers nor distorting the story those numbers tell. It is, however, an acknowledgment that there is often a bigger picture than what’s inside the frame.
Here is a quote from a scout:
I’ve got 23 years in the business,’’ Wren said, “and now clubs don’t want that experience? I look at teams now, and they’re hiring guys who aren’t really scouts. They’re sabermetric guys from the office, and they put them in the field like they’re scouts, just to give them a consensus of opinion.
That’s dangerous for a baseball team. It could be fatal for you. You’re up!
There is an interesting case that was argued before the Supreme Court the other day and it just might have an impact on your business. There was also a lawsuit filed in an unrelated matter that could have the same effect. A third item is a study that’s kind of scary. Let’s have a quick look at them and think about what they might mean to anyone who gathers information about their customers.
First, the case before The Supremes. It involves Spokeo, one of the large data aggregators. Spokeo’s information about a consumer was almost 100% wrong. As Justice Kagan said, “They basically got everything wrong about him. They got his marital status wrong. They got his income wrong. They got his education wrong. They basically portrayed a different person.” The plaintiff was seeking a job when he filed suit, and worried that the errors in the report would affect his job search. The other suit involves Ashley Madison. They were sued for allegedly misleading users by inflating the number of women who belonged to the service. As we have found out from the data hack, only a small percentage of the profiles belonged to actual women who used the site. The company hired employees whose jobs were to create thousands of fake female profiles.
I suspect that a third form of data abuse will be in the courts shortly, as a recent study found that the average Android app sends potentially sensitive data to 3.1 third-party domains, and the average iOS app connects to 2.6 third-party domains. None of the apps notify users that their information is being shared with third parties. Data that’s wrong, data that’s fake, and data that’s shared without permission. I suppose if we could get the fake guys to populate the wrong guys, sharing it without permission wouldn’t be a big deal. Since it’s your personal information, it is.
If you gather data (and who doesn’t), you have a responsibility to keep it secure and not to use it for purposes beyond what the owner of the data (that would be you and me) reasonably expects you’ll be doing with it. If you’re disseminating data, especially data that could impact someone’s life and not just your own business, you need to be sure it’s accurate. And if you’re making stuff up, please just go away.
They’re not just data points, folks. They’re people. Maybe they’re lawsuits in waiting, or maybe they’re your spouse, kids, or parents. Let’s be careful out there, ok?