Tag Archives: Data

July 18, 2016 · 11:02 am

How Facts Can Be Fiction

I was discussing some numbers with someone the other day. It was clear from the conversation that she was taking every bit of data as gospel. I tried to explain a few important things to keep in mind when working with data and as I thought about it perhaps my thinking could be helpful to some of you out there in screed-land.

We all want as much certainty in our business lives as we can get. Part of that is wanting all of our numbers to be facts. They’re not. You may be familiar with the term “sampling error.” Basically, it means that the data is off because the sample from which the data is drawn is not representative of whatever it is you’re trying to measure. While you might think that, for example, your analytics measure everyone, they don’t. Most of the data we read uses some sampling. Sometimes it’s a timing issue – financial data, in particular, can be skewed based on where we might be in a business calendar or where those who pay us are in theirs.

The point is that there are error rates involved with many of these “facts” because these facts are really just estimates.  TV ratings, for example, are probably the most widely known estimates and multi-billion dollar businesses involving networks, agencies, and marketers revolve around numbers everyone knows are not particularly accurate. There are error rates.

Here is the advice I give people. Figure out what questions you’re trying to answer and then find as many different sources of data as you can. If possible, see if you can get multiple people to interpret those data sets. In theory, they should all come up with the same answers. It’s critically important that you NOT tell them what position you’re trying to support (can you find me some information that says we should do XYZ). That is a recipe for disaster because it encourages people only to look at data or interpretations of data that supports what you or they already think is true. That is turning “facts”, which are already often on shaky ground, into a larger fiction, and that’s not what we’re after, is it?

Leave a comment

Filed under Consulting, Thinking Aloud

Tagged as , , , , , , ,

May 31, 2016 · 11:10 am

GIGO

The Memorial Day weekend gave me a little time to get caught up on some reading. Some of what I was reading were analytics reports (I know – get a life) and while I very much appreciate the cycle of continual improvement Google fosters within their analytics product, that cycle yields a continuously growing amount of data. The problem that I have isn’t so much understanding what I’m reading but trying to figure out why any of it matters to my clients. I also spend time figuring out which of the numbers are lying to me. 

It’s no secret that there are an awful lot of bad actors in the digital world. Once it becomes clear how fraud is detected those bad actors move on to another form. If viewability is important, they create sites where there is 100% viewability but no content of any value. I had a client get all excited about an increase in referral traffic until I pointed out that most of that traffic was coming as a result of referrer spam. When we filtered it out, traffic was flat. Another prospect got excited by the large “stickiness” – time on site and pages viewed – that her site has. They were impressive until you filtered out the IP addresses of her employees, who spent hours a day on the site.

Silly things, I know, but it points to a common problem. An IDG study of a couple of years ago pointed out that nearly half of marketers said they struggle to make sense of the vast amount of data they get. The other half thinks they know what the numbers mean, yet many of their plans are built to achieve unrealistic metrics. The problem is compounded by what the paper identifies as the accuracy problem I mentioned above:

Why is data accuracy still such a big issue? One possible reason is a lack of investment in a defined data management process that includes ongoing, consistent data migration, data maintenance, quality control and governance. Too often data is held and managed in multiple organizational silos. This results in inconsistency, duplication, gaps and errors.

So while “garbage in, garbage out” isn’t a particular revelation, it does serve as an excellent reminder to take out the trash as best you can while compiling all of that data.  You with me?

Leave a comment

Filed under Consulting, Reality checks, Thinking Aloud

Tagged as , , , ,

November 4, 2015 · 11:31 am

Who’s Working For Whom?

Ever encounter a situation where things seem backwards? Maybe you’ve seen a parent being told what to do by a child or a customer being berated by a service rep. It makes you wonder who is in charge or who is working for whom. I have another thought along those lines today, and it has to do with data. There was a post from AdAge by their data reporter, Katie Kaye who wrote the following about the NY Times piece on Amazon: 

The article should inspire us to question the value of decisions based entirely on data to create business efficiencies at the expense of human empathy and the arguable imperfections that can benefit any organization or project.

I like that. It makes you ask who is in charge here: the humans or the numbers. We all ingest more data than we can consume, and, unfortunately, some of us allow that massive intake to be regurgitated as unconsidered decisions. That’s a bad idea. The data is there to serve us, not the other way around.

I’m the first to say that we need lots of data. Without impartial feedback, we’re flying blind, and data can help us make better decisions. The key there is “help US”.   Data without the context of a plan is useless. Data that’s not actionable is useless.  Data that causes us to overreact, however, is  dangerous.  If you watched any election coverage last night, you probably heard a lot about early results and the need to wait for data from key precincts.  How many times has someone in your organization overreacted to an early piece of data, only to find out that it was not at all typical of the overall results?  We need a plan, we need context, and we need a little patience.

When we chase after outliers, we’re working for the data.  That’s backward.  Data, and all the other technological tools in our arsenals, needs to work for us.  Make sense?

Leave a comment

Filed under Helpful Hints

Tagged as , , , , ,