I was discussing some numbers with someone the other day. It was clear from the conversation that she was taking every bit of data as gospel. I tried to explain a few important things to keep in mind when working with data and as I thought about it perhaps my thinking could be helpful to some of you out there in screed-land.
We all want as much certainty in our business lives as we can get. Part of that is wanting all of our numbers to be facts. They’re not. You may be familiar with the term “sampling error.” Basically, it means that the data is off because the sample from which the data is drawn is not representative of whatever it is you’re trying to measure. While you might think that, for example, your analytics measure everyone, they don’t. Most of the data we read uses some sampling. Sometimes it’s a timing issue – financial data, in particular, can be skewed based on where we might be in a business calendar or where those who pay us are in theirs.
The point is that there are error rates involved with many of these “facts” because these facts are really just estimates. TV ratings, for example, are probably the most widely known estimates and multi-billion dollar businesses involving networks, agencies, and marketers revolve around numbers everyone knows are not particularly accurate. There are error rates.
Here is the advice I give people. Figure out what questions you’re trying to answer and then find as many different sources of data as you can. If possible, see if you can get multiple people to interpret those data sets. In theory, they should all come up with the same answers. It’s critically important that you NOT tell them what position you’re trying to support (can you find me some information that says we should do XYZ). That is a recipe for disaster because it encourages people only to look at data or interpretations of data that supports what you or they already think is true. That is turning “facts”, which are already often on shaky ground, into a larger fiction, and that’s not what we’re after, is it?
Consult Keith 