Today is one of those screeds in which I point out a problem but don’t offer a real solution. I apologize in advance. Maybe just ringing the alarm bell a bit is enough of a help but you’ll be the judge.
Like you, I read a lot articles published in trades. Most of what I see comes to me in the form of emailed articles and/or newsletters. There’s a lot of research cited in these pieces and many of them offer opinions with respect to a good course of action one should take to avoid a problem or improve performance. What I find interesting is how often I’ll finish the piece, look at the author’s bio, and realize that I just spent a couple of minutes reading a self-serving puff piece. For example, a nice article citing research on how content marketing can drive sales was offered by a guy who runs a content marketing company, which also commissioned the research. Funny how often the research conducted by “independent” firms says great things about the company that commissioned it, isn’t it?
That’s the problem I offer up today. It’s hard to know how meaningful research is when those who pay to have it done have a vested interest in the outcome. We saw this during the last political season. There were “Republican” polls that showed the presidential race one way, “Democratic” polls that had it the other way, and “independent” polls that were a mixed bag. Usually, the party-sponsored polls had their guy winning, and you’re probably familiar that the only entity that called the race almost perfectly was Nate Silver of The New York Times who uses a “poll of polls” methodology that wiped out the inherent biases.
We need to question those who ask the questions. That doesn’t mean ignore or even discount the research. What it does mean is to think about what vested interest the sponsor of any research has in the outcome and look for places where a question can be phrased in such a way as to twist the outcome. All reputable research will show you how the question was asked. It’s up to you to consider the inherent bias before taking anything as gospel. Even the blather put out in this space!