Posts Tagged ‘interpreting questionnaire data’

When Should I Believe Survey Results?

When it comes to survey results, people often go to one of two extremes:  They either decide that “all surveys lie” and they should never believe them, or they haplessly believe the results of every blog poll on the Internet.

As with most things, the best approach lies somewhere in the middle.

Why should you care?  Although we stress the importance of conducting regular surveys by putting a survey system in place in your business, you should go even farther in harnessing the power of surveys:  You should be paying attention to survey results related to your niche and using those data effectively, because very simply, it can help you make more money.

To do that, though, you have to know what results to believe.

The simple fact is that people CAN’T pull the wool over your eyes if you understand some basic information about surveys.  In fact, if you know the information we’re providing in the “Turn Surveys Into Cash” knowledge modules, you’ll have everything you need to be a good “survey consumer.”

But because we want EVERYONE to be able to use survey information effectively in their businesses, we’re providing some of the tips here.

To make it clear, I’m going to use an example from the social media space.  One of my Google Alerts (and if you’re not using Google Alerts to find data in your niche, you should be) highlighted a press release describing some surprising results from a new research study:  It shows that people who provide care to a disabled spouse actually live LONGER than those who don’t.

This important study puts a new twist on social support, because it shows that not only RECEIVING social support extends your life (we’ve known that for a long time) but GIVING support does, as well.

If this is true, it’s definitely something I want to report to my social networking audience on my blog.  But if I report results from a BAD survey, I’ll look stupid.  So how do I decide what to do?

  • The first thing I did was to make sure it was a “real” survey, not some blog poll.  It was.
  • The second thing I did was to see who conducted it.  Turns out it was done by the Institute for Social Research (ISR) at the University of Michigan and funded by the National Science Foundation (NSF).  ISR is a well-known research institute, one of the best in the nation (they were smart in the press release–they gave a description of ISR at the end).  This certainly doesn’t GUARANTEE the results are solid, but it enhances my confidence.  And NSF is an independent agency whose standards for funding research are really high (I know because I’ve served on NSF review panels).
  • So far, so good.  Now I want to know something about the sample.  The information in the press release looks good, and when I get more detailed information, it still looks good–the sample was large (1,688 independently-living couples) and drawn according to solid, scientific principles.
  • Third, I wanted to know if they controlled for other factors.  We could see a relationsip, or correlation, between caregiving and living longer that was really due to some other factor the researchers didn’t control.  So, for example, if the caregivers were all wealthier than the non-caregivers, we could really be seeing an effect of the economic situation, not caregiving itself.  In the press release, the authors told us they controlled for such things as age, gender, health, net worth, education, employment status, and race.  Those are the critical factors in this kind of research and they enhance my confidence that the relationship between caregiving and living longer is “real.”

In the end, I decide the study looks really solid, so I report on it to my blog readers.

These are some of the steps you should take to make sure that survey results are solid.  Knowing how a sample was drawn, what the response rate was (the percentage of “invited” respondents who completed the survey), how the analysis was done are all critically important.

It’s also important to know who funded the survey.  I’m much less likely to believe research results if the study was funded by a group with a “vested interest,” whether that’s a drug company or a political party/interest group.

So the next time you see survey results reported somehwere, look carefully.  If they don’t provide the information you need to ask these questions, then ask for it.  Be an “informed consumer” of data.

And USE data regularly in your business, whether it’s to report on new findings in your niche or to get ideas for new products and markets.  Once you get in the habit of doing that, you’ll understand what a powerful tool you have at your disposal.