Dan Ariely and Qualtrics

An interesting interview with Dan Ariely in part about his use of surveys. Particularly interesting for students in Behavioral Econ and BUS 230. From the interview:

WHAT ADVICE DO YOU HAVE FOR MARKET RESEARCHERS?


What people do is not what they say. In general, most of our findings suggest that we should not think very highly of focus groups.


I suspect that many market research companies use focus groups because they want a statement to put in a PowerPoint presentation for a client. But the problem is that what people say doesn’t always reflect what they think or, even more important, the reasons for their actions.


SO, THE QUESTION IS, WHAT SHOULD YOU ASK?


You can ask people what they have done, or what they think they will do in the future. They can answer those questions. But, the moment you get into reasons, into why, interpreting their answers as correct becomes much more tricky.


It is also important to ask about categories that are ones which people can accurately quantify. The more concrete your response scale is, the more likely people are to answer your questions in an informative way.


HOW WOULD YOU RECOMMEND RESEARCHERS USE SCALES?


Take the question, “How often have you done X?”


If you create a scale from “very rarely” to “very frequently”, that’s not as useful as offering, “2 times last week” or “3 times last week”.


The more concrete you get, the more people feel inclined to answer accurately and honestly.


ANY MORE ADVICE ON HOW TO USE SCALES?


Often people use a 5-point response scale, but we find most people have an aversion to the extremes. This means that when we use a 5-point scale, effectively we are using a 3-point scale. That makes our sensitivity of our measurement less useful.


I would encourage researchers to think whether people will have that extreme aversion, and if that is the case, to have more levels. A continuous scale with just 2 anchors in the extremes in such cases is ideal.