Aug 13

Asking customers what you want to hear

Check out my latest post on Inside Intercom:

There is nothing more insightful than learning how to improve your business directly from your customers. The challenge is asking the right questions.

Qualitative research data is only as good as the questions asked. Bad questions mean bad data, no matter how helpful and enthusiastic your customers are in response…

Nov 11

Why I speak in absolutes

More than once in the past I’ve been criticized for over-simplifying and over-generalizing research findings and for making bold claims that go beyond the scope of the research. But I’ve continued to do it, because I think when done correctly, it’s the best way to get people to act on the insight you generate.

I’ve just finished reading The Thank You Economy by Gary Veynerchuk (which by the way is an excellent read and has lots of content you can re-use if you’re trying to persuade others of the value of social media) and he makes similar claims about speaking in absolutes and the value of over-generalizing research findings:

Why I speak in absolutes

Because if I give you an inch, you’ll run a mile with it. When I said in 1998, “You’re dead if you don’t put your business on the Internet and get in on ecommerce,” was that true? No. But boy, can you imagine trying to be in business in 2010 with zero web presence? I’d rather shock you into paying attention, and admit later that business rarely requires an all-or-nothing approach, than take the chance that you won’t take the situation seriously enough.

The trick is to explain that your bold claims are your opinion, based on the research you did: “Here is what I found, here is what I think it means, and here is what I think you should do”.

All communicated research should include those three elements. Far too many people stop at “Here is what I found”. All research is subjective to some degree, and open to interpretation. So be bold, make claims that will make people listen, be open to feedback, and be prepared to be wrong.

Oh, and stop communicating your research in reports and slide decks. Reports are useful for research synthesis. Reports are for researchers. To get other people to act, communicate face to face.

Mar 09

Fake social networks for research

Last week we ran some research in the lab. We were testing comprehension of some new concepts in the communication space. Usually this would be straightforward – create screens/flows for key tasks and see if people understand what is going on and what to do next. The issue though is that sociability is full of subtleties. I may do one action for one friend, but something different for another. And when looking at a user interface, I might think one thing if I have one friend in mind, but something different if I have another friend in mind. For example, if I have a close friend in mind I may be concerned about privacy of our conversation, but maybe less so for a friend of a friend. So to try and uncover some of that subtlety, we did an exercise to make the tasks as “real” as possible. We created a fake “real-life” social network…

…and asked research participants to substitute real people in their life for these fake people. So they took the print-out of this diagram and scribbled names of real people in their life on top of the fake people. Then when they encountered a mocked up design with the fake names, they could substitute the real people. Likewise, when they were asked to do a task, they substituted the real people and we learned a lot more about the context of that task in their real life. For example with fake names we may get this feedback on a particular part of U.I.:

“…I’d type the message here and then click ‘send’…”

But with real people substituted in, we got this instead:

“…well, I could type in here and click ‘send’ but I wouldn’t do that because Joe never checks his email, so I’d phone him to make sure…”

We were worried that the cognitive load of substituting people would be too high, but it wasn’t – it worked out really well and we learned a lot about the broader context of what we were proposing. One thing we did have to do was constantly remind people to reference their sheet of real names. People slip into “hypothetical mode” very easily.

I think this tactic of “substitution” could work well in many lab research studies. Try it out!