User interviews generate lies

As designers, our job is to make sure our team really understands their target audience. We have many questions we want to answer: What needs do our customers have? How do they complete the task today? Why do they make those decisions, and act that way? Do they like our product? Will they use this new feature?

To answer these questions, we turn to one of many methods we have stolen from social science, simplified the nuances out of and used outside of their intended context: User interviews. Unfortunately, the answers you get during a user interview will not give you a trustworthy solution to these questions – they will mostly just generate lies and half-truths. This is because of, amongst other things, two weaknesses with the method:

Every time we “remember” we retrieve a memory by reconstructing it. The product of this reconstruction is affected by both how the question was asked in the moment, and any new information we have received after the memory was first created that could affect our understanding of it. Even memories as important and emotional as how people first heard the news of a catastrophe are often misremembered. When researchers have studied these “Flashbulb memories” (for example how Americans heard about the Challenger disaster or the 9/11 attacks), they found that people are very sure they remember exactly how they heard the news – even years later. But in reality, they remembered only about half of the details correctly when compared to their own accounts taken only a few days after the catastrophe took place.

If people don’t remember the details around such an important experience, how can we expect them to remember exactly how they went about changing suppliers or using a service five years ago?

The vast majority of choices we make are unconscious. That means that if you ask someone five times why they did something, they really don’t know the answer themselves. In most cases, the choice they made or action they completed happened without conscious thought. Most people are terrible at introspection and have little to no insight into their own thought processes – let alone be able to communicate to you what those processes actually were.

That doesn’t mean that they won’t give you an answer though – partially because there is a social expectation to answer when asked a question, and perhaps that gift card you promised in compensation gives them extra motivation to keep talking… The “answers” given are often a rationalization after the fact to justify their behavior and maintain a consistent self image when our unconscious actions go against our conscious values. We want to view ourselves and rational, thoughtful people, so we make up a thought process that never happened.

Asking people if they will use or buy something in the future is pointless for the same reason. For example, when Storebrand asked their customers if they wanted a world of sustainable investment options they got a clear yes. Customers loved the idea and said they wanted to invest immediately. But when they launched, few ended up actually buying the new products.

First and foremost, be realistic with what you can get out of a user interview. We can learn something about an individual's preferences – or at least how they would express them to a stranger. If we can catch them just after they did a task we can get a decent description of how it was executed – or at least that person’s perception of the fraction of the situation that they paid attention to. You will NOT, however, get the whole truth of how it happened, an answer to how most people would do it, and especially not trustworthy insight as to why they chose what they did or reacted the way they did.

Instead, prioritize direct observation and measurements. Be inspired by ethnography and field studies – get out to where your target audience is to observe what they are doing and ask questions in the moment. Set up measurements in your product to see what users are actually doing. Become a user or customer yourself! Buy the product, experience the service or attempt to solve the task yourself – but always remember your own bias and that you are but one of many diverse users. And if you find out that user interviews are the best you can do with the constraints you have – schedule them as close as possible to the event you want to learn about, do the interview in the context where users would naturally do the task, and take everything they say with a grain of salt.

🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥

Did this article spark something in you? Do you agree or disagree? Join the discussion on UX Norge's slack (Norwegian) or share this article with your colleagues and start the debate! Remember that the goal of Brannfakler is to start a discussion.