Problem interviews vs solution interviews: why a good demo proves nothing yet

Problem interviews vs solution interviews is mostly a question of order: run the problem interview first, unless you can already describe how people handle the problem today. A problem interview investigates what someone does now; a solution interview explores how they read and judge a proposed approach. A warm reaction to a demo can't tell you whether the problem mattered before you showed up with a product.

Let the open question pick the method. If you can't describe a recent event, the current workaround and its consequences, start there. If you understand the workflow and need to examine a proposed change, bring a concept or prototype. If you need to know whether people can use it, give them a task instead of asking for an opinion.

1. Use a decision tree to pick the method

This is a planning tool, not a fixed sequence for every feature.

1. Can we name the person, the triggering event and the current response? If not, study recent examples through problem interviews or observation.
2. Do we understand the consequences of that response, including whether anything needs to change? If not, reconstruct the steps and consequences before proposing a fix.
3. Is the question what our proposal means to people? Show a concept and ask them to explain it in their own words.
4. Is the question whether they can complete a task? Run a usability session with a realistic goal and a prototype that supports it.
5. Is the question whether they will adopt or buy? Examine the decision process and test a concrete next commitment. A friendly interview leaves that open.

Existing evidence can let you start further down: a team with carefully analyzed support cases and observed workflows may already know the problem. What counts is how strong and relevant that evidence is, not whether someone performed a ritual number of interviews.

GOV.UK's guidance separates learning about people's needs from defining a preferred solution: a need should describe the outcome or problem, not restate a proposed feature (GOV.UK guidance on learning user needs).

2. Start with a recent, specific problem

Here is a made-up example. A product team is considering a shared queue for delivery exceptions, and its first idea is a dashboard that flags delayed deliveries to a dispatcher. That dashboard is a proposal. The real problem might be deciding who is allowed to promise a new delivery time.

Start with one recent exception. A script the team could adapt:

• "Think of the most recent delivery that missed its planned window. When did you first learn about it?"
• "What did you do immediately after that?"
• "Who else got involved, and what did each person need to decide?"
• "Where did the updated information go?"
• "Did any part take more effort than you expected? What happened?"
• "How did you know it was resolved?"

Follow up on their details. If they mention messaging an account manager, ask what they needed from that person. Don't replace their story with your theory of why dispatch software is broken.

GOV.UK's in-depth interview guidance recommends neutral, open questions and concrete stories over general accounts of what should happen (GOV.UK interview guidance). That keeps the conversation anchored in experience.

3. Change the script when the solution comes out

Say the early research suggests dispatchers lose track of exceptions while waiting for someone else's decision. The team now has a narrower proposal: an exception card showing the outstanding decision, its owner and the last update.

A concept conversation could ask:

• "What do you think this card is for?"
• "Which information would you need to check before acting?"
• "Where would this fit in the delivery incident you described?"
• "What would still happen outside this screen?"
• "What would make this approach wrong for your team?"

These test interpretation and fit, not whether a dispatcher can use the interface under pressure.

For that, set a task: "This delivery will miss its window. You need the account manager to decide whether to offer a revised time. Show what you would do." Don't mention the button you hope they will press. (Pointing at it is the research version of grading your own exam.)

GOV.UK defines moderated usability testing as observing people attempt specific tasks, with a believable goal and no instructions that give away how to complete it (GOV.UK usability testing guidance).

4. Keep the evidence from each part separate

You can combine methods in one session, but mark the moment the concept appears. After someone sees it, their description of the original problem may bend toward it. You can't get the unprompted version back in that conversation.

Use three note sections:

Keep observations apart from interpretations. "Asked where to change the owner" is an observation. "The ownership model is too complex" is an explanation you may need to investigate.

If you have to help because the prototype is incomplete, record the help. The rest of the session still counts; it just can't show the assisted step works on its own.

5. Read mixed results without forcing a verdict

Someone can describe an important problem and dislike your solution. That is a reason to rethink the approach, not to throw out their account. Equally, someone can enjoy a polished demo for a situation they rarely face. Enjoyment isn't urgency.

In the delivery example, imagine a dispatcher likes the exception card but says only an account manager may change customer commitments. The next step isn't necessarily another dashboard feature; the team needs to understand that handoff and talk to the person on the other side of it.

Write a short decision record after each round:

• Supported: What did we observe or reconstruct?
• Challenged: Which assumption did the evidence contradict?
• Unanswered: What couldn't this method establish?
• Next: Which person, artifact or behavior would answer the remaining question?

The best outcome may be a smaller proposal: show the pending decision and notify its owner, before building a complete dispatch workspace. Good research changes your next move without having to end in a yes or no on the whole business.