Research methods
Generative research: what to do before you know the problem
Generative research finds the problem worth solving before you design anything: open interviews, field visits and diary studies with people who do the job.
Generative research is the research you do before you know which problem to solve: you study how people do a job today, what goes wrong, and what they wish were different, so you can decide what to build. Evaluative research comes later and checks whether something you built, or sketched, works. If you can't yet write the problem down in one sentence that a customer would agree with, you need generative research, and testing a prototype will only tell you how usable the wrong thing is.
The methods are simple to name: open interviews, watching people work, and diary studies. The hard parts are talking to the people who actually do the job and turning a pile of notes into a short list of problems worth solving.
Generative research asks what to build, and evaluative research asks whether it works
Nielsen Norman Group's guide to choosing UX research methods splits product work into three phases. In the first, the goal is to find new directions and opportunities, and the methods are "often generative in nature, because they help generate ideas and answers about which way to go." It lists field studies, diary studies, interviews, surveys, participatory design and concept testing for that phase.
The later two phases evaluate a design. NN/g calls methods that improve a design formative (card sorting, usability testing) and methods that measure how well a finished product performs summative (benchmarking, A/B tests, analytics). Many teams group both under the name evaluative research. Maze, which sells a research platform, defines evaluative research as assessing a specific product solution to make sure it is easy to use and works as intended.
| Generative research | Evaluative research | |
|---|---|---|
| Question | What problem should we solve, and for whom? | Does this design or product solve it? |
| You need first | A group of people and a job they do | Something to show: a sketch, prototype or live product |
| Typical methods | Open interviews, field visits, diary studies | Usability tests, A/B tests, benchmarks |
| Useful output | A short list of problems, in customers' words | A list of fixes, or a measure of performance |
Short version: if you have nothing to show yet, or you are not sure the thing you have is aimed at a real problem, you are doing generative research. If you have a design and want to know whether it works, run a usability test instead.
Start generative work when the brief is a group of people, not a feature
Some signs you are at the generative stage:
- The brief names a market ("small accounting firms") and no specific problem.
- Each person on the team describes the customer's main pain differently.
- Your list of ideas is long, and nobody can say which problem each idea fixes.
- A big customer asked for a feature, and you don't know whether anyone else has the problem behind it.
The trap is jumping to evaluation because it feels like progress. A prototype gives you something to point at in meetings. It also narrows every conversation to reactions to your idea, when what you need is the story of the job.
This does not fit every project. If the problem is well understood and you are choosing between two designs for it, generative work is a detour.
Three methods do most of the work
GOV.UK's service manual says early research should find out who your likely users are and what they are trying to do, how they do it now, and what problems or frustrations they run into. It recommends observing people, interviews and visits, and reading existing data such as support logs. In practice that means three methods.
1. Open interviews about past events. Ask people to walk you through the last time they did the job, from start to finish, and follow whatever surprises you. Avoid questions about your idea; there isn't one yet. In-depth interviews covers how to plan and run them.
2. Contextual inquiry. Watch someone do the real work where they do it, and ask questions as they go. NN/g's research methods glossary defines it as a field study in which the researcher watches the user perform their task naturally, and notes it is typically used in the early discovery stages. It catches the steps people forget to mention because they stopped noticing them years ago. Contextual inquiry covers how to run a session.
3. Diary studies. Participants record what they do and experience as it happens, over time. NN/g's diary study guide describes reporting periods from a few days to a month or longer, chosen by how often the behavior happens. Use one when the problem comes and goes, such as a monthly close, a seasonal rush or a task that takes ten minutes twice a week and is easy to forget by the time you interview someone.
Pick by the question:
| If you need to learn... | Start with |
|---|---|
| How people decided, what they tried and why | Open interviews |
| The real steps, tools and workarounds in a complex task | Contextual inquiry |
| What happens across weeks, and how often | A diary study |
NN/g's list also includes surveys. Hold them back until interviews give you the words and answer options to ask about; otherwise you will be counting your own guesses. Customer research methods compares the wider set.
Talk to the people who do the job, not the people who describe it
Managers, consultants and your own sales team can tell you how the work is supposed to go. The person doing it can show you how it goes. Generative research depends on the second version, because that is where the workarounds are, and workarounds are where problems hide.
So recruit by what people did recently, not by title. "Prepared at least 20 individual tax returns last season" finds the right people more reliably than "accountant." If the person who pays for a product is different from the person who uses it, talk to both, and keep their notes separate.
A made-up example: the client who sends a photo of a shoebox
Here is a made-up example. A three-person team wants to build software for small accounting firms, meaning firms with 2 to 15 staff. That is the entire brief.
They plan three pieces of work:
- Eight open interviews with tax preparers and managers at small firms, each starting with "Walk me through the last client return you finished. Where did it start?"
- Two half-day visits to watch a preparer work through returns in March.
- A two-week diary study in the busy season, where six preparers log each time they wait on a client for something.
The interviews keep returning to one moment: collecting documents from clients. The visits show a preparer keeping a spreadsheet of missing items per client, color-coded by how many times she has asked. (One client sent a phone photo of a shoebox and considered the matter closed.) The diaries show the waiting is spread across the whole season, with most of it falling on the same handful of clients each time.
Nobody in the interviews asked for document-collection software. Several did describe the spreadsheet with some feeling.
Turn findings into opportunity areas, not features
Teresa Torres, who coaches product teams, defines an opportunity as an unmet customer need, pain point or desire (Product Talk), and says opportunities emerge from the specific stories people tell about past behavior. That is the shape generative findings should end up in.
- Cluster your notes. Put every observation, quote and diary entry on its own card and group them by what the person was trying to do. Affinity mapping is the usual name for this step.
- Write each cluster as a need in the customer's terms. "I can't start a return until the client sends everything" is an opportunity area. "Automated document reminders" is a solution.
- Check for solutions in disguise. Torres's test: if there is only one way to address it, it is a solution. "I lose track of what each client still owes me" has many possible fixes.
- Attach evidence. Under each area, note how many people described it and one story that shows it. Report counts of people ("six of eight preparers"), not percentages.
- Rank. Put first the areas that came up across interviews, visits and diaries, and that cost people the most time or stress.
For the accounting team, the top areas might be "I don't know what each client still owes me," "the same few clients cause most of the waiting" and "I re-key figures from client documents by hand." That short list is the output. Choosing among the areas and testing ideas against them is product discovery.
Stop when you can name the problem and the people who have it
GOV.UK's guidance says you have done enough when you understand the different kinds of people who use your service. A practical version: you can write the top opportunity areas in a sentence each, name who has each one, and back each with several stories from different people. Once new conversations mostly repeat what you have heard, switch to evaluative work and start testing ideas.
Your next step
Write down the group of people you want to serve and the job they do, then book open interviews with five of them this week, asking each to walk you through the last time they did that job. Instant Expert can find people matching a description such as "tax managers at accounting firms with fewer than 15 staff," and send your invitations with the offer you set. The offer is what each person receives, Instant Expert's fee is added on top, and you pay only for calls that get booked. The directory pages for CPA firm tax managers and user research experts are good places to start.