Interviews versus surveys: match the method to the decision

Use interviews when you need to understand an experience in detail and follow up on what you do not yet understand. Use a survey when you have clearly defined questions and a sampling approach suitable for the comparisons or estimates you want to make.

Neither method automatically proves demand. Interview accounts do not establish population prevalence, and a large collection of survey responses does not become representative merely because it is large.

Start with the sentence you want the research to support. The wording of that claim will often reveal whether you need an interview, a survey, direct observation, existing records, or a combination.

Write the claim before choosing the tool

Compare four possible questions about a product onboarding process:

The right method depends on the uncertainty. A survey asking “Was onboarding easy?” cannot substitute for observing where someone gets stuck. An interview explaining one failure cannot tell you how frequently that failure occurs across the user base.

NN/g distinguishes research methods partly by what they reveal about reported attitudes and observed behavior. Use that distinction to avoid claiming that an answer describes an action you actually watched. Read the research-methods overview.

Use interviews to learn the shape of the problem

An interview can help you discover vocabulary, decision steps, constraints, and alternatives you did not anticipate. The interviewer can ask what a term means, request a specific example, or follow an unexpected branch.

Suppose a participant says they abandoned setup because they lacked “approval.” A conversation can investigate whether that meant manager permission, access to company data, a purchasing decision, or a technical authorization. Those are different problems.

Keep the account anchored in a recent event where possible. Label estimates and secondhand information. Ask about cases where the process worked as well as cases where it failed.

Interviews are not automatically unbiased or deep. Leading questions, poor participant fit, and selective interpretation can produce a convincing story that the evidence does not support.

Use surveys when the question can be measured clearly

Before building the form, define the population, eligibility, measure, and period. “New users” might mean people who created an account during a particular month, people who attempted setup, or people who completed it. The distinction affects the denominator.

AAPOR’s best practices emphasize specific research objectives, sampling, question design, and transparent methodology. They also recommend considering whether a survey is the right method at all. Read AAPOR’s survey guidance.

A useful survey question needs concepts the audience understands. If “approval” remains ambiguous, the resulting count may combine several different experiences. Interview work or questionnaire pretesting can help identify that problem before the survey is fielded.

A survey can include open-ended questions, but those answers require interpretation and coding. A long form with many open questions is not automatically an efficient substitute for a conversation.

An illustrative sequence

Imagine a team investigating setup abandonment. It first examines whether the product records the relevant setup steps reliably. The team then interviews people who recently attempted setup to understand what the events do not explain.

Suppose the illustrative conversations identify several possible meanings of “waiting for approval.” The team can define those categories more carefully, test whether prospective respondents interpret the wording as intended, and decide whether a survey would answer an important remaining frequency question.

A later task-based session could investigate whether a proposed change helps a person complete setup. That activity answers a different question from either the interview or the survey.

This is an example of sequencing methods, not a mandatory order. Existing evidence may already answer part of the question, or the decision may be small enough that a narrower study is sufficient.

Do not turn sample size into a substitute for design

A survey distributed through a founder’s social network may reveal the views of those respondents. It does not automatically estimate the views of every potential customer. Report how people were recruited and which population the results can reasonably describe.

Likewise, selecting interview participants because they are easy to reach can omit an important role or context. Increasing the count from the same source may leave the same gap.

Avoid attaching a conventional margin of sampling error to a sample that does not justify it. If the decision requires defensible population estimates, obtain appropriate survey design and analysis expertise rather than borrowing a generic sample-size calculator.

For exploratory work, report the accounts and their context without disguising them as statistical validation.

Compare effort across the whole study

Interviews require recruiting, scheduling, preparation, moderation, records, and analysis. Surveys require question design, testing, recruitment or sampling, monitoring, cleaning, and interpretation.

The number of minutes a participant spends is only one part of the research workload. A short survey with a poor measure can create more uncertainty than it resolves. A focused interview can answer a narrow question without becoming evidence for every market claim.

Choose the smallest study that can responsibly inform the decision, and state what it will not answer.

Finish with a method statement

Before launching, write:

“We need to decide [action]. The uncertain claim is [claim]. We will collect [evidence] from [defined people or records] using [method]. This can establish [scope], but it cannot establish [limit]. We will use [additional method or next step] if [specific gap] remains.”

That statement makes it harder to choose a method solely because the team already owns the software. It also gives the final report a boundary: the conclusion should fit the evidence the study was designed to collect.