Market research survey: how to design one and when to follow up with interviews

A market research survey asks a defined group of people the same questions so you can count how common each answer is. It works well when you already know what to ask and whom to ask. It works badly as the first step, because a survey can only measure the answer options you thought to offer, and it cannot ask a follow-up question.

A reliable order is: a few interviews to learn the words and options people use, then the survey to count them, then a handful of follow-up conversations to explain anything surprising.

A worked example: an ordering portal for restaurant customers

Suppose you run a regional food distributor with about 600 independent restaurants on your customer list. Most orders arrive by phone call or text message. You are considering an online ordering portal and want to know who places orders at each restaurant, when, and what goes wrong. This example is hypothetical and used throughout the page.

Decide whether a survey is the right tool

AAPOR's best practices for survey research start before any question is written. They ask whether your objectives are specific, whether other surveys already collected the data, whether another method would fit better, and whether a survey alone is enough or you also need records.

In the distributor example, your order records already show when orders arrive and through which channel. Do not ask restaurants what you can already count. Use the survey for what the records cannot show: who at the restaurant places the order, what goes wrong with it, and how much that matters to them.

Write the decision the survey supports in one sentence before drafting questions. For example: "Build the portal if a large share of restaurants report ordering problems that a portal would fix." Every question should help answer that sentence. Cut the ones that do not.

Write questions people can answer accurately

Pew Research Center's guide to writing survey questions and the AAPOR page agree on a few rules that matter most for business surveys:

• Ask about one thing at a time. A question that combines two ideas, such as ordering and delivery, gets answers you cannot interpret.
• Make answer options complete and non-overlapping. Include "don't know" or "does not apply" where people may need them.
• Keep lists short. Pew suggests four or perhaps five answer choices at most for most opinion questions, since people struggle to keep more in mind.
• Avoid agree/disagree statements. Pew notes that some respondents tend to agree with whatever statement they are shown, and recommends offering a choice between alternative statements instead.
• Prefer a yes/no question per item over "select all that apply." Pew found that forced-choice questions tend to give more accurate answers and generally avoids select-all lists.

Questions about what someone did recently are usually easier to answer accurately than predictions about what they would do. A predicted "yes, I'd use a portal" costs the respondent nothing.

Build answer options from interviews, not guesses

The answer options you offer shape the result. Pew describes an experiment in which 58% of respondents chose the economy as their top voting issue when it appeared on a list, but only 35% volunteered it when asked the same question without a list. In the open-ended version, 43% gave an answer that was not on the closed list at all.

That is why Pew notes researchers sometimes run an open-ended pilot first and build the closed answer options from the most common responses. For a small business, five or six short interviews do the same job. If you had written the distributor's problem list from your desk, you might have missed the one restaurants mention most. Customer interview questions has wording for those first conversations.

Order the questions and keep it short

AAPOR recommends putting general questions before specific ones, starting with easy questions, keeping the questionnaire short, and letting people skip a question or answer "don't know," because forcing an answer increases the risk they quit. Pew adds that demographic questions should not open a survey unless they are needed to check eligibility.

For the distributor, that means: first how the restaurant placed its last order and who placed it, then which problems happened recently, and only at the end a description of the portal and a question about it. If the portal description came first, it would color every answer after it.

End with one optional question: "May we contact you for a 15-minute follow-up call?" You will need that list later.

Test it before you send it

AAPOR recommends pretesting the questionnaire with people similar to your respondents, often through cognitive interviews, where the person explains how they understood each question and arrived at an answer. It also recommends a pilot run to check that the whole process works.

In practice, sit with three or four restaurant managers and watch them fill it in. You will find questions they read differently than you intended, and options that do not fit their situation.

Choose who receives it and read the numbers carefully

AAPOR calls the list you draw respondents from the sampling frame. The distributor's customer list covers only its customers. Restaurants that buy from a competitor are not on it, so the survey cannot tell you how the whole local market orders.

Treat percentages with care. Pew explains that a simple random sample of 1,067 people has a margin of error of about plus or minus 3 percentage points, while a subgroup of about 160 people has a margin of about plus or minus 8 points. If you want to compare, say, full-service and quick-service restaurants, each group needs enough responses on its own. And a survey emailed to a customer list, answered by whoever chooses to respond, is not a random sample. AAPOR notes such nonprobability samples need special care in analysis and reporting. Read the results as a rough picture of your respondents, and say so when you share them.

Quantitative market research covers sample planning and other ways of measuring in more depth.

Follow the survey with interviews

A survey tells you how many. It rarely tells you why. Schedule follow-up conversations when:

• A result surprises you. Suppose 40% of restaurants say the chef places orders by text late at night. Is that a preference, or the only time anyone is free?
• Many people choose "Other" and write similar things. That is an answer option you missed.
• Two groups answer very differently and you need to know what separates them.
• You need to understand how a decision is made, such as who would approve paying for a portal.

Call five to eight people from the "may we contact you" list, and ask about a recent specific event rather than their opinion of the survey topic. Interviews vs. surveys explains how the two methods fit together.

Your next step

Write the decision your survey supports in one sentence, then list the three to five questions that answer it. Before drafting options, run a few interviews with people like your respondents.

If the people you need to hear from are not on your own list, such as purchasing managers at restaurants that buy from someone else, Instant Expert can find people who match a description you write. You review who it finds, it sends your invitations, and you pay only for calls that get booked. The directory page for procurement professionals in restaurants is one place to start.