How to write survey questions that produce decision-grade data

Write survey questions backwards. Start with the decision the survey will inform, list the few facts that would change that decision, and write one plain question for each fact. Then test every question by asking: if the answer came back the other way, would I do anything differently? If not, cut it.

Most weak surveys fail before the wording. They ask about everything the team is curious about, and the result is a pile of percentages nobody can act on. The Nielsen Norman Group's survey best practices puts "ask only questions that you need answered" first, and says to keep surveys short.

Step 1: Write the decision in one sentence

Name the choice, the options and the threshold. "Learn about our customers" is not a decision. "Build the lien-waiver feature before the change-order feature if at least a third of respondents handled lien waivers in the past month" is.

AAPOR's best practices start with the same discipline: are the objectives unambiguous and specific, have other surveys already collected the data, and would another method fit better? AAPOR also says surveys should not be used to produce predetermined results or to sell. If the real goal is a quote for your pitch deck, you will write leading questions without meaning to.

Step 2: List what you need to know, then remove what you already know

Under the decision, list the facts that would move it. Then cross off anything you can get elsewhere. NN/g advises against asking questions you can answer yourself, such as demographics you could look up for existing customers. Your product analytics, CRM or order history often already hold the "how often" and "how much."

What remains is usually a short list: what people did recently, how they do it now, what it costs them, and how they feel about specific options.

Step 3: Ask about recent behavior, not predictions

NN/g gives a useful three-way comparison:

Future prediction (unreliable): "How likely are you to use this product?"
Present estimate (still shaky): "How often do you use this product in an average week?"
Past estimate (better): "Approximately how many times did you use this product in the past 7 days?"

People are poor predictors of their own behavior, and "an average week" invites them to remember an unusual one. Ask about a specific recent period, include "approximately," and offer ranges.

Step 4: Choose the question format that fits the fact

A count or category needs a closed question with complete, non-overlapping options. A degree of opinion needs a balanced rating scale. A priority needs a ranking or a forced trade-off. Something you cannot predict the answers to needs an open-ended question, used sparingly. Types of survey questions walks through each format, and Likert scale questions covers agreement scales in detail.

Step 5: Write in the respondent's words

Pew's guide to writing survey questions asks for simple, concrete language, one question at a time, no double negatives and no unfamiliar abbreviations or jargon. Your internal names for features mean nothing to respondents. "Change order" means something to a construction subcontractor; "CO workflow module" does not.

Pew also notes that researchers sometimes run open-ended pilots first and build answer lists from the most common responses. A handful of conversations does the same job for a small business, and it stops you from writing answer options that reflect your assumptions.

Finally, check each question against biased survey questions: no leading adjectives, no built-in assumptions, no pleading context.

Step 6: Add the exits

NN/g recommends an opt-out on every multiple-choice question ("Not applicable," "Don't know," "Other," "Prefer not to answer"), kept as separate options because they mean different things. It also recommends making most questions optional: a required question someone cannot answer produces either a random answer or a dropout. AAPOR agrees that requiring an answer increases the risk that people leave early.

Step 7: Test the questions on real people

AAPOR recommends pretesting with people similar to your respondents, often through cognitive interviews where they explain how they understood each question and how they arrived at an answer. Even three or four sessions can expose a question that people read differently from the way you meant it.

A worked example: a subcontractor billing survey

This example is hypothetical. A founder is building billing software for electrical and plumbing subcontractors and must choose which of two features to build first. The decision sentence: "Build lien-waiver tracking first if at least a third of respondents handled one in the past month and more than half of those say it caused a payment delay."

The questions follow directly:

"In the past 30 days, approximately how many lien waivers did your company send or receive?" None / 1-5 / 6-20 / More than 20 / Don't know
(If any) "In the past 30 days, was any payment to your company delayed because of a missing or incorrect lien waiver?" Yes / No / Don't know
"In the past 30 days, approximately how many change orders did your company submit?" None / 1-5 / 6-20 / More than 20 / Don't know
(If any) "In the past 30 days, was any payment delayed because a change order was disputed or unsigned?" Yes / No / Don't know
"Is there anything else about getting paid on jobs that you'd like us to know?" (optional, open)

Suppose 150 people respond and 60 handled a lien waiver. That is 60 ÷ 150 = 40%, above the one-third threshold. If 36 of those 60 report a delay, that is 36 ÷ 60 = 60%, above half. The decision rule says build lien-waiver tracking first. Five questions, one decision. The founder did not ask "Would you use lien-waiver tracking?" because a yes costs the respondent nothing.

When to interview instead of survey

NN/g's article Should you run a survey? places surveys as the one common research method that is both quantitative and attitudinal. They measure how many people hold a view. They are weak at "why" questions and at observing behavior. NN/g also warns against running a survey just to give a skeptical executive a big number when the underlying question is qualitative.

Interview first when:

You cannot yet write the answer options, because you do not know how people handle the problem.
The question is "why": why deals stall, why customers churn, why a workaround persists.
The decision depends on how a purchase is approved and who is involved.
Your audience is small and specialized, such as finance leads at mid-size subcontractors, and a survey would reach too few of them to count anyway.

In those cases, five to eight 30-minute conversations will often tell you more than a survey, and they will give you the vocabulary and answer options for a survey later. Interviews vs. surveys covers the choice in more depth, and customer interview questions has wording for the calls.

Your next step

Write your decision sentence and the threshold that would change it. List the three to six facts that matter, cross off what you already have, and draft one question per fact using the steps above.

If the people you need are not on your own list, Instant Expert can find people who match a description you write, such as controllers at specialty subcontractors. You review who it finds, it sends your invitations, and you pay only for calls that get booked. The directory page for finance professionals in construction shows the kind of people you can reach.