Open-ended survey questions: the text box people skip, and how to fix it

Open-ended survey questions with examples that work, what they cost in skipped answers and reading time, and when a 30-minute interview beats a text box.

Instant Expert EditorialPublished Updated 6 min read

Open-ended survey questions let people answer in their own words instead of picking from a list. Use them when you can't predict the answers, when you want the words customers actually use, or to catch the issue your closed questions missed. Use few: people skip them far more often than multiple-choice questions, and someone has to read every answer. If most of what you want to learn needs a follow-up question, run interviews instead.

They find what your list forgot

The Nielsen Norman Group's article on open-ended vs. closed questions puts it simply: open-ended questions let you find more than you anticipated. Closed questions return only the options you thought of.

Pew Research Center's guide to writing survey questions shows how big that gap can be. After the 2008 U.S. presidential election, 58% of respondents picked the economy as their most important issue when it appeared on a list, but only 35% volunteered it when asked the open-ended version. And 43% of open-ended answers named something that wasn't on the list at all. The list shaped what people said.

Examples that work, and three that don't

Good ones are specific and point at a recent event:

  • "What were you using before you started using us?"
  • "What almost stopped you from signing up?"
  • "Describe the last time an order from a supplier went wrong. What happened?"
  • "What is one thing you would change about how your team tracks shipments today?"
  • "What else did you consider before choosing your current tool?"
  • "Why did you choose that rating?" (after a key rating question)
  • "Is there anything else you'd like to tell us?" (at the end)

And the ones that rarely work in a survey:

  • "What features would you like to see?" People describe solutions instead of problems, and without a follow-up you can't ask what they would give up for them.
  • "What do you think of our product?" Too broad. Answers run from one word to an essay.
  • "How could we improve?" Answered mainly by people who already had something to say.

The text box people skip

Pew's analysis of nonresponse on open-ended questions in its American Trends Panel found closed-ended questions averaged between 1% and 2% nonresponse, while open-ended ones averaged around 18%, ranging from 3% to just over 50% depending on the question. Questions asking for several sentences had a higher median nonresponse rate (18%) than those asking for a word or phrase (13%). The strongest factor was how much thought the question required. Nonresponse was also consistently higher on mobile devices.

NN/g adds that surveys with many open-ended questions usually have lower completion rates. AAPOR's best practices recommend limiting them to keep respondent burden low, and note the answers must be coded before you can count them.

A hypothetical example of what that means: you survey logistics coordinators and get 300 responses. If your skip rate matched Pew's panel average of about 18%, you would have roughly 300 × 0.82 = 246 written answers per open-ended question. Three open-ended questions means around 740 answers to read and tag. Budget the afternoon before you add the third text box.

How to write ones people answer

  • Ask for a short answer when a short answer will do. "In a word or two, what did you use before us?" beats "Describe your previous workflow."
  • Point at a recent event. Remembering takes less thought than forming a new opinion on the spot.
  • Say what kind of answer you want. Pew says it should be clear that people can answer in their own words and what type of answer to give: a problem, a tool name, a number of days.
  • Place them carefully. Pew generally asks open-ended questions on topics such as national problems near the start of a questionnaire, because closed questions on the same topic placed before them make people repeat those ideas. NN/g recommends one broad optional question at the end for anything the survey missed.
  • Make them optional. A forced text box fills up with "n/a" or makes people quit.

How to analyze the answers

  1. Read 30 to 50 answers before creating any categories.
  2. Write a short list of categories in plain words, including "Other."
  3. Tag every answer; one answer can have more than one tag.
  4. Count the tags, and keep two or three representative quotes for each.
  5. Check that someone else would tag a few answers the same way.

Treat the counts as rough: people who bother to write are not a random subset of respondents.

When a 30-minute interview beats a text box

A text box gets one answer. An interview gets the answer and the next five questions. NN/g notes that leaning heavily on open-ended questions may be a sign an interview would serve you better. Choose interviews when:

  • You need the story. "What went wrong with the last shipment?" in a survey gets "carrier was late." On a call you can ask what happened next, who noticed, what it cost and what they changed.
  • You need to follow up on surprises. A survey can't ask "What do you mean by that?"
  • You need to understand a decision. Who was involved and what nearly stopped it rarely fit in a text box.
  • Your audience is small. If you can reach only 40 people in the role, eight conversations may teach you more than 40 short text answers.

A good sequence: a few interviews to learn the words and options, a mostly closed survey to count them, then a few follow-up calls with people whose answers surprised you. See qualitative research interview questions for the call guide and types of survey questions for the closed formats.

Your next step

Sort each open-ended question in your draft. A short answer you need stays, made easy. Keep one catch-all at the end. Anything that needs a conversation moves to interviews.

For those interviews, describe the people, such as operations managers at freight brokerages, and Instant Expert finds people who match. The directory page for operations professionals in logistics technology shows the kind of people you can reach.

There is also a middle option between the text box and the interview: a paid written or voice answer to one open-ended question. The person sees your question and the amount before agreeing, answers on their own time, and you're charged when the answer arrives. If an answer doesn't address your question or was written by AI, write to us and we can refund it. Our fee is added on top of what they receive, so a $40 offer costs $50.