Concept testing: how to test a product idea with buyers before you build it

Concept testing means showing a short description of a product idea, sometimes with a sketch and a price, to people in your target market, and measuring whether they understand it, find it relevant and would consider buying it. It comes after you know the problem is real and before you build. A good concept test tells you which of several ideas to pursue and what to change. A careless one produces enthusiastic scores that say little about what people will buy.

The main risk is taking stated interest at face value. Jakob Nielsen of Nielsen Norman Group sums up the rule: watch what people actually do, and put little faith in what they say they do, still less in what they predict they will do (NN/g). A concept test is still useful, as long as you use it to compare ideas and find problems, and follow it with a test of behavior.

Where concept testing fits

If you have not yet confirmed the problem, start with problem interviews. Otherwise a concept test measures reactions to a solution for a problem you have not confirmed.

A worked example

This example is hypothetical, and we use it throughout. A small team wants to build software for independent dental practices. From earlier interviews, they know front-desk staff lose hours to two problems: checking patients' insurance before appointments and filling last-minute cancellations. They have three ideas:

• A. Automatic insurance eligibility checks for every patient two days before their appointment.
• B. A texting assistant that offers cancelled slots to patients who asked for an earlier time and books the first one who replies.
• C. A monthly report comparing the practice's collections with similar practices.

Write each concept the same way

A concept statement should be short and in the same format for every idea, so you compare the ideas and not the writing. Include:

• The problem, in the buyer's words.
• What the product does, in one or two sentences.
• How it differs from what they do today.
• A simple picture or mockup, if the idea depends on it.
• A price, if price is part of the decision.

For idea B: "When a patient cancels the day before, your front desk spends an hour calling the waiting list. This assistant texts patients who asked for an earlier appointment and books the first one who replies. It works with your current scheduling system. $149 a month per location."

Write idea A and C to the same length and level of detail. A common way to bias a concept test is to describe your favorite idea more vividly than the others.

Choose interviews or a survey

Concept interviews suit early work: a few people per buyer type, one concept at a time, with time to ask why. They tell you what confuses people, what worries them and what they would compare the idea with.

Concept surveys suit comparing ideas across more people. SurveyMonkey describes three designs (SurveyMonkey):

• Monadic: each respondent sees one concept.
• Sequential monadic: each respondent sees several, one after another, and answers the same questions about each.
• Forced choice: each respondent sees all of them and picks a favorite.

SurveyMonkey recommends monadic designs because they mirror the real world, where a buyer sees one product, keep the survey short enough for careful answers and avoid order effects. It cites studies showing that people rate the first concept they see more favorably, and that a strong first concept makes later ones look worse. The trade-off is sample size, since each concept needs its own group. If respondents are scarce, a sequential design with the order rotated is a compromise, but SurveyMonkey warns that rotation may not balance out in a small sample.

Questions to ask

In interviews, ask:

1. "In your own words, what does this do, and who is it for?" This checks comprehension before appeal.
2. "When did you last deal with this problem? What did you do?" This ties the concept to real behavior. The Mom Test is a short book on asking questions like this so you avoid polite, biased feedback.
3. "What would this replace, and what would you stop doing?"
4. "What would worry you about using it?"
5. "Who else would need to agree before the practice paid for it?"
6. "What would you need to see before you tried it?" Then ask for a small next step, such as a follow-up demo or an introduction to the practice owner.

In a survey, use the same measures for every concept: an open question on what it does (scored for accuracy), how relevant it is, how different it is from what they do today, how believable it is, how likely they would be to consider buying it, and an open question on what would stop them.

Read the results

• Check comprehension first. If many people describe a concept wrongly, fix the description before judging its appeal.
• Compare concepts with each other, on the same measures, from the same test. Treat "likely to buy" scores as a ranking, not a sales forecast. Nielsen gives a hypothetical example: if half of survey respondents say they would buy more from sites with 3-D product views, all it shows is that 3-D sounds cool.
• Read the open answers for objections that repeat.
• Look at who responded most strongly. In the example, practices with three or more dentists might prefer idea B while solo practices prefer A. That tells you which segment to test next.
• Then test behavior. Put the leading idea in front of buyers in a way that asks for action: a waitlist, a fake door or a request for a paid pilot. Market validation covers how to ask for a commitment.

Common mistakes

• Testing with the wrong people. Opinions from outside the target market are noise. Screen respondents on the job they do; research screener questions covers how.
• Leading questions. "How much would this help your practice?" assumes it would. Biased survey questions has fixes.
• Asking only fans. Customers who already like you will be kinder to every concept.
• Stopping at stated interest. Treat a high score as a reason to run a behavior test before you start building.

Your next step

Write each idea as a concept statement in the same format, with a price if price matters. Show them to five people who match your target buyer, one at a time, and ask each to explain the concept back to you before you ask what they think of it.

If you do not know people in the field, Instant Expert can find people who match a description such as "office managers at independent dental practices with two to five dentists." You review who it finds, it sends your invitations and you pay for each call that gets booked. The directory pages for operations professionals in dental services and dental consultants are one place to start.