Research methods
Message testing: how to test B2B positioning and messaging with buyers
How to test B2B messaging: write variants that make different claims, test them in buyer interviews and monadic surveys, size any A/B test first and read the results.
Message testing means showing target buyers different ways of describing your product and finding out which one they understand, believe and care about, before you spend money on a launch, a website or a sales deck. In B2B, the most useful tests are usually small. A handful of conversations in which real buyers read your copy and explain it back can tell you more than a large survey of people who would never buy. Live A/B tests on ads or pages come later, and only if you have the traffic to read them.
Positioning comes first
April Dunford separates positioning from messaging. In her definition, positioning is about what your product leads at delivering and which clearly defined customers care a lot about it. She is explicit that positioning is not messaging, a tagline or a brand story (April Dunford). Her method starts with competitive alternatives, meaning what a customer would do if your product did not exist, then moves to your differentiated capabilities, the value they create, the customers who care most and the market category.
That order matters when you test. If every variant falls flat, the problem may be the positioning: the wrong alternative, the wrong buyer or the wrong category. Rewording will not fix that. Write each message against the alternative your buyers actually use today, which, as Dunford notes, can be a spreadsheet, a manual process or doing nothing at all.
A worked example
This example is hypothetical. A startup sells software that collects evidence for security audits such as SOC 2. Its buyers are heads of security and compliance managers at software companies with 100 to 1,000 employees, most of whom collect evidence by hand with spreadsheets and screenshots. The team writes three messages:
- A, time: "Get through your SOC 2 audit without weeks of screenshots."
- B, risk: "See which controls would fail today, before your auditor does."
- C, sales: "Answer a prospect's security questionnaire the day it arrives."
Each one reflects a different choice about the main pain and, in practice, the main buyer. That is what makes the test worth running.
Write variants that make different claims
Swapping synonyms tests very little. Each variant should make a different claim about who the product is for and why it matters. Keep the format identical so you are testing the idea and not the layout: a headline, a two-sentence explanation and one proof point each, at roughly the same length.
Method 1: buyer interviews
Start with a few conversations with each type of buyer, and add more until the reactions repeat. In each one:
- Ask about today first. How do they handle the problem now? When did it last cause trouble? This gives you their words and the alternative you are really competing with.
- Show one variant at a time. Ask them to read it and then say, in their own words, what the company does and who it is for. If they cannot, the message is unclear, however much they like it.
- Use a highlighter. In a content highlighter test, people mark text that is clear in one color and text that is confusing in another, then explain why (18F). For messaging, you can ask for convincing and hard-to-believe instead.
- Ask what they would need to see to believe it, and what they would compare it with.
- Rotate the order between participants. SurveyMonkey cites studies showing that people tend to rate the first concept they see more favorably than later ones (SurveyMonkey).
Pay attention to what people do more than what they say. Jakob Nielsen of Nielsen Norman Group points out that people bend their answers toward what they think you want to hear (NN/g). Stronger signals are an accurate restatement, a specific follow-up question about price or integration, or a request to send the page to a colleague. "I like that one" is a weak signal.
Pay for participants' time, not for a favorable reaction; paying for participation without paying for positive feedback explains how to keep those separate.
Method 2: a monadic survey
When you need more than a handful of reactions, a survey can compare variants. SurveyMonkey describes three designs:
| Design | What each respondent sees | Main drawback |
|---|---|---|
| Monadic | One variant only | Needs a separate group of respondents for each variant |
| Sequential monadic | Several variants, one after another | Order effects and fatigue from answering the same questions repeatedly |
| Forced choice | All variants together, then picks one | Unlike the real world, where a buyer sees one message |
SurveyMonkey recommends monadic designs for this kind of test: they mirror what buyers will see in the market, avoid order bias, and leave room for more follow-up questions. It also warns that randomizing the order in a sequential design may not balance things out with a small sample.
Useful questions for each variant: what does this company do (open text, which you score for accuracy), how relevant is it to your work, how believable is it, and what would you want to know next. The hard part in B2B is the sample. A panel of "IT decision makers" may include people who do not make the decision, so screen carefully; research screener questions covers how.
Method 3: live tests on ads, emails or pages
A live A/B test shows variants to real prospects and measures what they do. It is the most realistic test, and it needs a lot of traffic. Evan Miller's rule of thumb for sample size is n = 16σ²/δ², where σ² is p(1 − p) for a conversion rate p and δ is the smallest change you want to detect (Evan Miller). For a landing page that converts at 3%, detecting a lift to 3.6% works out to about 13,000 visitors for each variant by our calculation. Many B2B pages never reach that.
Miller also describes a mistake that is easy to make: checking results repeatedly and stopping as soon as a difference looks significant. That makes the reported significance meaningless. Decide the sample size in advance and wait for it.
For most early B2B teams, interviews plus a small survey are more realistic. If you do run a live test, use it to confirm a message that already did well in conversations.
Read the results
| What you see | What it suggests |
|---|---|
| Buyers restate the message accurately and ask about price or setup | The message is clear and relevant |
| Buyers like it but describe it wrongly | Clarity problem: rewrite before judging appeal |
| The same objection comes up in several conversations | Add proof or change the claim |
| Different buyer types prefer different variants | A positioning decision about who the main buyer is |
| No variant lands with anyone | Revisit the positioning, starting with the competitive alternative |
In the example, suppose heads of security responded most to message B and compliance managers to message A. That is a decision about who the primary buyer is, which belongs in your buyer persona work as much as in your copy.
Your next step
Write three messages that each make a different claim, all in the same format. Then book five conversations with buyers who match your target, and ask each one to explain every message back to you.
If those buyers are outside your network, Instant Expert can find people who match a description such as "heads of security at software companies with 100 to 1,000 employees." You review who it finds, it sends your invitations and you pay for each call that gets booked. The directory pages for chief information security officers and marketing professionals in cybersecurity are one place to start.