Expert opinion: how to get one you can use for a business decision

An expert opinion is worth asking for when the person has recently done the exact thing you are deciding, and worth trusting in proportion to the facts they can show you, not how sure they sound. To get a usable expert opinion for a business or product decision, ask a narrow question of three people who were close to similar decisions, then weigh their answers with a simple score instead of by seniority.

(One note on scope: in a US courtroom, "expert opinion" is a legal term for testimony that must meet Federal Rule of Evidence 702, and that is a question for a lawyer, not this post.)

1. Count someone as an expert only for the question in front of you

An expert for your decision is a person who did the thing recently, did it with their own hands and saw how it turned out. A strong title is a weak substitute. A former VP of Sales is an expert on how her team bought a sales tool in 2024. She is not automatically an expert on whether your pricing page works.

There is research behind the "saw how it turned out" part. Daniel Kahneman and Gary Klein reviewed when professional intuition can be trusted, in a 2009 American Psychologist paper. Their conclusion: judge how predictable the setting is and whether the person had a real chance to learn its patterns. They also wrote that a person's feeling of confidence is not a reliable sign of accuracy.

In practice, ask three things about each candidate:

• Did they personally make or carry out this kind of decision?
• How recently? A buying process from eight years ago may not exist anymore.
• Did they see the result? Someone who signed the contract and then left the company never learned whether it worked.

Here is a made-up example. Dana runs operations at a regional food delivery company with 40 refrigerated trucks and one dispatcher who refuses to retire his whiteboard. She is deciding whether to buy route planning software. A logistics director who rolled out a similar tool two years ago and stayed through the first winter is a strong expert. A consultant who writes comparison guides about route software is a weaker one. A supply chain professor is great on theory and weak on whether drivers will actually use the app.

The posts on evaluating expert relevance and what a subject matter expert is go further on checking experience before you book anyone.

2. Ask for what happened before you ask for a verdict

Ask experts to describe what they saw first, and only then ask what they would do. Facts are easier to check and easier to compare across people. A verdict on its own ("I'd buy it") tells you the answer without the reasons, so you cannot tell whether those reasons apply to you.

Use this template, in this order:

1. Context: "We are a [size] company deciding whether to [decision] by [date]. The main worry is [one sentence]."
2. Their experience: "When you faced something similar, what did you do, and what happened in the first six months?"
3. The surprise: "What turned out differently than you expected?"
4. The verdict: "In our position, what would you do?"
5. The check: "What would have to be true for that advice to be wrong for us?"
6. Confidence: "How sure are you, from 0 to 100?"

The fifth question does most of the work. People who know a topic well can usually name the conditions where their advice breaks. If someone cannot think of any, weigh their answer less. (They may still be right. They just have not shown you why.)

Send the context line before the conversation. Otherwise the first part of every call goes to explaining what your company does.

3. Get three opinions from people who saw the problem from different sides

Get three expert opinions for a decision that matters, and pick people who would not have talked to each other. One opinion is a story. Two that agree might share the same blind spot. Three from different positions (for Dana: someone who bought route software, someone who used it every day and someone who sold it) usually show you where the real uncertainty is.

This is a rule of thumb, not a statistical result. Add a fourth only when two of the three disagree about a fact, and choose that person specifically to settle the disputed fact. Stop when a new conversation adds nothing you have not heard.

The tradeoff: three conversations take longer than one. For a decision you can cheaply reverse, one good opinion is enough.

4. Weigh opinions with a score, then look at the fact they disagree on

Score each opinion before you compare them, so the loudest or most senior person does not win by default. Give each one 0 to 2 points on four questions:

Back to Dana. The logistics director scores 7 of 8 and says buy. A dispatcher who used similar software scores 6 and says buy, but only with two weeks of training. The software sales rep scores 3: he knows the product well but never saw a customer's results firsthand, and he says buy with no conditions. The weighted answer is "buy, and budget two weeks for training," which comes from the two high scorers. The rep's unconditional yes carries the least weight.

When the high scorers disagree, do not average them. Find the single fact behind the disagreement. Often they are describing different conditions: a team of 10 drivers versus a team of 200. The post on what to do when experts disagree walks through checking whether they measured the same thing, and testing the one fact still in dispute.

Short version: score closeness, recency, outcome and reasoning. Ignore title and confidence until the end.

5. Discount long range predictions, especially confident ones

Expert opinion is strongest on what happened and weakest on what will happen. The best known evidence comes from Philip Tetlock, who over about 20 years collected forecasts from 284 people who made their living commenting on political and economic trends. As the Journal of Accountancy summarized, he gathered predictions on 82,361 questions and found the forecasts were no more likely to be accurate than dart-throwing monkeys. The same summary reports that the more famous the forecaster, the more overblown the forecasts.

Tetlock's book did find a difference between forecasters. In the publisher's description of Expert Political Judgment, the "fox" (who knows many small things and draws on several traditions) predicts better than the "hedgehog" (who knows one big thing and works within one tradition).

What this means for a business decision:

• Ask experts about past events and current practice. "How long did your rollout take?" is a question they can answer well.
• Treat five year forecasts ("everyone will use this by 2030") as one input with little weight.
• Prefer the expert who says "it depends on X, and here is how to check X" over the one with a single theory that explains everything.

Fair warning: Tetlock studied political and economic forecasts, not product decisions. The lesson that transfers is narrower: confidence and fame are not evidence of accuracy.

6. Decide what you will do with the opinion before you ask

Write down your current lean and the answer that would change it before the first conversation. Without that, expert opinions tend to become decoration for a decision you already made. Dana might write: "I lean toward buying. I will not buy if two of three people say drivers stopped using the app within three months."

This matters even more if you pay for the opinion, because paying makes it tempting to follow the advice to get your money's worth. The post on paid advice covers a study on exactly that effect, its caveats, and how to set a fair price for a short call.

Ask three people who did it recently

Pick the decision, write the six question template, and find three people who made a similar decision in the last two years from different sides of it. Score what they tell you, and chase the one fact they disagree on. If you need opinions from specific people you do not know, such as board advisors or startup advisors, Instant Expert can invite them to a short call or a written or voice answer, and you pay only if someone books or answers.