What to do when experts disagree in research: check they measured the same thing

When experts disagree in your research, check whether they described the same work under the same conditions before you pay for another call. They may be measuring different things. Start with your notes: what did each person say, when did it happen, and what did they personally do?

Keep their accounts apart from your explanation of the difference. GOV.UK's analysis guidance makes that distinction between observations, findings and next actions.

"One day" and "six weeks" can both be true

Here is a hypothetical example. You are planning customer support for a software product. One expert says onboarding takes a day. Another says six weeks. You need to decide how much staff time to reserve and what launch date to promise.

Check what each estimate includes. When does the clock start and stop? Is it working time, or calendar time including waits? Look in your notes before deciding anyone is wrong.

Suppose the notes show that the first person meant one day configuring a standard account after clean data arrived. The second meant six weeks from signing the contract to several departments using the product, including approvals and training.

They measured different work, and neither gives a typical duration for your customers. Staffing needs the work involved in your kind of setup. A launch date also needs time for waiting on data, decisions and people. Averaging the two answers neither question (and describes nobody's onboarding).

When it really is the same fact, check the fact

Now suppose both people are discussing the same product version, import route and settings. One says the file must contain a customer ID column. The other says that column is optional.

This one you can check. If a current public import guide exists for that version, its required-column list shows what the provider documents. If the dispute is whether an upload actually succeeds without it, a permitted test with nonprivate sample data can check that under the tested conditions.

Run only the check that answers the question you still have. Nielsen Norman Group's guidance on using multiple research methods recommends turning to other sources or methods to fill gaps in the evidence.

Give the next call one specific job

Before choosing another person, finish this sentence: "I still need to find out..."

Here, that might be what happened when someone recently uploaded a file without the customer ID column on the relevant version and route. A person who handled that upload can describe the file and the result. Another opinion on whether onboarding is easy doesn't help.

Use the expert-selection guide to find someone whose work fits that missing detail. If the evidence still doesn't settle it, keep the question open in your decision notes. A third person agreeing with one side adds a vote. It does not settle the fact.