How to combine customer interviews with product analytics (and not fool yourself)

To combine customer interviews with product analytics, check what customers told you against the records before you change anything. The records show where to look. They cannot explain why anyone waited.

Here is a hypothetical example. Your product requires an initial data import, and several customers say they waited for a manager's approval before importing. Is this the problem to work on next?

1. Compare the same task over the same period

Define the behavior first: did each new account complete its first successful import within seven days of account creation? A login is not an import, and activity elsewhere in the customer's company may belong to a different account.

Use only accounts that have had the full seven days, and leave out every newer one, including those that already finished. Keeping recent successes while dropping recent unfinished accounts flatters the result. Seven days fits this example; choose a period that suits your product.

Check that your records reliably capture account creation and the first successful import. Until you rule out a tracking gap, a missing import event is not evidence the customer failed.

Mixpanel's funnel documentation explains counting a sequence of events within a defined period; the same care with task, account and clock applies in any tool.

2. Check that you actually know who needed approval

Your interview notes may establish this for the people you spoke with, not for every other account in the dashboard.

Split accounts into "needed approval" and "did not" only when you have reliable information for both groups, and keep unknown cases separate. If approval was never recorded consistently, the dashboard cannot answer this yet.

Suppose the approval group did finish less often within the week. It might also have had more complicated imports, so the gap is something to investigate. It does not prove that removing approval would help. Microsoft's product-research example distinguishes monitoring the current experience from testing a proposed change.

3. Give the next conversation a specific question

Find an account you know required approval but still finished within the week. Start with its notes; if the sequence is still unclear, talk to someone who handled that account's setup.

Ask: "Walk me through what happened between creating the account and the first successful import." Then ask where approval fit into that sequence. Do not assume it caused a delay. (A leading question is how a hunch ends up on the roadmap.)

That conversation might show how this account got through setup. It will not show how common that is, or whether approval slows other accounts. A general industry expert cannot fill in this customer's history either.

That is the point of combining them: the records hand you a more specific person and question. Mixpanel's practitioner guide describes choosing between the methods based on what remains unanswered.

Check your notes and customer contacts first. If you still need another person's experience, use the expert-selection guide to work out whose experience answers the question before you pay for a conversation.