Heuristic evaluation: three reviewers, Nielsen's 10 heuristics and one afternoon

A heuristic evaluation is a review where a few people inspect an interface on their own, check it against a short list of usability principles (the heuristics), and write down every place the design breaks one. Run a heuristic evaluation with three to five evaluators using Nielsen's 10 heuristics, rate how severe each problem is after the sessions, and then put the worst problems in front of real users, because a heuristic evaluation predicts problems and a usability test confirms them.

Nielsen Norman Group calls it a method for identifying design problems in a user interface, and says it is useful for stretching a limited research budget. It needs no participants. It does need more than one pair of eyes, which is where most quick reviews go wrong.

The example we will use

Here is a made-up example. Dana is the product manager at a small company that makes booking software for dog groomers. Groomers say rebooking a regular client takes too long, and Dana has a redesign in a prototype. She wants a quick review before she spends money on a usability test. (The regular client in every test is a poodle named Biscuit who comes in every six weeks.)

1. Use Nielsen's 10 heuristics as written, because they are general on purpose

Jakob Nielsen developed the heuristics with Rolf Molich in 1990 and refined them in 1994, based on a factor analysis of 249 usability problems (NN/g). NN/g updated the explanations and examples in 2020, and Nielsen says the heuristics themselves have not changed. Here they are, with a plain summary of each:

1. Visibility of system status. Keep people informed about what is happening, with timely feedback.
2. Match between the system and the real world. Use words and concepts your users know, and follow real-world conventions.
3. User control and freedom. Give a clearly marked way out of an action someone did not mean to take, such as undo and cancel.
4. Consistency and standards. The same word or action should mean the same thing everywhere, and match what people know from other products.
5. Error prevention. Design so mistakes are hard to make, and ask for confirmation before risky actions.
6. Recognition rather than recall. Show options and information so people do not have to remember them from one screen to the next.
7. Flexibility and efficiency of use. Offer shortcuts for experienced users and let them speed up frequent actions.
8. Aesthetic and minimalist design. Remove information that is irrelevant or rarely needed, because it competes with what matters.
9. Help users recognize, diagnose and recover from errors. Error messages should say in plain language what went wrong and how to fix it.
10. Help and documentation. When help is needed, it should be easy to search, focused on the task and give concrete steps.

NN/g describes these as broad rules of thumb rather than specific usability guidelines. That is why they work on a groomer's booking screen as well as on a banking app. It is also why a reviewer needs judgment: a design that bends a heuristic is not automatically wrong. NN/g's example is a hidden hamburger menu on a retailer's mobile site: it breaks recognition rather than recall, yet can be an acceptable tradeoff on a small screen.

Keep the standard list instead of writing your own. Your team already has opinions; the list's job is to make everyone look at the same things.

2. Use three to five evaluators working alone, because one person finds about a third

Averaged over six of his projects, Nielsen found that single evaluators found only 35 percent of the usability problems in the interfaces. Different people find different problems. The hard ones were sometimes found by evaluators who did not otherwise find many problems, so you cannot fix this by picking your single sharpest reviewer.

His recommendation is normally three to five evaluators, because more people add little new information. In his words, about five, "but certainly at least three." In one worked cost example on the same page, four evaluators came out as the best value.

Each evaluator works alone. NN/g's how-to says to keep evaluators' notes hidden from each other until everyone is done. A group walkthrough feels faster, but the first confident voice in the room decides what everyone else notices.

Short version: three people working alone beat one expert, and four or five is about where extra people stop paying for themselves. The tradeoff is time: three evaluators means three people's afternoons, plus someone to merge the lists.

3. Rate severity after the sessions, alone, and average the ratings

A list of 40 problems with no ranking is a list nobody fixes. Nielsen's severity scale gives each problem a number:

Severity combines three things: how often the problem occurs, how hard it is to overcome when it does, and whether people keep running into it after they know about it. Nielsen adds market impact as a fourth consideration, since some problems that are easy to get past can still hurt a product's popularity.

Collect the ratings after the evaluation sessions, not during them, because evaluators are busy finding new problems. Nielsen suggests sending each evaluator the full merged list, described well enough that someone who did not spot a problem can still judge it, and having each person rate it alone. That takes about 30 minutes per evaluator. One person's ratings are, in his words, "too unreliable to be trusted," while the average of three evaluators is good enough for most practical purposes.

Back to Dana's made-up review. One problem: after a groomer picks a new date for Biscuit, the screen gives no sign the booking saved until they reopen the calendar (visibility of system status). Her three evaluators rate it 3, 4 and 3, which averages to 3.3. Another: the button says "Reschedule" on one screen and "Move" on another (consistency and standards), rated 2, 1 and 2, an average of 1.7. The first goes to the top of the list.

4. Run the whole thing in one afternoon with a fixed schedule

NN/g suggests narrowing the scope to one task, one section, one user group or one device type (NN/g). Dana's scope is one task on desktop: rebook a regular client for six weeks from today. Then the afternoon looks like this:

1. Practice round, 20 minutes. NN/g suggests a short group practice on something simple, such as a weather app, so everyone knows what is expected. Do it together; this is the only group part before the debrief.
2. Independent evaluation, 60 to 90 minutes. NN/g suggests timeboxing to about one to two hours, and Nielsen recommends going through the interface at least twice. The first pass is for learning the flow without judging it. The second is for noting each problem, one per line in a shared spreadsheet, with the heuristic it breaks and a screenshot.
3. Merge, 30 minutes. One person combines the lists and groups duplicates. NN/g uses affinity diagramming, which means clustering similar notes together.
4. Severity ratings, 30 minutes. Everyone rates the merged list alone, using the 0 to 4 scale.
5. Debrief, 45 minutes. Discuss the highest averages, where people disagreed, and what needs more data.

Bring fixes to the debrief, because the method will not produce them on its own. Nielsen says heuristic evaluation "does not provide a systematic way to generate fixes," and suggests using the debriefing to brainstorm redesigns. The tradeoff of an afternoon format is depth: one task gets a careful review, and the rest of the product gets none.

5. Follow it with a usability test, because experts can only predict what users will do

A heuristic evaluation finds the problems that people who know the principles can see coming. It cannot tell you that, in Dana's made-up case, groomers rebook while holding a wet dog and need the whole thing done with one hand. NN/g is direct that heuristic evaluations cannot replace user research, because design depends heavily on context.

Use the two together. NN/g suggests running a heuristic evaluation before an upcoming usability test to decide which parts of the design to target. Fix the obvious problems first, so five test participants do not spend their sessions tripping over a missing save message you could have caught at your desk. Then the usability test shows what the experts missed, and a System Usability Scale survey at the end gives you one number to compare before and after.

If the problem is where things live in the menu, rather than how a screen behaves, a tree test answers that more directly. And if you want to know how competitors handle the same task, a UX competitive analysis runs the same heuristics across several products.

6. Bring in an outside UX expert when your team knows the product too well

Your own team is the cheapest set of evaluators, and NN/g notes that people new to UX can use heuristic evaluations to train themselves to spot common issues. The weakness is familiarity. People who built the rebooking flow know where the save button is, so they do not miss it.

Add an outside evaluator in three situations. The first is when nobody on the team has done one before, because NN/g says a first-time team should learn the heuristics and do a practice round first, and someone who has run one can lead that round. The second is when the review decides something expensive, such as whether a redesign ships. The third is a specialized domain: Nielsen says evaluators who do not know the domain need help, and suggests giving them a typical usage scenario built from realistic tasks. A designer who has worked on scheduling software needs less of that help.

Make the outsider one of your three to five, not a replacement for them. The tradeoff is cost and coordination; the gain is someone who sees Biscuit's rebooking screen for the first time, the way a new groomer will.

Your next step

Pick one task, write it in a sentence, and book three people for one afternoon this week (the schedule in section 4 takes under four hours) with the 10 heuristics and a shared spreadsheet open. If you want an outside evaluator, Instant Expert can find people matching a description such as "product designers who have worked on booking or scheduling software" and send them paid invitations for a short call or a written answer, and you pay only if someone books or answers. The directory pages for UX consultants, user research experts and heads of design are one place to start.