8. Conducting Heuristic Evaluation

Heuristic evaluation (also known as expert review, usability audit, usability inspection) is a usability evaluation of a product, which involves a small set of expert evaluators who examine the level of its compliance with recognized usability principles, also called heuristics. Expert evaluators are needed, although it could be carried out by less experienced people, provided that they receive some training and a checklist, however, fewer problems get discovered that way. Each evaluator inspects the interface individually, the findings are then aggregated. It differs from user testing, because the responsibility of analyzing the interface is on evaluators, not a researcher or moderator.

Benefits

There are multiple benefits of letting experts evaluate usability:

• They provide quick feedback to designers, relatively easy to do compared to other methods;
• Cheaper than many other methods, such as user testing.
• Could be carried out before user testing, so that biggest usability problems are discovered and addressed before they prevent participants from discovering harder to spot, their workflow specific issues. Doing this would save a lot of money, since you would need fewer rounds of user testing.
• Reviews are also excellent in competitive benchmarking, since they allow comparing the usability of your product to your competitor’s product.

Heuristic Evaluation vs User Testing

Heuristic evaluation is never a substitute for user testing, since it does not provide any insights on how actual users use the system, it is hard even for experts to predict it.

Nielsen’s Usability Heuristics

Nielsen’s Heuristics are commonly used usability principles in expert reviewing.

Visibility of system status

The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

Match between system and the real world

The system should speak the users’ language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

User control and freedom

Users often choose system functions by mistake and will need a clearly marked “emergency exit” to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

Recognition rather than recall

Minimize the user’s memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

Flexibility and efficiency of use

Accelerators — unseen by the novice user — may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

Aesthetic and minimalist design

Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

Help users recognize, diagnose, and recover from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out, and not be too large.

Other usability standards

• Iso’s 7 dialogue principles (http://www.userfocus.co.uk/resources/iso9241/part110.html): suitability for the task, suitability for learning, suitability for individualization, conformity with user expectations, self descriptiveness, controllability, error tolerance.
• Schneiderman’s 8 golden rules of interface design (https://www.cs.umd.edu/users/ben/goldenrules.html): strive for consistency, cater to universal usability, offer informative feedback, design dialogs to yield closure, prevent errors, permit easy reversal of actions, support internal locus of control, reduce short-term memory load.
• Arnie Lund’s “Expert Ratings of Usability Maxims” (http://www.theuxbookmark.com/2009/11/ux-articles-and-research-papers/usability-engineering/expert-ratings-of-usability-maxims/).
• Bruce Tognazzini’s “Principles of Interaction Design” (http://asktog.com/atc/principles-of-interaction-design/)

Most UX practitioners have their favorite sets of usability principles; most of these principles overlap.

Typical process of heuristic evaluation

1. Identify the business goals of the product being evaluated, create short profiles of users, typical tasks. When carrying out an evaluation for a client, it is a good practice to send them a quick questionnaire that asks about the users and typical tasks or just call them to discuss it.
2. Decide which heuristics you are going to use.
3. Create a checklist for that particular application domain.
4. Review the product: firstly use the product to get a general overview of its structure and functionality, then go through each task and note usability issues.
5. You could also explore the elements not related to the tasks, e.g. additional web pages. It is a good practice to go through the interface several times.
6. Invite 2-4 additional evaluators, brief them on the product, the tasks, the checklist etc.
7. Ask each evaluator to evaluate the product.
8. Put the problems found by each evaluator together, remove duplicates.
9. Write up a report that describes the problems, assigns severity ratings to them and suggests solutions. The report should be easy to understand to the audience who are not usability professionals, terms such as “lack user control and freedom” are not straightforward, you should clearly describe each problem. Often experts do not map each problem directly to a specific heuristic, since the audience is less interested in the heuristics than in how they can address the problems.
10. Present the findings to the design team.

Evaluation checklists (templates)

It is a good practice to use detailed evaluation checklists when carrying out an evaluation since using only high level principles risks missing some usability issues, also high level principles (such as Nielsen’s heuristics) might appear too vague when applied to new technologies, thus domain specific checklists are needed.

A checklist for evaluating usability of websites – UserFocus web usability checklist. It is very detailed, thus covers all aspects of web usability.

You can download the Excel checklist here http://www.userfocus.co.uk/resources/guidelines.html. The document has several worksheets that correspond to the main elements and aspects of websites: Homepage, Task Orientation, Navigation, Forms, Trust, Content, Layout and others.

Also, it calculates usability scores, which is excellent for competitor benchmarking, you could compare usability scores of your website to competitors’.

How to use it: For each checklist item, enter a rating between -1 and 1; -1 means that it does not comply with the guideline, 0 – kind of complies, +1 means complies. If a guideline is not applicable, leave it blank. Each guideline has a field for entering comments when needed. After you do that for all checklist items, you can see the scores – just go to the “Results” worksheet and you will get numerical ratings of compliance there.