How do you do usability evaluation: Usability inspection

There are two common approaches to usability inspection: heuristic evaluation and cognitive walkthrough. As conceived by Jakob Nielsen (1993), arguably the world's most famous usability expert, the heuristic evaluation method employs a set of principles (termed heuristics) which have been defined prior to the evaluation. Heuristic evaluation is usually done with experts such as human-computer interface design specialists, digital library designers, or graphic artists. The experts independently examine the product and judge its compliance with a set of heuristic principles. Here are the original ten heuristics listed at Neilsen's website (http://useit.com):

Visibility of system status:	The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
Match between system and the real world:	The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
User control and freedom:	Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
Consistency and standards:	Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Error prevention:	Even better than good error messages is a careful design which prevents a problem from occurring in the first place.
Recognition rather than recall:	Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
Flexibility and efficiency of use:	Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
Aesthetic and minimalist design:	Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
Help users recognize, diagnose, and recover from errors:	Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
Help and documentation:	Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the users' task, list concrete steps to be carried out, and not be too large.

As each expert spends time interacting with the digital library, usually two to four hours depending on the complexity of the library and its functions, he or she make notes of the features of the library interface and functionality that violate one or more of the heuristics on a predetermined list. The expert may also identify usability flaws that do not obviously match one of the predefined heuristics.

After reviewing the system, each expert usually goes back through all of the problems identified to rate each one according to its frequency and severity. Subsequently, the various experts may be brought together for a debriefing in which they compare the problems found and attempt to come to an overall recommendation concerning each problem area. The consensus of the experts might be guided by using a final rating scale as represented below.

Usability Problem Rating Scale

0	This is not a usability problem.
1	This is a cosmetic usability problem only, and it need not be fixed unless extra time is available on project.
2	This is a minor usability problem and fixing this should be given low priority.
3	This is a major usability problem, and it is important to fix so it should be given high priority.
4	This is a usability catastrophe, and it is imperative to fix this before the digital library can be released.

Another method used in usability inspection is the cognitive walkthrough. In a cognitive walkthrough, a group of expert evaluators (e.g., composed of graphic user interface specialists, software developers, etc.) work through a paper mock-up, prototype, or full version of the digital library, with the goal of completing a set of realistic tasks while evaluating the library's ease of learning, user-friendliness, and understandability. Before beginning the walkthrough, evaluators are informed of important issues such as:

Who will be the users of the system?
What types of tasks will typically be done?
What is the status of the digital library (e.g., early prototype versus beta version about to go public)?

While completing the walkthrough, evaluators ask themselves questions such as:

Will the user know what to do here?
Will the user associate the correct action with the desired effect?
If the correct action is chosen, will the user know he or she is on the right path?

In doing a cognitive walkthrough, evaluators should put themselves in the user's �shoes� so to speak. The goal of the walkthrough is to have the evaluators look at the digital library through a user's eyes, trying to create scenarios of where and why a user might be successful in completing a task, and also scenarios of where and why a user might experience difficulty in completing a task. Evaluators usually speak their feedback aloud as they go through the digital library. What they say can be recorded for later transcription or another evaluator may take notes as the expert is doing the cognitive walkthrough.

There are several other methods of conducting usability inspections. These include pluralistic walkthroughs, feature inspections, perspective-based inspections, and claims analysis.

Pluralistic walkthroughs (Bias, 1994) are similar to cognitive walkthroughs. But instead of just using experts, the walkthroughs are conducted with a mix of typical users, subject matter experts, and usability experts who are expected to discuss and hash out their different reactions to the program.

Feature inspections (Kahn & Prail, 1994) are expert or user reviews that are focused on specific features of a system. For example, if a digital library is being redesigned to include a much richer set of search delimiters, different approaches to enabling these functions might be targeted for inspection and feedback.

Perspective-based inspections (Zhang, Basili, & Shneiderman, 1999) involve reviewing the interface design and functionality from a variety of different viewpoints. For example, viewpoints could be from the perspective of the novice visitor, the frequent patron, or a content expert.

Claims analysis (Keith, Blandford, Fields, & Theng, 2002) focuses on identifying the positive and negative effects of a feature that may influence the usability of a digital library. The goal of this variant of usability inspection is to be able to describe the benefits and disadvantages of features, and then consider and propose alternatives that could improve the design.