How do you do usability evaluation: Usability Testing

Usability testing involves having a number of individuals, who are considered representative of typical users, complete routine and/or special tasks in the digital library while evaluators observe and collect results to see how the interface supports the users in completing the tasks. Usability testing can occur in formal settings such as a professional usability laboratory or it can occur in virtually any other setting using portable usability equipment or only a regular video camera.

Professional usability labs generally consist of two rooms separated by a one-way glass window. In one room, a computer user sits at a desk and interacts with the application being evaluated, e.g., a digital library. Two or three video camera s mounted in the room are focused on the user from different perspectives. For example, one camera might be focused on the user's hands whereas another might be recording the user's facial expressions. In the other room, usability evaluators sit at control panels where they can simultaneously observe the user in the room through the one-way glass or any of the video screens displaying selected aspects. The user may be instructed to “think aloud ” as he or she uses the digital library, e.g., talk about why certain choices are made or describe any confusion about the library's interface. Alternatively, the evaluators may question the user via headsets or speakers about why he or she has acted in certain ways.

In setting up a usability test, users are informed that they will be observed, and they have the right to discontinue a test at any time for any reason. Typically, these sessions are videotaped for later analysis and documentation. Some professional usability labs actually have a third room where clients can observe through a one-way glass the usability testing as it is being conducted

A portable usability lab is much simpler than commercial usability laboratories, and it has the advantage of allowing users to stay in their own environment rather than forcing them to come to a lab and test the digital library in an artificial environment. This may increase the validity of the usability test.

You can even do usability testing with only a single video camera, especially if you cannot afford to rent a professional laboratory or buy a portable usability lab. For example, a single video camera can be used to record two users while one explains to the other how to use a digital library. This has some unique advantages. The two people have to talk as one explains how the digital library works and the other asks questions. Reviewing the video record of these interactions can be very informative. You can interpret the adequacy of the mental model of the library held by the person doing the instruction and also estimate how hard the interface is to learn by the questions asked by the other user.

Regardless of what type of lab is used (professional or portable), usability testing enables evaluators to collect both quantitative and qualitative data related to issues such as user interface, mental models, navigation, ease-of-learning, documentation utility, effectiveness, and efficiency. There are a variety of protocols used in usability testing. One of the most common usability testing methods is the think aloud protocol.

The think aloud protocol is useful in gathering qualitative data about users' mental models of a digital library, as well as their general impressions and feelings about library factors such as layout, navigation, and design. Below are guidelines for implementing the think aloud protocol:

You may want to ask questions to users during observations, but asking questions during a usability test can change what the user would naturally do. An alternative is a delayed think aloud approach whereby you video the user, and later play the tape back to the user. During the playback, you ask the user to state what he or she was thinking while interacting with the digital library. You can ask specific questions such as “Why did you decide to use those search terms?” The tape assists the user in recalling the recorded session. Later, the same tape can be shown to other experts for their advice and interpretations. Alternatively, a focus group of designers can review the videotapes of users to stimulate new ideas about enhancing the interface of the digital library.

There is a lot more to usability testing than can be described in this brief Guide. If you are going to engage in serious usability testing, we recommend going to a workshop on the subject and perhaps hiring a consultant your first time out. If you get even more serious about usability testing, you should join the Usability Professionals Association (http://www.upassoc.org/) and attend one of their annual conferences.