How do you do usability evaluation: Usability Inquiry

Usability inquiry is much like usability testing, except that in the former the evaluator observes users working with the digital library while trying to complete their own real work rather than tasks defined by the evaluator. You will want to use usability inquiry methods if you are interested in gathering information about users' likes, dislikes, needs, and understanding of a digital library. Indeed, some experts maintain that usability inspection methods are better than usability testing approaches because only the former allow you to evaluate the user, the tasks, and the working environment at the same time (Hackos & Redish, 1998). There are a number of evaluative methods that fit within the usability inquiry framework including: field observations, use logs, focus groups, interviews, and questionnaires.

Field observations require you to schedule visits with real or potential users in the workplace or home where they would normally access a digital library. For example, if you want to understand how teachers would access a digital library, it would be useful to observe them in their schools. You may want to begin your observations of each teacher with a brief “get-acquainted” interview, spend some time observing the teacher using the digital library, and then conduct a debriefing interview.

User logging is a method that involves having computers record what people actually do when they are using a digital library. It is relatively easy to collect user statistics with user logs, but much more difficult to make sense of the data so that actual enhancements to an interface can be made. Some statistics should be routinely logged such as error messages or time patrons spend using specific resources. When and where people go for online help can also be useful data.

Focus groups are useful when you want to collect information about a digital library's usability from a group of people who have already been using it for a while. This normally requires at least two people, one to moderate the discussion and the other to take notes. The session may also be recorded. The focus group method is useful in terms of getting users' reactions to an interface as they use it over time. But it has the disadvantage of only collecting information about what users say they do, and not what they actually do.

Interviews are similar to focus groups in terms of utility and limitations, except that instead of interviewing a group, interviews are normally with one user at a time. An interviewer questions the user, the user replies, and the interviewer records those responses using either written notes or a recording that is later transcribed. Interviewing can be relatively unstructured, although for usability inquiry, structured interviews are more common.

Questionnaires are probably the most common form of evaluation instrument used in evaluations in general, but they have limited utility in usability evaluations. One form of questionnaire that can be useful sometimes is a “pop-up” questionnaire that is programmed to appear whenever the user does something unexpected in a digital library. Alternatively, a brief on-screen questionnaire about usability issues may be initiated after a user has been using the library for a certain period of time or upon exiting the library. Questionnaires may also be emailed to users of a digital library if accessing the library requires some sort of identification protocol that would give you the email addresses of users. Regardless of how they are presented, questionnaires employed in usability inquiry should be brief and clear if you expect many people to respond. Consider including an incentive to respond such as entry into a drawing or a coupon good for online shopping.