How should an evaluation plan be organized: Sample

The Sample section of the evaluation plan specifies the digital library users, information scientists, subject matter experts and other people from whom data will be collected. They are also called the evaluation participants. Except in rare situations, it is not possible to collect information from everyone in any given population of potential participants. Therefore some sort of sampling is required whereby a subset of the population is selected to represent the information that would be collected from everyone if that was feasible. One way of sampling is to use some sort of random selection process, but this is impractical in most real world evaluation contexts. You will want to put some serious thought into your sampling plan. Involving people in an evaluation should not be done carelessly because you are asking for their valuable time and energy.

The nature of your sampling strategies will vary considerably depending upon the methods selected and the status of the digital library being evaluated. For example, early in the stages of development of a digital library, fewer participants will be involved for longer and more intensive evaluation sessions. On the other hand, when a digital library is ready for beta testing, it can be shared with large numbers of reviewers who might try it out and complete a pop-up questionnaire about it. Here is a brief example of a Sample section:

SAMPLE:

The participants in this evaluation will include a:

non-random sample of people who log into DLEE during first quarter 2004
panel of digital library experts identified by the editor of D-LIB Magazine
panel of engineering education experts identified by the NAAEP Board
usability testing expert from Usability Gurus, Inc.

The Sample section should answer the following questions:

Who will participate in the evaluation?
How will the participants be identified and recruited?