Interviews and focus groups case study

Peterson and York (2003) report their user evaluation of the Montana Natural Resource Information System (NRIS), a digital library of natural resource information used by diverse user groups including federal, state, and local government employees, academicians and scientists, and private citizens. To obtain a representative sample of the various user groups, the evaluators employed snowball sampling whereby people interviewed early in the process nominated others who should be interviewed.

For this digital library evaluation, a total of fifty interviews were conducted throughout the state of Montana. Although transaction log analysis had already indicated a wealth of hits for NRIS (nearly 2,000 sessions per day), the interviews allowed the evaluators to provide decision makers with valuable information about how NRIS was actually used. Among the more surprising results were that many users preferred to access raw data from NRIS rather than the data processed with the NRIS-supplied applications. This evaluation study illustrates the importance of triangulating your evaluation results with multiple methods of data collection. Relying upon transaction log analysis alone in this case might have yielded misleading interpretations of the use of this digital library.