How do you do interviews and focus groups?

Interviews provide participants in an evaluation more opportunities to speak in their own voice instead of merely responding to the categories of questions that others have defined for them, as they might with a questionnaire. Getting started with interviews and focus groups inevitably involves preparing a protocol of questions. It is important to plan your interview protocol carefully. Refinement of an interview protocol will often involve several trial interviews and subsequent revisions of the questions. Interview protocols generally have two types of questions. The first are the major questions you wish to address. Under each of these, there are usually several secondary questions that can be used to prompt participants when they are not adequately responsive to the primary question. For example, suppose you asked a user, “What are the primary reasons you use our digital library?” If the user is unable to articulate specific reasons, you might ask secondary questions such as “Do you use the digital library for education? For research? Or for something else?”

Recording interview and focus group data presents challenges. Most interviews and focus groups are recorded. Audio or video recorders may be used, but you should be aware that these devices may intimidate interviewees to such a degree that their responses are limited. Some evaluators prefer to take brief notes during an interview, and subsequently record a more detailed transcription of the interview immediately after its conclusion. If you adopt the notes strategy, it is advisable to write everything you can recall concerning one interview before starting to conduct another. Otherwise, you are apt to confuse the responses of one person with those of someone else.

Here are ten steps to follow in carrying out interviews and focus groups:

1. Organize a team of colleagues to assist in developing the interview/focus group protocol.
2. Determine the purposes of the interview/focus group (e.g., collecting digital library user input into a redesign process).
3. Identify a representative sample with whom to conduct the interviews or focus groups.
4. Generate a list of draft questions, initially focusing on “brainstorming” as many good questions as possible, and later selecting the best ones.
5. Construct a draft interview or focus group protocol with the questions in the order that seem to make the most sense.
6. Test the protocol with a small sub-sample of your representative sample, and look for misunderstandings and dead-ends. Expect to make changes.
7. Revise the protocol and retest if necessary.
8. Carry out the interviews or focus groups with the rest of your sample.
9. Process and analyze the data using qualitative data analysis methods.
10. Report and use the results to influence decisions in a timely manner.

Analyzing the qualitative data involves going through the transcripts of what the respondents said (and/or your notes) to look for themes, patterns, or categories. Suppose you are conducting interviews with scientists to determine their willingness to trust the data provided by digital libraries of real time data. Some themes that might emerge are concerns about validity, reliable access, and transmission errors, as well as appreciation for the value of having such data for their particular scientific community. It is beyond the scope of this guide to go into much detail about qualitative data analysis, but Miles and Huberman (1994) provide extensive guidance in this area.