Service evaluation case study

An example of an evaluation that was done using the multi-attribute, stated-preference technique is the evaluation of the Comprehensive Access to Printed Materials (CAPM) project at John Hopkins University (Choudhury et al., 2002). The CAPM project involves a robotic retrieval system that provides users with the ability to view, full-text search, and scan materials that are shelved off-site. The CAPM system allows a user to control a robot at an off-site shelving location to retrieve materials, and bring the materials to a scanning station where the user can actually browse the materials to determine whether to request the materials be delivered or returned to the shelf.

The evaluation team used the multi-attribute, stated-preference technique to determine users' preferred combination of service level and price from 36 possible options. Factors considered in these options included: presence or absence of digital images; presence or absence of full-text search; delivery time to receive materials from off-site location; and price per semester for using CAPM service. A Web-based choice survey was constructed and refined. A total of 2,000 randomly selected John Hopkins faculty, students, and staff were invited to participate. An incentive for participation was offered (a chance to win a $500 travel certificate), and eventually 603 people responded, reflecting a 30% response rate.

From the survey results, the evaluation team was able to determine an approximate amount of money users were hypothetically willing to pay for services provided by CAPM (Choudhury et al., 2002). This information was used to inform the decision to continue development of the CAPM robotic system at John Hopkins University. More information about the project is available at: http://dkc.mse.jhu.edu/CAPM/.