Unmasking a phantom:: A psychometric assessment of mystery shopping

Because of increasing concern over whether they are really satisfying their customers, more retail and service firms are using mystery shoppers (sometimes also referred to as secret, phantom, or anonymous consumer shoppers) to monitor their frontline operations, to assess their customer service, and to benchmark their competitors' performance. But virtually nothing is known about the psychometric quality of the data collected in mystery shopping studies, and how it compares with that of customer survey data. Yet such information is necessary for organizations to know when to use mystery shopping, how to design mystery shopping studies, and what weight to place on their results. Here we use a generalizability theory approach to assess the psychometric quality of mystery shopping data. First, we design a study to compare the use of mystery shopping and a traditional customer survey when collecting data to assess the service quality of competitive stores. Second, we evaluate the psychometric performance of a set of secondary mystery shopping data collected to evaluate and benchmark the performance of branches of a retail network. Finally, in a follow-up study, we examine whether mystery shopping data collected to scale the more objective store environment are more reliable than data for service quality.