The effect of search procedures on utility elicitations

Objective. Elicited preferences for health states vary among scaling methods, manners of describing health states, and other features of the elicitation process. The authors examined the effects of changing the search procedure for a subject's utility on mean utility values. Methods. A randomized controlled trial of two search procedures (titration and "ping-pong") using two otherwise identical computer programs that describe health states related to Gaucher's disease, then measuring subjects' preferences. Setting. Paid, healthy volunteers recruited from the community through advertisements. Results. The mean time tradeoff (TTO) and standard gamble (SG) utility values for life with severe anemia and splenomegaly and life with chronic bone pain from Gaucher's disease were between 0.10 and 0.15 higher with the titration search procedure than with the ping-pong procedure. Effects of the search procedure were additive with variability due to scaling methods, resulting in mean differences in utility ratings for the same health state of as much as 0.28 among procedures and scaling methods. Effects of search procedures on utility values persisted on repeated testing at week 2 and week 3; there was no evidence of convergence to a single "true" utility value over time. Conclusions. The procedure used to search for subjects' utility values strongly influences the results of preference-assessment experiments. Effects of search procedures persist on repeated testing. The results suggest that utility values are heavily influenced by, if not created during, the process of elicitation. Thus, utility values elicited using different search procedures may not be directly comparable.