Assessment of medical devices: How to conduct comparative technology evaluations of product performance

Objectives: U.S. expenditures on medical devices ($70 billion in 2003) are one of the fastest growing components of hospital costs. Physicians' selection of medical devices lacks an evidence base on the comparative clinical effectiveness of these products. Comparative studies (e.g., vendor 1 versus vendor 2, technology A versus technology B) are increasingly promoted in the public sector as a means of cost containment, value-based purchasing, and quality improvement. This study illustrates how hospitals and physicians can conduct comparative technology assessments of product performance. Methods: Surgeons evaluated comparable medical devices manufactured by eight different vendors in standardized surgical procedures. Devices included sutures and endomechanical products, which account for $2.5 billion of total device spending. Evaluations covered multiple performance dimensions, including ergonomics, functionality, clinical acceptability, and vendor preference. Results: One vendor's products garnered consistently high ratings from surgeons, while two other vendors garnered consistently low ratings. Differences in ratings were statistically significant and persist when controlling for physician background characteristics and prior experience. Study results were used by a large hospital group purchasing organization to select which vendors to contract with for these products. Conclusions: Comparative technology evaluations assist physicians and hospitals in making cost-effective purchases of devices. These evaluations provide robust information on the performance of products routinely used by clinicians. Such evaluations can be carefully designed to have scientific rigor and clinical credibility.