How Useful Are Early Economic Models? Comment on "Problems and Promises of Health Technologies: The Role of Early Health Economic Modelling"

Early economic modelling has long been recommended to aid research and development (R&D) decisions in medical innovation, although they are less frequently published and critically appraised. A review of 30 innovations by Grutters et al provides an opportunity to evaluate how early models are used in practice. The evidence of early models can be used to inform two types of decision: to continue development ("stop or go") or to alter future R&D activities. I argue that early models have limited use in stop or go decisions, as less resource and data undermine the reliability of the models' indicative estimates of cost-effectiveness. Whilst they are far more useful for informing future R&D directions, the best techniques available from statistical decision science, such as value of information analysis, are not regularly used. It is highly recommended that early models adopt these methods to best deal with uncertainty, quantify the potential value of further research, identify areas of study with the greatest potential benefit and generate recommendations on study design and sample size.