Ecological Validity and Pervasiveness in the Evaluation of Ubiquitous Computing Technologies for Health Care

The difficulties associated with the evaluation of ubiquitous computing (Ubicomp) technologies increase in application domains such as hospitals, where human life can be at risk, privacy of personal records is paramount, and labor is costly and highly distributed across space and time. For the last 6 years numerous Ubicomp technologies in support of hospital work have been created and pilot-tested. In this article, the lessons learned from these evaluations are discussed, using two criteria to classify them. The first criterion is ecological validity, namely, the extent to which the evaluation is conducted under realistic conditions. Alternatives range from controlled experiments to in situ evaluations. The article argues in particular for the advantages of intermediate approaches, which is referred to as in silico and in replica. The second criterion relates to the degree of integration of the technology with the environment, which is referred to as its pervasiveness. The evaluation grid that comes out of this exercise highlights the importance of ecological validity in evaluating ambient computing technology that supports the activities conducted in complex health care settings such as hospitals. This provides a framework for evaluating Ubihealth, which can be used to select appropriate techniques as a function of the technological and environmental complexity as well as to devise novel evaluation techniques.