Using cognitive task analysis to explore issues in the procurement of intelligent decision support systems

Government statistics and various news reports suggest that upwards of half of all large-scale information technology (IT) development projects fail to meet expectations for facilitating cognitive work. Many of the failures point to the neglect of human-centering considerations during the development of sociotechnical systems. The groups of people who create IT themselves constitute a sociotechnical system. Therefore, laws of cognitive work apply to the cognitive work of IT development, and these laws include the "reductive tendency" for people to form simplified understandings when confronted with domains of dynamics and complexity. In this article, we report a study in which we "turned the tables" on IT systems development. Rather than using cognitive task analysis to study some work domain for which an envisioned IT system would be developed, we used cognitive task analysis to study the work domain of IT systems development itself. Through documentation analysis and critical decision method procedures, we sought to reveal specific challenges with regard to human-centering, and ways in which principles, methods, and tools of ergonomics (human factors, cognitive systems engineering) might help the developers of IT systems better address the human and social aspects of cognitive work. The findings highlight the outstanding challenges and barriers to the procurement and development of usable, useful, and understandable IT for sociotechnical systems. Challenges include the following: the need for better coordination mechanisms; the need to locate cognitive systems engineers, as advocates for workers, in key leadership roles; the need to reconceive concepts and methods of requirements and requirements specification; and the need for better negotiation of the trade-offs of cost/schedule considerations with human-centering considerations. © 2008 Springer-Verlag London Limited.