Assessment of medical students' communicative behaviour and attitudes: estimating the reliability of the use of the Amsterdam attitudes and communication scale through generalisability coefficients

It is widely accepted that adequate attitudes and communicative skills are among the essential objectives in medical education. The Amsterdam attitude and communication scale (AACS) was developed to assess communicative skills and professional attitudes of medical students. More specifically, it was designed to evaluate the clinical behaviour of clerks to establish their suitability for the medical profession. The AACS covers nine dimensions. Moreover, an overall judgement of the student's performance is included. The present paper reports first results on the reliability of the use of the AACS. Data were collected in the course of an AACS training programme for future judges: senior medical and nursing staff members (N = 98). Participants judged three videotapes of clerks interviewing patients at the bedside. For the assessment of videotapes, the first four dimensions of the AACS and the overall judgement are relevant. By applying Generalisability Theory to the training data we can forecast the reliability of the AACS in practice and gain insight in the number of raters that is needed to achieve sufficient reliability in clinical practice. If clerk behaviour is rated by six judges, summative assessment is sufficiently precise, i.e. <0.25. When using the full AACS, covering 10 items, the same number of judges is needed. Scores on individual AACS items are not sufficiently reliable. In conclusion, the results indicate that students' behaviour can be evaluated in a reliable manner using the AACS as long as enough judges and items are involved. (C) 2001 Elsevier Science Ireland Ltd. All rights reserved.