The development of a formative scenario-based computer assisted assessment tool in psychology for teachers: The PePCAA project

Formative computer assisted assessment has become increasingly attractive in Higher Education where providing useful feedback to large numbers of students can be difficult. However, the nature of such assessments has often been limited to objective questions such as multiple-choice. This paper reports on the development and initial trialling of a more innovative, formative use of computer assisted assessment in a Higher Education context. The European funded PePCAA (Pedagogical Psychology Computer Assisted Assessment) project developed a series of scenario-based computer-delivered formative assessments of pedagogical psychology for teachers and trainee teachers, using a range of software features, including the addition of confidence measurement. The project had a two-fold aim: to provide a tool to improve understanding of pedagogical psychology and to explore the potential of more innovative techniques of computer assisted assessment to motivate students and to assess deeper learning. The combination of computer-delivered formative assessment with innovative question styles and confidence ratings is believed to be unique for pedagogical psychology. Scenarios were based on realistic classroom situations and focused on problem solving or on utilising best practice. The PePCAA Learning Assessment Circle (PLAC) provided a framework for indexing the kinds of processes required of users. In the UK, small scale trialling involved a total of 23 teacher trainees such that each assessment was attempted by about seven participants. Participants completed evaluation questionnaires after each assessment. Responses from learners indicated that the UK scenarios were generally very well received and had at least partly achieved the aim of stimulating deeper learning. Transfer of assessments between countries proved more difficult than expected. The next stage of development should be to conduct a larger pilot, thus allowing full investigation of the reliability and validity of the assessments. There is also scope for further development of the PePCAA approach and for its application in other subjects. (c) 2007 Elsevier Ltd. All rights reserved.