Supervised learning events: direct observation of procedural skills pilot

Background Workplace-based assessment (WBA) is the assessment of specialist competence based on what a trainee doctor actually does in the workplace. Between January 2014 and January 2015, all UK occupational medicine (OM) trainees were invited to test a suite of direct observation of procedural skills (DOPS) tools designed in a supervised learning event (SLE) format. The Faculty of Occupational Medicine (FOM) Workplace-Based Assessment Advisory Group (WBAAG) studied feedback on the new format. Aims To assess the utility of the redesigned tools, including their acceptability, feasibility, usability and key aspects of their reliability and validity. Methods The face and content validity of the new forms were assessed by a comprehension trial (CT), inter-rater reliability by a video scoring exercise and usability and acceptability by an electronic survey of trainees and trainers. Results The CT of trainees and trainers indicated that the face and content validity of the revised tools were acceptable. Inter-rater reliability video assessments indicated there was consistency of grading among trainers. Sixty-eight per cent of trainees and 95% of trainers agreed that the redesigned tools were an improvement on the current WBA DOPS tools and 83% of trainees indicated the new tools encouraged them to reflect on their performance. Conclusions The results from this pilot study provided evidence to support a request to the General Medical Council (GMC) for the new SLE-DOPS forms to be used for WBA in OM. These changes were accepted by the GMC in January 2016 for implementation in April 2016.