Measuring Team Performance in Complex and Dynamic Military Environments: The SPOTLITE Method

The Scenario-based Performance Observation Tool for Learning In Team Environments (SPOTLITE) provides a systematic method for developing team performance measurement instruments comprised of behaviorally anchored rating scales that are tied to observable behaviors that tap critical knowledge and skills and can be assessed at specific intervals during a training scenario. We developed a measurement instrument for four-person teams of F-16 pilots training for air-to-air combat in a high-fidelity simulation environment and implemented it in a handheld computer to support fast and accurate data entry as a team executes a scenario. An experiment demonstrated the sensitivity, reliability, and validity of the instrument.