Analysis of UTM Tracking Performance for Conformance Monitoring via Hybrid SITL Monte Carlo Methods

Conformance monitoring supports UTM safety by observing if unmanned aircraft (UA) are adhering to declared operational intent. As a supporting system, robust cooperative tracking is critical. Nevertheless, tracking systems for UAS traffic management (UTM) are in an early stage and under-standardized, and existing literature hardly addresses the problem. To bridge this gap, this study aims to probabilistically evaluate the impact of the change in tracking performances on the effectiveness of conformance monitoring. We propose a Monte Carlo simulation-based method. To ensure a realistic simulation environment, we use a hybrid software-in-the-loop (SITL) scheme. The major uncertainties contributing to the stochastic evaluation are measured separately and are integrated into the final Monte Carlo simulation. Latency tests were conducted to assess the performance of different communication technologies for cooperative tracking. Flight technical error generation via SITL simulations and navigational system error generation based on flight experiments were employed to model UA trajectory uncertainty. Based on these tests, further Monte Carlo simulations were used to study the overall impacts of various tracking key performance indicators in UTM conformance monitoring. Results suggest that the extrapolation of UA position enables quicker non-conformance detection, but introduces greater variability in detection delay, and exacerbates the incidence of nuisance alerts and missed detections, particularly when latencies are high and velocity errors are severe. Recommendations for UA position update rates of >= 1 Hz remain consistent with previous studies, as investments in increasing the update rate do not lead to corresponding improvements in conformance monitoring performance according to simulation results.