The DARLENE XR platform for intelligent surveillance applications

Closed-circuit television (CCTV) systems play a significant role in the prevention and handling of criminal events. Although multiple cameras can reduce blind spots in surveillance areas, they tend to lack tools for automatic scene comprehension that could eliminate distractions and missed events. A novel intelligent surveillance system based on eXtended Reality (XR) and Computer Vision (CV) is developed to help analysts identify security incidents while improving their situational awareness by presenting them with a global representation of the area in VR. In addition, AR supervision and the ability to communicate with officers in the field, help headquarters to improve operational planning and simplify the coordination processes. The proposed platform offers real-time 3D human tracking and pose estimation in a virtual photo-realistic 3D environment equipped with a transparency layer that allows visibility in all premises of a building. In addition, informative visualizations overlay camera streams regarding 2D pose estimation, instance segmentation and activity recognition. The developed application has been tested in simulated security events, evaluating the provided functionalities.