Machine Learning based Video Processing for Real-time Near-Miss Detection

Video-based sensors are ubiquitous and are therefore indispensable in understanding traffic behavior at intersections. Deriving near-misses from large scale video processing is extremely useful in assessing the level of safety of intersections. In this paper, we develop real-time or near real-time algorithms for detecting near-misses for intersection video collected using fisheye cameras. We propose a novel method consisting of the following steps: 1) extracting objects and multiple object tracking features using convolutional neural networks; 2) densely mapping object coordinates to an overhead map; 3) learning to detect near-misses by new distance measures and temporal motion. The experimental results demonstrate the effectiveness of our approach with real-time performance at 40 fps and high specificity.