Real-time tracking of recreational boats in coastal areas using deep learning

To effectively manage and conserve coastal ecosystems, accurate spatial data on marine recreational activities are crucial. This study introduces a deep-learning-based system designed for real-time detection and tracking of recreational vessels in coastal environments using cameras. We fine-tuned and evaluated two object detection and classification algorithms, YOLOv5 and YOLOv7, for automated, real-time vessel detection, classification and positioning within Marine Protected Areas (MPAs). Additionally, we optimized two multiple object tracking algorithms, StrongSORT and ByteTrack, for tracking the movements of the detected vessels in sequential timeframes. We implemented the best combination (YOLOv5 and ByteTrack) on an NVIDIA Jetson platform, an edge computing device specifically designed for AI applications, conducting thorough benchmarking across various simulated hardware configurations to determine its minimal computational and power needs. Then, we conducted field tests by positioning the system on a coastal cliff overlooking a recreational fishery located in a partial MPA. These tests aimed to validate the system's real-time operational viability and to acquire precise vessel trajectories. The results confirmed the system's efficacy and its data collection capabilities within a real marine environment. Finally, we evaluated two camera calibration techniques for converting image trajectories to geographic coordinates: a projective transformation with homography for accurate perspective adjustment, and an innovative neural network-based approach. The system we have developed could markedly enhance the monitoring and surveillance capabilities within MPAs, generating spatial-temporal data of recreational fishing effort that can be easily transferred to other case studies.