Fusion of Multi-Modal Underwater Ship Inspection Data with Knowledge Graphs

With recent advances in underwater inspections of ships with remote sensing technologies the need for automated data annotations and analysis becomes apparent. During underwater ship inspections, various data such as video, positioning information, and other telemetry data are collected and combined with the results of computer vision models. The variability in the modalities of data makes the automatic analysis across multiple data sources challenging. We propose the use of a Knowledge Graph in combination with industry standards in the ship inspection domain for the taxonomy. This enables automated data analysis for underwater ship inspection videos which is the requirement for different downstream use cases. In this work, we demonstrate the applicability of our approach on 12 ship inspections in two downstream tasks. First, we aim at supporting a detailed ship status report generation, and second, we demonstrate big data analytics for several inspections. We use the fused data to compare different ships by identifying patterns in the findings aided by computer vision algorithms.