Realtime semantic enrichment of video streams in the age of big data

This paper describes AgileRAI, a framework for searching, organizing, and accessing multimedia data in a fast and semantic-driven way. AgileRAI supports realtime ingestion of video streams on which different machine-learning techniques (such as global and local visual features extraction and matching) are applied in a parallel and scalable way. Extracted features are matched to a reference database of visual patterns (e.g., faces, logos, and monuments) in order to produce a set of metatags describing the ingested contents. Furthermore, these tags are semantically enriched using open semantic data repositories. The system is designed with a scaleout pattern architecture based on Apache Spark, ensuring high performance in Big Data management environments. © 2019 SMPTE.