Smart Monitoring and Automated Real-Time Visual Inspection of a Sealant Applications (SMART-VIStA)

Computer Vision (CV) assisted robotics applications are receiving significant attention with increased automation on the factory floor. The automated solutions provide a pathway to smart manufacturing and improve the overall product quality inspection process. We propose a closed-loop framework - Smart Monitoring and Automated Real-Time Visual Inspection of Sealant Application (SMART-VIStA) to address critical challenges in process automation and optimized process parameters feedback. This novel modular approach combines CV-based robotic path planning, deep learning-based classification, image segmentation, and real-time recommendations for corrective actions in a sealant deposition process. Specifically, the system includes pose detection and localization for the rectangular deposition plate, predicting and segmenting the glue dot class through few-shot learning, quantifying the quality of the glue dot through the unique shape quality index for circular artifacts, and an advisory recommendation system through Bayesian Decision Network. Monitoring of real-time results through a graphical user interface (GUI) allows the flexibility to change the parameters for subsequent cycles. A prototype for this process has been set up at the Boeing manufacturing facility to demonstrate its effectiveness. With accuracies extending beyond 90% in multiple tasks, this framework has promising applications in sealant inspection and can extend to different manufacturing scenarios, such as robotic welding and robotic painting. (c) 2023 The Authors. Published by ELSEVIER Ltd. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0)