Smart inspection for assembly states of connectors in wiring harness assisted by AR glasses

To replace the manual inspection of connecting states between wiring harness and connectors in the assembly process of aerospace products, a smart inspection method assisted by Augmented Reality (AR) glasses for assembly states of connectors in wiring harness was proposed. The smart-inspection automatically conducted three major processes of pin-hole detection, pin-hole sequencing and mismatched pin-hole visualization. For pin-hole detection, the detection model combining the Feature Pyramid Networks (FPN) architecture and the Bi-directional Long Short-Term Memory (BiLSTM) component effectively located pin-holes' location in images and recognized their actual assembly states. These detected pin-holes were clustered into different annuluses and numbered according to their polar angle in the operation of pin-hole sequencing. Traversing all numbered pin-holes in order, mismatched pin-holes that were missing with wires or contain extra mismatched wires were identified by comparing the ground truth of assembly results and actual assembly states. AR glasses presented the wireframe image which could highlight these mismatched pin-holes for visualization. For eleven different connectors, the pin-hole detection model achieved a competitive performance with the figure for accuracy up to 99.00% in mean Average Precision (mAP) which was the metric in PASCAL VOC dataset. Meanwhile, the pin-hole sequencing method even against the negative effect of image deformation in some extends. © 2021, Editorial Department of CIMS. All right reserved.