An optical flow-based method for condition-based maintenance and operational safety in autonomous cleaning robots

Autonomous professional cleaning robots are ubiquitous today, coexist with humans, and the demand continues to improve hygiene and productivity, especially in large indoor workplaces. Hence, a proper maintenance strategy is essential to monitor the robot's health, assuring flawless operation and safety. Manual supervision and periodic maintenance are adopted in general, which is challenging to detect failures in advance, resulting in higher maintenance costs and operational hazards. Anomalous vibration due to system degradation or environmental factors is an early symptom of failure or potential threats. Hence, predicting the sources of such abnormal vibration will help prompt maintenance action or set a hazard-free environment. However, such condition-monitoring research studies are not common for indoor cleaning robots. To fill this gap, we proposed an automated Condition Monitoring (CM) system, predicting the abnormal vibration sources that will enhance Condition-based Maintenance (CbM) and Operational Safety. A novel vibration-based CM method is presented for mobile robots, suitable for indoor environments under typical illumination states, using a monocular camera and optical flow technique instead of their usual perception-related applications. The sources of abnormal vibration were classified as terrain, collision, loose assembly, and structural imbalance. We modelled vibration as sparse optical flow, and the change in optical flow vector displacement due to the robot's vibration is derived from consecutive camera frames used as vibration data. A practical framework is developed adopting the One Dimensional Convolutional Neural Network (1D CNN) model to fit this vibration data and tested predicting the vibration source class with an average accuracy of 93.8%. In addition, a vibration source map (CbM map) is proposed by fusing the predicted class in the workplace environment map for real-time monitoring. The case studies conducted using our in-house-developed cleaning robot show that the proposed CM framework will help the maintenance team for CbM, operational safety, and select proper maintenance strategies.