Computer Vision Based Robotic Arm Controlled Using Interactive GUI

This paper presents the design and implementation of a robotic vision system operated using an interactive Graphical User Interface (GUI) application. As robotics continue to become a more integral part of the industrial complex, there is a need for automated systems that require minimal to no user training to operate. With this motivation in mind, the system is designed so that a beginner user can operate the device with very little instruction. The application allows users to determine their desired object, which will be picked up and placed by a robotic arm into the target location. The application allows users to filter objects based on color, shape, and size. The filtering along the three parameters is done by employing a Hue-Saturation-Value (HSV) mode color detection algorithm, shape detection algorithm, size determining algorithm. Once the target object is identified, a centroid detection algorithm is employed to find the object's center coordinates. An inverse kinematic algorithm is used to ascertain the robotic arm's joint positions for picking the object. The arm then goes through a set of preset positions to pick up the object, place the object, and then return the arm to the initial position. The joint coordinates are forwarded to a microcontroller that sets the arm's joint angle at each position.