Shape Reconstruction of Soft Manipulators Using Vision and IMU Feedback

In recent times, soft manipulators have gardened immense interest given their dexterous abilities. A critical aspect for their feedback control involves the reconstruction of the manipulator shape. The research, for the first time, presents shape reconstruction of a soft manipulator through sensor fusion of information available from Inertial Measurement Units (IMUs) and visual tracking. The manipulator is modeled using multi-segment continuous curvature Pythagorean Hodograph (PH) curves. PH curves are a class of continuous curvature curves with an analytical expression for the hodograph (slope). The shape reconstruction is formulated as an optimization problem that minimizes bending energy of the curve with a length constraint and the information from IMUs and/or visual markers. The paper experimentally investigates the robustness of shape reconstruction for scenarios when position of all visual markers, or slope at all the knots (placement of sensors) are known. Occlusion of manipulator segments is frequent, hence, this scenario is simulated by fusing information of available slopes (IMUs) at all knots and position (vision) at some knots. The experiments are performed on a planar tensegrity manipulator with IMUs feedback and visual tracking. The robustness study indicates reliability of these models for real world applications. Additionally, the proposed sensor fusion algorithm provides promising results where, for most cases, the shape estimates benefit from additional position information. Finally, the low dimensionality of the optimization problem argues for extension of the approach for real-time applications.