HoloNeedle: Augmented Reality Guidance System for Needle Placement Investigating the Advantages of Three-Dimensional Needle Shape Reconstruction

Augmented reality (AR) systems have been proposed for medical needle procedures as alternatives to restore the physician's natural perspective and head motion parallax that are lost when using standard imaging methods. AR systems have assumed a rigid needle, but in practice, a needle can experience large deflections that can lead to significant errors. We combine an instrumented needle that measures bending deflections with optical tracking of the needle base for precision and an AR system to provide users with precise three-dimensional rendering of needle motion and deflection inside a body. In controlled tests, users guided a needle through 7 cm of tissue phantom to reach a 2-mm diameter target. We found that displaying needle shape allows users significantly reduce placement errors when compared to the rigid needle assumption. In addition, a new display technique called tip tangent extrapolation, based on the sensed needle endpoint and direction, allowed users to further reduce errors resulting in 26% less error when compared to a rigid needle assumption.