Real-time monitoring of manual acupuncture stimulation parameters based on domain adaptive 3D hand pose estimation

Manual acupuncture (MA) is a widely used type of therapy method in the world, its treatment result and clinical safety are highly related to the selection of stimulation parameters (needling amplitude and frequency) of acupuncturists. However, to date, there is no stimulation parameter measurement solution that can be conveniently used in the clinic. Thus, there is an urgent need to develop a single camera-based real-time monitoring system for MA operation. This monitoring system is expected to give the result of both the amplitude and frequency of MA. Considering that constructing the labeled MA monitoring dataset is laborious and time-consuming and that there is a large amount of unlabeled data, we propose an adaptive orientation-based domain adaptation framework to alleviate the domain shift and achieve better performance. Moreover, we contribute a benchmark that contains 20 videos of on-body MA operation and 30 videos of on -simulator MA operation with 3D coordinates to facilitate the future development of real-time MA monitoring. Extensive experiments on the proposed benchmark demonstrate the superiority of the proposed methods on both movement estimation and frequency estimation of hand acupuncture. The application prospects of this framework for the clinical work of MA included the investigation of dose-effect relationship of MA, enhancement of its operation safety, etc. Our data is publicly available at https://github.com/SHUTCM-tcme/SUTCM-AM.