A continuous-action deep reinforcement learning-based agent for coronary artery centerline extraction in coronary CT angiography images

The lumen centerline of the coronary artery allows vessel reconstruction used to detect stenoses and plaques. Discrete-action-based centerline extraction methods suffer from artifacts and plaques. This study aimed to develop a continuous-action-based method which performs more effectively in cases involving artifacts or plaques. A continuous-action deep reinforcement learning-based model was trained to predict the artery's direction and radius value. The model is based on an Actor-Critic architecture. The Actor learns a deterministic policy to output the actions made by an agent. These actions indicate the centerline's direction and radius value consecutively. The Critic learns a value function to evaluate the quality of the agent's actions. A novel DDR reward was introduced to measure the agent's action (both centerline extraction and radius estimate) at each step. The method achieved an average OV of 95.7%, OF of 93.6%, OT of 97.3%, and AI of 0.22 mm in 80 test data. In 53 cases with artifacts or plaques, it achieved an average OV of 95.0%, OF of 91.5%, OT of 96.7%, and AI of 0.23 mm. The 95% limits of agreement between the reference and estimated radius values were -0.46 mm and 0.43 mm in the 80 test data. Experiments demonstrate that the Actor-Critic architecture can achieve efficient centerline extraction and radius estimate. Compared with discrete-action-based methods, our method performs more effectively in cases involving artifacts or plaques. The extracted centerlines and radius values allow accurate coronary artery reconstruction that facilitates the detection of stenoses and plaques.