U-net-based deformation vector field estimation for motion-compensated 4D-CBCT reconstruction

Purpose For four-dimensional cone-beam computed tomography (4D-CBCT), its image quality is usually degraded by insufficient projections at each respiratory phase after phase-sorting. Recently, we developed a simultaneous motion estimation and image reconstruction (SMEIR) technique, which can improve lung 4D-CBCT reconstruction quality by incorporating an interphase motion model generated as deformation vector fields (DVFs). Simultaneous motion estimation and image reconstruction uses an intensity-driven two-dimensional (2D)-three-dimensional (3D) deformation technique to estimate these DVFs by intensity-matching 2D projections. However, 2D-3D deformation may fail to generate accurate intra-lung DVFs, since the motion of intricate, small lung structures only leads to subtle intensity variations on 2D projections that are insufficient to drive accurate DVF optimization. This study is to develop convolutional neural network (CNN)-based methods to fine-tune the 2D-3D deformation DVFs to improve the efficiency and accuracy of 4D-CBCT reconstruction. Methods We built two U-net-based architectures for this study. The first architecture (U-net-3C) uses 2D-3D deformation-estimated DVFs (in three cardinal directions) as the input with three channels (3C), and outputs are fine-tuned DVFs. For the second architecture (U-net-4C), the reference phase CBCT image reconstructed by SMEIR was added as an additional input channel (4C) to represent patient-specific heterogeneous properties of the lung. The output fine-tuned high-quality DVFs of both models were input again into the SMEIR workflow, as an optimized motion model, to generate the final 4D-CBCT. Both methods were evaluated on 11 lung patient cases, using fivefold cross-validation. We also reconstructed 4D-CBCTs by the original SMEIR and the SMEIR-Bio (SMEIR with biomechanical modeling) algorithms for comparison. The 4D-CBCT accuracy was quantitatively assessed through metrics including root-mean-square-error (RMSE), universal quality index (UQI), and normalized cross-correlation (NCC). The DVF accuracy was evaluated by manually tracked lung landmarks. We also evaluated our proposed methods on the SPARE challenge dataset based on reconstructed 4D-CBCT quality using the above metrics. Results The average (+/- standard deviation) residual DVF errors of SMEIR-U-net-3C, SMEIR-U-net-4C, SMEIR-Bio, and SMEIR were 3.88 +/- 3.12 mm, 3.71 +/- 2.90 mm, 3.75 +/- 3.40 mm, and 5.73 +/- 4.61 mm, respectively. The SMEIR-U-net-3C and SMEIR-U-net-4C generated images of generally improved RMSE, UQI, and NCC as compared to the other methods. Compared with SMERI-U-net-3C, SMEIR-U-net-4C has slightly higher 4D-CBCT reconstruction and DVF estimation accuracy. For the SPARE dataset, the UQI for SMEIR-U-net-3C, SMEIR-U-net-4C, SMEIR-Bio, and SMEIR were 0.96, 0.97, 0.96, and 0.94. Conclusion The CNN-based models can achieve fast (10 s) and accurate DVF fine-tuning to improve the efficiency and accuracy of 4D-CBCT reconstruction.