Research progress on polarimetric imaging technology in complex environments based on deep learning (invited)

Significance Polarization information, as one of the fundamental physical characteristics of light waves, can provide information about the intrinsic properties of the target. Polarimetric imaging technology digitizes the polarization information of the measured target field through digital processing. This approach effectively reduces the interference from the light propagation environment, thereby improving the imaging quality of the target and enhancing perception of its characteristics. In complex environments, polarimetric imaging has significant advantages. However, in complex environments such as scattering and low illumination, the degradation mechanism of polarized images exhibits nonlinear characteristics, leading to high complexity in polarized information interpretation methods. Deep learning methods possess powerful feature extraction and learning capabilities, enabling the recovery of polarized information by learning the mapping rules hidden in the large-scale collected data. This approach is particularly suitable for complex signal processing problems like polarimetric imaging, which involves multiple dimensions and interrelated signals. Progress First, the basic theory of the polarimetric imaging is introduced, including the principles of polarimetric imaging and a macroscopic description of polarimetric imaging issues in complex environments. Next, the general workflow of deep learning polarization imaging technology in complex environments is introduced. Based on deep learning, polarimetric imaging technology in complex environments uses the multidimensional polarimetric parameters collected by the polarimetric imaging system as input data. It leverages the nonlinear feature-fitting capabilities of neural networks to obtain image restoration results. Essentially, this approach transforms the nonlinear inverse problem of polarimetric imaging restoration in complex environments into a pseudo-forward problem, avoiding the challenges associated with solving nonlinear inverse problem algorithms. The representative developments of research in deep learning polarimetric imaging technology in response to scattering and noise, two of the most representative complex imaging environments, have been elaborated. From the inception of research in this field, the developmental trajectory of the field has been systematically outlined. In the early stages, polarimetric imaging technology in complex environments based on deep learning primarily relied on supervised training. Due to the challenges in collecting real-world data, researchers explored solutions using unsupervised, self-supervised, transfer learning, and simulation algorithms. Researchers also delved into the incorporation of prior knowledge and physical models into networks, leading to training approaches embedded with physical models or guided by prior knowledge. Overall, these representative works have made significant contributions to addressing the difficulties in constructing large-scale datasets, enhancing the generalization performance of networks, and exploring the interpretability of the networks. To better illustrate the connections and distinctions among research works, and to streamline the developmental process in this field for reader convenience, a summary has been compiled in the form of a table. The table provides task types, training methods, and characteristics of representative works for easy reference. Conclusions and Prospects With the rapid development of deep learning, polarimetric imaging technology in complex environments has achieved remarkable research progress. Existing studies indicate that, due to the multiple parameters and inherent correlations in polarized information, this multi-dimensional and interrelated signal processing problem is well-suited for the application of deep learning. The combination of deep learning and polarimetric imaging technology enables further improvement in optical imaging quality, meeting the imaging demands of complex environments and demonstrating more prominent advantages. The generalization ability, interpretability, and parameter lightweighting of deep learning technology remain areas that require further in-depth research. There is a continued need for refinement in multimodal fusion strategies, exploration of the underlying principles of network polarimetric parameter image restoration, and the design of network structures tailored for polarized multidimensional data to enhance real-time performance. Further efforts are essential to consolidate the feasibility of deep learning models in polarimetric imaging within complex environments, to enhance the adaptability of models to changes in complex environmental conditions, and to make them more universally applicable across different scenarios. © 2024 Chinese Society of Astronautics. All rights reserved.