FFT pattern recognition of crystal HRTEM image with deep learning

Rapid analysis and processing of large quantities of data obtained from in-situ transmission electron microscope (TEM) experiments can save researchers from the burdensome manual analysis work. The method mentioned in this paper combines deep learning and computer vision technology to realize the rapid automatic processing of end-to-end crystal high-resolution transmission electron microscope (HRTEM) images, which has great potential in assisting TEM image analysis. For the fine-grained result, the HRTEM image is divided into multiple patches by sliding window, and 2D fast Fourier transform (FFT) is performed, and then all FFT images are inputted into the designed LCA-Unet to extract bright spots. LCA-Unet combines local contrast and attention mechanism on the basis of U-net. Even if the bright spots in FFT images are weak, the proposed neural network can extract bright spots effectively. Using computer vision and the information of bright spots above mentioned, the automatic FFT pattern recognition is completed by three steps. First step is to calculate the precise coordinates of the bright spots, the lattice spacings and the inter-plane angles in each patch. Second step is to match the lattice spacing and the angles with the powder diffraction file (PDF) to determine the material phase of each patch. Third step is to merge the patches with same phase. Taking the HRTEM image of zirconium and its oxide nanoparticles as an example, the results obtained by the proposed method are basically consistent with manual identification. Thus the approach could be used to automatically and effectively find the phase region of interest. It takes about 3 s to process a 4 K x 4 K HRTEM image on a modern desktop computer with NVIDIA GPU.