To investigate the relationship between metal magnetic memory testing (MMMT) signals and stress concentration factors (SCFs), four-level sinusoidal constant-amplitude load tension-tension fatigue tests were carried out on 45CrNiMoVA steel specimens with different SCFs. The normal component of MMMT signals, Hp(y), was collected during the fatigue tests, and three characteristics were extracted and analyzed during the tests, including the peak-to-peak value of abnormal peaks (Delta Hp(y)), the slope coefficient of the fitting line of Hp(y) (K1), and the slope coefficient of the fitting line of Hp(y) between abnormal peaks (K2), and a back-propagation (BP) neural network was developed to differentiate the SCF of the specimens. The results showed that both fatigue load and fatigue cycle number influenced MMMT signals, and the characteristics remained stable as the fatigue cycle number increased for the same fatigue load but increased significantly as fatigue load increased. In addition, all the characteristics increased as the distance between the scan line and the center line increased, but none of them could be used to differentiate the SCF of the specimens. With properly selected input vector and hidden nodes, the established BP neural network can quantitatively recognize the SCF of specimens.