SpectraTr: A novel deep learning model for qualitative analysis of drug spectroscopy based on transformer structure

The drug supervision methods based on near-infrared spectroscopy analysis are heavily dependent on the chemometrics model which characterizes the relationship between spectral data and drug categories. The preliminary application of convolution neural network in spectral analysis demonstrates excellent end-to-end prediction ability, but it is sensitive to the hyper-parameters of the network. The transformer is a deep-learning model based on self-attention mechanism that compares convolutional neural networks (CNNs) in predictive performance and has an easy-to-design model structure. Hence, a novel calibration model named SpectraTr, based on the transformer structure, is proposed and used for the qualitative analysis of drug spectrum. The experimental results of seven classes of drug and 18 classes of drug show that the proposed SpectraTr model can automatically extract features from a huge number of spectra, is not dependent on pre-processing algorithms, and is insensitive to model hyperparameters. When the ratio of the training set to test set is 8:2, the prediction accuracy of the SpectraTr model reaches 100% and 99.52%, respectively, which outperforms PLS_DA, SVM, SAE, and CNN. The model is also tested on a public drug data set, and achieved classification accuracy of 96.97% without pre-processing algorithm, which is 34.85%, 28.28%, 5.05%, and 2.73% higher than PLS_DA, SVM, SAE, and CNN, respectively. The research shows that the SpectraTr model performs exceptionally well in spectral analysis and is expected to be a novel deep calibration model after Autoencoder networks (AEs) and CNN.