A Textural Deep Neural Network Combined With Handcrafted Features for Mechanical Failure Classification

Periodic fractographic analysis of fracture surfaces helps improve performance of mechanical pieces and avoids economical and security problems in many industries, such as the automotive industry. Classifying a fracture into a failure mode is necessary to determine the causes that generated the fracture in the first place. Experts in fracture classification of metallic materials usually use texture and surface marks to determine the type of fracture. Deep Learning is a machine learning technique that learns features directly from input data and has achieved outstanding results in object classification. However, when it comes to texture classification, the results of deep learning are not as good as in other classification tasks. This paper proposes to improve the performance of deep learning for texture analysis in the context of fractographic classification by extracting handcrafted features (Haralick, fractal dimension and local binary patterns) from the output of the convolutional layers of the VGG-19 model. Four datasets are used in this paper. Two common textural databases, KTH-TIPS and KTH-TIPS2-B, are used as benchmark for the texture recognition problem, and two datasets of fractures are used for evaluation of the proposed methods. One of the datasets for evaluation corresponds to real-scale images of ductile, brittle and fatigue fractures; and the other dataset corresponds to images acquired with a Scanning Electron Microscopy (SEM) of ductile, brittle, fatigue and corrosion fatigue fractures. The best performance for the KTH-TIPS and KTH-TIPS2-B datasets was attained with local binary patterns (LBP) extracted from the first feature map of the third convolutional layer with an F1-score of 1.0 and from the second feature map of the fifth convolutional layer group with an F1-score of 0.77, respectively. In the case of the fracture database of real-scale images, the best performance was attained with fractal dimension features extracted from the feature maps of the first convolutional layer with an F1-score of 0.72, and with LBP extracted from the first feature maps of the fourth convolutional layer for the SEM images.