Malware image classification: comparative analysis of a fine-tuned CNN and pre-trained models

A crucial part is played by malware detection and classification in ensuring the safety and security of computer systems. In this work, a comprehensive study has been presented for the classification of harmful or malware images that uses a Convolutional Neural Network (CNN) which has been finely tuned and its performance has been compared with five pre-trained models: ResNet50, InceptionResNetV2, VGG16, Xception and InceptionV3. The suggested CNN framework has been trained using the dataset MalImg_9010, consisting of 9,376 grayscale images resized to 128 × 128 pixels. The models have been evaluated based on their F1 score, recall, precision, and accuracy. The experiments that were conducted demonstrate that the fine-tuned CNN model achieves an impressive 0.965 as the F1 score and a 95.57% accuracy. Furthermore, the comparison with pre-trained models reveals the dominance of the presented framework concerning the F1 score and accuracy. The output of the conducted simulation suggests that the fine-tuned CNN approach shows promise for accurate malware image classification. Additionally, the paper discusses potential improvements, such as increasing the number of training epochs and incorporating larger and more diverse malware datasets, including RGB images and a broader range of malware families. The current research article gives valuable observations on various models’ effectiveness for classifying malware images and highlights the future scopes for research incorporating this domain. © 2023 Informa UK Limited, trading as Taylor & Francis Group.