Empowering Diagnosis: Vision Transformers, CNNs, and Machine Learning in Mammogram Classification for Breast Cancer

Breast cancer is a leading cause of cancer-related deaths globally, emphasizing the need for accessible and accurate diagnostic tools. In this paper, we present an AI-driven system designed to classify mammogram images as benign or malignant. Building upon our previous work on a breast cancer treatment expert system, which utilized forward-chaining inference based on NCCN guidelines integrated with ChatGPT, this study expands the system's capabilities to include diagnostic support. We evaluated a combination of machine learning (ML) models, a fully trained custom convolutional neural network (CNN), and pretrained CNNs and Vision Transformers used as feature extractors for breast cancer classification. These models were trained and optimized for robust performance, with results compiled for comparative analysis. We also deployed our model on Azure with an automated pipeline, enabling seamless integration with a user-friendly website. Patients can upload mammogram images and receive instant classification results, bridging diagnostic insights with treatment recommendations. This work represents a significant step toward a comprehensive AI solution for breast cancer diagnosis and treatment, aiming to improve accessibility and personalized care.