Introduction
Breast cancer is increasingly recognized as a major global health concern, affecting a significant number of people worldwide. This type of cancer begins with the unchecked growth of abnormal breast cells, leading to the formation of tumors that can spread and become life-threatening if not addressed. The causes of breast cancer are multifaceted, encompassing genetics, environmental factors, reproductive complexities, and hormonal imbalances. Risk factors include age, hormone replacement therapy, family history, and lifestyle choices. The seriousness of breast cancer is underscored by stark statistics that emphasize its alarming prevalence and extensive impact, despite considerable advances in research and treatment. In 2020, breast cancer was responsible for approximately 685,000 deaths globally [
1]. This statistic paints a somber tableau of the profound repercussions this malady imposes on individuals, families, and communities around the world. A poignant observation is that nearly half of all breast cancer cases occur in women who do not have obvious risk factors, underscoring the complex nature of this disease. Although it predominantly affects women, breast cancer also occurs in men, accounting for about 0.5% to 1% of cases. As we explore the origins, progression, and effects of breast cancer, it becomes clear that addressing this issue effectively requires more than medical expertise. A comprehensive approach involving public health education, early detection strategies, and improved access to treatment is essential for the effective management and mitigation of this global health challenge [
2].
Figure 1 presents a choropleth map that visually depicts the distribution and intensity of breast cancer incidence, with Asia being the most affected continent.
The impetus for our research stems from the urgent need for precise and early breast cancer detection methods [
3]. Current diagnostic techniques depend on the subjective visual assessments of medical professionals, which are prone to errors due to variations in interpretations and personal biases. To address these challenges, our study utilizes the intricate features extracted from deep convolutional neural network (CNN) models. These models enhance diagnostic accuracy by capturing complex patterns in medical images, providing more objective and reliable insights. The features are combined to create a comprehensive and informative feature set [
4]. To increase the efficacy of our classification model, we have integrated an extreme gradient boosting (XGBoost)-based feature selection approach that identifies and retains the most influential features. Our methodology ultimately employs an XGBoost classifier that accurately distinguishes between cancerous and non-cancerous breast lesions. This tool is significant for aiding medical practitioners in their decision-making processes. Our multifaceted approach aims to deliver a dependable and automated system that supports oncologists in providing precise and timely assessments, potentially leading to improved patient outcomes and even life preservation [
5]. Regular breast self-exams, clinical assessments, and mammography screenings are crucial in detecting breast cancer at manageable stages. Meanwhile, the incorporation of state-of-the-art technologies, particularly deep learning (DL), has opened promising avenues for enhancing detection accuracy. DL frameworks equipped with sophisticated algorithms can analyze extensive patient data and medical image datasets to detect complex patterns and anomalies that may indicate cancer. According to Murtaza et al. [
6], radiologists can use these models to refine their image interpretations, which will reduce the likelihood of false positives and negatives and ultimately improve patient outcomes. This study has the potential to enhance the precision and effectiveness of automated diagnostic tools, benefiting oncologists and patients worldwide.
The structure of the remaining paper follows a systematic framework. Section 2 offers a concise overview of relevant prior research, critically analyzing and comparing various classification techniques used in diagnosing breast cancer. Section 3 details the dataset utilized in this research. Section 4 explores the methodology, highlighting key aspects such as the characteristics of the dataset, the pre-processing steps applied to the data, the details of feature extraction using deep CNNs (D-CNNs), and an examination of the classification methods being considered. Section 5 presents empirical results, providing a comprehensive analysis of how the classifiers performed across various evaluation metrics. Section 6 concludes the research and discusses the future scope of the work.
Literature Review
Breast cancer diagnosis and classification have been significantly advanced by machine learning (ML) and DL techniques. In this comprehensive literature survey, we explore various research efforts to understand their methodologies, performance, and potential impact on improving breast cancer classification [
7]. These studies demonstrate the ongoing pursuit of more accurate and efficient diagnostic tools, which hold promise for timely detection. Bhardwaj and Tiwari [
8] utilized the Wisconsin Breast Cancer database (WBCD) dataset and applied the Genetically Optimized Neural Network method, achieving high classification accuracy. Das et al. [
9] employed a gene expression dataset from Mendeley to generate synthetic 2D datasets. Their innovative ensemble DL model included 3 CNNs as base classifiers, with a multilayer perceptron (MLP) used for the final classification. Their findings showed that the ensemble model surpassed the performance of single models. The GeneViT model featured a stacked autoencoder, Improved DeepInsight for image transformation, gene selection via t-SNE, and a channel expansion algorithm, all contributing to its superior performance on benchmark datasets. Adem [
10] adopted a hybrid approach to analyze a microarray dataset for breast cancer, combining a stacked autoencoder with the Subspace k-nearest neighbors (KNNs) algorithm. On a smaller dataset from the Kent Ridge-2 database, this approach achieved an accuracy of 91.24%.
Alruwaili and Gouda [
11] employed transfer learning to fine-tune pre-trained models such as ResNet50 and Nasnet-Mobile for evaluating their system on the Mammographic Image Analysis Society dataset. Their approach, which incorporated robustness augmentation techniques, achieved an accuracy of 89.5% with ResNet50 and 70% with Nasnet-Mobile. Mohammed et al. [
12] utilized 184 ultrasound images to automate the characterization of breast lesions, achieving notable levels of specificity, sensitivity, and precision. Cai et al. [
13] enhanced mammogram-based cancer diagnosis, achieving high accuracy through a comprehensive approach that included image quality improvement, segmentation, feature extraction, and an optimized CNN. Choudhury and Perumalla [
14] combined CNNs with a vision transformer (ViT) for feature encoding to extract features from BUSI1311 and breast histopathological images. Their model’s performance was further improved by a transfer learning strategy that incorporated BERT pre-training of Image Transformers. Additionally, Razavi et al. [
15] achieved strong mitosis detection and high segmentation accuracy using conditional generative adversarial networks across multiple datasets for mitosis and nuclei segmentation. These studies highlight the effectiveness of using various image modalities to address medical issues. In another study, Stephan et al. [
16] introduced a highly accurate hybrid artificial bee colony with whale optimization algorithm, which was subsequently evaluated on multiple breast cancer datasets.
In a similar vein, Budak et al. [
17] developed a model that combines a fully convolutional network for advanced feature extraction with a bidirectional long short-term memory to detect breast cancer using the BreaKHis database. Their model achieved a remarkable average accuracy of 91.90%. Additionally, Kadam et al. [
18] proposed a feature ensemble learning approach that combines Sparse Autoencoders with Softmax Regression on the WBCD dataset, achieving an impressive accuracy of 98.60%. Togacar et al. [
19] introduced the innovative BreastNet CNN model, which utilizes the BreaKHis dataset and demonstrated exceptional abilities in detecting breast cancer. Furthermore, Ayana and Choe [
20] developed the BUViTNet model, employing a multistage transfer learning approach with visual information, using publicly available ultrasound breast image datasets. This framework achieved remarkable accuracies of 98.58% and 97.87% for binary and multi-class classification, respectively, utilizing INbreast mammograms.
Strelcenia and Prakoonwit [
21] employed the WBCD without images and introduced the Kullback-Leibler Divergence Conditional GAN method to generate synthetic data. This method proved stable and accurately represented the original dataset. Dheeba et al. [
22] developed an abnormality detection algorithm using a clinical database of 216 mammograms, achieving an area under the receiver operating characteristic curve score of 0.96853. Ferreira et al. [
23] utilized RNA-sequencing (RNA-Seq) datasets along with image-derived features from blood smears and breast cell nuclei images, exploring various autoencoders and training approaches. Their research consistently outperformed baseline models, achieving notable F1 scores in RNA-Seq and image-derived data classification. Xiong et al. [
24] used a micro-computed tomography dataset to predict breast cancer bone metastases, introducing a temporal variational autoencoder that demonstrated superior predictive accuracy compared to existing models. Tummala et al. [
25] utilized the BreaKHis dataset and achieved exceptional performance with an ensemble of Swin transformers, recording average test accuracies of 96.0% for 8-class and 99.6% for binary classification. Wang et al. [
26] introduced a model for recognizing colorectal cancer in pathological images through the use of semi-supervised DL. They utilized well-established CNN models, including VGG19, ResNet101, DenseNet201, and the ViT base model. Among these, ResNet101 demonstrated the highest accuracy (95.59) and F1 score (94.76). Qin et al. [
27] explored breast cancer lesion segmentation in dynamic contrast-enhanced magnetic resonance imaging by employing the TR-IMUnet model. This 2-stage framework integrated the U-Net architecture with a transformer module to improve segmentation accuracy. Su et al. [
28] developed the YOLO (You Only Look Once) and LOGO (Local-Global) model for detecting and segmenting masses in mammography, utilizing the CBIS-DDSM and INBreast datasets. This model surpassed the performance of baseline methods. Supriya and Deepa [
29] introduced an optimized artificial neural network that incorporated data pre-processing, feature selection using the Modified Dragonfly algorithm, and optimization with the Gray Wolf Optimization algorithm. Their bit plane complexity segmentation showed superior performance compared to the Improved Weighted-Decision Tree classifier.
Abdar and Makarenkov [
30] conducted a comprehensive analysis of the WBCD using the CWV-BANNSVM ensemble model, which achieved a 100% accuracy rate in depicting overfitting. Chiu et al. [
31] analyzed a dataset from the University Hospital Centre of Coimbra, employing principal component analysis for dimension reduction and using an MLP for feature extraction. Their method achieved an impressive accuracy of 86.97% through 10-fold cross-validation. Wang [
32] explored the use of microwave imaging combined with ML for breast cancer detection, showing promising results. The study involved a training dataset of 3-dimensional contrast-source inversion images and demonstrated exceptional performance on both synthetic and experimental data. Prinzi et al. [
33] applied transfer learning to YoloV3, YoloV5, and YoloV5-Transformer using the CBIS-DDSM, INbreast, and a proprietary dataset. The smaller YoloV5 model exhibited superior performance with a mean average precision of 0.621, and the integration of YOLO predictions with Eigen-CAM-generated saliency maps effectively reduced false negatives in clinical settings.
In conclusion, the studies reviewed in this literature survey introduce innovative methods for diagnosing and classifying breast cancer. However, these studies also exhibit several limitations and drawbacks that must be taken into account when considering their implementation in clinical settings. These limitations include small and specific dataset sizes, limited diversity within datasets, difficulties in generalizing findings to clinical environments, challenges in interpreting complex models, potential biases in imbalanced datasets, a lack of sufficient external validation, ambiguous clinical relevance, and inadequate attention to ethical and privacy issues. In the following section, we introduce our solution designed to address these challenges-the Breast Cancer Ensemble Diagnosis Network (BCED-Net). Our model leverages the capabilities of transfer learning by integrating a range of pre-trained models for feature extraction. Additionally, we utilize ML classifiers to improve the accuracy of breast cancer classification. BCED-Net not only seeks to enhance the robustness, adaptability, and interpretability of breast cancer diagnosis models but also addresses ethical and privacy concerns by incorporating a diverse array of pre-trained models.
Materials and Methods
The RSNA Breast Cancer Detection dataset, publicly available on Kaggle, forms the basis of this research. It comprises over 60,000 mammograms from women, totaling more than 110,000 images, thus providing a substantial resource for analysis. Each mammogram is meticulously annotated to show whether a breast lesion is present or absent [
34]. The dataset is divided into 3 subsets: 10% is allocated for validation, 10% for testing, and the remaining 80% for training purposes.
Figure 2 presents 2 sample images from the RSNA dataset: one from a cancerous subject and one from a normal subject.
The primary objective of this study was to utilize the dataset to create a robust model that can accurately classify images into their respective categories. The main focus was on developing a model that can effectively differentiate between cancerous and normal cases using the provided images.
Methodology
The research’s step-by-step methodology is graphically represented by the flowchart in
Figure 3, which provides a clear and concise overview of the methodology used in this paper by elucidating the systematic procedures, techniques, and processes that were employed throughout the research process.
Data Pre-Processing
To enable essential model training and performance optimization, the first step of the process involved data pre-processing of the input images.
Resize
Images were scaled to a uniform size of 64×64 pixels. This standardization ensured that each image could be processed consistently in subsequent stages. As a result, the model benefited from learning from uniform image representations.
Scaling
Every image’s pixel values were adjusted to fall between 0 and 1. This scaling was necessary for ML algorithms to process numbers effectively, as it standardized the intensity levels of the pixels.
These pre-processing stages, which converted the input images into a consistent and optimized format, enabled more effective and precise extraction of information from the data.
Feature Extraction
The scaled and resized images were input into various configurations of pre-trained CNNs to extract features. These D-CNNs were selected based on their diverse architectures and capabilities. The feature values were derived by removing the final convolutional layer from these networks. This removal exposed the activations from intermediate convolutional layers, after which the max pooling technique was applied to produce feature maps of the input images. The resulting feature vectors were then utilized for classification using ML algorithms. This approach involved examining a variety of pre-trained CNN models, each with a unique architecture, to capture a comprehensive range of image features [
35].
Fusing Complex Deep Features
Considering the complex patterns found in breast lesion imagery, a standalone D-CNN may struggle to fully capture the intricate details present in mammographic images. To tackle this complexity, the current study introduces a method that integrates features from a variety of D-CNN models, thereby capturing the diverse patterns found in input dermoscopic images. We systematically explored different combinations of deep features, with a specific focus on those obtained from pre-trained models, including EfficientNetB3, ResNet50, VGG19, ConvNeXtTiny, and DenseNet121 [
36-
40].
The feature extraction process involves removing the fully connected layers from these models and directing the input through their final convolutional layers to global average pooling layers. The dimensionality of the features varies among these models, as shown in
Figure 4.
The aggregation of these distinctive features results in a composite feature vector with a dimensionality of 11,776. Given the dimensionality of the synthesized feature vector, a careful evaluation is performed to identify features that may have limited informative value. Following this assessment, a streamlined feature vector is created, thereby improving the ML classifier’s discriminatory capabilities and overall performance.
Feature Selection and XGBoost Classification
This section covers the strategic process of feature selection followed by the application of the XGBoost classifier for practical classification tasks. A variety of ML classifiers, such as support vector machine, random forest, KNN, XGBoost, and CatBoost, were assessed using a combined feature vector. The XGBoost classifier exhibited the best performance, leading to its selection as the preferred classifier by Chen and Guestrin [
41].
To address the challenge of high-dimensional features, we employed a dimension reduction technique that assigns importance scores to individual features using the intrinsic evaluation mechanism of the XGBoost classifier. We then selected the top K-ranked features based on the validation accuracy achieved during classifier training. Subsequently, the XGBoost classifier was comprehensively trained using these selected features. In parallel, we constructed a combined feature set for the test set images, which was then used to evaluate the classifier's real-world performance.
Results and Discussion
The performance of various classifiers was evaluated using a dataset containing 64×64 images [
42]. This assessment utilized different performance metrics, as presented in
Table 1.
The feature extraction process involved experimenting with various combinations of pre-trained models. Different sets of features, derived from these model combinations, were used to train an XGBoost classifier. The performance of each set was assessed based on validation accuracy.
Table 2 displays the combinations of pre-trained CNN models along with the number of concatenated features.
Table 3 presents the values of several evaluation metrics—accuracy, precision, recall, and F1-score—used to assess the performance of feature extraction from proposed combinations of pre-trained CNN models and classification using ML classifiers.
The confusion matrix shown in
Figure 5 demonstrates the performance of a model that utilized feature extraction. This model combined ResNet50, EfficientNetB3, and ConvNeXtTiny, and integrates these with an XGBoost classifier.
Examining the values from the provided table, distinct performance trends emerged across various combinations of pre-trained models and classifiers. These variations in performance may have been due to potential overfitting or underfitting within the models, which could have impacted the complexity of the features extracted. Combinations that included Resnet50, EfficientnetB3, and ConvNeXtTiny appeared to capture more intricate patterns within the data, resulting in higher accuracy, precision, recall, and F1-score metrics. In contrast, combinations involving Densenet121 and VGG19 seemed to struggle with representing the necessary complexity within the dataset, adversely affecting the classifiers' predictive capabilities. The discrepancies observed in feature extraction and the alignment of complexity with classifier requirements might explain the varying performance metrics observed across the combinations of pre-trained models and classifiers. Improving the models’ architectures and refining feature engineering techniques could potentially improve the representativeness and discriminative power of the extracted features, thereby boosting overall predictive capabilities.
Comparison with State-of-the-Art Algorithms
This section compares the proposed ensemble technique, which classifies breast lesions in mammography images as benign or malignant, with existing state-of-the-art algorithms.
Table 4 demonstrates that the proposed model surpasses existing techniques in classifying breast lesions. The use of transfer learning with pre-trained D-CNN, ResNet50, and Nasnet-Mobile resulted in low accuracy, primarily due to data mismatches and limited fine-tuning on a scarce training dataset. The subpar performance of MiNuGAN with cGAN and focal loss can be attributed to the introduction of noise by GANs, complicating the discrimination between real and synthetic images. ResNet-SCDA-50, enhanced with SCDA data augmentation, achieved an accuracy of 86.3% due to dataset variations caused by the data augmentation. In another approach that involved feature extraction through CNNs and ViT, coupled with transfer learning through BERT, the lower accuracy resulted from the combination of different neural networks, which led to suboptimal feature integration. The algorithm that utilized metaheuristics for classification was significantly affected by the choice of parameters, accounting for its lower accuracy. Finally, the proposed model attained an accuracy of 89% as it extracted intricate features from the input images using D-CNN models and then leveraged an ML classifier for effective decision boundary modeling.
Key Findings of the Research Work
Consistent high performers
Models such as Resnet50, EfficientnetB3, and ConvNeXtTiny consistently excelled in producing superior results across various classifiers. Their exceptional ability to effectively extract and represent complex data patterns is likely the key to their success.
Among these combinations, Resnet50+EfficientnetB3+ConvNeXtTiny stood out with its impressive accuracy of 0.89, precision of 0.86, recall of 0.86, and F1-score of 0.86. These results demonstrate the model's robust predictive capabilities.
Underperforming combinations
Pairings involving Densenet121 or VGG19 tended to underperform, showing lower metrics across classifiers. This suggests that these models may have limitations in extracting highly representative and discriminative features, potentially leading to decreased predictive power.
One notable example is the impressive results achieved by combining Resnet50, EfficientnetB3, and Densenet121 with XGBoost. This powerful combination resulted in an accuracy of 0.79, precision of 0.74, recall of 0.71, and an F1-score of 0.72.
Influence of the selection of pre-trained models
The choice of pre-trained models had a profound impact on the success of feature extraction and directly affected the overall predictive ability of classifiers.
