A Predictive Model for Pathologic Complete Response in Breast Cancer Patients Treated with Neoadjuvant Chemotherapy Using Machine Learning ()
1. Introduction
Neoadjuvant Chemotherapy (NAC) has long been used for decreasing the tumor size to either increase operability [1] [2]. In patients with NAC, pathological Complete Response (pCR) has been proposed as a surrogate endpoint for the prediction of long-term clinical benefits, such as Disease-Free Survival (DFS) and Overall Survival (OS) [3] [4]. Especially, there was the strongest association between pCR and long-term outcome in patients with aggressive breast cancer subtypes (triple negative, HER2-positive and hormone-receptor-negative) [5].
In previous reports, various ways were used for the prediction of pCR in breast cancer patients treated with NAC. Magnetic resonance imaging had a predictive value [6] [7] and high Tissue Infiltrating Lymphocyte (TIL) status was an independent factor for prediction [8]. In addition, pathologic factors such as Ki 67 proliferation index [9] or transcripts such as long non-coding RNAs were associated with pCR [10]. Meanwhile, the machine learning method has recently emerged as a new way of a prediction tool for effective and accurate decisions [11].
In this study, we present an easy-to-use prediction tool for pCR using machine learning. We used data from clinical characteristics and gene expression profiles. Gene profiles came from Cancer SCANTM, a targeted sequencing platform designed at Samsung Medical Center [12].
2. Methods
2.1. Study Population
We performed a retrospective chart review of 1308 breast cancer patients who underwent NAC and surgery between August 2008 and June 2017 at Samsung Medical Center in Seoul, Korea. Among them, 377 patients who underwent Cancer SCANTM were included. Cancer SCANTM test was conducted only on patients who agreed to provide genetic information. As part of this study, DNA sequencing results and electronic medical records including pathology reports were reviewed. 139 cases were excluded from analysis due to incomplete medical data and 238 cases were included for analysis. We used additional retrospective data from 72 patients who underwent NAC, surgery and Cancer SCANTM between July 2017 and April 2018 for internal validation. This study adhered to the tenets of the Declaration of Helsinki and was approved by the institutional review board (IRB) of Samsung Medical Center (IRB No. 2018-05-035).
The available data for the cohorts included age at diagnosis, subtype (e.g., Hormone Receptor [HR] positive/Human Epidermal growth factor 2 [HER2] receptor negative, HR positive/HER2 positive, HR negative/HER2 positive, HR negative/ HER2 negative), histopathology (e.g., Invasive Ductal Carcinoma [IDC], Invasive Lobular Carcinoma [ILC], mixed), menopausal status, family history for breast cancer, regimen for NAC (e.g., AC [adriamycin, cyclophosphamide], AC + D/T [docetaxel/taxol], AC + D/T + Herceptin [H], AC + Paclitaxel + Carboplatin, TCHP [docetaxel, carboplatin, trastuzumab, pertuzumab], others), Multiplicity, pathological T-stage, axillary nodal evaluation (clinical N0, axillary fine needle aspiration [FNA] result), results of supraclavicular and internal mammary lymph node (IMLN) FNA, Ki67 status, tumor marker level (carcinoembryonic antigen [CEA], carcinoma antigen15-3 [CA15-3]) and gene profile. We defined pCR as breast and also axillary pCR simultaneously. Breast pCR was defined as no invasive disease (ypT0 or ypTis) on final pathologic results. Axillary pCR was defined as no metastasis (ypN0) or isolated tumor cell on final pathologic results.
2.2. DNA Extraction and Sequencing
Genomic DNA (250 ng) from each tissue was sheared in a Covaris S220 Ultrasonicator (Covaris, Woburn, MA) and used with CancerSCAN™ probes and a Sure Select XT reagent kit HSQ (AgilentTechnologies) for construction of a library according to the manufacturer’s protocol [12].
This panel is designed to enrich exons of 81 genes, covering 366.2 kb of the human genome. After enriched exome libraries were multiplexed, the libraries were sequenced on a HiSeq 2500 sequencing platform (Illumina). Briefly, a paired-end DNA sequencing library was prepared through gDNA shearing, end-repair, A-tailing, paired-end adaptor ligation, and amplification. After hybridization of the library with bait sequences for 27 hours, the captured library was purified and amplified with an index barcode tag, and library quality and quantity were assessed [12]. We defined mutation as single nucleotide variants or copy number variation or translocation.
2.3. Statistical Analysis
Variables were compared between pCR (−) and pCR (+) groups using chi-squared test or Fisher’s exact test, while mean age was compared between the two groups via Mann-Whitney U tests with SAS version 9.4 (SAS Institute, Cary, NC, USA). Receiver Operating Characteristic (ROC) curves and Areas Under the ROC Curve (AUC) were calculated. All tests were two-sided and a p-value of <0.05 was considered statistically significant.
2.4. Machine Learning
Azure Machine Learning (Azure ML; Microsoft, Redmond, WA, USA) is a cloud service that enables the execution of machine learning processes. The Azure Machine Learning Studio (Microsoft, Redmond, WA, USA) is also available as a workspace to help users build and test predictive models [13]. We built a supervised machine learning classification model using the Azure ML platform. This was accomplished using the steps of: 1) edit the data; 2) split the data; 3) train the model; 4) score the model; and 5) evaluate the model (Figure 1). We split the modeling data (238 cases) into training and testing sets using a randomized 60 - 40 split. We then trained our training set using a Two-class Bayes point machine method [14] for the prediction of pCR.
3. Results
3.1. Patient Characteristics
The clinicopathologic characteristics of included patients are summarized in Table 1. The pCR (−) group had 200 patients and the pCR (+) group had 38 patients. The median age was older in pCR (+) group (p-value = 0.038) and pCR (−) group had more premenopausal patients than pCR (+) group (p-value = 0.045). IDC, AC/AC + Taxane regimen and triple negative breast cancer (HR-/HER2-) subtype were majority in both groups. There was no difference in both groups according to family history, subtype, multiplicity, T stage, axillary nodal status, Ki-67 and tumor marker status. In gene profile results, only BRCA2 mutation was associated with pCR (+) status statistically (p-value = 0.014). Patients with BRCA2 mutation were more in pCR (−) group (36.5%) than pCR (+) group (15.8%). We developed a predictive model with 238 cases using the Azure ML platform (Figure 1) using various classification algorithms, such as Two-class Decision Forest, Two-class Decision Jungle, Two-class Decision Forest, Two-class Support Vector Machine, and Two-class Neural Network. Among them, Two-class Bayes Point Machine was the most suitable method for prediction of pCR. We assessed Area Under the Curve (AUC). The AUC of the Receiver Operating Characteristic (ROC) curve was 0.909 and accuracy was 0.875 (Figure 2(a)). In addition, we developed a predictive model without gene profiles. We used only clinical data but patients pool (n = 238) and process were same with previous model. Through additional model, the AUC of ROC curve was 0.743 and accuracy was 0.800 (Figure 2(b)).
3.2. Predictive Model
We developed a predictive model with 238 cases using the Azure ML platform (Figure 1) using various classification algorithms, such as Two-class Decision Forest, Two-class Decision Jungle, Two-class Decision Forest, Two-class Support Vector Machine, and Two-class Neural Network. Among them, Two-class Bayes Point Machine was the most suitable method for prediction of pCR. We assessed Area Under the Curve (AUC). The AUC of the receiver operating characteristic (ROC) curve was 0.909 and accuracy was 0.875 (Figure 2(a)). In addition, we developed a predictive model without gene profiles. We used only clinical data but patients pool (n = 238) and process were same with previous model. Through additional model, the AUC of ROC curve was 0.743 and accuracy was 0.800 (Figure 2(b)).
3.3. Validation
We also conducted internal validation using 72 patients who underwent NAC and Cancer SCANTM during July 2017 and April 2018. When we applied a 0.4 threshold value, accuracy was 0.806 in predictive model with gene profiles and 0.778 in model without gene profiles respectively (Table 2). As threshold value was decreased, sensitivity was increased but specificity was decreased.
3.4. Clinical Application
The Azure ML platform provides a function for the set-up of web services: (http://docs.microsoft.com/en-us/azure/machine-learning/studio/consume-web-services).
![]()
Table 1. The baseline characteristics of enrolled patients.
Abbreviation: FHx, family history; AC, adriamycin + cyclophosphamide; TCHP, docetaxel + carboplatin + trastuzumab + pertuzumab; IDC, invasive ductal carcinoma; ILC, invasive lobular carcinoma; HR, hormone receptor; HER2, human epidermal growth factor receptor 2; FNA, fine needle aspiration; SCN, supraclavicular lymph node; IMLN, internal mammary lymph node; CEA, carcino embryonic antigen; CA15-3, carcinoma antigen 15-3. *HER2− means HER2 1+ or 2+ with SISH negative, #HER2+ means HER2 3+ or 2+ with SISH positive.
![]()
Table 2. The predictive results of validation with 72 patients.
Sensitivity = true positive/true positive + false negative; specificity = true negative/ false positive + true negative; Precision, positive predictive value = true positive/ true positive + false positive; accuracy = true positive+ true negative/total.
![]()
Figure 1. The workflow of modeling using Azure ML (Microsoft, Redmond, WA, USA). It consists of establishing a dataset, editing the metadata, employing an algorithm (e.g., Two-Class Bayes Point Machine), splitting the data, training the model, scoring the model, and evaluating the model.
After using the Azure ML predictive model as a web service, we used a Representational State Transfer application programming interface to send data and obtain predictions in real-time. For example, when we input data according to each variable excluding the final value “pCR,” an external application communicated with a machine learning workflow scoring model in real-time, enabling the predicted value to be calculated in only a few seconds (Figure 3).
4. Discussion
The prediction of pCR in breast cancer patients treated with NAC is important in terms of management. The scope of surgery could vary depending on whether pCR or not and it is possible to consider novel NAC in the case of non-responder.
In previous study, various methods were used for prediction of pCR in patients treated with NAC. One of them was prediction using breast MRI. Weber et al. studied predictive value of MRI before and after NAC in128 patients [7]. MRI had a positive predictive value of 63.4% and negative predictive value of 84.1% for in-breast pCR. Moreover, Positive predictive value of axillary pCR was 65.6% and negative predictive value was 66.7% [7]. Lee et al. conducted retrospective study in 74 patients treated with NAC and underwent breast MRI before NAC [6]. They showed that perfusion parameters of tumor, background parenchyma of contralateral breast and their combination in pretreatment breast MRI allow early prediction for pCR of breast cancer. The highest predictive power
![]()
Figure 2. (a) The Receiver Operating Characteristic (ROC) curve of our predicted model with gene data. The Area Under the Curve (AUC) of ROC curve was 0.909; (b) The receiver operating characteristic (ROC) curve of our predicted model without gene data. The area under the curve (AUC) of ROC curve was 0.743.
![]()
Figure 3. An illustration of web service usage for our predictive tool. For example, when we input data according to each variable excluding the final value “pCR”, the predicted value to be calculated in only a few seconds. The meaning of each variable was shown in Supplementary Table A1.
for pCR was 0.807 of AUC (p-value = 0.002) [6].
Ki67 was also predictive value for pCR in previous studies [9] [15] [16]. Cabrera et al. showed that no reduction of Ki67 significantly increased the hazard ratio of recurrence and death by 3.39 (95% confidence interval [CI] 1.8 - 6.37) in OS and 7.03 in DFS (95%CI 2.6 - 18.7) [9]. Brown et al. conducted scoring of Ki67 expression for prediction of response to NAC and showed that both the average and maximum score was directly correlated to pCR (average p-value = 0.0002; maximum p-value = 0.0011) [16].
In addition, Tumor infiltrated lymphocytes (TIL) was associated with pCR in patients treated with NAC [8] [17] [18]. For example, Denkert et al. investigated intratumoral lymphocytes in a total 1058 pretherapeutic breast cancer biopsy from two NAC study [17]. Results showed that the percentage of intratumoral lymphocytes was a significant independent parameter for pCR (training cohort;p-value = 0.012; validation cohort p-value = 0.001) [17].
Our data showed that menopause and BRCA2 mutation were associated with pCR. The pCR (+) group had less premenopausal patients (p-value = 0.045) and also less BRCA2 mutation case (p-value = 0.014) than pCR (−) group. There were some study about association between BRCA mutation and pCR [19] [20]. Minckwitz et al. revealed that BRCA mutation was predictor for higher pCR rates after NAC (anthracycline/taxane based) in TNBC [19]. According to Arun et al., BRCA1 status was independently associated with higher pCR rates [20]. Among 317 patients who underwent BRCA testing and NAC, 26 of 57 (46%) BRCA1 carriers achieved pCR, compared with 3 of 23 (13%) BRCA2 carrier and 53 of 237 (22%) BRCA non-carriers (p-value < 0.001) [20]. However, the association between menopause and pCR was not confirmed in the previous studies and our study alone cannot sufficiently explain the relationship between menopause and pCR.
We included DCIS in pCR definition. In Mazouni’s study, residual DCIS in patients treated NAC does not adversely affect survival or local recurrence rate therefore inclusion of patients with residual DCIS in the definition of pCR is justified [21]. And also, the definition for pCR has been not standardized in clinical trials [22] [23].
Among previous articles, there were some studies revealed that it is difficult to reflect pCR with only clinical variables. Bear et al. insisted that there is no clinically useful molecular predictor of response to any cytotoxic drug used in the treatment of breast cancer [24]. Hortobagyi et al. also reveal that clinical parameters such as tumor size, estrogen or HER-2 receptor status, histologic or nuclear grade, or the expression of single molecular markers (i.e., Bcl-2, p53, MDR-1, and so on) show weak association with response and are not regimen-specific, which limits their utility in selecting chemotherapy treatment [25]. Our study contains data from gene profiles and it was our one of advantage. There were other studies about prediction of pCR with gene data in patients treated with NAC. Wang et al. used lncRNA signature to predict pCR rate [10] and Ayeret al. selected a 74-gene k-NN model for predictors of pCR to T/FAC neoadjuvant therapy [26]. Overall, a 78% (14 of 18) predictive accuracy was observed, with a 100% (three of three) positive predictive value for pCR, a 73% (11 of 15) negative predictive value, a sensitivity of 43% (three of seven), and a specificity of 100% (11 of 11) [26].
Our study is valuable as the analysis contained not only clinical findings, but also gene profiles, and was developed with machine learning. After deploying the Azure ML predictive model as a web service, we used a Representational State Transfer application programming interface to send data and obtain predictions in real-time. Meanwhile, variable factors were measured according to the official international standard but there could be minimal differences among centers. Our predictive model can incorporate data from other centers and still provide proper results for each center, so any disparity among centers or hospitals could be diminished. In addition, our predictive model showed reliable result. The accuracy was 0.875 in modeling and 0.810 in validation group. Through additional predictive model without gene profiles, the accuracy was 0.806 in modeling and 0.778 in validation group. If the center is not able to use gene data, you had better use the second model. Moreover, we made a prediction model using only the variables that can be obtained before NAC. Our model did not require data during NAC therefore it is more consistent with the meaning of prediction before NAC.
Our study has several limitations. First, the number of patients enrolled in this study was relatively small. Because we used data from patients underwent NAC, surgery but also Cancer SCANTM. If we did not use gene data, we would have enrolled more than one thousand patients treated with NAC followed by surgery. Second, only internal validation was performed. To increase study reliability, an analysis with a larger number of patients and external validation regardless of race is needed.
5. Conclusion
Our predictive model presented a useful and easy-to-access tool for the prediction of pCR in breast cancer patients treated with NAC. After additional evaluation with a larger patient group and external validation, our model could be more widely used.
Acknowledgements
This study was presented for oral presentation session in Global Breast Cancer Conference 2019.
Supplementary Table A1. Variables for Clinical Application