Predicting Response to Neoadjuvant Therapy in Breast Cancer Using Deep Learning on Primary Core Needle Biopsy Slides

doi:10.12114/j.issn.1007-9572.2023.0512

Abstract

Abstract:

Background

Preoperative neoadjuvant therapy (NAT) is a standardized treatment for locally advanced breast cancer. However, only a portion of patients are sensitive to NAT, hence it is very important to predict the treatment efficacy before NAT. Previous studies have used statistical methods combined with clinical data or deep learning methods combined with medical imaging to predict the efficacy of NAT in breast cancer, but without good results.

Objective

A deep learning model based on core-needle biopsy whole slide images (WSI) of breast cancer (DL-CNB) was trained using the multiple instance learning (MIL) method to predict pathological complete response (pCR) and visualize related tumor areas.

Methods

A retrospective study was conducted to collect the clinical data and biopsy hematoxylin-eosin (HE) stained slides of breast cancer patients who received NAT in Beijing Chaoyang Hospital from April 2019 to April 2022. A total of 195 patients were selected according to the inclusion and exclusion criteria. Patients were divided into pCR group (MP=5, n=40) and non-pCR group (MP=1-4, n=155) according to Miller-Payne (MP) grading. The clinical data were analyzed and the Logistic regression model of pCR influencing factors was constructed. All WSI images were randomly divided into training set and test set in a ratio of 4∶1, and 25% of the data from the training set was taken as verification set. All tumor cell regions in each WSI were labeled, and the training set was prepared by sliding window extraction, data screening, data enhancement, and normalization. Compared with five convolutional neural network models, the optimal model was selected as the feature extractor of DL-CNB. Parameters were set to train the DL-CNB model. The predictive value of DL-CNB was evaluated by using independent test set. To realize the visualization of the important regions related to prediction in the WSI, heat map was drawn according to the weights obtained by the attention-based module.

Results

The proportion of patients with high histological grade, ER negative, PR negative, HER2 positive and Ki-67 high expression in pCR group was higher than that in non-pCR group, and the difference was statistically significant (P<0.05). Compared with the HR+/HER2-, HR-/HER2+ (OR=10.189, 95%CI= 3.225-32.187) and HR+/HER2+ (OR=3.349, 95%CI=1.152-9.737) predicted patients' achie pCR (P<0.05). The AUC of the logistic regressmodel is 0.769, with an accuracy of 81.000%. The AUC of DL-CNB model in the independent test set was 0.914, and the accuracy was 84.211%. Pieces of tumor region labeled non-pCR and pCR in the independent test set were randomly selected for visual display.

Conclusion

The DL-CNB model enables the prediction of pCR in neoadjuvant therapy and visualization of important regions by WSI of breast cancer biopsies. The prediction results are better than the clinical data Logistic regression method. Therefore, we can provide clinical decision-making reference for breast cancer patients who meet the indications of NAT, and assist the realization of individualized precision treatment, which is of great significance to improve the quality of life and survival expectancy for patients.

Key words: Breast cancer, Neoadjuvant therapy for breast cancer, Biopsy pathological WSI, Deep learning model, Multiple instance learning algorithm, Precision therapy

摘要：

背景

术前新辅助治疗（NAT）是治疗局部晚期乳腺癌的标准化手段，但只有部分患者对NAT敏感，在NAT前对患者进行疗效预测至关重要。既往研究利用统计学方法结合临床数据或深度学习方法结合影像学图像预测乳腺癌NAT疗效，效果欠佳。

目的

利用多示例学习（MIL）方法训练基于乳腺癌粗针穿刺全切片图像（WSI）的深度学习（DL-CNB）模型，实现对病理性完全缓解（pCR）的预测和相关肿瘤区域的可视化。

方法

采用回顾性研究模式，收集北京朝阳医院2019年4月—2022年4月收治的经NAT的乳腺癌患者的临床资料和NAT前穿刺苏木精-伊红（HE）染色切片。依据纳排标准共筛选出195例患者。根据Miller-Payne（MP）分级将患者分为pCR组（MP=5级，n=40）和non-pCR组（MP=1~4级，n=155）。首先对临床资料进行分析，构建pCR影响因素的Logistic回归模型。将所有WSI图像按照4∶1的比例随机划分为训练集和测试集，并从训练集中取出25%的数据作为验证集。标记每张WSI中全部肿瘤细胞区域，通过滑动窗口取块、数据筛选、数据增强、归一化处理等步骤准备训练集。对比5种卷积神经网络模型，选择最优模型作为DL-CNB的特征提取器。设置参数训练DL-CNB模型。利用独立测试集测试模型，评价DL-CNB的预测价值。根据由注意力模块获得的权重绘制热力图，实现WSI中与预测相关重要区域的可视化。

结果

pCR组组织学分级高、ER阴性、PR阴性、HER2阳性、Ki-67高表达的患者占比高于non-pCR组（P<0.05）。与HR+/HER2-相比，HR-/HER2+（OR=10.189，95%CI=3.225~32.187）和HR+/HER2+（OR=3.349，95%CI=1.152~9.737）可测预患者达到pCR状况（P<0.05）。Logistic回归模型的受试者工作特征曲线下面积（AUC）为0.769，准确率为81.000%。DL-CNB模型独立测试集AUC为0.914，准确率为84.211%。随机选取独立测试集中某张标签为non-pCR和某张标签为pCR的WSI肿瘤区域进行可视化展示。

结论

DL-CNB模型实现了通过乳腺癌穿刺WSI对新辅助治疗pCR的预测和重要区域的可视化，其预测结果优于临床数据预测模型。由此，本研究能够为符合NAT适应证的乳腺癌患者提供临床决策参考，辅助实现个体化精准治疗，对改善患者生活质量及生存预期具有重大意义。

关键词: 乳腺肿瘤, 乳腺癌新辅助治疗, 穿刺病理全切片图像, 深度学习模型, 多示例学习算法, 精准治疗

CLC Number:

R 737.9

LUO Yunzhao,JIANG Hongchuan,XU Feng. Predicting Response to Neoadjuvant Therapy in Breast Cancer Using Deep Learning on Primary Core Needle Biopsy Slides[J]. Chinese General Practice, 2025, 28(19): 2407-2413. DOI: 10.12114/j.issn.1007-9572.2023.0512.
罗云昭,蒋宏传,徐峰. 基于深度学习模型辅助穿刺病理图像预测乳腺癌新辅助治疗疗效的研究[J]. 中国全科医学, 2025, 28(19): 2407-2413. DOI: 10.12114/j.issn.1007-9572.2023.0512.

Figures/Tables 6

References 26

[1]	GRADISHAR W J, MORAN M S, ABRAHAM J, et al. Breast cancer，version 3.2022，NCCN clinical practice guidelines in oncology[J]. J Natl Compr Canc Netw，2022，20(6)：691-722. DOI：10.6004/jnccn.2022.0030.
[2]	SPRING L M, FELL G, ARFE A, et al. Pathologic complete response after neoadjuvant chemotherapy and impact on breast cancer recurrence and survival：a comprehensive Meta-analysis[J]. Clin Cancer Res，2020，26(12)：2838-2848.
[3]	AGARWAL R, UNNIKRISHNAN U G, KEECHILAT P, et al. Pathological complete response in locally advanced breast cancer after neoadjuvant chemotherapy：survival outcome and its relevance as a surrogate end point[J]. South Asian Journal of Cancer，2020，9(3)：136-140. DOI：10.1055/s-0040-1721238.
[4]	TALEGHAMAR H, ALI JALALIFAR S, CZARNOTA G J, et al. Deep learning of quantitative ultrasound multi-parametric images at pre-treatment to predict breast cancer response to chemotherapy[J]. Sci Rep，2022，12(1)：2244.
[5]	MASSAFRA R, COMES M C, BOVE S, et al. Robustness evaluation of a deep learning model on sagittal and axial breast DCE-MRIs to predict pathological complete response to neoadjuvant chemotherapy[J]. J Pers Med，2022，12(6)：953.
[6]	DAMMU H, REN T, DUONG T Q. Deep learning prediction of pathological complete response，residual cancer burden，and progression-free survival in breast cancer patients[J]. PLoS One，2023，18(1)：e0280148. DOI：10.1371/journal.pone.0280148.
[7]	SKARPING I, LARSSON M, FöRNVIK D. Analysis of mammograms using artificial intelligence to predict response to neoadjuvant chemotherapy in breast cancer patients：proof of concept[J]. Eur Radiol，2022，32(5)：3131-3141.
[8]	CAMPANELLA G, HANNA M G, GENESLAW L, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images[J]. Nat Med，2019，25(8)：1301-1309. DOI：10.1038/s41591-019-0508-1.
[9]	MERCAN E, MEHTA S, BARTLETT J, et al. Assessment of machine learning of breast pathology structures for automated differentiation of breast cancer and high-risk proliferative lesions[J]. JAMA Network Open，2019，2(8)：e198777.
[10]	WOERL A C, ECKSTEIN M, GEIGER J, et al. Deep learning predicts molecular subtype of muscle-invasive bladder cancer from conventional histopathological slides[J]. Eur Urol，2020，78(2)：256-264. DOI：10.1016/j.eururo.2020.04.023.
[11]	SKREDE O J, DE RAEDT S, KLEPPE A, et al. Deep learning for prediction of colorectal cancer outcome：a discovery and validation study[J]. Lancet，2020，395(10221)：350-360.
[12]	ZHAO K, LI Z H, YAO S, et al. Artificial intelligence quantified tumour-stroma ratio is an independent predictor for overall survival in resectable colorectal cancer[J]. EBioMedicine，2020，61：103054. DOI：10.1016/j.ebiom.2020.103054.
[13]	LUCAS M, JANSEN I, VAN LEEUWEN T G, et al. Deep learning-based recurrence prediction in patients with non-muscle-invasive bladder cancer[J]. Eur Urol Focus，2022，8(1)：165-172. DOI：10.1016/j.euf.2020.12.008.
[14]	LI F L, YANG Y Q, WEI Y N, et al. Deep learning-based predictive biomarker of pathological complete response to neoadjuvant chemotherapy from histological images in breast cancer[J]. J Transl Med，2021，19(1)：348.
[15]	OGSTON K N, MILLER I D, PAYNE S, et al. A new histological grading system to assess response of breast cancers to primary chemotherapy：prognostic significance and survival[J]. Breast（Edinburgh，Scotland），2003，12(5)：320-327.
[16]	ILSE M, TOMCZAK J M, WELLING M. Attention-based deep multiple instance learning[Z]. International conference on machine learning，2018：2127-2136.
[17]	WU Y N. Cross entropy[M]. Computer Vision：A Reference Guide. Boston：Springer International Publishing，2021：225-226.
[18]	THEODORIDIS S. Stochastic Gradient Descent[M]. Machine learning. Oxford：Academic Press，2015：161-231.
[19]	LOSHCHILOV I, HUTTER F. SGDR：Stochastic gradient descent with warm restarts[Z]. International Conference on Learning Representations，2017：1-16.
[20]	GOORTS B, VAN NIJNATTEN T J A, DE MUNCK L, et al. Clinical tumor stage is the most important predictor of pathological complete response rate after neoadjuvant chemotherapy in breast cancer patients[J]. Breast Cancer Res Treat，2017，163(1)：83-91. DOI：10.1007/s10549-017-4155-2.
[21]	XIAO Y, DING J H, MA D C, et al. Predicting pathological complete response in neoadjuvant dual blockade with trastuzumab and pertuzumab in HER2 gene amplified breast cancer[J]. Front Immunol，2022，13：877825. DOI：10.3389/fimmu.2022.877825.
[22]	CHEN P X, WANG C, LU R L, et al. Multivariable models based on baseline imaging features and clinicopathological characteristics to predict breast pathologic response after neoadjuvant chemotherapy in patients with breast cancer[J]. Breast Care（Basel，Switzerland），2022，17(3)：306-315.
[23]	DENKERT C, VON MINCKWITZ G, DARB-ESFAHANI S, et al. Tumour-infiltrating lymphocytes and prognosis in different subtypes of breast cancer：A pooled analysis of 3771 patients treated with neoadjuvant therapy[J]. Lancet Oncol，2018，19(1)：40-50. DOI：10.1016/s1470-2045(17)30904-x.
[24]	HAQUE W, VERMA V, HATCH S, et al. Response rates and pathologic complete response by breast cancer molecular subtype following neoadjuvant chemotherapy[J]. Breast Cancer Res Treat，2018，170(3)：559-567.
[25]	METI N, SAEDNIA K, LAGREE A, et al. Machine learning frameworks to predict neoadjuvant chemotherapy response in breast cancer using clinical and pathological features[J]. JCO Clin Cancer Inform，2021，5：66-80. DOI：10.1200/cci.20.00078.
[26]	徐春燕，谢嘉伟，杨春霞，等. 基于病理穿刺切片组织形态学分析的乳腺癌新辅助化疗疗效预测[J]. 四川大学学报（医学版），2021，52(2)：279-285. DOI：10.12182/20210360505.

组别	例数	年龄		组织学分级		cT分期		cN分期		ER
组别	例数	<50岁	≥50岁	Ⅰ~Ⅱ级	Ⅲ级	1~2期	3~4期	0期	1~2期	阴性	阳性
non-pCR组	155	61（39.4）	94（60.6）	106（68.4）	49（31.6）	138（89.0）	17（11.0）	48（31.0）	107（69.0）	34（21.9）	121（78.1）
pCR组	40	13（32.5）	27（67.5）	19（47.5）	21（52.5）	37（92.5）	3（7.5）	16（40.0）	24（60.0）	22（55.0）	18（45.0）
检验统计量值		0.634		6.028		0.124^a		1.176		16.980
P值		0.426		0.014		0.725		0.278		<0.001
组别	PR		HER2		Ki-67		分子分型
组别	阴性	阳性	阴性	阳性	≤20%	>20%	HR/HER2-	HR-/HER2+	HR+/HER2+	TNBC
non-pCR组	38（24.5）	117（75.5）	110（71.0）	45（29.0）	74（47.7）	81（52.3）	90（58.1）	12（7.7）	33（21.3）	20（12.9）
pCR组	25（62.5）	15（37.5）	16（40.0）	24（60.0）	10（25.0）	30（75.0）	7（17.5）	13（32.5）	11（27.5）	9（22.5）
检验统计量值	20.975		13.336		6.706		28.231
P值	<0.001		<0.001		0.010		<0.001

组别	例数	年龄		组织学分级		cT分期		cN分期		ER
组别	例数	<50岁	≥50岁	Ⅰ~Ⅱ级	Ⅲ级	1~2期	3~4期	0期	1~2期	阴性	阳性
non-pCR组	155	61（39.4）	94（60.6）	106（68.4）	49（31.6）	138（89.0）	17（11.0）	48（31.0）	107（69.0）	34（21.9）	121（78.1）
pCR组	40	13（32.5）	27（67.5）	19（47.5）	21（52.5）	37（92.5）	3（7.5）	16（40.0）	24（60.0）	22（55.0）	18（45.0）
检验统计量值		0.634		6.028		0.124^a		1.176		16.980
P值		0.426		0.014		0.725		0.278		<0.001
组别	PR		HER2		Ki-67		分子分型
组别	阴性	阳性	阴性	阳性	≤20%	>20%	HR/HER2-	HR-/HER2+	HR+/HER2+	TNBC
non-pCR组	38（24.5）	117（75.5）	110（71.0）	45（29.0）	74（47.7）	81（52.3）	90（58.1）	12（7.7）	33（21.3）	20（12.9）
pCR组	25（62.5）	15（37.5）	16（40.0）	24（60.0）	10（25.0）	30（75.0）	7（17.5）	13（32.5）	11（27.5）	9（22.5）
检验统计量值	20.975		13.336		6.706		28.231
P值	<0.001		<0.001		0.010		<0.001

变量	B	SE	Waldχ²值	P值	OR（95%CI）
组织学分级	0.285	0.345	0.683	0.409	1.330（0.676~2.617）
Ki-67	0.013	0.009	1.907	0.167	1.013（0.995~1.031）
分子分型（以H R+/HER2-为参照）	—	—	15.854	0.001	—
H R-/HER2+	2.321	0.587	15.643	<0.001	10.189（3.225~32.187）
H R+/HER2+	1.209	0.545	4.926	0.026	3.349（1.152~9.737）
TNBC	1.158	0.659	3.084	0.079	3.183（0.874~11.592）

变量	B	SE	Waldχ²值	P值	OR（95%CI）
组织学分级	0.285	0.345	0.683	0.409	1.330（0.676~2.617）
Ki-67	0.013	0.009	1.907	0.167	1.013（0.995~1.031）
分子分型（以H R+/HER2-为参照）	—	—	15.854	0.001	—
H R-/HER2+	2.321	0.587	15.643	<0.001	10.189（3.225~32.187）
H R+/HER2+	1.209	0.545	4.926	0.026	3.349（1.152~9.737）
TNBC	1.158	0.659	3.084	0.079	3.183（0.874~11.592）

分组	AUC	95%CI	ACC（%）	SENS（%）	SPEC（%）	PPV（%）	NPV（%）
AlexNet
T	0.918	0.814~0.974	78.947	93.750	73.171	54.692	96.774
V	0.702	0.460~0.883	65.000	50.000	71.429	42.857	76.923
ResNet101
T	0.951	0.859~0.991	80.702	93.750	75.610	60.000	96.875
V	0.833	0.601~0.960	80.000	66.667	85.714	66.667	85.714
DenseNet121
T	0.938	0.840~0.984	89.474	81.250	92.683	81.250	92.683
V	0.833	0.601~0.960	75.000	58.333	82.143	58.333	82.143
Inception-v3
T	0.946	0.888~0.979	90.517	90.000	90.625	66.667	97.753
V	0.777	0.615~0.894	73.333	66.667	76.190	54.545	84.211
VGG19
T	0.931	0.832~0.981	80.702	78.049	87.500	94.118	60.870
V	0.881	0.659~0.981	90.000	92.857	83.333	92.857	83.333
I-T	0.914	0.694~0.993	84.211	85.714	80.000	92.308	66.667