HEAD AND NECK RADIOLOGY / ORIGINAL PAPER
Figure from article: An attention-based...
 
KEYWORDS
TOPICS
ABSTRACT
Purpose:
To develop and evaluate an attention-based multimodal model for improving the prediction of overall survival in patients with glioblastoma by integrating clinical and imaging features.

Material and methods:
Data from 358 glioblastoma patients were obtained from The Cancer Imaging Archive. The dataset included clinical data and three magnetic resonance imaging (MRI) sequences. Three categories of features – clinical, radio­mics, and deep features – were used for survival prediction. Radiomics features were extracted using PyRadiomics, while deep features were extracted from MRI images using a Swin UNEt TRansformers. An attention-based multi­modal predictive model was designed to integrate these three feature sets. The DeepHit framework was used for survival modeling. Model performance was evaluated using the concordance index (C-index) and the Brier score. In addition, accuracy, precision, recall, area under the curve (AUC), and F1-score were calculated to predict mortality risk at different time intervals.

Results:
The proposed model achieved a C-index of 0.91 and a Brier score of 0.14. For identifying mortality risk within 6 months, the model achieved an accuracy of 0.93, precision of 0.87, recall of 0.92, AUC of 0.96, and an F1-score of 0.89. For mortality risk within 12 months, the corresponding values were 0.91, 0.87, 0.85, 0.92, and 0.88, respectively. For mortality risk within 24 months, the model achieved an accuracy of 0.75, precision of 0.71, recall of 0.70, AUC of 0.94, and an F1-score of 0.80.

Conclusions:
The proposed model, leveraging a combined radiomics-based approach, demonstrated the capability to predict overall survival in patients with glioblastoma. However, to ensure generalizability, further validation using multicenter datasets is required.
REFERENCES (30)
1.
Luo C, Yang J, Liu Z, Jing D. Predicting the recurrence and overall survival of patients with glioma based on histopathological images using deep learning. Front Neurol 2023; 14: 1100933. DOI: 10.3389/fneur.2023.1100933.
 
2.
Kesler SR, Harrison RA, Rao V, Dyson H, Petersen M, Prinsloo S. Predicting overall survival in diffuse glioma from the presurgical connectome. Sci Rep 2022; 12: 18783. DOI: 10.1038/s41598-022-22387-7.
 
3.
Lobato-Delgado B, Priego-Torres B, Sanchez-Morillo D. Combining molecular, imaging, and clinical data analysis for predicting cancer prognosis. Cancers (Basel) 2022; 14: 3215. DOI: 10.3390/cancers14133215.
 
4.
Whitfield BT, Huse JT. Classification of adult-type diffuse gliomas: impact of the World Health Organization 2021 update. Brain Pathol 2022; 32: e13062. DOI: 10.1111/bpa.13062.
 
5.
Agravat RR, Raval MS. A survey and analysis on automated glioma brain tumor segmentation and overall patient survival prediction. Arch Comput Methods Eng 2021; 28: 4117-4152. DOI: 10.1007/s11831-021-09559-w.
 
6.
Wen PY, van den Bent M, Youssef G, Cloughesy TF, Ellingson BM, Weller M, et al. RANO 2.0: update to the response assessment in neuro-oncology criteria for high- and low-grade gliomas in adults. J Clin Oncol 2023; 41: 5187-5199.
 
7.
Patel SH, Poisson LM, Brat DJ, Zhou Y, Cooper L, Snuderl M, et al. T2-FLAIR mismatch, an imaging biomarker for IDH and 1p/19q status in lower-grade gliomas: a TCGA/TCIA project. Clin Cancer Res 2017; 23: 6078-6085.
 
8.
Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 2014; 5: 4006. DOI: 10.1038/ncomms5006.
 
9.
Chato L, Latifi S. Machine learning and radiomic features to predict overall survival time for glioblastoma patients. J Pers Med 2021; 11: 1336. DOI: 10.3390/jpm11121336.
 
10.
Gomaa A, Huang Y, Hagag A, Schmitter C, Höfler D, Weissmann T, et al. Comprehensive multimodal deep learning survival prediction enabled by a transformer architecture: a multicenter study in glioblastoma. Neurooncol Adv 2024; 6: vdae122. DOI: 10.1093/noajnl/vdae122.
 
11.
Truhn D, Schrading S, Haarburger C, Schneider H, Merhof D, Kuhl C. Radiomic versus convolutional neural networks analysis for classification of contrast-enhancing lesions at multiparametric breast MRI. Radiology 2019; 290: 290-297.
 
12.
Vale-Silva LA, Rohr K. Long-term cancer survival prediction using multimodal deep learning. Sci Rep 2021; 11: 13505. DOI: 10.1038/s41598-021-92799-4.
 
13.
Babaei Rikan S, Sorayaie Azar A, Naemi A, Bagherzadeh Mohasefi J, Pirnejad H, Wiil UK. Survival prediction of glioblastoma patients using modern deep learning and machine learning techniques. Sci Rep 2024; 14: 2371. DOI: 10.1038/s41598-024-53006-2.
 
14.
Chen RJ, Lu MY, Weng WH, Chen TY, Williamson DF, Manz T. Multimodal co-attention transformer for survival prediction in gigapixel whole slide images. Proceedings of the IEEE/CVF International Conference on Computer Vision 2021; 3995-4005. DOI: 10.1109/ICCV48922.2021.00398.
 
15.
He K, Gan C, Li Z, Rekik I, Yin Z, Ji W, et al. Transformers in medi­cal image analysis. Intell Med 2023; 3: 59-78.
 
16.
Mohammadpour S, Emami H, Rabiei R, Roshanpoor A, Tabatabaeefar M, Faeghi F. Scoping review of image-based overall survival prediction in glioma using machine learning. Pol J Radiol 2025; 90: e571-e581. DOI: 10.5114/pjr/209963.
 
17.
Shortliffe EH, Cimino JJ (eds.). Biomedical informatics: computer applications in health care and biomedicine. 3rd ed. Springer; 2006.
 
18.
Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein KH. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 2021; 18: 203-211.
 
19.
Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, et al. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging 2015; 34: 1993-2024.
 
20.
Pereira S, Pinto A, Alves V, Silva CA. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 2016; 35: 1240-1251.
 
21.
Hatamizadeh A, Nath V, Tang Y, Yang D, Roth HR, Xu D. Swin UNETR: Swin transformers for semantic segmentation of brain tumors in MRI images. Brainlesion 2022; 12962. DOI: 10.1007/978-3-031-08999-2_22.
 
22.
Yan J, Zhang B, Zhang S, Cheng J, Liu X, Wang W, et al. Quantitative MRI-based radiomics for noninvasively predicting molecular subtypes and survival in glioma patients. NPJ Precis Oncol 2021; 5: 72. DOI: 10.1038/s41698-021-00205-z.
 
23.
Lao J, Chen Y, Li ZC, Li Q, Zhang J, Liu J, et al. A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Sci Rep 2017; 7: 10353. DOI: 10.1038/s41598-017-10649-8.
 
24.
Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology 2016; 278: 563-577.
 
25.
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Advances in Neural Information Processing Systems 2017; 30: 5998-6008.
 
26.
Mienye ID, Sun Y, Wang Z. An improved ensemble learning approach for the prediction of heart disease risk. Inform Med Unlocked 2020; 20: 100402. DOI: 10.1016/j.imu.2020.100402.
 
27.
Lynch CM, Abdollahi B, Fuqua JD, De Carlo AR, Bartholomai JA, Balgemann RN, et al. Prediction of lung cancer patient survival via supervised machine learning classification techniques. Int J Med Inform 2017; 108: 1-8.
 
28.
Dagogo-Jack I, Shaw AT. Tumour heterogeneity and resistance to cancer therapies. Nat Rev Clin Oncol 2018; 15: 81-94.
 
29.
Lee C, Zame W, Yoon J, van der Schaar M. DeepHit: a deep learning approach to survival analysis with competing risks. Proceedings of the AAAI Conference on Artificial Intelligence 2018; 32. DOI: 10.1609/aaai.v32i1.11842.
 
30.
Katzman JL, Shaham U, Cloninger A, Bates J, Jiang T, Kluger Y. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Med Res Methodol 2018; 18: 24. DOI: 10.1186/s12874-018-0482-1.
 
ISSN:1899-0967
Journals System - logo
Scroll to top