External validation of the Brain Tumour Reporting and Data System (BT-RADS) in the multidisciplinary management of post-treatment gliomas

Introduction

Gliomas constitute a group of heterogenous malignant primary brain tumours, with a median survival duration that spans from 7 years for low-grade gliomas to 18 months for high-grade gliomas, such as glioblastoma [1,2]. Standard therapy involves gross total/near total resection, conformal radiotherapy and alkylating agents like temozolomide (TMZ) [3].
Magnetic resonance (MR) imaging has been instrumental in the surveillance of post-treatment gliomas; however, the complex clinical course, varied patterns of presentation, overlapping features of tumour progression and treatment effect, and the heterogeneous nature of each case present challenges in the interpretation and reporting of the findings [4]. Several criteria for response assessment in gliomas have been proposed, including Macdonald and Response Assessment in Neuro-Oncology criteria [5-7]. These systems have limited utility in clinical practice due to their complexity, lack of objectivity, and time-intensive nature. In order to address this clinical gap, a structured reporting system, the Brain Tumour Reporting and Data System (BT-RADS), was proposed by the Emory neuroradiology group, wherein MRI studies are interpreted and reported systematically to provide objective clarity to the referring clinicians in line with the current standard management recommendations [8]. Our objective was to independently and externally validate the BT-RADS scoring system, and evaluate its interobserver variability and accuracy in directing management. We also attempted to investigate whether the BT-RADS score has any predictive value for survival outcomes.

Material and methods

Patient selection and interpretation

This retrospective study was approved by our institutional review board (project no. 900621), which waived the requirement for written informed consent. We included post-treatment glioma patients who underwent contrast-enhanced MRI between November 2018 and June 2019. We curated the following information from the patient’s chart in electronic medical records (EMR) and the Picture Archiving and Communication System (PACS): patient demographics, diagnosis and follow-up details, treatment history, and imaging parameters for two consecutive MRIs. Two-year follow-up data were collected.

MRI protocol

All patients underwent MR brain examination using dedicated phased-array head coils on one of the following scanners: Ingenia, Philips (1.5T), Signa, or GE (1.5T and 3T). A standardised MR brain tumour protocol was followed, and the pulse sequences that were used are summarised in Table 1.

Image analyses

Image interpretation was performed independently by two neuro-radiologists (RD1 with 10 years and RD2 with 4 years of neuro-oncologic imaging experience) on a dedicated clinical workstation using picture archiving and communication systems (PACS). The readers were aware that all patients were post-treatment glioma cases but were blinded to all other outcomes. The readers assigned a BT-RADS score to each patient after assessing two consecutive post-treatment MRIs.

Eligibility criteria and scoring system

We included all postoperative glioma patients undergoing adjuvant therapy (radiation therapy ± TMZ) and excluded paediatric (≤ 18 years) brain tumours and patients without histological diagnosis as per electronic medical records. The study design is depicted in the graphic abstract shown in Figure 1. The adapted scoring lexicon system and its definitions are provided under supplementary Tables S1 and S2. Any discordance between the two assigned scores was then resolved through mutual discussion, and a final “consensus” score was assigned. The consensus score was considered as the ground truth for all analysis. Each of these scores was then correlated with the multidisciplinary meeting (MDM), and agreement statistics were derived. We defined “clinically significant” observer variability in cases where there was a substantial difference in the management guidelines linked to the scores assigned by the 2 readers separately for the same patient. We grouped categories that indicate tumour progression (3b, 3c, and 4) and the remaining categories (1-3a) for assessment of survival outcomes.

Statistical analysis

Data were analysed using IBM SPSS v25. Interobserver agreement was categorised as small for a kappa (κ) value of 0.01-0.20, fair for a κ value of 0.21-0.40, moderate for a κ value of 0.41-0.60, substantial for a κ value of 0.61-0.80, and almost perfect for a κ value of 0.81-1.00 [9]. The overall survival among the BT-RADS scoring categories was assessed using Kaplan-Meier analysis. Death was considered an event, and the time until death/last follow-up was calculated from the date of the subsequent MRI used for assigning scores within the study duration.

Results

Demographics

Table 2 depicts the basic demographic details of our study cohort. The observed frequencies of the scores, listed in descending order, were as follows: BT-RADS 2 in 67 cases, BT-RADS 1 in 9 cases, BT-RADS 3c in 8 cases, BT-RADS 3a in 6 cases, BT-RADS 4 in 5 cases, BT-RADS 0 in 4 cases, and one case of BT-RADS 3b.

Agreement between radiologists and the multidisciplinary meeting decision

Kappa statistics showed low interobserver variability in BT-RADS scores, both between RD1 and RD2 and between each radiologist and the consensus. The overall agreement rate between RD1 and RD2 was 62.7%, with a κ of 0.67. The agreement rate between RD1 and the consensus was 83.3%, with a κ value of 0.85, while the agreement between RD2 and the consensus was 69.3%, with a κ value of 0.79 (Table 3). Among the radiologists, the highest agreement was observed for score 2, and no concordance was observed for score 3b (n = 1) between consensus and both RD1 and RD2. When comparing individual reader findings with the consensus score, there was no clinically significant difference in the assigned scores between RD1 and the consensus. However, RD1 and RD2 assigned differing scores to 3 patients, which was clinically significant (Table 4). As illustrated in Figures 3-5, these 3 patients probably represent true interobserver variability, which can be attributed to the complex nature of post-treatment glioma imaging. The agreement between Joint Clinic (JC) management decision and BT-RADS-linked management recommendation for each score (consensus BT-RADS score) was 97.9% (in 94 out of 96 cases, because 4 cases were scored 0), with only 2.1% showing disagreement.

Correlation of BT-RADS scoring with overall survival (OS)

Patients were followed up for a median duration of 13.5 (12.2-16.4) months. Categories indicative of tumour progression (3b, 3c, and 4) were grouped together for comparison with the remaining categories (1-3a).
Overall, the one-year survival rate was 87.1% (95% CI: 80.5-94.2) including all patients under surveillance following recent MRI (p < 0.001). The probability of 12-month survival with a score ≤ 3a was 94.8% (95% CI: 89.9-99.8), whereas for scores 3b or higher it was only 31.5% (95% CI: 4.0-59.0). This disparity in survival rates was statistically significant (p < 0.001) (Table 5 and Figure 2).

Discussion

This study is the first in literature to externally and independently validate the use of the BT-RADS scoring system in response assessment of post-treatment gliomas. In our evaluation, a substantial agreement was observed between the two reporting radiologists, yielding a  value of 62.9% ( = 0.67) for RD1 and RD2. Furthermore, there was a robust 97.9% consensus between the recommendations from MDM and BT-RADS score-linked management recommendation. Similar results were reported by Cooper et al.: an overall agreement of 82.2% was reported between radiologist 1 and radiologist 2, and 79.0% between the initial review and the consensus of the tumour board [10].
Interobserver variability was lowest for scores ≤ 3a, while it was highest for score 3b, followed by 3c. It again dropped for score 4, indicating low variability. This observation highlights that the BT-RADS scoring system functioned well at the extreme scores but with relative ambiguity at the mid-range score, i.e., score 3; assigning this score was intuitive and based on experience, considering the heterogenous nature of the post-treatment imaging and difficulties in differentiating the subsets of pseudoprogression from true progression. This suggests that the scoring system may not be completely objective and may require further refinement or training to improve consistency. The concordance between JC management decisions and BT-RADS-linked management recommendations for each score was 97.9%, indicating a high level of accuracy in this scoring system and underscoring its potential for clinical translation. As a subset of our secondary objective, we calculated the predictive value of BT-RADS. Patients with score ≤ 3a exhibited an expected one-year overall survival probability of 94.8 (95% CI: 89.9-99.8); in contrast, those with scores ≥ 3b had a one-year OS probability of only 31.5 (95% CI: 4.0-59.0). This predictive value brings these management recommendations more in line with other structured reporting systems, like BIRADS or NIRADS.
We encountered certain issues during implementation of the scoring system. Firstly, we faced ambiguity when comparing immediate post-operative MRI as the baseline to any post-CRT (chemoradiotherapy) scan, in which there was resolution of haemorrhagic changes in the resection cavity, subdural/extradural haematoma, and decrease in oedema or perioperative ischaemic changes. These changes tended to settle down with time, as seen on the 6-week post-CRT scan, leading to an “apparent” improvement in imaging findings, even though there may not have been any change in the tumour burden per se. The guidelines in the BT-RADS standard scoring template are unclear about improving imaging findings due to a decrease in the post-surgical findings, and hence the scores of either 2 or 1a were interchangeably assigned by each radiologist. Secondly, a potential limitation was the omission of advanced imaging techniques such as perfusion imaging and spectroscopy [12,13]. These techniques have been proven to offer additional insights as adjunctive tools to differentiate pseudoprogression versus true progression. Finally, time since radiotherapy (90 days) was the only parameter taken into consideration when assigning score 3a or 3b irrespective of the type of enhancement. This, in our opinion, was an oversimplification because a new unequivocal solid enhancing lesion must be given the benefit of doubt regarding disease progression (as demonstrated in our patient, Figure 3), even if it occurs within 90 days of radiation treatment. Our study has certain limitations. Firstly, the retrospective nature of the study is inherently susceptible to selection bias, and secondly, we had a limited sample size.

Conclusions

The BT-RADS structured reporting system was inde­pendently and externally validated to have good agreement between reporting radiologists. Despite the overall good agreement, variation rates escalated with worsening findings. The BT-RADS management recommendations for each score also showed near perfect concordance with decisions taken by our multidisciplinary team. There also seemed to be a potential predictive role in overall survival; however, additional data are required for validating the same.

Disclosure

The authors report no conflict of interest.

References