Inter-observer agreement of the Coronary Artery Disease Reporting and Data System (CAD-RADS^TM) in patients with stable chest pain

Introduction

Coronary artery disease is the leading cause of morbidity and mortality throughout the world. Clinical assessment is important in preliminary assessment of chest pain and in determining whether it is due to stenotic coronary artery disease. The degree of coronary artery stenosis is an important issue for treatment planning, whether by medical therapy or by revascularisation [1-3]. Different imaging modalities are used for the evaluation of the coronary arteries. Echocardiography is used for the examination of patients with chest pain, but it cannot delineate the course of the coronary arteries [4, 5]. The clinical application of coronary magnetic resonance angiography is reduced by the limited temporal and spatial resolution, partial volume artefacts, reliance on multiple breath-holds, and poor visualisation of the entire coronary artery [6, 7]. Invasive coronary angiography is considered as the gold standard technique for evaluation of the coronary arteries. However, it is an invasive and expensive procedure, with some periprocedural morbidity and mortality [1-5].
Coronary computed tomography angiography (CTA) is the most common method for assessment of coronary arteries. Coronary CTA allows for fast and safe triage of patients. This procedure is a relatively fast and simple procedure that does not require arterial access or hospital admission. The images can be post-processed and reconstructed during different cardiac phases to allow retrospective selection of the phases with the fewest motion artefacts [8-12].
Different reporting systems in medical imaging have introduced standardised reporting linked with information guiding subsequent steps in patient management. BI-RADS standardised reporting of screening mammograms allows clinicians to interpret the clinical relevance of imaging findings and to take action. Other reporting systems for evaluating thyroid, prostate, and parotid glands have also been developed [13-16]. The Coronary Artery Disease Reporting and Data System (CAD-RADSTM) is a classification system suggested by the Society of Cardiovascular Computed Tomography, which is used for grading the severity of coronary arteries stenosis. CAD-RADS is a standardised reporting system for coronary CTA results on a per-patient basis. It is applicable to coronary CTA in patients with suspected or known coronary artery disease. The CAD-RADS classification is meant to be complementary to the final impression of the report, particularly because the report will provide specific information regarding the location and extent of coronary plaque and stenosis. It includes suggestions regarding further patient management [17-20].
The aim of this work is to assess the inter-observer variability of CAD-RADS for classifying the degree of coronary artery stenosis in patients with stable chest pain.

Material and methods

Patients

The institutional review board committee approved the study protocol, and all patients provided written, informed consent. The prospective study was conducted upon 102 patients clinically suspected to have coronary artery disease, who were referred for coronary CTA. We excluded six patients from the study: due atrial fibrillation in four patients and impaired renal function in two patients. The final patients included in this study comprised 96 patients (82 male, 14 female, age range 45-74 years, mean age 63 years) who underwent coronary CTA.

Computed tomography angiography

Coronary CTA was performed using a 128-detector CT scanner [Ingenuity Core TM, Philips, Netherlands]. A -blocker (100 mg propranolol) was administrated orally one hour before the examination in patients with heart rate greater than 65 beats per minute, and no adverse reactions to -blocker were recorded. Non-contrast scan was done for calcium scoring to decide if the patient was fit for a contrast scan. A bolus of contrast medium (Ultravist, 370 mg of iodine per millilitre) was injected into an antecubital vein at the rate of 5.5 m/s, followed by a 50-ml saline flush with a dual-barrel power injector. The amount of contrast medium was determined according to the patient’s body weight and scanning time. The bolus tracking technique was used with the region of interest (ROI) positioned in the ascending aorta. After injection of the contrast bolus, monitoring scans at the level of the ascending aorta were obtained and attenuation values within the ROI are measured. Once the attenuation value reached 190 HU, the patient was automatically instructed to perform a breath hold during which the scan was started. Retrospective electrocardiographically gated CTA was performed with collimation of 128 × 0.6 mm, pitch 0.6 mm, tube current 800 mA, and tube voltage 100 kV.

Post processing

The images were archived as Digital Imaging and Communications in Medicine images (DICOM) and transferred to an Ingenuity CoreTM, Philips, Netherlands; 2010 CT apparatus (128 slices) workstation. Axial, cross-sectional, curved planar reformation, multiplanar reformation, and three-dimensional maximum intensity projection (MIP) images were reconstructed.

Image analysis

Analysis of the CTA images was performed by two radiologists (R.M. and A.G.) with five and three years of coronary imaging, respectively, who were blinded to the other clinical findings. They independently reviewed the coronary CTA images and analysed the degree of stenosis of the left main coronary artery, the left circumflex artery, the left anterior descending artery, and the right coronary artery. The CAD-RADS classified results into: CAD-RADS 0 (absence of atherosclerosis), CAD-RADS 1 (1-24% – minimal stenosis or plaque with no stenosis), CAD-RADS 2 (25-49% – mild stenosis), CAD-RADS 3 (50-69% stenosis), CAD-RADS 4 (4A – 70-99% stenosis or 4B – left main more than 50% or 3-vessel obstructive more than70%), and CAD-RADS 5 (the presence of at least one total occlusion). The modifiers of CAD-RADS were: CAD-RADS-N non-diagnostic, CAD-RADS-V for vulnerable plaques, CAD-RAD-G for graft patency, and CAD-RAD-S for stents. Each graft was assessed for course, patency of the ostia, body, and anastomotic segments, and each stent was assessed for its site, patency, and distal filling. The associated coronary anomalies, non-coronary cardiac findings, and extra-cardiac findings were detected. Finally, the standardised reporting in terms of the CAD-RADS reporting and data system was used.

Statistical analysis

The statistical analysis of data was done using SPSS software (Statistical Package for Social Science version 20). The kappa coefficient (K) including 95% confidence interval (CI) with percentage agreement was calculated to estimate the level of agreement between both reviewers for the degree of stenosis of coronary arteries, dominance of the coronary artery, each CAD-RADS category, CAD-RADS modifiers and associated congenital coronary anomalies, and non-coronary cardiac and extracardiac lesions. The k values were interpreted as follows: k values between 0.00 and 0.20 represented poor; k values between 0.21 and 0.40 were interpreted as fair; k values between 0.41 and 0.60 were interpreted as moderate; k values between 0.61 and 0.80 were interpreted as good; and k values between 0.81 and 1.00 were interpreted as excellent. A p value less than 0.05 indicated a statistically significant difference.

Results

Table 1 shows the inter-observer agreement of coronary arteries. Stenosis detected by both reviewers was seen in the left main coronary, the left anterior descending artery, the left circumflex artery, and the right coronary artery with excellent inter-observer agreement of CTA for detection of coronary artery and a percentage agreement 99.7% (k = 0.99, 95% CI = 0.98-1.00, p = 0.001). Table 2 shows the inter-observer agreement of the coronary dominance with a percentage agreement of 94.8% and excellent inter-observer agreement for coronary predominance (k = 0.852, 95% CI = 0.724-0.98, p = 0.001).
Table 3 shows the inter-observer agreement of CAD-RADS. The CAD-RADS of both reviewers were: CAD-RADS 0 (Fig. 1), CAD-RADS 1 (Fig. 2), CAD-RADS 2 (Fig. 3), CAD RADS 3 (Fig. 4), CAD-RADS 4 (Fig. 5), and CAD-RADS 5 (Fig. 6) (n = 16 and 12). There was excellent inter-observer agreement for CAD-RADS 0 (k = 1.0, 95% CI = 1.0-1.0, p = 0.001), CAD-RADS 1 (k = 0.92, 95% CI = 0.80-1.00, p = 0.001), CAD-RADS 3 (k = 0.808, 95% CI = 0.64-0.97, p = 0.001), CAD-RADS 4 (k = 0.826, 95% CI = 0.68-0.97, p = 0.001), and CAD-RADS 5 (k = 0.833, 95% CI = 0.67-0.99, p = 0.001) and good inter-observer agreement for CAD-RADS 2 (k = 0.76, 95% CI = 0.58-0.95, p = 0.001). The percent agreement of both reviewers for CAD-RADS 0 was 100%, for CAD-RADS 1 it was 97.9%, for CAD-RADS 2 it was 93.8%, for CAD-RADS 3 it was 94.8%, for CAD-RADS 4 it was 94.8%, and for CAD-RADS 5 it was 95.8%.
Table 4 shows the inter-observer agreement of modifiers of CAD-RADS. The modifiers of CAD-RADS of both reviewers were modifier N, modifier V (Figure 7), modifier G (Figure 8) and modifier S (Figure 9). There was good inter-observer agreement for the modifier N (k = 0.79, 95% CI = 0.4-1.0, p = 0.001), moderate inter-observer agreement for the modifier V (k = 0.59, 95% CI = 0.24-0.94, p = 0.001), and excellent inter-observer agreement for the modifier G (k = 1.0, 95% CI = 1.0-1.0, p = 0.001) and the modifier S (k = 1.0, 95% CI = 1.0-1.0, p = 0.001). The agreement between both reviewers for the modifier N was 98.9%, for the modifier V was 94.8%, for the modifier G was 100.0%, and for the modifier S was 100.0%.
Table 5 shows the inter-observer agreement of associated coronary anomalies, and non-coronary cardiac and extra-cardiac findings. The associated coronary artery anomalies detected by both reviewers revealed agreement of 94.8% and excellent inter-observer agreement for associated congenital anomalies (k = 0.845, 95% CI = 0.712-0.98, p = 0.001]. The associated non-coronary cardiac findings of both reviewers were myocardial infarction, thrombus (Figure 10), left ventricle aneurysm, and aortic valve calcification with agreement of 95.8% and excellent inter-observer agreement (k = 0.857, 95% CI = 0.72-0.99, p = 0.001). The associated extra-cardiac findings of both reviewers were lipoma, aortic coarctation, and pulmonary nodules with agreement of 91.7% and excellent inter-observer agreement (k = 0.81, 95% CI = 0.684-0.94, p = 0.001].
The suggested further investigations and management of patients with coronary artery disease were determined based on the CAD-RADS results. Patients with CAD-RADS 0 need no further investigation. Patients with CAD-RADS 1 and 2 do not undergo further investigation and are treated with preventive medical therapy. Patients with CAD-RADS 3 are referred for functional assessment with stress echo, and patients with CAD-RADS 4 and 5 are referred for conventional angiography.

Discussion

The present results show excellent inter-observer agreement between both reviewers for the degree of coronary artery stenosis with coronary CTA and the classification of stenosis according to CAD-RADS. There is an excellent inter-observer agreement for CAD-RADS as a standardized reporting system for coronary CTA. The CAD-RADS is reproducible between reviewers and can used by radiologists and clinicians in the future.
In this study, there is excellent inter-observer agreement between both reviewers for the degree of coronary stenosis. Previous studies reported that there is a good correlation between coronary CTA and conventional coronary angiography in the grading of stenoses, with a slight tendency to overestimate the degree of stenosis when using coronary CTA. Current coronary CTA systems are capable of accurately evaluating coronary vessels for the presence of stenosis for vessel diameters as small as about 1 mm [2, 21]. The main determinant of the false positive results for diagnosing more than 50% coronary luminal stenosis is small vessel size, and the diameter of stenotic segments tends to be underestimated by coronary CTA [3, 22].
In this study, there is excellent inter-observer agreement between both reviewers for the CAD-RADS categories as a standard method for reporting coronary CTA. The CAD-RADS categories range from CAD-RADS 0 for a complete absence of stenosis and plaque to CAD-RADS 5 for the presence of at least a totally occluded coronary artery. Categories should reflect the most clinically relevant finding per patient [18-20]. The k values for agreement between fellows are 0.48 (95% CI: 0.42-0.54] at one year. The k value of intra-observer agreement in the reading of conventional coronary angiography is 0.95 (95% CI: 0.92-0.98) for experts with more than 10 years’ experience [23].
The CAD-RADS categories are complemented with modifiers that indicate the presence of stents, grafts, and vulnerable plaque [19]. The presence of vulnerability markers, such as positive remodelling, napkin-ring sign, low CT attenuation, and spotty calcification, are associated with adverse clinical events [24, 25]. CT imaging of coronary stents depends on patient and stent characteristics; however, about 8% of stents are not accessible, mostly due to blooming or motion artefacts. Ideal candidates have stents with a diameter of 3 mm and more [26, 27]. Coronary CTA can detect the patency and course of by-pass grafts [28, 29]. In this study, there is excellent inter-observer agreement for the detection of modifiers of CAD-RADS, including stents, grafts, and vulnerable plaques.
In this study, there is excellent inter-observer agreement of associated coronary anomalies, and non-coronary cardiac and extra-cardiac findings. Coronary CTA has the added ability to depict extracardiac lesions of the chest and scanned upper abdomen. These incidental lesions can often present a challenge to physicians because of the potential benefits and risks of identifying such lesions [30-32].
The main goal of CAD-RADS is to standardise reporting of coronary CTA results and to facilitate the communication among physicians along with suggestions for further investigations and management [17, 18]. Thus, CAD-RADS provides a basis for the clinical use of CAD-RADS for the further investigation, diagnosis, management, and treatment planning of patients with stable chest pain [19, 20]. In this study, patients with CAD-RADS 1 and 2 were treated with preventive medical therapy, patients with CAD-RADS 3 underwent functional assessment, and patients with CAD-RADS 4 and 5 underwent invasive angiography.
The advantages of CAD-RADS are a standardised method of reporting coronary CTA: providing a single, per-patient severity score and providing recommendations for further investigation of the patients [8-10]. Thus, this reporting system provides a common language between radiologists and clinicians to increase the clarity in reporting and recommendation for the management of coronary artery stenosis. The achieved standardisation of reporting will be of benefit in education, research, peer review, and quality assurance and may ultimately result in improved quality of care [11, 12].

Limitations of the study

There are a few limitations of this study. First, this study was done in one centre that includes a small number of patients, and the degree of the stenosis was divided in increments of 25%, which resulted in relatively small groups; prospective multicentre studies of large numbers of patients will enhance the application of CAD-RADS among radiologists and cardiologists. Second, this study was done using 128-slice multi-detector CT scanners. Further studies using dual-energy CT scanners and higher multi-detector scanners such as 320-slice detectors are recommended [33, 34]. Third, there was no correlation between the results of coronary CTA and conventional angiography. Further studies to correlate the degree of stenosis with conventional angiography and MR angio­graphy are recommended [35-39]. Fourth, the authors did not exclude any segment from the investigation and did not discuss the possible relationship between heart rate, calcium score, and inter-observer variability. Further studies are recommended with correlation of CAD-RADS with calcium scoring and its relation to heart rate.

Conclusions

We concluded that there is inter-observer agreement of CAD-RADS in categorising the degree of coronary arteries stenosis, system modifiers, and associated cardiac and extra-cardiac findings, and CAD-RADS can be a standard method for reporting coronary CTA in the future.

Conflict of interest

The authors declare no conflict of interest.

REFERENCES

1. Eisen A, Giugliano RP, Braunwald E. Updates on Acute Coronary Syndrome: A Review. JAMA Cardiol 2016; 1: 718-730.
2. Mack M, Gopal A. Epidemiology, traditional and novel risk factors in coronary artery disease. Cardiol Clin 2014; 32: 323-332.
3. Hollander JE, Than M, Mueller C. State-of-the-Art Evaluation of Emergency Department Patients Presenting With Potential Acute Coronary Syndromes. Circulation 2016; 134: 547-564.
4. Balfour PC Jr, Gonzalez JA, Kramer CM. Non-invasive assessment of low- and intermediate-risk patients with chest pain. Trends Cardiovasc Med 2017; 27: 182-189.
5. Earls JP, Woodard PK, Abbara S, et al. ACR Appropriateness Criteria Asymptomatic Patient at Risk for Coronary Artery Disease. J Am Coll Radiol 2014; 11: 12-19.
6. Saremi F. Cardiac MR Imaging in Acute Coronary Syndrome: Application and Image Interpretation. Radiology 2017; 282: 17-32.
7. François CJ. Current state of the art cardiovascular MR imaging techniques for assessment of ischemic heart disease. Radiol Clin North Am 2015; 53: 335-344.
8. Yoo SM, Chun EJ, Lee HY, et al. Computed Tomography Diagnosis of Nonspecific Acute Chest Pain in the Emergency Department: From Typical Acute Coronary Syndrome to Various Unusual Mimics. J Thorac Imaging 2017; 32: 26-35.
9. Bittencourt MS, Hulten EA, Veeranna V, et al. Coronary Computed Tomography Angiography in the Evaluation of Chest Pain of Suspected Cardiac Origin. Circulation 2016; 133: 1963-1968.
10. Maffei E, Seitun S, Guaricci AI, et al. Chest pain: coronary CT in the ER. Br J Radiol 2016; 89: 20150954.
11. Chlett CL, Hoffmann U, Geisler T, et al. Cardiac computed tomography for the evaluation of the acute chest pain syndrome: state of the art. Radiol Clin North Am 2015; 53: 297-305.
12. Nieman K, Hoffmann U. Cardiac computed tomography in patients with acute chest pain. Eur Heart J 2015; 36: 906-914.
13. Rao AA, Feneis J, Lalonde C, et al. A Pictorial Review of Changes in the BI-RADS Fifth Edition. Radiographics 2016; 36: 623-639.
14. Grant EG, Tessler FN, Hoang JK, et al. Thyroid Ultrasound Reporting Lexicon: White Paper of the ACR Thyroid Imaging, Reporting and Data System (TIRADS) Committee. J Am Coll Radiol 2015; 12: 1272-1279.
15. Greer MD, Brown AM, Shih JH, et al. Accuracy and agreement of PIRADSv2 for prostate cancer mpMRI: A multireader study. J Magn Reson Imaging 2017; 45: 579-585.
16. Abdel Razek AA, Ashmalla GA, Gaballa G, et al. Pilot study of ultrasound parotid imaging reporting and data system [PIRADS]: Inter-observer agreement. Eur J Radiol 2015; 84: 2533-2538.
17. Chandrashekhar Y, Min JK, Hecht H, et al. CAD-RADS: A Giant First Step toward a Common Lexicon? JACC Cardiovasc Imaging 2016; 9: 1125-1129.
18. Cury RC, Abbara S, Achenbach S, et al. Coronary Artery Disease – Reporting and Data System (CAD-RADS): An Expert Consensus Document of SCCT, ACR and NASCI: Endorsed by the ACC. JACC Cardiovasc Imaging 2016; 9: 1099-1113.
19. Cury RC, Abbara S, Achenbach S, et al. CAD-RADSTM Coronary Artery Disease – Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology. J Cardiovasc Comput Tomogr 2016; 10: 269-281.
20. Cury RC, Abbara S, Achenbach S, et al. CAD-RADSTM: Coronary Artery Disease – Reporting and Data System: An Expert Consensus Document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology. J Am Coll Radiol 2016; 13:1458-1466.
21. Gu H, Gao Y, Wang H, et al. Difference of coronary stenosis severity between systolic and diastolic phases in quantitative CT angiography. J Cardiovasc Comput Tomogr 2017; 11: 105-110.
22. Plank F, Burghard P, Friedrich G, et al. Quantitative coronary CT angiography: absolute lumen sizing rather than %stenosis predicts hemodynamically relevant stenosis. Eur Radiol 2016; 26: 3781-3789.
23. Pugliese F, Hunink MG, Gruszczynska K, et al. Learning curve for coronary CT angiography – what constitutes sufficient training. Radiology 2009; 251: 359-368.
24. Saremi F, Achenbach S. Coronary plaque characterization using CT. AJR Am J Roentgenol 2015; 204: W249-60.
25. Szilveszter B, Celeng C, Maurovich-Horvat P. Plaque assessment by coronary CT. Int J Cardiovasc Imaging 2016; 32: 161-172.
26. Piccolo R, Giustino G, Mehran R, et al. Stable coronary artery disease: revascularisation and invasive strategies. Lancet 2015; 386: 702-713.
27. Suh YJ, Hong YJ, Lee HJ, et al. Accuracy of CT for Selecting Candidates for Coronary Artery Bypass Graft Surgery: Combination with the SYNTAX Score. Radiology 2015; 276: 390-399.
28. Tesche C, De Cecco CN, Vliegenthart R, et al. Coronary CT angiography-derived quantitative markers for predicting in-stent restenosis. J Cardiovasc Comput Tomogr 2016; 10: 377-383.
29. Amanuma M, Kondo T, Sano T, et al. Assessment of coronary in-stent restenosis: value of subtraction coronary computed tomography angiography. Int J Cardiovasc Imaging 2016; 32: 661-670.
30. Öztürk E, Kafadar C, Tutar S, et al. Non-coronary abnormalities of the left heart: CT angiography findings. Anatol J Cardiol 2016; 16: 720-727.
31. Venkatesh V, You JJ, Landry DJ, et al. Extracardiac findings in cardiac computed tomographic angiography in patients at low to intermediate risk for coronary artery disease. Can Assoc Radiol J 2010; 61: 286-290.
32. Shuaib W, Arepalli C, Vijayasarathi A, et al. Coronary anomalies encountered in the acute setting: an imaging review. Emerg Radiol 2014; 21: 631-641.
33. Tabari A, Lo Gullo R, Murugan V, et al. Recent Advances in Computed Tomographic Technology: Cardiopulmonary Imaging Applications. J Thorac Imaging 2017; 32: 89-100.
34. Machida H, Tanaka I, Fukui R, et al. Current and Novel Imaging Techniques in Coronary CT. Radiographics 2015; 35: 991-1010.
35. Abdel Razek AA, Denewer AT, Hegazy MA, et al. Role of computed tomography angiography in the diagnosis of vascular stenosis in head and neck microvascular free flap reconstruction. Int J Oral Maxillofac Surg 2014;43:811-815.
36. Abdel Razek A, Ashmalla G, Samir S. Clinical value of classification of venous malformations with contrast enhanced MR angiography. Phlebology 2017; 32: 628-633.
37. Razek AA, Gaballa G, Megahed AS, et al. Time resolved imaging of contrast kinetics (TRICKS) MR angiography of arteriovenous malformations of head and neck. Eur J Radiol 2013; 82: 1885-18591.
38. Romeih S, Al-Sheshtawy F, Salama M, et al. Comparison of contrast enhanced magnetic resonance angiography with invasive cardiac catheterization for evaluation of children with pulmonary atresia. Heart Int 2012; 7: e9.
39. Razek AA, Saad E, Soliman N, et al. Assessment of vascular disorders of the upper extremity with contrast-enhanced magnetic resonance angiography: pictorial review. Jpn J Radiol 2010; 28: 87-94.