This retrospective, observational, cross-sectional study was conducted in strict adherence to the principles outlined in the Declaration of Helsinki and in accordance with Good Clinical Practice standards. Authorization for this research was granted by the Swedish ethical review authority (Gothenburg Regional Ethical Review Board, DNR 2021-00441) and for the Swedish CArdiopulmonary BioImage Study (SCAPIS) (Gothenburg and Umeå regional ethical review boards, Dnr 2010-228-31M). In compliance with the ethical regulations governing Swedish registries and national legislation, the SCAPIS study subjects were duly informed about their involvement in a registry and provided the option to decline participation or request the removal of their data. The need for written consent was waived for the current study.

All included subjects had undergone CSCT within SCAPIS [21] at Linköping University Hospital between October 8, 2015, and June 12, 2018, amounting to a total of 5057 individuals, 50–64 years of age. Subject data and population demographics (Table 1) were retrospectively gathered from the SCAPIS database. To this date, a total of 45 articles have reported on parts or the entire included study population [22]. None of these articles addressed AI evaluation of CSCT scans.

Exclusion criteria were declination of the CSCT scan (n = 45) and those previously established by Wolterink et al [23]. This led to the exclusion of CSCT scans registered as exhibiting severe motion artifacts or high noise levels (n = 114), missing images from the CSCT scan (n = 1), incomplete scan length coverage (n = 4) and missing CACS results tables from the semi-automatic evaluation (n = 56). Subjects with a self-reported medical history, including myocardial infarction, angina pectoris, heart failure, heart valve disease, coronary bypass surgery, percutaneous coronary intervention (n = 122) or missing information about any of these conditions (n = 148), were also excluded to follow guideline recommendations [11, 12] for application of CACS as risk assessment in individuals asymptomatic of CAD.

All CSCT scans were performed using a SOMATOM Definition Flash dual source CT system (Siemens Healthineers). Depending on heart rate and pulse variability, a prospectively ECG-triggered high-pitch spiral (Flash scan) or a sequential scan technique was employed. Both scan protocols had vendor-recommended scan and reconstruction settings, utilizing 120 kVp tube voltage and automated tube current modulation (CARE Dose4D, Siemens Healthineers) set to 80 quality reference mAs. Additional scan parameters included a gantry rotation time of 0.28 s, pitch 3.4, 128 × 0.6 mm collimation. Scan initiation for Flash scans was set at 60% of the cardiac cycle, while sequential scans were conducted at 70%.

Image reconstructions were executed using weighted filtered back projection (WFBP, Siemens Healthineers), a convolution kernel (B35f) dedicated for calcium score reconstructions, 3.0 mm image slice thickness, 1.5 mm increment. Sublingual nitroglycerine and beta-blockers (Metoprolol, i.v. and/or oral) were administered if the heart rate exceeded 60 bpm and no contraindications to these pharmaceuticals were found. Following the CSCT scan, a coronary computed tomography angiography (CCTA) was performed during the same imaging session.

The semi-automatic CACS evaluations conducted within SCAPIS were made by thoracic radiologists or cardiologists using dedicated CACS post-processing software (syngo.Via VB10A, Siemens Healthineers). All readers had a minimum level 1 training in accordance with the American College of Cardiology Foundation/American Heart Association Clinical Competence Statement on cardiac CT for cardiac CT reading [24] and between 1 and > 10 years’ experience in reading clinical CCTAs. In addition to this, the readers attended yearly training and information sessions within the frame of SCAPIS to assure consistency in reading and reporting.

CACS results charts from the semi-automatic evaluation were saved in the local PACS (Sectra Workstation IDS 7, Sectra). These were pseudonymized and exported for automatic readout. The readout was made with a custom-made MATLAB script (R2023a, Mathworks) that utilized optical character recognition for the automated tabulation. All values were manually controlled. This process rendered access to AS as well as MS and VS, number of lesions and lesion location from the semi-automatic evaluation. No complete pre-control was made for the current study to discover errors in the reference standard results. However, an audit was made of cases misclassified by the AI software by a cardiac imaging radiographer (L.H.) with level 1 training for cardiac CT reading as defined by the Society of Cardiovascular CT [25] and 6 years of experience in cardiac CT research reading. A second read was also performed (by L.H.) on a subset of 200 semi-automatic evaluations acquired consecutively between May 17, 2016, and October 18, 2016, for subsequent reliability analysis.

The automatic evaluation was conducted by exporting the CSCT images from the PACS for post-processing by a newer version of the same CACS post-processing software, which enables AI evaluation (syngo.Via VB60A, Siemens Healthineers) and has been described in previous studies [18, 19, 26]. The charts from this fully automatic evaluation were pseudonymized and exported for automatic readout in the same manner as the semi-automatic evaluation charts and rendered the same type of information.

Based on the AS, CACS values from both AI and semi-automatic evaluations were classified into five commonly used CV risk categories: AS 0 (No identifiable plaque, very low cardiovascular risk), AS 1–10 (Minimal plaque burden, low cardiovascular risk), AS 11–100 (Mild atherosclerotic plaque burden, moderate cardiovascular risk), AS 101–400 (At least moderate atherosclerotic plaque burden, moderately high cardiovascular risk), and AS > 400 (Extensive atherosclerotic plaque burden, high cardiovascular risk) [27].

Continuous data were reported as mean ± standard deviation. Categorical data were presented as counts and percentages. Normality was assumed due to the large sample size (central limit theorem).

Correlation and agreement between the standard reference and the automatic software with respect to the AS, VS, MS, number of lesions and lesion location were evaluated with Pearson correlation (r) and intraclass correlation coefficient (ICC). p-values < 0.05 were considered statistically significant. Bland–Altman plots were utilized to visualize bias and limits of agreement within a 95% confidence interval. Discrepancies in CV risk classifications were examined through weighted kappa analysis (κ) and accuracy. Kappa coefficients were assessed as 0.01–0.20, slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; and 0.81–0.99, almost perfect agreement [28]. The second read subset analysis was conducted with ICC for total and per-vessel AS and weighted kappa analysis (κ) and accuracy for CV risk classification.

Bland–Altman plots were carried out using Microsoft Excel (Microsoft Office 365, MSO, version 2302), while all other statistical analyses were performed using IBM SPSS version 27 (IBM).