Introduction

Clinical trials have established that the use of more potent and prolonged antithrombotic medications after an acute coronary syndrome (ACS) can reduce subsequent ischaemic events.1–3 However, the benefit is at least partly offset by the increased risk of major bleeding.1–4 Because there is a wide range of individual ischaemic and bleeding risk, post-ACS guidelines recommend the use of multivariable risk scores to estimate the risk of adverse ischaemic and major bleeding outcomes to facilitate optimal management.5–7

In current practice, patients are recommended to receive dual antiplatelet therapy (DAPT) with aspirin and a P2Y12 inhibitor for at least a month post-ACS. The recommended duration of DAPT beyond the first month is based on the estimated risks of bleeding and of recurrent ischaemic events.7 For patients who have recurrent ischaemic or bleeding events in the first month, subsequent antithrombotic treatment needs to be individualised, but in the majority, who are event free by 1 month, it would be useful to have a pair of risk scores which estimate the ischaemic and bleeding risks over the remainder of the year, to inform DAPT duration. Although there are separate risk scores that can stratify patients’ post-ACS ischaemic and major bleeding risk,8–14 no scores have been developed within the same population using a common methodology. Unless both ischaemic and bleeding risk scores are validated in the same target population, calibration errors may lead to substantial overestimation or underestimation of both the absolute levels of risk and the ratio of ischaemic to bleeding risk15 potentially leading to inappropriate clinical decision making. A further complexity is that ischaemic and bleeding risks vary with time after the ACS event, with the highest event rates in the first month post-ACS.16 17 Currently, available risk scores include those early events and may therefore overestimate risk beyond the first month. To be most clinically useful, scores need to be customised for the cohort and for the time period over which they are to be applied.

In Aotearoa New Zealand, virtually all patients with ACS who are managed invasively with coronary angiography are captured in the All New Zealand ACS Quality Improvement (ANZACS-QI) registry and are linked to well-validated national administrative datasets for outcome ascertainment. The aim of this study was to use this real-world ACS cohort to develop a pair of ischaemic and major bleeding risk scores to estimate ischaemic and bleeding risk up to 1 year post-ACS in patients who are event free at 1 month. The risk scores are designed specifically to assist clinicians and their patients in deciding on subsequent DAPT duration.

MethodsCohort and data sources

The ANZACS-QI programme is a web-based prospective registry of New Zealand (NZ) residents hospitalised with ACS and undergoing angiography. A mandatory dataset is collected. The registry collects data regarding demographic, clinical history and presentation, investigations and management available by the time of discharge from hospital. A detailed description of data collection has been previously published.18 The registry undergoes monthly auditing to ensure capture of >99% of all patients admitted with suspected ACS who are investigated with coronary angiography, and annual audit to check the accuracy of data entry.

A cohort which included all those aged 18–84 years who received coronary angiography during the index ACS admission and were discharged alive between 1 January 2012 and 31 August 2020 was created from the ANZACS-QI registry. The index admission was defined as the entire episode of care, from hospital admission to discharge home, and included interhospital transfers. This cohort was linked to national public hospitalisations, mortality and pharmaceutical dispensing databases, using encrypted National Health Index (NHI) numbers.18 The association between atrial fibrillation (AF) and either ischaemic risk (which includes stroke) or bleeding risk will be confounded by variable anticoagulation use which cannot be adequately addressed in a model where the majority of patients did not have AF. Patients were therefore excluded if they had prior hospitalisation with AF or were dispensed anticoagulant medication in the 6 months before the admission or 3 months after discharge (n=7123). In addition, 83 patients with incomplete data were excluded, as were patients who had an ischaemic or major bleeding event in the first 28 days postdischarge (n=568).

OutcomesPost-ACS ischaemic model

The primary outcome was cardiovascular disease (CVD) mortality or rehospitalisation for myocardial infarction (MI) or ischaemic stroke between 29 days and 1 year postdischarge. Hospitalisations were identified using primary or secondary International Classification of Diseases, 10th Revision, Australian Modification (ICD-10-AM) diagnosis codes.

Post-ACS bleeding model

The primary outcome was bleeding mortality or rehospitalisation for a primary ICD-10-AM bleeding diagnosis or a secondary bleeding diagnosis with an associated blood transfusion, between 28 days and 1 year postdischarge.19 Major bleeding events associated with coronary artery bypass grafting (CABG), other surgical procedures or trauma were excluded.

The end of follow-up for both models was 31 December 2020.

ICD-10 AM codes used to define all outcomes are shown in online supplemental appendix table 1. Sixty patients experienced both ischaemic and major bleeding events within 1 year follow-up, which were therefore outcomes in both models.

Details of data quality checks performed by the NZ Ministry of Health have been published previously.20 The ANZACS-QI registry has previously been used to validate the accuracy of ICD-10-AM coded MI hospitalisations.21

Predictor variables

Our aim was to develop risk scores for clinical use, requiring a parsimonious set of variables with the greatest predictive ability in each risk score. We identified the predictor variables included in prior published ischaemic and bleeding risk scores, and based on our prior research and clinical experience, we also identified several other possible predictors. From this process, 24 risk predictors were selected a priori for the ischaemic and bleeding models (online supplemental appendix table 2), and exploratory analysis was conducted on the cohort with outcomes available in 2019 (2012–2018 patients). Age and sex were retained as minimum predictors in each model. Additional ‘core’ variables were defined as those with a strong independent association with each outcome (p<0.001), then ‘borderline’ variables were assessed for inclusion in a backward selection process using Akaike information criterion (AIC)22 for selection, where the model with the lowest AIC was selected as the final prediction model.

For the ischaemic risk score, the ‘core’ variables were age, sex, ethnicity, estimated glomerular filtration rate (eGFR), severity of coronary artery disease (CAD), coronary intervention during admission, prior CVD, diabetes, ACS type and worst Killip class, and the additional ‘borderline’ variables retained were NZ Deprivation index, smoking, haemoglobin, left ventricular ejection fraction (LVEF) and heart rate. Prior hospitalisation for bleeding, ratio of total cholesterol to high-density lipoprotein cholesterol (TC:HDL), systolic blood pressure and prior heart failure were eliminated from the ischaemic model.

For the bleeding risk score, the ‘core’ variables were age, sex, ethnicity, eGFR, prior hospitalisation for bleeding, severity of CAD, haemoglobin and index admission bleeding, and the additional ‘borderline’ variables retained were NZ Deprivation index, coronary intervention during admission, heart rate and TC:HDL. Systolic blood pressure, non-steroidal anti-inflammatory medication/corticosteroids or gastric protection medication were eliminated from the bleeding model. Values of TC:HDL and LVEF were missing for 7% and 23% of the cohort, respectively, and assumed missing at random based on prior registry analyses. Missing TC:HDL and LVEF were included as categories to allow application of the risk scores to patients where it is similarly unavailable due to variability in practice or resource availability.

Statistical analysis

See online supplemental appendix for full description. Comparisons between patients with and without 1-year events were made with χ2 test, two-sample t-test or the Mann-Whitney U test (when the normality assumption was not met) as appropriate. Multivariable Cox proportional hazards regression was used to develop models from 28 days postdischarge for the index ACS event to the first subsequent MI, ischaemic stroke or major bleeding event, date of death or end date (data extraction). The correlation between estimated ischaemic and bleeding risk was assessed using the Pearson correlation coefficient.

To facilitate a discussion of clinical implications, the risk scores are presented in categories of risk ≤2% (low), >2 to <4% (intermediate), ≥4 (high). The 2% and 4% cut-points were based on previously proposed cut-points for high-risk bleeding. A PRECISE-DAPT score ≥25, equating to a 1-year Thrombolysis in Myocardial Infarction (TIMI) major or minor bleeding risk of just under 2%, has been proposed as a high-risk criterion for patients undergoing percutaneous coronary intervention (PCI).23 In contrast, a recent consensus statement from the Academic Research Consortium for High Risk Bleeding (ARC-HBR) proposed that a 1-year bleeding risk of ≥4% be considered high risk for patients undergoing PCI.24 For comparison, the same categories were applied for ischaemic risk.

Predictive performance was assessed using calibration, global model fit and discrimination. Calibration was also assessed for subgroups: age (<70 years vs ≥70 years), sex, ACS type (ST elevation MI vs non-ST elevation ACS), coronary intervention (PCI/CABG vs none), on DAPT beyond 3 months postdischarge versus not, and geographical region. Model fit was measured with Nagelkerke’s R2 and 95% CIs derived from 1000 bootstrap samples. Model discrimination was quantified by Harrell’s c-statistic and Gönen and Heller’s K-statistic. Internal validity was assessed via 1000 bootstrap samples.

Models from complete case analysis of patients with no missing data (n=21 255 and 25 803 for ischaemic and bleeding risk scores, respectively) were compared with those for the full cohort.

Model development and assessment was completed in accordance with the recommendations of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement25; external validation has not been performed due to the absence of a relevant external cohort at this stage. All analyses were performed using R software, V.4.0.0, and packages.

Results

The cohort comprised 27 755 patients admitted to hospital with ACS who had coronary angiography and were event free by 28 days postdischarge. Baseline characteristics of the cohort are shown in table 1. By 1 year postdischarge, there were 1200 (4.3%) ischaemic events and 548 (2.0%) bleeding events (figure 1). The ischaemic events comprised 919 (76.6%) MI, 124 (10.3%) stroke and 157 (13.1%) deaths. Of the major bleeding events, 376 (68.6%) were gastrointestinal, 45 (8.2%) intracerebral/intraocular and 127 (23.2%) other aetiologies.

Figure 1

Kaplan–Meier cumulative incidence plots of (A) ischaemic and (B) major bleeding events for patients with ACS who are event free at 28 days postdischarge. ACS, acute coronary syndrome.

Table 1

Baseline characteristics of patients with versus without ischaemic or major bleeding events at 28 days to 1 year after ACS discharge

Risk scores and performance

The adjusted multivariable HRs from the final ischaemic and bleeding models are shown in table 2. The coefficients and baseline survival for each risk score and example calculations are in table 3. There was excellent calibration of the scores throughout the range of predicted risk (figure 2). The ischaemic model had a slope of 0.999 (0.947, 1.053) and an intercept of −0.0002 (−0.057, 0.056), and the bleeding model had a slope of 1.000 (0.889, 1.111) and an intercept of −0.0001 (−0.084, 0.084).

Figure 2

Calibration plots for observed versus predicted 28 day to 1 year risk. Calibration performance of ANZACS-QI risk prediction scores developed in the full cohort. The diagonal line represents perfect calibration. Both the ischaemic and bleeding scores were well calibrated in all subgroups (online supplemental appendix figure 1).

Table 2

Adjusted multivariable HRs from the final ischaemic and bleeding models

Table 3

Clinical example calculations of 28 days to 1 year absolute risk

The scores stratified risk of ischaemia and major bleeding well, with an approximately 10-fold range of risk between the lowest and the highest deciles of predicted risk. The mean 1-year predicted and observed ischaemic risks in the lowest decile were 1.0% and 0.9%, and in the highest decile 17.2% and 17.6%. In contrast, the mean 1-year predicted and observed major bleeding risks in the lowest decile were 0.7% and 0.7%, and in the highest decile 6.1% and 5.8%. Measures of model performance are shown in online supplemental appendix table 3. Both models had moderate discrimination, with Harrell’s c-statistic 0.75 (95% CI, 0.74 to 0.77) and 0.69 (95% CI, 0.67 to 0.71), respectively.

Sensitivity analyses

Both the ischaemic and bleeding risk scores were well calibrated in all subgroups (online supplemental appendix figure 1). HRs in the models developed in the full cohort with categories for missing data were very similar to those for the complete case analysis (online supplemental appendix table 4), as was model performance during internal validation. The mean (95% CI) of the bootstrap validated Harrell’s c-statistic for the ischaemic and bleeding models, respectively, was 0.75 (0.74 to 0.77) and 0.69 (0.67 to 0.71) (online supplemental appendix table 3).

Ischaemic versus bleeding risk

The distributions of 28-day to 1-year risk and the individual differences between ischaemia and major bleeding risks are shown in figure 3. The median absolute 1-year risk of cardiovascular death/MI/stroke event was 2.7% (IQR 1.8%–4.9%) and of major bleeding was 1.6% (IQR 1.1%–2.4%). For 85% of patients, the risk of dying or having an ischaemic event outweighed the major bleeding risk (median absolute difference 1.2%, IQR 0.5%–2.8%), and 42% had an ischaemic risk at least double that of the major bleeding risk. There was a moderate correlation between individual ischaemic and major bleeding risk estimates (r=0.643, 95% CI 0.636 to 0.650). The median ischaemic risk increased from 2.2% to 4.3% to 9.2% for low (≤2%), intermediate (>2% to <4%) and high bleeding risk (≥4%) categories (figure 4). When each patient’s individual ischaemic and bleeding risks were classified in these categories (table 4), 34% had an intermediate or high bleeding risk, of whom most (91%) had an intermediate or high ischaemic risk. Of the 66% with a low bleeding risk, 55% had a medium or high ischaemic risk.

Figure 3

Distribution of absolute 28-day to 1-year predicted risks (left panel) and differences between individual patient ischaemic and major bleeding risk scores (right panel). The dashed lines represent the medians of the distributions. The x-axis has been truncated at 20% (left panel) and at −5% and 20% (right panel) for clarity.

Figure 4

Distribution of ischaemic risk by bleeding and ischaemic risk categories. This box and whisker plot shows the median inside a box defined by the 25th and 75th percentiles, with whiskers running from the ninth percentile to the 91st percentile.

Table 4

Risk classification table stratified by ischaemic and major bleeding risk scores

Discussion

In this real-world national ACS cohort, paired risk scores have been developed to estimate the 28-day to 1-year postdischarge risk of ischaemic and major bleeding events. The risk scores are well calibrated, both overall, and in demographic and relevant clinical subgroups. Median 1-year ischaemic risk is nearly twofold higher than major bleeding risk, but for 15% of patients the risk of a major bleeding event outweighed that of an ischaemic event. Of the two-thirds of patients with a low bleeding risk, half had an intermediate or high ischaemic risk. Of the remaining third with an elevated bleeding risk, most also an elevated ischaemic risk.

To our knowledge, there are no other paired multivariable risk scores available to predict postdischarge ACS ischaemic and bleeding risk. There are two prior post-PCI studies where ischaemic and major bleeding risks were modelled, either in the 2-year post-PCI in the PARIS registry26 or beyond 1 year in the DAPT study.27 However, in addition to estimating risk over different time periods, these PCI cohorts were at lower overall ischaemic risk than our post-ACS cohort. Only 12% of patients had troponin-positive ACS in PARIS, and although half of patients in the DAPT study were post-ACS, they were enrolled 12 months post-PCI. Other investigators have developed either post-ACS ischaemic or bleeding risk scores. The GRACE investigators presented a model to estimate the risk of death or MI within 6 months postdischarge.28 More recently, the Epicor risk score estimates 1-year mortality risk postdischarge after ACS.8 The BleeMACS postdischarge ACS bleeding model, like ours, utilised rehospitalisation for bleeding to define bleeding endpoints.14 The Trilogy bleeding score was developed from a clinical trial cohort enrolled 10 days after ACS without revascularisation.11 Other bleeding scores have either included in-hospital bleeding10 12 or were in post-PCI cohorts.26 27 Similarly, other post-ACS scores included in-hospital events.9 29

The finding that most patients have an ischaemic risk which exceeds their major bleeding risk has implications regarding DAPT use after ACS. Because the benefits of medical therapies for cardiovascular prevention including anti-platelet agents are known to be proportional to the absolute ischaemic risk,30–32 high ischaemic risk patients might be expected to benefit most from more prolonged DAPT. However, several recent studies suggest that the ischaemic risk in patients at high bleeding risk may be less modifiable by prolonged DAPT. A meta-analysis of coronary stenting trials assessing short versus longer duration DAPT found that ischaemic events were reduced by longer DAPT for patients at low bleeding risk, but in those at high bleeding risk, defined using the Precise-DAPT score, longer DAPT duration was associated with similar ischaemic event rates but higher bleeding rates.23 In the subgroup with acute coronary syndromes, they reported a similar result although with relatively small numbers of events. Two subsequent clinical trials in patients at high bleeding risk treated with modern generation stents have reported similar findings.33 34

Weighting of ischaemic and bleeding events

In benefit–harm models, ischaemic and major bleeding outcomes have been weighted equivalently.26 In this analysis, we have endeavoured to identify only the most serious ischaemic and bleeding events. In this study, 15% of patients had a bleeding risk higher than the ischaemic risk. If ischaemic events were subjectively weighted more than bleeding events, the proportion with net harm would be less than 15%. Treating the ischaemic and bleeding events as equivalent is lent support by a recent study which reported that after ACS both postdischarge bleeding and postdischarge MI were associated with a similar increase in subsequent all-cause mortality.35 The equivalence of ischaemic and bleeding risk does not imply that changing the intensity or duration of DAPT will modify bleeding and ischaemic risk by the same amount. For example, clinical trials of short compared with long-term DAPT report a greater relative increase in bleeding events than decrease in ischaemic events with long-term DAPT.36

In the current 2020 European Society of Cardiology (ESC) guidelines ‘algorithm for antithrombotic treatment in NSTEACS patients without AF undergoing PCI’, it is not possible to quantitatively compare ischaemic and bleeding risks for an individual patient. The guideline defines high bleeding risk using either of two different scores (PRECISE-DAPT and the ARC-HBR criteria) while ischaemic risk prediction is more qualitative.7

Clinical implications and implementation

In an era of shared decision making, it is important to provide patients and clinicians with objective risk estimates as a basis for deciding on management.37 The ANZACS-QI risk scores presented here are calculated at hospital discharge using data consistently available at that time point. They provide an estimate of risks from 28 days to 1 year as long as the patient remains event free at 1 month. Consequently, they can be used at discharge to plan DAPT duration beyond a month. In clinical practice, if an event does occur within the first 28 days the estimated risks are no longer relevant and the decision to stop or continue DAPT must be individualised.

The evidence around DAPT duration post-ACS continues to evolve,38 and the role of ischaemic and bleeding risk scores requires justification in prospective clinical trials. In table 5, we have proposed a potential clinical and research algorithm based on the recent ESC recommendations applied to the ANZACS-QI risk categories.7 The one-third of the cohort at elevated bleeding risk were also mostly at elevated ischaemic risk. As discussed above, recent clinical trials suggest that in these patients the efficacy of DAPT for reducing ischaemic events may be less than for patients at lower bleeding risk and an abbreviated DAPT duration is recommended. The ESC currently recommend a default 12 months of DAPT in all those at low bleeding risk. However, the ESC also recommends that for those at low risk of major bleeding and an elevated ischaemic risk (37% of our cohort), more prolonged DAPT may be justified, and for those with low bleeding and ischaemic risk (29% of our cohort) a shorter period of DAPT may be justified. In clinical practice, there are other factors which might also influence the decision regarding DAPT duration. These include procedural variables such as stent type, lesion location and length, and vessel size, and specific clinical situations such as the need for non-cardiac surgery. In addition to informing DAPT duration decisions, the bleeding risk estimates are useful to decide on whether to use a potent P2Y12 receptor inhibitor (ticagrelor or prasugrel) or the less potent clopidogrel.7

Table 5

Using the ischaemic versus bleeding risk matrix to inform DAPT duration: a potential clinical algorithm and future research directions

All variables used in the current scores are routinely collected in the ANZACS-QI electronic registry for every New Zealander who has a coronary angiogram post-ACS. Consequently, each patient can have bleeding and ischaemic risks automatically generated prior to discharge. We plan to develop separate risk scores for patients with AF post-ACS and for those who do not have a coronary angiogram.

Limitations

These risk scores have not been externally validated but are well validated in the population in which they are intended for use. While the current lack of external validation limits implementation of our scores in other jurisdictions, this study demonstrates the feasibility and potential accuracy of developing paired quantitative ischaemic and bleeding risk scores within a national registry linked to administrative health data. The scores’ coefficients are provided to enable other groups to validate and potentially recalibrate the scores for their use. A comprehensive set of predictors were considered a priori to optimise discrimination and calibration. However, several recognised risk factors for bleeding were not included in development of the ANZACS-QI bleeding score: platelet and white cell count were not available for this study, and others occurred with very low frequency in the cohort—liver disease (1.1%), cancer (0.6%) and prior intracerebral bleeding (0.5%). The ANZACS-QI risk scores are both very well calibrated, but measures of global discrimination are slightly better for the ischaemic score. This is consistent with other studies in the literature predicting similar outcomes.9 14 Outcome data in this study were obtained by de-identified linkage to national health datasets. It is therefore not possible to independently validate these events, and events not associated with re-hospitalisation or death are not captured.

Conclusion

In this real-world national ACS registry cohort, paired risk scores have been developed which simultaneously estimate the 28-day to 1-year postdischarge risk of ischaemic and major bleeding events. In the era of personalised medicine, application of these scores may help inform the appropriate intensification of management after ACS.