We report our findings according to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis reporting guidelines (TRIPOD) statement [20].

We performed a retrospective cohort study using the electronic medical records of all patients admitted to the general ICU at Addenbrooke’s hospital, Cambridge University Hospitals (CUH), between 1 February 2017 and 30 September 2021. CUH is a large teaching hospital, providing emergency and elective, medical and surgical care to a large diverse population [21]. CUH is also a leading liver transplant centre and provides tertiary hepatology services for the East of England.

All adults (≥ 18 years of age) admitted to the ICU during the study period were eligible for inclusion. The following exclusion criteria were applied according to the original study: (1) patients who developed delirium within the first 24 h of ICU admission, (2) ICU admission less than 24 h, and (3) patients who were persistently comatose (having Richmond Agitation-Sedation Scale (RASS) score of − 4 or − 5) throughout the first 24 h in the ICU [22]. We additionally exclude: (4) readmissions, as recorded in Intensive Care National Audit & Research Centre (ICNARC) audit data [23].

ICU patients were assessed for delirium twice daily by critical care nurses as part of routine clinical care, using the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) assessment [24]. CAM-ICU is a standardised evidence-based tool for delirium assessment in the ICU [25]. Delirium was defined in the primary analysis as either a positive CAM-ICU assessment or the new administration of any antipsychotic medication during the patient’s ICU stay. While the original studies included only haloperidol in the definition of delirium [7, 8], we also included atypical antipsychotic drugs since both have been previously recommended by our ICU guidelines for the management of agitated delirium [6]. We also consider alternative definitions of delirium in sensitivity analyses (see Statistical analysis section below). RASS was recorded hourly in the ICU.

Clinical data were extracted from Epic, the integrated electronic health record system used at the study hospital [26], and linked ICNARC audit records. All data were recorded in the electronic health records system by physicians and nurses in the participating ICUs, and were extracted by a clinical informatics team who did not participate in clinical care.

We aimed to reproduce the definitions from the original study as closely as possible. PRE-DELIRIC is comprised of ten predictors, measured in the first 24 h of an ICU admission: age, Acute Physiology and Chronic Health Evaluation II (APACHE II) score, coma category, admission category, infection, metabolic acidosis, morphine use, sedative use, urgent admission and serum urea [7]. Age, admission APACHE II score, admission category and urgency of admission were extracted from local data collected for ICNARC [23]. Medication usage was extracted from the electronic medication administration record in Epic [26]. Infection status was defined as any use of any antibacterial medication during the first 24 h of the ICU stay, reflecting clinician belief on whether infection was present. In the study ICU the use of antibiotics is discussed in a daily multi-disciplinary meeting with a Microbiologist. Sedation status was defined as any administration of propofol, midazolam or lorazepam. A patient was diagnosed with metabolic acidosis if the pH was under 7.35 and base excess was under −2. The highest serum urea measurement during the first 24 h in the ICU stay was used.

Comatose patients (defined as RASS of −4 or −5) who were administered any of fentanyl, remifentanil, propofol or midazolam were eligible for classification as “comatose with use of medication”. Non-pharmacological causes of coma were identified from ICNARC data. Comatose patients with both pharmacological and non-pharmacological causes for coma were classified in the “combination” category. While the original PRE-DELIRIC model specifies “morphine usage” as a predictor [7], the most commonly used opioids in the study ICU are fentanyl and remifentanil, so we converted opioid administration into morphine equivalent doses [27].

All available data in the database, during the study period, were used to maximise the power and generalisability of the results [28].

Only two risk factors had missing values; metabolic acidosis and urea. Metabolic acidosis had 17 (0.5%) missing values that were imputed as no acidosis present. Urea had 11 (0.3%) missing values and were imputed with the mean value of the respective group; delirious or not delirious.

For all analyses presented in the current study, we used R statistics version 4.1.3 [29]. For the clinical characteristics of ICU patients, mean (SD) was employed for normally distributed characteristics, whereas the median (interquartile range (IQR)) was used for non-normally distributed characteristics. Binary and multiple categorical risk factors are presented as frequency and percentage.

In the primary analyses, we used the latest version of the PRE-DELIRIC model, specifically the “recalibrated coefficients” version [8]. The predictive ability of the PRE-DELIRIC model was evaluated based on both discrimination and calibration. Regarding the discrimination ability, the AUROC curve was employed. We also plot the precision-recall (PR) curve, showing precision (positive predictive value (PPV)) against recall (sensitivity), and calculate the area under the PR curve (AUPRC). Calibration was assessed by examining the agreement between predicted and observed values, and the calibration intercept and slope from the model logit(probability of delirium) = alpha + beta × logit(p), where p is the PRE-DELIRIC score [20].

As sensitivity analyses, we examined performance when delirium was defined as either a positive CAM-ICU assessment or administration of haloperidol only (i.e. not including other antipsychotic medications); and when delirium was solely defined by CAM-ICU assessments (not including administration of any antipsychotic medications). We also examined performance using the original model coefficients [7], rather than the recalibrated coefficients; and assessed a strict definition of “morphine use” that included only administration of morphine itself (i.e. we exclude other opioids) to exactly match the original PRE-DELIRIC paper [7].

As secondary analyses, we assessed PRE-DELIRIC performance in three specific subgroups of ICU patients in CUH: (1) patients with liver disease who had not had a liver transplant; (2) patients who received sedative medication and (3) patients who received varying doses of opioid drugs.

We also explored relationships between key clinical variables and delirium incidence, including ammonia levels, bilirubin concentration and opioid administration.

The study was approved by a UK Health Research Authority ethics committee (19/SC/0541). Patient consent was waived because the de-identified data presented here were collected during routine clinical practice; there was no requirement for informed consent.