This study was a quality improvement initiative as part of a larger multicenter stepped wedge pragmatic trial [10]. The sub-study is a pre/post cohort study of ED patients at a large academic tertiary care hospital in the Midwest United States with 80,000 ED patient visits a year. Standards for quality improvement reporting excellence (SQuIRE) were followed [11]. Ethical approval for the primary study was granted by the New York University Grossman School of Medicine Institutional Review Board (ID: i18-00607) with a waiver of consent. This single-site study was approved by the institution’s review board (#2020H0004). Publication of data was withheld until the final parent study and data analysis were completed in 2024 to avoid potentially biasing other study sites or the analysis team. The primary study results are now available [12].

The site implementation team included an ED physician, nurse educators, palliative medicine physicians, and management support from the central trial team. The educational component included a 4-h simulation workshop in end-of-life communication (Education in Palliative and End-of-Life Emergency Medicine (EPEC-EM)) for faculty and advanced practice providers and one hour of online didactic training (End of Life Nursing Education Consortium Critical Care (ELNEC) for ED nurses. Education was completed by 79% of ED clinicians (n = 62/78) and 67% (n = 98/134) of the ED nurses. Educational emails and reminders were sent during the implementation month (September 2019). Staff joining the department after September 2019 received brief training on EHR changes and templates but did not undergo EPEC-EM or ELNEC.

For the clinical decision support component of the intervention, a new GOC discussion documentation template was developed (Fig. 1) to standardize documentation and correspond to billing needs for the Centers for Medicare and Medicaid 99497 code for ACP conversations [13]. This included a more prominent link to the patient’s code status and advance care planning documents in an Epic navigator tab [8]. The intervention team also investigated and troubleshooted advance care planning documentation in the EHR [8]. No palliative care needs screening or alerts were implemented.

Template for ACP discussion documentation can be modified by the user with every use. The text bracketed by @ symbols are sectioning smart links that list this text in the Epic’s ACP activity. This allows future providers to easily find the information, instead of searching through the text of multiple notes. The drop-down menus in brackets allow for quick, guided documentation. For example, for participants, the drop-down allows the provider to easily pick from family, daughter, spouse, HPOA, chaplain, social worker and patient, among others. The sections with three asterixis (***) are easy to tab to and then add free text information

The patients were identified by EHR query (Epic, Epic Systems Corporation, Verona, Washington) for ED encounters September 2018-April 2020 that met any of 3 criteria: (1) a referral order to hospice within 48 h of ED arrival, (2) death within 48 h of ED arrival, or (3) ICU admission from ED. These criteria were chosen to identify patients with critical illness that may have warranted GOC discussion during their ED visit. As few patients qualified under criteria 1, all were included. Many encounters were potentially qualifying under criteria 2 and 3 and so these encounters were randomized within each criteria group for study selection. An interim check was done to evaluate the group composition and equalize the proportions from each criteria group in the pre and post cohorts. The charts were excluded if there was no ED encounter, incomplete documentation, had an encounter date during the time of the intervention (September or October 2019), or if the ICU admission was not immediately from the ED encounter (for example, admitted to a regular ward for two days and later upgraded to ICU admission).

Based on a prior study of trauma patients at our site, we assumed a pre-cohort GOC conversation rate of 15% [14]. If the intervention increased the rate to 45%, we would need 47 patients per group to be 90% powered to detect this difference with an alpha of 0.05. As we expected variability due to the 3 types of inclusion criteria, the sampling goal was 75 charts per cohort for at least 150 participants.

Chart abstraction was completed by trained study abstractors (medical students, residents, faculty) using REDCap, a HIPAA compliant research database [15]. Blinding to study group was impossible as the date of ED encounter was necessary to abstract the correct information. To avoid bias, chart abstractors were not funded by or involved with the parent PRIM-ER study, and all data collection was completed and analyzed prior to the final PRIM-ER findings. An abstraction codebook was developed during training and refined during chart review (see Supplemental Data A). All ED notes from the index encounter, including from physicians, nurses, advanced practice providers, social workers, and consultants were reviewed. The team preselected data elements at higher risk of misclassification for dual abstraction: type of emergency, presence of life-limiting illnesses, presence of a GOC conversation, and reason for the GOC conversation. The discrepancies were adjudicated by an independent third abstractor and inter-rater reliability was calculated.

Data elements included demographics, any life-limiting medical condition(s), type of emergency, initial and final code status, level of care provided (ICU, intermediate, or floor), ED disposition, and documentation of a GOC discussion. GOC conversation was defined as any of: (1) documentation of a discussion with a patient or their decision maker(s) regarding code status, advance care planning decisions or documents, or (2) a note containing one or more of the following terms (do not resuscitate, cardiopulmonary resuscitation (CPR), advance directive, power or attorney, Do Not Resuscitate (DNR), code status, advance directive, living will, POLST, goals of care, advance care planning), or (3) a positive search for documents containing advance directive, DNR, or hospice, or a specifically labeled “Advance Care Planning” note occurring during the ED stay. There had to be evidence of a conversation or discussion—routine note templates saying “Code status: Full” did not count as a documented GOC conversation. Key elements of a GOC conversation (decision-making capacity of the patient, legal next of kin, participants, presence of ACP documents, and code status) were noted if present. As a data check, the abstractor copied the specific text from the chart that defined “GOC conversation”. The reasons for GOC conversations were an actively dying patient, a newly diagnosed life-limiting illness, admission to the ICU, need for a procedure, or patient or family initiated.

Type of emergency was defined as the medical event warranting a visit to the ED, categorized as sepsis, trauma, stroke, other neurologic disease, cancer complications, COVID-19, heart failure, end stage renal disease, respiratory disease, cardiac arrest, or other. The life-limiting conditions present in the ED were categorized as oxygen-dependent lung disease, congestive heart failure, end-stage cancer, end-stage renal disease, stroke, severe injury, or other life-limiting conditions. More than one could be present.

Code status in Ohio is divided into Full Code (proceed with all disease directed therapies), Do Not Resuscitate–Comfort Care Arrest (DNRCC-A, do not resuscitate in the case of cardiac arrest), and Do Not Resuscitate–Comfort Care (DNRCC, comfort care measures only) [16].

Descriptive analysis of the patient population is reported as means and standard deviation and proportions with 95% confidence intervals as appropriate. The pre-and post-cohorts were compared using chi2 tests for proportions. Univariate logistic regression was used to look for factors associated with GOC conversation. The variables were pre-selected by the team and included age, gender, cohort, and type of emergency. The missing data were reported as missing.