Remember me
Despite the increasing volume of patient-reported outcome measures (PROMs) data collected in hospitals [], clinicians’ use of these data remains suboptimal [,]. PROMs are tools designed to capture patients’ own perceptions of their health, functioning, and quality of life [,], often through electronic surveys [,]. In outpatient care, clinicians can use a patient’s PROM data to screen and monitor important outcomes and involve patients in medical decisions []. In this way, PROMs facilitate value-based health care (VBHC) [,], aiming to optimally allocate resources to achieve outcomes that matter to patients.
Achieving and maintaining consistent use of PROM data by clinicians presents difficulties, requiring them to adapt their data use behavior and acquire new skills [,,]. Therefore, to realize the full benefits of PROMs, it is imperative to understand the factors influencing clinicians’ use of PROMs and develop strategies that facilitate and motivate their use in outpatient care [,,].
Previous WorkClinicians’ use of PROM data has been proposed as an important metric to evaluate implementation success of PROMs []; however, it has received limited attention thus far. The underuse of PROM data is concerning because the effectiveness of PROMs relies on clinicians acting upon the received data [-]. Moreover, clinicians’ explicit use is essential for encouraging continued patient participation in future PROMs [,].
While comprehensive overviews of barriers to and facilitators of PROMs implementation exist [,-], these often lack a specific focus on clinicians’ perspectives and behaviors [] and leave gaps in understanding how to respond to the identified factors. While some studies propose hypothetical strategies on the basis of identified barriers [,], insights from real-world experiences are limited. A few exceptions provide insights into clinicians’ experiences with PROMs [,-], their self-reported use [], and implemented strategies [,,]. Thus, there remains a critical need to learn from both successful and less successful implementation experiences aimed at enhancing clinicians’ use of PROMs [,,,].
CaseThis study draws attention to clinicians’ use of PROMs in a Dutch university hospital, Erasmus Medical Center (Erasmus MC), and the organizational-level strategies initiated to enhance this practice. Adopting a stepwise approach, the first specialty departments began collecting electronic PROMs from their outpatients around 2017, aimed at fostering VBHC [-]. New subdepartments continued to join this initiative, continuing into 2024. A central support team (CST) facilitated and coordinated this change, comprising around 6 full-time equivalent and an integrated IT team.
At Erasmus MC, patients complete PROMs before some of their outpatient consultations. To prevent survey overload, a maximum frequency for PROMs completion is set in agreement with subdepartments. This means that during outpatient consultations, clinicians will have a subset of patients for whom new PROM data are available. Clinicians are expected to review these data and discuss them with the outpatient to inform care decisions and actions. This process is facilitated by a consultation room dashboard that visualizes outcomes over time and compares them to reference values. This dashboard shows summary scores as trend lines and allows clinicians to access patient responses to specific items. Clinicians can access this dashboard, which is fully integrated into the patient’s electronic health record (EHR) since 2020, both during preparation for consultations and during the consultations themselves. For the first time, patients will receive feedback on their completed PROMs during their consultation. Individually, they can only access item-level responses without visualizations that aid in interpretation. In future, a separate patient dashboard is planned to offer these additional features.
[-] outlines the process of using PROMs, the specific tools used, and it provides a visual representation of the dashboard. In addition to the PROMs items, the patient survey includes an open-ended question designed to capture topics that the patients wish to discuss during their consultation. provides further information on dashboarding.
Figure 1. The process of patient-reported outcome measures (PROMs) completion and discussion (top-left), the 3 tiers of PROMs tools used (top-right), and an impression of the dashboard designs (bottom). For the generic tier, the hospital used the Patient-Reported Outcome Measure Information System (PROMIS; version 1.2; Global Health). Within the domain-specific tier, PROMIS short forms (SFs) were used among the nononcological population, including version 2.0, Physical function 4a; version 1.0, Anxiety 4a; version 1.0, Depression 4a; version 1.0, Fatigue 4a; version 1.1, Pain interference 4a; version 1.0, Sleep disturbance 4a; and version 1.0, Satisfaction with participation in social roles 4a. The European Organisation for Research and Treatment of Cancer Core Quality of Life Questionnaire (EORTC QLQ-C30 CAT) was used for the oncological population. Subdepartments could select additional disease-specific PROMs. By the end of 2023, PROMs were delivered to nearly one-quarter of the outpatient population from 70 subdepartments. However, clinicians’ use of PROM data remained limited. This is despite numerous strategies organically and pragmatically applied by the CST to facilitate and encourage use, including those previously reported as supportive [,,]. This discrepancy highlights the need for further investigation and makes Erasmus MC an intriguing case for investigation.
ObjectivesThe objectives of this study are 3-fold. First, we aimed to outline the issues with clinicians’ underuse of outpatients’ PROM data in Erasmus MC around 2023, focusing on adoption, implementation, and maintenance. Second, we aimed to develop knowledge on the organizational-level strategies implemented to enhance clinicians’ use of PROM data from 2020 to 2023. This includes understanding the underlying rationales for these strategies and identifying any strategies that appear missing to address barriers or leverage facilitators. Third, we aimed to investigate the key factors driving and constraining clinicians’ use of PROMs in 2023.
This approach helps understand how implementation unfolds in a real-life context using standardized frameworks to ensure that findings can be compared with other cases and replicated in different settings []. In addition, the findings may inform future strategies aimed at facilitating and promoting clinicians’ use of PROMs.
The study was conducted in accordance with the Declaration of Helsinki and exempted from formal approval under the Dutch Medical Research Involving Human Subjects Act by the Medical Research Ethics Committee of the Erasmus School of Health Policy & Management (ETH2223-0225). All participants consented to participation and the publication of quotes. Survey data were collected in a way that ensured participants remained anonymous, and interview data were pseudonymized. All data were stored in a secured space. Participants did not receive compensation.
Data SourcesOverviewThis study used 4 sources of data: implementation outcome indicators, a survey, hospital documents, and interviews. Data collection was part of a broader study on the implementation of VBHC []. Implementation outcome indicators were used specifically for objective 1, whereas the other data sources were used to address objectives 2 and 3.
Implementation Outcome IndicatorsThe hospital monitored the degree to which clinicians accessed the PROM consultation room dashboard. We extracted the percentage of completed PROMs accessed by clinicians from the hospital’s monitoring system, covering the 12-month period from February 2023 to January 2024. We used the aggregated data that eventually included 70 subdepartments. Investigating subdepartmental differences in the use of PROMs was outside the scope of this study. We analyzed both the average percentage of completed PROMs accessed by clinicians during the week of the patient’s consultation and on the day of the consultation. While the former indicates the use of PROMs to prepare for consultations, the latter serves as a proxy for discussion with the patient.
Hospital DocumentsInternal hospital documents were accessed through the digital workspace of the CST, to which author VvE was granted access. This workspace was established around 2020. A systematic search was conducted using the following keywords: compliance, evaluation, barrier, facilitator, challenge, plan, intervention, and strategy. This yielded 56 files. These were then screened for data pertaining to factors facilitating or impeding clinicians’ use of PROMs or strategies aimed at enhancing their use, followed by data extraction. If files referred to other files, these were also considered for inclusion. Subsequently, the list of extracted strategies was cross-validated with author MDH-A, a member of the CST, to distinguish between planned and executed strategies.
SurveyIn January 2023, a collective survey, named EMC23, was digitally distributed to all 194 clinicians across the 35 subdepartments that collected PROM data from outpatients in 2022. One clinician involved in the survey design was excluded from participation. The participants remained anonymous and untraceable. The survey comprised 26 items (), developed based on preliminary insight from document analysis. The items delved into clinicians’ self-reported use of PROMs, prominent factors influencing this, and their overall satisfaction with the implementation process and outcomes achieved. In addition, the respondents were asked to evaluate a subset of implemented strategies and had the opportunity to provide comments. Two reminders were sent. As shown in , 47 responses were included in the analysis.
Table 1. Data sources, sample sizes, and participant characteristics.Data source, theme, and descriptionValuesSurvey in January 2023 (N=194)aPROM: patient-reported outcome measure.
bVBHC: value-based health care.
InterviewsFor a process evaluation on the implementation of VBHC [], author VvE conducted 20 semistructured interviews with individuals involved in the central implementation effort during the past decade (). Two individuals refused participation for personal reasons. Participants were identified through documents and snowball sampling. Interviews were recorded and transcribed verbatim. Given the hospital’s focus on implementing PROMs in their move to VBHC, interviewees provided valuable insights into the hospital’s efforts and experiences in enhancing clinicians’ use of PROMs. The portion of the interview guide that elicited relevant comments for this study is presented in . We reanalyzed these transcripts, extracting information on barriers and facilitators for clinicians to use PROMs, strategies and their rationales, and areas for improvement. These data complemented the other sources mentioned in the Data Sources section, enriching the findings of this study.
Theoretical Lenses and Data AnalysisOverviewGiven the intricate nature of the implementation of PROMs, adopting a multifaceted research approach is imperative [], typically including contextual factors, the innovation (PROMs), strategies to support effective implementation of the innovation, and implementation outcomes interact, as illustrated in . These elements align with the objectives of this study. Therefore, our methods were plural, incorporating the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework []; the Expert Recommendations for Implementing Change (ERIC) taxonomy [] and associated 9 clusters []; and the COM-B constructs []. We reviewed multiple strategy taxonomies and evaluation frameworks as summarized by Nilsen [], selecting the aforementioned frameworks due to their alignment with this study’s objectives and data, widespread use, compatibility with the health care setting, and clarity in presenting results. These frameworks were used for post hoc, deductive data analysis; these frameworks are explained in subsequent sections.
Figure 2. Study objectives. PROM: patient-reported outcome measure. Objective 1: Implementation Outcomes Among Clinicians in 2023The RE-AIM framework [] guides the planning and evaluation of programs by examining 5 outcomes: reach, effectiveness, adoption, implementation, and maintenance. This study focused on the latter 3 outcomes at the individual level rather than at the department or hospital level. While the collection of PROMs among outpatients was a departmental choice, individual clinicians subsequently needed to adopt, implement, and maintain using PROMs. In this study, adoption denoted individual clinicians’ willingness to initiate the use of PROM data from their patients. Implementation denoted the consistency of clinicians’ use of PROMs as intended, that is, acknowledging outcomes and discussing results with the patient during their consultation to inform decisions. Maintenance referred to the clinicians’ ongoing use of PROMs. Reach and effectiveness have not been directly studied in this research but were indirectly addressed, as they are contingent upon the extent to which clinicians adopt, implement, and maintain PROM use [,].
We examined the outcomes adoption, implementation, and maintenance through survey data, reporting item-level frequencies for two key questions: (1) How frequently do you examine patients’ responses to PROMs? with the following response options: I never used PROMs, I stopped looking at PROMs, occasionally, as often as possible, and always; and (2) To what extent do you discuss these outcomes with the patient during the consultation? (for those who examine PROMs) with the following response options: never, occasionally, as often as possible, and always. We supplemented these data with insights from the hospital’s implementation outcomes indicators, for which no further analyses were required.
Objective 2: Implemented and Missing Strategies From 2020 to 2023We retrospectively analyzed Erasmus MC’s implementation efforts. First, we documented all strategies applied by the hospital from 2020 to 2023 based on document and interview data, focusing on understanding why these strategies were chosen, particularly in response to barriers or facilitators. For each strategy, we aimed to deductively specify the actor, action, target of the action, temporality, dose, and implementation outcome affected and provide a justification () [].
Besides describing the practical application of strategies at Erasmus MC, this study mapped discrete ERIC strategies [] to the identified strategies using a coding manual developed by Fridberg et al []. ERIC provides a compilation of 73 discrete strategies that one can use to effectively implement an innovation in a health care setting [], which have been mapped into 9 strategy clusters []. These clusters are mentioned in the Results section. The manual excluded the discrete strategy of facilitation due to overlap with other strategies. This study also included 10 strategies proposed as additions to the ERIC taxonomy: provide stakeholders with the possibility to attend educational meetings [], recruit clinicians with competence in the innovation [], provide stakeholders with resources [], act as a role model [], assess and redesign workflow [], create web-based learning communities [], engage community resources [], plan for outcome evaluation [], obtain worker feedback about the implementation plan [], and communication as a strategy category [].
To deepen our understanding, we included stakeholder reflections on strategies from documents, interviews, and survey data, including comments on their feasibility and perceived utility. We coded these data by mapping them to the respective strategy they pertained to, allowing us to cluster relevant information accordingly. Subsequently, we integrated these qualitative data into the results per strategy. The dataset was too limited to conduct thematic analyses on a per-strategy basis. Furthermore, we assessed whether identified barriers or facilitators in our data were addressed through these strategies. If not, these cases were labeled as missing strategies.
We observed that our data aligned effectively with the constructs of capability, opportunity, and motivation, which are components of the COM-B model []. Therefore, we structured our results around these constructs, finding them intuitive and straightforward for conveying our findings.
Objective 3: Key Factors Influencing Clinicians’ Use of PROMs in 2023On the basis of document analysis, we identified key barriers and facilitators to clinicians’ use of PROMs. In the survey, clinicians rated the perceived influence of these factors on their PROM use (). They also had the opportunity to identify additional factors.
We analyzed item-level scores for the entire survey population by grouping agree and partly agree into a single category. Likewise, disagree and partly disagree were grouped into a single category. The categories neutral and no opinion remained unchanged. Moreover, we conducted subsample analyses to explore differences among different user profiles. Clinicians were allocated to 3 groups depending on their self-reported use of PROMs: nonusers, frequent users, and occasional users. Nonusers included clinicians who reported never attempting to use PROMs or ceasing to use PROMs. Frequent users comprised those who reported using PROMs as often as possible or always. Clinicians who reported occasional use remained unchanged.
The hospital monitored that, on average, from February 2023 to January 2024, clinicians accessed PROM data for nearly 3 out of 20 patients (14%; SD 1.6%) on the day of each patient’s consultation. This data served as a proxy for discussing PROMs with the patient. During the week of the patient’s consultation, which included the use of PROM data for triage and consult preparations, this percentage was slightly higher, averaging at 19.5% (SD 2.4%). While variations in the use of PROM data were apparent across subdepartments and individual clinicians, these specifics were outside the scope of this study.
Yearly data suggested a slight, albeit minimal, increase in the percentage of completed PROMs of unique patients accessed by clinicians using the dashboard (). It is important to note that during 2024, the number of participating subdepartments grew from 38 to 70, implying that around half of the clinicians were in the adoption and implementation stages, while the other half were potentially moving toward the maintenance stage.
Figure 3. Patient-reported outcome measure (PROM) dashboard use rates among clinicians from February 2023 to January 2024. Survey DataClinicians’ self-reported use of PROMs, as derived from our survey in early 2023, revealed issues across all 3 RE-AIM outcomes. Adoption issues were apparent, with around one-tenth (5/47, 11%) of the respondents never attempting to review their patients’ PROMs responses (). Implementation issues, characterized by inconsistent use of PROMs as intended, became evident, with more than half (27/47, 58%) of the clinicians examining completed PROMs infrequently. Moreover, among those who examined PROMs, less than one-third (10/35, 29%) reported always discussing PROM data with patients. Less than one-fifth of the respondents (6/35, 17%) did not engage in this activity at all. Furthermore, 15% (7/47) ceased examining PROMs, indicating a maintenance issue.
Table 2. Clinicians’ self-reported use of patient-reported outcome measures (PROMs).Items and answersRespondents, n (%)Frequency of examining patients’ responses to PROMs (n=47)summarizes the strategies that the hospital implemented alongside their corresponding ERIC strategies, structured according to the COM-B constructs of capability, opportunity, and motivation. The hospital used 27 (37%) out of 73 discrete ERIC strategies to enhance clinicians’ use of PROMs alongside 5 proposed additions to the ERIC taxonomy. These are elaborated upon in the subsequent section and specified in .
Strategies were drawn from 8 (89%) out of 9 strategy clusters, including using evaluative and iterative strategies, providing interactive assistance, adapting and tailoring to the context, developing stakeholder interrelationships, training and educating stakeholders, supporting clinicians, engaging consumers, and changing infrastructure. No strategies from the cluster financial strategies were used. Qualitatively, it appears that clinicians who used PROMs more extensively rated the supportiveness of the initiated strategies higher than those who used PROMs less. A few strategies were found to be missing, leaving barriers unaddressed or facilitators not leveraged.
Tracking and generalizing strategy specifications, such as temporality and dose, for all strategies was challenging due to the organic and pragmatic application of strategies tailored to specific subdepartment needs and the lack of a comprehensive documented program theory. Having been developed during preceding pilot phases and, most of the strategies were available from the onset of the hospital-wide implementation program in 2020 and refined in the course of time.
Table 3. Strategies applied by the hospital, structured around the COM-B constructs of capability, opportunity, and motivation and corresponding strategies from the Expert Recommendations for Implementing Change (ERIC) taxonomy.COM-B construct targeted by the strategy and strategies by the CSTa, except otherwise notedERIC strategies (except when referenced and noted with Nb)CapabilityaCST: central support team.
bNumbered strategies as per Waltz et al [].
cPROM: patient-reported outcome measure.
dVBHC: value-based health care.
eEHR: electronic health record.
Capability-Associated StrategiesTo enhance clinicians’ knowledge and skills in using PROMs, the CST organized kick-off presentations at each department when PROMs were installed. Clinicians received written instructions on using the PROMs dashboard and interpreting outcomes via email, which were later supplemented with video instructions. In 2022, a pocket guide, with exemplary sentences to discuss PROMs was cocreated with clinicians (internal document dated July 14, 2022). Training possibilities, both internal and external, were highlighted. Internal training was often conducted in collaboration with the person-centered care group, focusing on using PROMs in shared decision-making and among patients with limited literacy. The latter aimed to address limitations from clinicians believing that “PROMs cannot be discussed with the patient in question” (internal document, July 14, 2022). Flexible learning resources, such as e-trainings, were developed in response to time constraints:
Attending a conference for half a day can already be challenging, let alone undertaking a lengthy training.In addition, efforts were made to align trainings with clinicians’ accreditation requirements.
However, interviewee 19 noted that training had not been an essential component of the program since 2020; interviewee 10 emphasized a tailored approach that addressed the specific needs of individual subdepartments. The CST increasingly reached out to subdepartments with low use of PROMs. They addressed barriers in perceived capability. One such barrier is clinicians feeling unable to influence certain PROMs outcomes that are affected by external factors, such as divorces impacting the quality of life. This perception led some clinicians to disregard these data (internal document, July 14, 2022). In response, the CST emphasized the importance of empathic listening and referring patients with critical PROMs outcomes to other physicians. Furthermore, since 2022, they provided on-the-job coaching, which proved valuable:
You must take them [a substantial portion of clinicians] by the hand for a moment. Where do you click?They also organized a few peer-to-peer discussion events each year, which interviewee 20 recognized as more impactful than traditional theoretical instruction. However, attendance issues arose with clinicians not always able to participate in these sessions. While survey results indicated that around 60% (26/43) of respondents were dissatisfied with the training or coaching received, skill issues were not a prominent factor impeding their use of PROMs ().
Table 4. Prominent reasons for clinicians to use patient-reported outcome measures (PROMs), both in total and as per 3 user profiles.Item, post hoc categories, and answer optionsAll respondents who used PROMs at least once (n=42)User profilesaThere are lower sample population values compared to the total population value because this question was posed to ceased users only (n=7), excluding those who never attempted PROMs (n=5).
Opportunity-Associated StrategiesAnother set of strategies targeted clinicians’ social and physical environments to create opportunities for using PROMs. Regarding clinicians’ social context, the CST aimed to create legitimacy and enable clinicians to discuss PROM data during multidisciplinary consultations, even with colleagues who had not yet initiated PROMs. They focused on raising hospital-wide awareness about PROMs and VBHC through mass communication via email, intranet, and marketing in public spaces. One clinician described a challenge in this endeavor:
Emails get buried under more important ones, and newsletters go unread. However, too little communication also doesn’t seem to work (you never do it right).Interviewee 14 also noted difficulties arising from the organization’s size, with hard-to-reach islands.
Anticipated future strategies include providing information about PROMs to general practitioners and external providers involved in patient care. Clinicians requested this strategy to facilitate communication with external health care professionals, such as making them aware of possible referrals on the basis of PROMs scores. However, this strategy was deemed feasible once PROMs were used among all patients, as partial implementation would require excessive manual effort (internal document, November 23, 2020; interviewee 19). Furthermore, a dashboard enabling patients to review their own outcomes will be developed to increase patient initiative in discussing PROMs:
If patients can see their outcomes, they’ll probably ask more questions about it. So, it will get used more by clinicals, both intentionally and unintentionally.Currently, to encourage patient initiative, patients are provided with resources such as an animated video explaining how to discuss PROMs during consultations.
Regarding the physical environment, several clinicians experienced or assumed PROM use in outpatient care as incompatible with their workload:
It takes far too much time, and we don’t get any extra.To mitigate this issue, the hospital sought to streamline the care process and simplify PROM use, as extending available consultation time was not feasible. Interviewee 10 explained as follows:
Small adjustments can make a real difference for a doctor who may not necessarily be receptive to changing their practice.As per interviewee 15, integration of the PROMs dashboard into the EHR was considered a major improvement. Two other features reduced the registration burden: quick copying of PROMs scores into patient notes and a referral template for easily referring patients needing further attention on the basis of PROM outcomes (internal document, May 11, 2023). However, 79% (33/42) of the survey respondents had no opinion on the usefulness of the referral letter, possibly indicating unawareness of this feature. In addition, the CST implemented a reminder by prominently displaying the PROMs completion status on the patients’ EHR front page. This also eased the workflow by eliminating the need to open the PROMs dashboard to verify completion. More than one-third (15/42, 36%) of the survey respondents perceived this feature as helpful.
Moreover, to save time, PROM data were used in triage and the developed PROMs-IT infrastructure was used to have patients report their medication and lifestyle. This extension reduced the necessity for discussions on these topics during consultations (internal document, May 11, 2023). Despite these efforts, perceived time constraints remained the most frequently cited barrier among clinicians as of early 2023 (). Loading time delays of the PROMs dashboard exacerbated these constraints, resulting in decreased commitment and temporary cessation of PROM use. These delays were resolved, and a dedicated campaign was organized to encourage the reuptake of PROMs.
Motivation-Associated StrategiesThe use of PROMs remained nonobligatory, focusing on genuine engagement rather than mere compliance. The inclusion of clinicians in the steering committee aimed to create “visible standard-bearers from within the healthcare sector” (internal document, May 20, 2019) to enhance buy-in. The CST frequently communicated about PROMs and VBHC to achieve and sustain interest. They shared stories highlighting the benefits of PROMs from both patients and clinicians via a monthly newsletter and intranet. One of the ideas was that “if it’s beneficial for the patient, it usually has added value for the clinician as well” (internal document, July 14, 2022). Stories of championing clinicians were noted as a key facilitator in documents and interviews (internal document, August 31, 2020; interviewee 3). However, approximately half (21/43, 49%) of the survey respondents did not feel that the stories of other health care professionals positively influenced their attitude toward working with PROMs, with only 16% (7/43)—primarily consistent PROM users—reporting that they did. In communication, the CST also emphasized the importance of clinicians’ discussion of PROMs on patient adherence to complete subsequent PROMs, supported by local evidence (internal document, June 7, 2023). Sometimes, the CST chose not to overtly reference VBHC, focusing instead on visible outcomes for clinicians, such as PROMs enhancing the patient-clinician conversation. Interviewee 3 explained as follows:
Sometimes your sales pitches need to be much flatter than what’s underneath it all.This adjusted framing aimed to address resistance among clinicians who viewed VBHC sceptically, perceiving it, for example, as a “dull, container concept” (interviewee 6) and “a hype” (survey respondent 11). As another reason, terminology shifts in the national policy created confusion and was overwhelming (interviewee 3). Nonetheless, some clinicians perceived the hospital’s communication efforts negatively. Survey respondent 60 mentioned, “extremely selective advertising,” and survey respondent 11 noted that “a hallelujah story is being told that overlooks the differences in patient populations.”
Concurrently, formal pressures increased: the executive board verbally committed to achieving VBHC, supported by video messages and integration into hospital aims. Externally, government programs supported PROMs and VBHC [,], and external audits increasingly evaluated PROM use (internal document, July 14, 2022). These factors directly motivated clinicians and empowered department heads to prioritize PROM use among their employees, according to interviewee 14. Since 2022, the executive board requested subdepartments to report on their VBHC implementation status thrice yearly and complimented high-performing departments (internal document, July 14, 2022). The CST proposed that the executive board explicitly request information on clinicians’ use of PROMs rather than merely their inquiries among outpatients (interviewee 19). However, this proposal has not been implemented thus far, and the reasons for this remain unknown to the CST. Survey respondent 11 expressed concerns about such an approach:
My biggest fear is that it [PROMs] becomes an outcome, that the departments that use it are deemed “better” than those that do not.This fear may have stemmed from clinicians’ belief that PROMs do not suit all patient groups or consultations equally (internal document, July 14, 2022).
To facilitate adoption, subdepartments were enabled to try PROMs with a subset of patients. Interviewee 19 highlights the importance of clinicians developing direct experience with PROMs:
[T]he true impact of PROM data becomes evident when you experience it.However, implementing PROMs among a subset of patients had an unintended consequence:
If we decide, “We’ll do it for X and Y but not for Z”... well, then you forget about it.Forgetting to use PROMs was the second-most cited barrier by clinicians (). To stimulate sufficient PROM survey volume, several strategies aimed to enhance patient response rates to PROMs. Furthermore, the CST aimed to evidence the benefits of PROMs and their impact on professionals’ work experience, including time investment. Interviewee 6 stated the following:
The key question is, do you have proof? That is the ultimate killer question.However, these efforts were hindered by limitations in defining measurable outcome measures, data availability, and difficulties in consolidating local evidence and making it convincing to other disciplines. Moreover, the hospital conducted both formal and informal evaluations of PROMs among clinicians to gain insights into their experiences, motivations, and needs, subsequently adapting implementation plans on the basis of these findings. This included prioritizing disease-specific PROMs, extending training opportunities, and expanding PROM data use to areas beyond outpatient care, such as triage, research, and care pathway improvements, tapping into different clinician motivations (interviewee 17). In addition, the CST alleviated clinicians’ concerns about accountability by adapting patient communication to clarify patients’ responsibility for timely contact in critical situations via traditional means.
To achieve and maintain effective implementation, the CST provided feedback on PROM use to subdepartments. The frequency and format of feedback was tailored to departmental preferences, with some favoring nonanonymous feedback, such as “[clinician’s name] has accessed this many PROMs” (interviewee 19), for a competitive aspect, while others preferred anonymity. However, nearly half of the survey respondents (20/43, 47%) experienced infrequent feedback. Furthermore, local departments were increasingly empowered to adapt PROMs and their application to their context, addressing limitations of uniform solutions on buy-in, perceived ownership, and local fit (interviewee 18). Subdepartments could incorporate disease-specific items that were deemed critical:
[a generic PROMs] is excellent for (big) data collection but lacks clinical value; it’s too general.Moreover, they could determine patient eligibility criteria, adjust inquiry frequency, and choose from multiple dashboard layouts (internal document, May 11, 2023). Individual clinicians were also enabled to manually exclude patients from PROMs if needed, aiming to mitigate challenges from upset patients:
The feeling of incompetence [to complete a PROM] among the less literate population takes a huge toll on their outpatient visits.Moreover, it aimed to overcome a specific drawback noted by survey respondent 4:
[Patient] inclusion in studies is less due to PROMs, so that irritates me.Overall, interviewee 20 perceived the following:
Once you give a team the feeling of autonomy or autonomy itself, then you already win 80% of the battle to get them to use it.Unaddressed barriers also appeared in relation to clinicians’ capability, opportunity, and motivation. Various interviewees highlighted shortcomings in contemporary education, noting a lack of emphasis on developing the professional attitudes necessary for effectively using PROMs in VBHC. This includes insufficient focus on understanding the impact of disease on the individual, their social environment, and work (interviewee 20), as well as limitations in shared decision-making, inclusive care, and data-driven improvement work (interviewee 19). Interviewee 14 pointed out that VBHC requires clinicians to take shared responsibility for patient outcomes rather than dismissing certain issues as outside their scope, observing that this attitude is “only really embraced by a few people.” Interviewee 18 emphasized another limitation in education, stating the following:
It has become checkbox medicine.... [Residents are] afraid to color outside the lines of those protocols, guidelines, and contractual agreements.In addition, there seem to be limitations in clinicians’ capacity for change and medical leadership (internal document, July 13, 2020). Interviewee 11 noted that clinicians are typically “quite conservative” and find changing behavior “very difficult.” Interviewee 20 observed that PROMs resonate more with nurses, who focus more on caring and benefit from longer consultation times, while physicians typically focus on curing. The CST found that departments with a dedicated nurse specialist or physician assistant achieved higher use of PROMs compared to those without such staff. Some subdepartments tasked these nurses with discussing patients’ PROM data and flagging any issues for the medical specialist who would see the patient next (internal document, May 11, 2023). However, the CST struggled to leverage this facilitator across the hospital due to budget constraints in local subdepartments to hire nurse specialists or physician assistants and the nonbillability of PROMs discussions by nurses. Overall, the CST believes that patients’ experiences of symptoms, functioning, and quality of life should be a central focus in all care activities rather than being narrowed to nurse consultations alone.
There also appeared limitations in the structural integration of PROMs for VBHC into the onboarding of new staff. Survey respondent 39 noted the following:
I started working here a year ago, there hasn’t been a structured introduction to this topic, I have no experience with it.Interviewee 19 explained that despite 2 attempts by the program team to include PROMs in onboarding, these efforts faced rejection due to PROMs not yet being uniformly adopted across all departments.
Behavior change was further hindered by limitations in opportunity and motivation. Clinicians’ contemporary feelings of work pressure and well-being were considered to have a negative impact:
People feel exhausted, like stretched rubber bands nearing their breaking point.... They think: “What now again? Will this add further strain to me?”Interviewee 19 noted the challenge of change fatigue alongside “a pervasive aversion to any form of registration and data handling.” A fine line existed between meeting data needs and experiencing an overwhelming availability of data. Interviewee 11 noted a lack of tension and urgency to use PROMs:
There is no fire. There are no patients dying if you don’t use PROMs.Interviewee 18 noted the challenge from the ongoing volume-based rather than value-based payment, impeding clinicians’ opportunity to use PROMs to make value-based decisions:
We ask people to play a different game of soccer without changing the rules. Resultantly, no one ends up playing differently.Despite implementing numerous strategies (objective 2), the hospital continued to face limitations in clinicians’ use of PROMs (objective 1). Therefore, as the third objective, we aimed to broaden our understanding of the key factors influencing clinicians’ use of PROMs in early 2023. We first highlight the primary motivations driving clinicians to use PROMs, followed by the key factors contributing to suboptimal use.
Reasons to Use PROMsSurvey respondents, excluding those who never attempted using PROMs, were asked to select the most prominent reasons for using (or having used) PROMs. Resp
Comments (0)