The LCME requires that “a medical school engage in ongoing strategic planning and continuous quality improvement (CQI) processes” (Element 1.1) [2], and further requires that the medical school report on elements chosen for monitoring in a structured manner using the Data Collection Instrument (DCI) [3]. We suspect that schools whose USMLE metrics are between the new benchmark and the national norm would be self-motivated to engage in CQI initiatives, and that schools with performance below the minimum benchmark during an accreditation visit will receive an adverse status classification for Element 8.4 [16] and be formally compelled to include USMLE performance in their CQI processes. However, we propose that even those schools already meeting USMLE benchmarks will also benefit from utilizing this reflection as a tool to stimulate discussion about curriculum design, strategic planning, resource alignment, and student support, given that USMLE considerations impact — and are impacted by — a school’s strategy and priorities.

To this end, we now address the medical school personnel responsible for USMLE-focused CQI. Because USMLE challenges and interventions are multifactorial, we recommend using a multidisciplinary, team-based approach that includes perspectives from students, staff, faculty, and administrators from across the medical education enterprise. A few key units or personnel that your school may engage, depending on your specific challenge, include representatives from accreditation, assessment and evaluation, curricular affairs and student academic support, student affairs, curriculum committee, course or clerkship directors, behavioral health, graduate medical education, registrar, finance, project management, and information technology.

We outline a series of reflection questions that your medical school team can use to determine its current compliance with the new USMLE norms of accomplishment and to reflect on a variety of domains that may be impacting exam readiness. The logic of this self-reflection employs the fishbone or Ishikawa diagram method [17], which has been successfully applied to determine the root causes of poor exam performance in the medical education space [18] (Fig. 1). Suboptimal school-level USMLE performance is the “problem” at the head of the fish and is broadly framed as “USMLE metrics below target.” After clearly articulating a USMLE performance concern, which will be school-specific (Stage 1), the causes of these USMLE performance issues can be carefully examined through discussion-based self-reflection (Stage 2). We present potential contributing factors (“sub-branches”), which we sort into a list of six broad domains (“branches”) that may underlie or contribute to below-target cohort-level USMLE performance: promotions policies, curricular alignment, curricular programming, diagnostic practices, provision of study resources, and academic support services. After completing this self-reflection, your medical school team is encouraged to use formal data-driven methods to test the causes identified as most probable, move to implement corrective actions, monitor the interventions to see if changes led to improvement, and adjust them accordingly.

Fig. 1

Fishbone/Ishikawa diagram summarizing potential causes (sub-branches) of below-target USMLE performance organized into six major domains (branches)

Stage 1: Defining the Problem

The first stage in this self-reflection is to generate your medical school’s precise problem statement. This process begins with analyzing your organization’s USMLE data against the LCME’s USMLE norms of accomplishment, as follows:

1.

Complete LCME DCI Table 8.4–3 [3] and LCME DCI Table 8.4–4 [3], which summarize the number of examinees as well as school-specific and national pass rates for Step 1 and Step 2CK, respectively, over the last 3 years. If the school’s passing percentage for first-time USMLE Step 1 test takers is below 85% for either of the last 2 years, or below 89% for Step 2 CK, the school is below the LCME’s national norm of accomplishment.

2.

Generate your initial problem statement. For example, “Academic year 2025 USMLE Step 1 scores are below the LCME norm of accomplishment.”

3.

Conduct subgroup analyses of USMLE pass rates and account for USMLE scheduling status (see Table 1) to get a more nuanced understanding of the problem. Organize this table by differentiating your student body into subgroups that you suspect might have meaningful differences in performance. For example, you may want to analyze by matriculation year if your school has recently gone through a substantial curriculum reorganization, or you may want to analyze by specialty programs (e.g., Pathway 1 students vs. Pathway 2 students). Is each subgroup where the school expects them to be relative to the USMLE timelines outlined for that class? Are there meaningful differences in the percentages of students who have attempted or passed the USMLE exam?

4.

Refine your problem statement according to your subgroup analyses. For example, “Only 70% of students in the cohort that matriculated in 2028 have sat for Step 1 by the time we suggested, and only 80% of those who took the exam passed.”

Table 1 Summarize USMLE Step 1 or Step 2 CK attempts by subgroup to analyze meaningful exam states leading up to a first-attempt scoreStage 2: Exploring Potential Causes of USMLE Metrics That Are Below Target

The second stage of this process is to explore the underlying causes that may contribute to the problem you outlined in Stage 1. The Ishikawa/fishbone diagram shown in Fig. 1 and described in detail below will guide you in reflecting as to how each of the six domain “branches” and the causes within them (“sub-branches) may contribute to your problem statement. Note that your medical school team may identify several potential causes contributing to the problem or that there are interactions among multiple causes. Read the introductory paragraph for each branch and consider your responses to the questions posed for each sub-branch:

Branch 1: Promotions Considerations

Use this branch to explore your medical school’s macro-level curricular policies that may interact with your students’ USMLE preparation. Consider that policies governing the timing of USMLE attempts have a profound effect on USMLE outcomes as well as student progress through the medical school curriculum [19, 20]. For example, requiring students to sit for USMLE Step 1 as an internal promotion gateway before students have had a sufficient opportunity to acquire and consolidate sufficient knowledge in the foundational sciences can lead to delays and/or lower pass rates.

Take/Pass Policies: Do students have to take and/or pass USMLE Step 1 and Step 2 CK for advancement and/or graduation (see LCME DCI Table 8.4–1 [3])? What is the rationale for these requirements? How might your program be affected if you changed any of these requirements to be more stringent or more permissive?

Timelines and Promotions Gateways: What are the timelines in the promotions policy that outline when the school recommends or requires students to take or take-and-pass each of the USMLE exams [14, 19,20,21,22,23]? What is your school’s rationale for creating these timelines? Do these timelines put students in the strongest position to sit for the exam when considering the content and cadence of the curriculum? What would be the effect of adding, removing, or changing these timelines on the program?

Delays: Do students typically sit for the exam when you recommend or require an attempt (i.e., what is your delay rate)? What are the policies and procedures that govern USMLE delays or re-attempts? Does your school prospectively collect and store data in the student information system about reasons for USMLE delays? Historically, what have been the reasons for delays? How can students request a delay? By whom and how quickly is the delay request adjudicated? What are the internal consequences of a delay (e.g., not allowed to start next phase of curriculum, financial aid implications)? Is there formal infrastructure (e.g., independent study courses, academic support) set up for students who delay or need to re-attempt?

Branch 2: Curricular Alignment

This branch will help you explore whether there is synergy between USMLE needs and other activities (e.g., preparing students for clerkships) within your formal curriculum. Considering the vast scope of physician education, many priorities compete for curricular footprint, including USMLE preparation. The degree to which USMLE preparation aligns with your medical school’s mission-specific curricular priorities can impact the way students spend their time [24].

USMLE Content: What role does the USMLE Content Outline [25] play in the development of curricular objectives for your school? How familiar are course directors and curriculum committee members with the USMLE Content Outline? What is the balance between “boards and wards” in curriculum mapping? What is the general attitude of curriculum planners at the school regarding USMLE preparation or “teaching to the test” [26, 27]?

USMLE-Style Assessment: How prominent are “USMLE-style” questions in your school’s internal assessments in both format and difficulty [28]? How much training do internal item writers have in developing USMLE-style questions? In considering how students spend their time studying, how are they incentivized to perform well on internal exams versus USMLE exams?

Dedicated Preparation Period: Does the curriculum provide students with a dedicated preparation period? If so, how long is this dedicated period? Has the school collected any feedback from students about the perceived utility of this time?

Branch 3: Curricular Programming

Medical schools can design the degree to which USMLE preparation is a part of the formal curriculum or a co-curricular activity. This branch explores the types of development your medical school provides.

Skills Development: What programming or resources does your school offer to support students in the following areas?

Exam overview (e.g., role of USMLE, exam format, when and how to schedule)

Skills related to exam preparation and testing (e.g., resource selection, study plan development and revision, exam pacing)

Review of USMLE content (e.g., knowledge of the cardiovascular system)

Exam-related wellness and stress management [29]

Format: In what formats is USMLE-related programming delivered (e.g., workshops vs lectures, asynchronous vs. synchronous, in-person vs virtual, small-group vs. whole-group)? Is the programming elective or required? What is the rationale underpinning the timing of when this programming is delivered? Who is responsible for the planning and delivery of this programming?

Parallel Curriculum: What tend to be attitudes regarding the “parallel” curriculum” (i.e., students using third-party resources) [30]? Are these resources welcomed and woven into the formal curriculum, or is their use discouraged?

Branch 4: USMLE Diagnostics

This branch explores whether your school gathers data to predict USMLE performance at various stages, and how you use that data to individualize the preparation of students or cohorts. Consider that there are meaningful quantitative predictors of USMLE performance well upstream of a USMLE attempt and additional predictors that become useful as students get closer to an attempt.

Early Intervention: Has your school conducted any studies to identify local predictors of USMLE performance? These may include pre-matriculation metrics (e.g., MCAT scores) [11], internal assessments (e.g., summative exams) [31], or student status information (e.g., students experiencing challenges in their life outside of school) [29]. How early can your school predict the likelihood of adverse USMLE outcomes? How quickly does your school intervene when a student likely to require additional support is identified?

Predicting Performance: Does your school administer any NBME Comprehensive Basic or Clinical Science Exams (CBSE/CCSE) and/or provide vouchers for students to take the NBME Comprehensive Basic or Clinical Science Self-Assessment (CBSSA/CCSSA) [32]? When are these administered relative to other key landmarks (e.g., at the beginning of dedicated study)? Is this done only for select students or for the entire cohort? Do proctoring practices for these practice exams align with USMLE norms (e.g., timing, testing center protocol)?

Data Analysis: How is performance on these assessments benchmarked? How is the data at the cohort level used to manage curricular programming? Is student-level data used to identify students who may need additional support? What performance information is shared with students (e.g., only individual data, cohort-level performance, historical data with predictive information)?

Branch 5: USMLE Resources

There is a wealth of resources devoted to preparing students for USMLE exams. The resources vary in quality and students need to be trained in how to select appropriate tools, how to incorporate them effectively into their studying, and how to use them alongside the medical school curriculum. This branch explores access that your medical school provides to these tools and how you approach training your students to use them.

Internal Resources: Does you school have any peer support resources [33]? These may include peer tutoring, organized study groups, or peer support groups. Who trains and supervises the peer tutors? How does the school know if the programming has been effective? Does your school develop internal resources to support USMLE preparation? These may include USMLE study plans that are mapped on top of the school’s curriculum, internally developed question banks, and content review resources. What training do students, faculty, and advisors receive in developing and utilizing internal resources effectively?

External (Third-Party) Resources: Does your school provide or facilitate any third-party resources specifically designed to support students in USMLE preparation? [34,35,36,37] These may include (i) resources that help with study organization (e.g., tools that generate student-specific study plans); (ii) access to USMLE question banks [38]; (iii) resources that cover USMLE-related content (e.g., spaced-repetition/flashcard platforms, video series, review books) [37]; (iv) USMLE preparation or remediation programs (e.g., in-residence programs, virtual preparation courses); (v) professional USMLE 1:1 tutoring services [39].

Access: What process does your school use to direct students to specific resources? If third-party resources are provided, are they purchased by the school, packaged into cost-of-attendance calculations for financial aid consideration, or supported using an alternative model? For resources purchased by students, do school administrators or student leaders organize initiatives to order in bulk and receive discounted pricing? Does your school monitor whether and how student and instructors are using the supported resources?

Branch 6: Academic Support Services

Advisors from academic support offices can help ensure that students requiring additional support for USMLE preparation receive it from informed professionals at appropriate points. This branch explores the expertise and capacity your medical school devotes to support students in USMLE preparation and how advisors manage your students’ needs.

Expertise: How do your advisors acquire and maintain expertise in USMLE preparation? How do they keep up with developments in exam content and study resources? Are there advisors who can cover all the programming, or does the school have individuals who specialize in specific areas (e.g., medical content tutoring vs study plan development)? What is the referral process for students requiring support beyond the advisor’s expertise?

Provision of Services: How do advisors identify students requiring additional support? Can students initiate the request for additional support? Who manages communication and touchpoints with the students in need of additional support? What services are these students eligible for above-and-beyond the general student body? Once students are offered support, is their participation elective or required? Are these services provided to individual students or to small cohorts? Are there any special conditions that students must agree to before receiving additional support or resources?

Capacity: Do academic advisors have sufficient capacity to support all students in need? Does that capacity align with surges in demand driven by intensive periods of USMLE preparation? What is the ratio of academic advisors to students? Can advisors access external resources for students requiring USMLE interventions that exceed the school’s internal capacity?