Study Design

This study used a multi-methods approach to inquiry, where qualitative coding methods were undertaken to describe narrative data from students, and descriptive statistics were calculated to quantify the codes. Quantitative analysis was necessary to distill and interpret our large narrative dataset of over 1000 statements.

Research Setting

This study took place at Oakland University William Beaumont School of Medicine (OUWB), an allopathic medical school with a class size of 125 per cohort. The medical program is 4 years long, with the majority of foundational biomedical sciences taught in the first 2 (preclinical, M1, and M2) years followed by 2 years of clinical rotations. The Individualized Online Lessons used in this study were offered during the fall semester (August through December) in the years 2020, 2021, and 2022. In 2020, during the COVID-19 pandemic, all preclinical lectures were delivered online due to the university’s physical distancing mandate. Since 2021, all preclinical courses have been delivered in a hybrid format, with some lectures in-person and others held remotely online. Each faculty member has the opportunity to intentionally choose the best modality for their learning objectives, resulting in a variety of delivery modalities used within a course. The current study describes and evaluates a novel method of asynchronous teaching basic medical sciences, using first year biochemistry, cell biology, and histology lectures as examples. Each of these disciplines was part of integrated foundational science courses and was interspersed with laboratories, case-based problem-solving sessions, team-based learning (TBL), patient panels, and synchronous lectures. Attendance was mandatory during instructional methods that relied on peer-to-peer teamwork and patients, while lecture attendance was generally optional, although the content was covered on assessments. Summative assessments in the form of midterm or final examinations were held approximately every 4 weeks, occurring within the respective preclinical course. The examinations were instructor-written, multiple choice assessments in the clinical vignette-style of the National Board for Medical Examiners [31].

Pedagogical Innovation: Asynchronous Lessons

The content and activities for the Individualized Online Lessons in this study were delivered using Moodle Lessons. While Moodle lessons were used at this institution, most learning management systems have analogous capabilities. Moodle (moodle.org) is a free, open-source learning management system (LMS) designed for education. Moodle Lesson is a tool in the LMS that allows the educator to custom-build content pages with branching scenarios. Content pages can display text, embedded multimedia resources, hyperlinks, and assessment questions (multiple choice, matching, true/false, short answer, or essay). Using Individualized Online Lessons, the teacher can pre-program which pages are displayed based on input from the learner. The teacher can view a report for each learner, including how they progressed through the Individualized Online Lesson in their individual scores on closed-ended assessment questions. Aggregate assessment data from all learners can also be accessed by the educator.

The authors each followed the same construction for the Individualized Online Lessons described in this study. Learners have some autonomy over how they view and interact with the asynchronous lesson. The workflow for our Individualized Online Lesson template is displayed in Fig. 2. First, learners are directed to view an Introduction page that displays the learning objectives, links to the eBook versions of required/optional textbooks, a PowerPoint file containing the slides from the instructional videos, tips for navigating the Individualized Online Lesson, and the teacher’s contact information. Next, the learner can view several mini-modules within the Individualized Online Lesson. A mini-module contains an embedded, short instructional video (5–15 min) with supplementary resources linked or embedded in the page allowing the teacher to “chunk” information for the learner rather than providing all session content as a single entity. We provided our instructional videos using YuJa (Yuja Inc., San Jose, CA), a video content management system designed for educational institutions. The platform can be used to stream, record, edit, store, and play videos. YuJa was selected for this study for its accessibility features, including a clear audio/video feed of the teacher and their computer screen, closed captioning, transcripts, user-controlled playback speeds, and embedding into the LMS.

Fig. 2

Individualized Online Lesson Structure and Workflow. Students first are presented with an introduction and description of learning objectives. Then they view several mini-modules containing short videos with supplementary learning materials. Students can choose to complete several formative assessments following each mini-module or advance to the next mini-module, or any chosen point within the lesson. Following the mini-modules and formative assessments, students can choose to submit a muddiest point where they will receive timely feedback from the instructor. Students can continue the dialogue as needed directly in the LMS

After each mini-module, learners have the option to complete formative assessment questions and immediately see their performance, with answer rationales. The questions ranged from assessing recall to the application of foundational concepts, using multiple-choice, matching, and short-answer formats. The learner can also decide to skip the assessment questions and progress to the next mini-module. Once the learner has finished the mini-modules, they are routed to the “Muddiest Point” prompt where they are encouraged to write about their most confusing point from the Individualized Online Lesson, following the prompt, “What is your muddiest (most confusing) point from this lesson? I will respond directly here.” The Muddiest Point was fully optional, and had no minimum or maximum word count. The Muddiest Point was created on the backend in Moodle using an “essay question” where the teacher can see the name of the student and their comment. The teacher can respond back directly through the LMS, and a copy of the teacher’s message is also sent to the learner’s email address. These questions are a private conversation between the teacher and learner, without an audience of their peers. The formative assessment questions and the “Muddiest Point” prompt in the Individualized Online Lesson allows for the teacher to provide individualized, just-in-time feedback to each individual learner. On average, students received a personal response to their Muddiest Point within 24–28 h. As seen in Fig. 2, learners can choose their own path through the mini-modules, formative assessment questions, and Muddiest Point, but at minimum are suggested to view the mini-modules. The Individualized Online Lessons were fully flexible. Although the mini-modules were designed to be consecutive and build concepts in a logical order, students could choose to complete them in any order and could re-access material and re-try activities on an unlimited basis.

Data Collection

This study was determined to be non-human subjects research by the Oakland University Institutional Review Board (IRB-FY2022-267) where the student activities and data examined were part of the planned curriculum. At OUWB, learners have the opportunity to complete an evaluation survey of a faculty member’s teaching when the faculty member has delivered their last session in a given course or clerkship. The survey instrument has five Likert-style items and three open-ended items. This study examined responses provided for the standard open-ended questions:

1.

Please comment on what the faculty member did well in the course.

2.

Please comment on what the faculty member could do to improve the instruction

3.

Please provide any additional comments you may have regarding the instructional materials used by this faculty member.

No questions asked the learners to reflect on our asynchronous Individualized Online Lessons specifically. Learners completed their surveys online from a location of their choosing using the OASIS (Schilling Consulting, LLC, Madison, WI) student scheduling and administration system, OUWB’s tool for giving and receiving learner and faculty evaluations. The data were collected anonymously.

Upon preliminary review of the raw data, we observed that it was common for a single narrative response from a learner to convey more than one distinct idea. During our second review, each learner’s narrative response was divided into distinct codable comments that represented a discrete idea or meaning. The comments were cut and pasted verbatim into a single Google Sheet (Google Inc., Mountain View, CA) and served as the units of analysis. In total, the data reflect the instruction delivered in three different teaching disciplines (biochemistry, cell biology, and histology), across 27 course offerings in the first year of the curriculum spanning the period between August 2020 to December 2022.

Data Analysis

First, the comments were sorted into the binary categories of “strengths” feedback and “constructive criticism” feedback. Descriptive coding was used to summarize the essence of each comment. A code is a short word or phrase applied to a portion of qualitative data to symbolically represent its meaning [28]. After independent initial review of the dataset, the authors met to create the initial codebook for the study. A codebook is a list of all codes, their definitions, and rules for when to apply or not apply the code. Each author thoroughly reviewed their own learner comments and applied codes as appropriate in Google Sheets. Coding one’s own evaluation comments was necessary during first-round coding. Some codes pertained to the level of detail of the content taught or content taught in other parts of the curriculum, which could only be accurately applied by the faculty subject matter expert. The authors met to discuss the first round of coding, and as a result some new codes had to be created. The dataset was reviewed again and adjustments were made to comply with the updated codebook. For a second round of coding, each author reviewed another author’s coded data. Coding disagreements or points of confusion were flagged by the second coder, and the discrepancies were discussed and reconciled during research team meetings. Where appropriate, some related (but still distinct) codes were grouped into larger categories. Descriptive statistics were calculated for each code and category.

Study Rigor

The low-stakes nature of the teaching evaluation forms contributed to the data’s authenticity. The responses were collected anonymously online, which fostered a safe and private environment where students could provide honest feedback without fear of repercussions. As the students were not prompted to write about the Individualized Online Lessons, we can infer that related comments were true thoughts that they wanted to share rather than fabricated responses to meet an evaluation requirement. Analysis of our own feedback from the students contributed to the trustworthiness of the results. Certain codes could only be accurately applied to the raw data by the faculty subject matter expert with knowledge of the curriculum and the context-specific jargon used by students. Finally, the use of second coders contributed to consistency in coding across the large dataset.