A novel simulation-based Pediatric Respiratory Distress Educational Curriculum (PReDEC) for emergency medicine residents
Article information
Abstract
Purpose
It is important for emergency physicians to be able to evaluate and manage pediatric respiratory distress, which is rare in general emergency departments. Despite this importance, the emergency departments show inconsistency in pediatric readiness, while emergency medicine (EM) residents express discomfort in caring for critically ill children due to limited exposure during their residency training. The purpose of this study was to meet this educational need by developing and implementing a curriculum to prepare EM residents to provide safe care for pediatric respiratory distress.
Methods
We prospectively assessed 20 senior EM residents after undergoing a 3-hour simulation-based curriculum at Indiana University School of Medicine. The curriculum was developed using the Kern’s methodology by content experts. Pre- and post-curriculum confidence, knowledge, competency, and situational awareness were assessed using the 5-point Likert scale, multiple-choice questions, checklists of a pediatric acute respiratory distress scenario, and a novel modified video version of the situational awareness global assessment technique, respectively.
Results
From the pre- to post-intervention phases, the confidence improved from 3.2 ± 0.4 to 3.9 ± 0.2 on the Likert scale (P < 0.001). The knowledge improved from 50.3% ± 12.9% to 75.3% ± 10.6% on the multiple-choice questions (P < 0.001). The competency improved from 46.8% ± 16.0% to 73.6% ± 10.6% (P < 0.001) with significant improvements in 7 of the 14 checklist items. The situational awareness raised both data perception (P < 0.001) and comprehension (P = 0.004) with no significant improvements in data extrapolation (P = 0.120).
Conclusion
Preliminary findings suggest that a 3-hour simulation-based curriculum for EM residents, including implementation of a novel situational awareness assessment tool, can improve learner’s confidence, knowledge, competency, and situational awareness of pediatric respiratory distress.
Introduction
Severe pediatric respiratory distress is a high acuity, low prevalence condition in general emergency departments (EDs), and the need for pediatric endotracheal intubation is 5 times less frequent than adult intubation (1,2). Additionally, pediatric patients who are intubated in general EDs have a longer length of stay while awaiting transport to free standing children’s hospitals, worsening outcomes (3,4). Although 90% of pediatric visits occur in general EDs, most of the EDs have shown inconsistency in both pediatric readiness scores and patient outcomes (5,6).
Despite expectations that emergency medicine (EM) residents must be trained to evaluate and treat pediatric emergencies, EM graduates have shown discomfort with many facets of pediatric critical care, including rapid sequence intubation and ventilator management (6,7). This is likely in part secondary to lack of exposure to critically ill pediatric patients, with EM residents seeing a median of only 6 pediatric patients with an Emergency Severity Index of 1 during their training (6). Poor pediatric readiness and EM physicians’ discomfort in caring for critically ill pediatric patients pose a vital area for improvement in EM training. This, in combination with worsening ED overcrowding and high-volume respiratory illness outbreaks, highlights a need for further training relating to pediatric respiratory distress (5-7).
Given low volume of critically ill patients seen by EM residents, simulation is an important educational modality to fill this gap. Simulation has been shown to effectively foster and improve critical care resuscitation, invasive procedure safety, crisis resource management, and ventilator management competencies (8-11). Notably, the Accreditation Council of Graduate Medical Education (ACGME) mandates the use of simulation in graduate medical education training (12), and simulation-based education has been shown to outperform traditional teaching in instruction of critical care competencies (11). While there are studies of training for pediatric respiratory distress at the undergraduate medical education level, there are no validated curricula for pediatric respiratory distress management for EM residents (13,14).
In this light, we aimed to develop, implement, and assess a Pediatric Respiratory Distress Educational Curriculum for senior EM residents. The objectives of the study were to test whether the curriculum could render the residents familiar with assessing changes in pediatric respiratory distress, skillful in ventilator management, understanding and appreciating the human factors for the learner’s performance, and able to apply non-injurious lung protective ventilator settings. Full details of goals and objectives are provided in Appendices 1 and 2 (https://doi.org/10.22470/pemj.2024.01109). We hypothesized that the use of a novel curriculum in conjunction with a simulated pediatric respiratory distress scenario would improve participant confidence, cognitive knowledge, simulated competency, and situational awareness with pediatric respiratory distress.
Methods
1. Study location and equipment
This prospective study was conducted from February through March 2022 in a dedicated 30,000 square-foot simulation center affiliated with Indiana University School of Medicine in the United States. We used a standard simulation room, a Maquet Servo-iTM ventilator (Maquet), a Laerdal’s SimBabyTM mannequin (Laerdal Medical), and an Ingmar Medical ASL 5000TM lung simulator (Ingmar Medical). The experimental environment has been previously validated and described in detail (8,9). We also used a set of supplies listed in Appendix 1. The study received an exemption waiver by the institutional review board at Indiana University School of Medicine (IRB no. 13793).
2. Participants
Participants were recruited from an ACGME accredited 3-year EM residency training program in the Midwestern U.S. The participants practiced in 2 level-1 trauma centers with over 90,000 annual visits each, as well as a level-1 pediatric trauma center with over 40,000 annual visits. Individuals were eligible to participate if they were in the latter half of their residency (post-graduate year 2-3) and had completed a required 4-week pediatric critical care rotation. We recruited 20 participants via emails and word-of-mouth with a faculty facilitator to participant ratio of less than 1:4. The facilitators were recruited from the Department of EM and the Division of Pediatric Critical Care Medicine of the Department of Pediatrics at Indiana University School of Medicine. The faculty were chosen based on their expertise and clinical context to manage pediatric respiratory distress. Technical support was provided by simulation specialists and technicians from The Simulation Center at the Fairbanks Hall in Indianapolis, IN, U.S.
3. Curriculum development
We developed a curriculum based on the Kern’s 6-step method, using an inter-professional needs assessment approach using structured interviews of pediatric emergency physicians, emergency physicians, and pediatric intensivists with regards to pediatric respiratory distress (15). We created a series of formative scenarios using a constructivist framework (16), using debriefing with good judgment and deliberate practice (17,18), combined with a didactic lecture on human factors in medicine (Appendix 2). The participants did not receive any debriefing or formative feedback after the assessments.
4. Educational intervention
The curriculum design process was targeted teaching senior residents with an intermediate level of expertise on pediatric respiratory distress, defined as having completed the pediatric critical care rotation and pediatric EM rotation. We focused on interpreting data from advanced respiratory monitoring sources (end-tidal CO2, ventilator, and advanced physical assessment), critical resource management, and applying the human factors to pediatric care in the ED (Appendix 2). The human factors specifically refer to the interaction between the individuals, teams, their equipment, and the environment in which they work. We used a constructivist learning model to apply these advanced concepts to the base of existing knowledge and competency derived from aforementioned learner clinical experience (16). The intervention involved lectures, formative simulation, and mixed-methods interactive exercises. Among these interventions was a rapid cycle deliberate practice set of scenarios (Appendix 2) designed to follow a ladder for troubleshooting post-intubation hypoxia (Fig. 1). Learners completed the 3-hour curriculum assessed by faculty from the Department of EM and the Section of Pediatric Critical Care at the simulation center.
The initial assessment of intubated hypoxia in pediatric patients. Modified from Pacheco et al. (20) with permission of Elsevier. ABCs: airway, breathing, and circulation, ETT: endotracheal tube, POCUS: point-of-care ultrasound.
5. Data collection
1) Pre-intervention assessment
A pre-intervention assessment was performed gathering data on the confidence, knowledge, competency, and situational awareness regarding pediatric care and respiratory distress. The confidence was assessed using a 5-point Likert scale, derived via expert opinion (Appendix 3, https://doi.org/10.22470/pemj.2024.01109). For the knowledge domains, 15 multiple-choice questions (MCQs) were developed and piloted on recent EM graduates. A performance competency checklist comprising 14 critical actions in a summative scenario was derived from the 2015 Pediatric Acute Lung Injury Consensus Conference guidelines for pediatric acute respiratory distress syndrome (19) and from Pacheco et al. (20).
2) Modified video situational awareness global assessment technique (vSAGAT)
We modified the previously validated situational awareness global assessment technique (SAGAT) by using trigger video scenarios, i.e., vSAGAT, followed by 5 sets of 3-level MCQs upon completion (15 questions in total) (21-26). Specifically, situational awareness was assessed using the vSAGAT where we used the trigger videos and 5 sets of 3 difficulty tiered-questions (15 questions in total). This video modification enabled increased reproducibility and decreased the time to completion from an estimated 20 minutes to 5 minutes. The questionnaire was piloted on pediatrics and EM experts, refined after several iterations, and used with the accompanying videos (Appendix 3).
3) Post-intervention assessment
After the curriculum intervention, the same assessment tools as above were used to evaluate the confidence, knowledge, competency, and situational awareness of the participants. Of note, the same simulation case was used both pre- and post-intervention without receiving an interceding debriefing. Additionally, the participants were asked to provide feedback on opportunities to improve this novel curriculum (Appendix 3). The total assessment time was approximately 1 hour, including the asynchronously completed MCQs.
6. Data analysis
We reported categorical variables using frequencies and percentages. Continuous variables were reported as means ± standard deviations. Differences between pre- and post-scores were estimated using a paired t-test. For checklist completion, Fischer’s exact tests were used. All statistical analyses were done using SAS ver. 9.4 (SAS Institute).
Results
Twenty post-graduate years 2 (n = 5) and 3 (n = 15) EM residents participated in the study, of whom 10 were women, and another 10 had completed 2 pediatric critical care rotation.
1. Confidence, knowledge, and competency levels
The participants felt confident with their ability to manage pediatric respiratory distress following training with improvement of the mean values in the 5-point Likert scale increasing from 3.2 ± 0.4 in the pre-intervention phase to 3.9 ± 0.2 in the post-intervention phase (Table 1). The cognitive knowledge MCQs scores improved from a mean of 50.3% ± 12.9% in the pre-intervention phase to 75.3% ± 10.6% post-intervention. The competency, which was measured by successfully completed critical actions during a simulated case, improved from a mean of 46.8% ± 16.0% to 73.6% ± 10.6% post-intervention. Of note, 7 of the 14 critical actions demonstrated significant mean performance improvements (Table 2).
Mean scores for pre- and post-intervention assessment
Table caption
2. Situational awareness
Situational awareness, as measured by the vSAGAT, showed improvement in several domains. In level 1, data perception, there was an increase in the mean number of correct MCQs, from 2.2 ± 1.1 pre-intervention to 3.7 ± 1.0 post-intervention. In level 2, data comprehension, there was an increase in the mean number of correct MCQs from 2.6 ± 0.8 pre-intervention to 3.6 ± 1.1 post-intervention. In level 3, data extrapolation, no change was observed.
Discussion
To our knowledge, there are currently no validated curricula for teaching pediatric respiratory distress to EM residents, and no prior studies that show incorporation of pediatric mechanical ventilation training into the simulation (9-11,13,14). We demonstrated efficacy of a novel curriculum designed for senior EM residents in the simulated environment. The combination of simulation-based scenario training, bedside instruction, lectures, and procedural training improved learner’s confidence, knowledge, competency, and situational awareness regarding the pediatric respiratory distress.
Pediatric respiratory distress is a complex medical condition that involves multiple components of critical care and situational awareness. Given this complexity, our educational objective was to help enhance hands-on skills rather than didactic training in novice learners. For this reason, we tailored our inclusion criteria to focus on residents who would be in an established zone of proximal development, or who would be capable of learning this information (22). Given that half of our participants had completed multiple pediatric critical care rotations, it was not surprising that many participants had high baseline levels of performance on task domain completion. Despite the initial high levels, there was still significant improvement in the half of all domains (Table 2), showing effectiveness of the curriculum related to the competency.
In designing our curriculum, we focused on constructivist theory, allowing for the residents to build knowledge throughout each subsequent component (16). This included the use of rapid cycle deliberate practice in multiple areas of the curriculum (Appendix 2) to help with the confidence and knowledge gains, which was shown to effectively teach critical concepts (27). To our knowledge, this is the first application of the rapid cycle deliberate practice in the setting of troubleshooting pediatric ventilator. Throughout the progression of cases, the participants were given less deliberate feedback and were encouraged to troubleshoot each scenario on their own based on the preceding instruction. This led to some unexpected findings. Notably, in the piloting of this curriculum, a pattern emerged that the residents disconnecting the mannequins from the ventilators and performing a bag-valve-mask ventilation via the endotracheal tubes. This action led to a deduction on checklist the item 7, “Does NOT disconnect patient from ventilator to bag (suction okay).” During the debriefing, the participants consistently explained that they decided to disconnect the mannequins from the ventilators owing to a concern for equipment malfunction or barotrauma. The faculty facilitator felt this decision to be reasonable. Future iterations of the curriculum may revise the item 7 to “Participant disconnects the circuit, checks equipment functionality, and ventilates the patient.”
Confidence, content knowledge, and skill competency are important factors in managing a pediatric patient in respiratory distress. However, emergency physicians may need to consider human factors associated with resuscitation such as situational awareness (22,23). While universally recognized as a critical element for providing effective care, we found limited evidence on how best to improve situational awareness (24-26). We chose to develop and deliver a lecture on human factors in medical resuscitation from content experts as part of our curriculum. To assess the effectiveness of instruction on the awareness, we modified the SAGAT to fit our assessment needs.
The SAGAT has traditionally been used with in-person simulations and has been successful in measuring situational awareness of EM providers and pediatric critical care providers (22-26). Modifying the SAGAT by introducing video-triggered scenarios, i.e., vSAGAT, allowed our group to have increased flexibility in assessment and instructional methods, which otherwise would not have been feasible. The findings from the vSAGAT demonstrated that we were able to improve situational awareness levels, likely through improving the knowledge, skills, and attitudes, and using a constructivist framework. We were able to improve the participant’s levels 1 and 2 of situational awareness, data perception and comprehension, respectively. Notably, the vSAGAT did not increase the level 3, data extrapolation. This might be explained by differences in cognition, or by the need for learners to experience automaticity, which indicates the ability to perform a task unconsciously in the field. Further studies are needed on how best to improve situational awareness in the simulation environment (28,29). While the video modification cannot be validated in this preliminary study, the use of vSAGAT warrants further investigation moving forward.
Our study has several limitations inherent to simulation research. First, the simulated context represents a potential confounder in that emergency physicians may be less careful than when having patient contact. Nevertheless, simulation offered access to events that cannot otherwise be observed directly in a safe and controlled environment. Second, the curricular evaluation was executed at a single institution with a cohort of the 20 residents. However, this number is comparable to the total number of senior residents at other programs across the U.S. who have completed a mandated ACGME pediatric critical care experience. Third, this study piloted this curriculum. Studies aimed at the retention or longitudinal effects of this study may be of interest. Lastly, the vSAGAT was modified from the SAGAT for more flexible application. Although the vSAGAT was thematically matched with the preexisting SAGAT, the vSAGAT has not yet been validated in this realm. Lastly, our curriculum benefitted from the advanced technology, and experienced simulation faculty and staff that may not be available at other simulation centers. One of the great barriers to ventilator management simulation is the relatively high financial expense, as well as significant time commitment by both simulation faculty and pediatric resuscitation experts. Future areas of research should consider curriculum design for low-resource educational settings that replicates the objectives and outcomes of this curriculum.
Notes
Author contributions
Conceptualization, Data curation, and Visualization: AC
Formal analysis: AC, ES, and RA
Investigation: PB, AB, LF, DC, MM, and AC
Methodology: PB, AB, LF, DC, MM, AC, NN, and MB
Project administration: AB, LF, DC, MM, AC, NN, and MB
Resources: ES, RA, and DC
Software: ES
Supervision: SA, NS, JE, and KA
Writing-original draft: AC
Writing-review and editing: SA, NS, JE, KA, PB, AB, LF, DC, MM, AL, NN, MB, and RA
All authors read and approved the final manuscript.
Conflicts of interest
No potential conflicts of interest relevant to this article were reported.
Funding sources
No funding source relevant to this article was reported.
Acknowledgements
This project was made possible through the work of the simulation specialists at Indiana University School of Medicine, Fairbanks Simulation Center, specifically John Cartwright and Chris Weidman. This work was presented at the International Meeting for Simulation in Healthcare in 2023, and at the Society for Academic Emergency Medicine Simulation Academy’s Fellow Forum in 2022. This work won best Simulation Research Project at the Simulation Academy’s Fellows Forum, listed above.
