CDEM Voice – Research Column


SIMplifying Your SIMulation Research

As medical educators, we all have been involved with simulation during our careers, whether it was during medical school training, residency competency assessment, or leading high-fidelity case scenarios and debriefs with our clerkship students. While it may seem that simulation has effortlessly weaved its way into the threads of undergraduate and graduate medical education, it has required extensive research and external validation in order to gain widespread influence. We understand what makes a simulation case memorable and high yield from a learners’ perspective, but it is much more challenging to translate it into a well-constructed research design. In order to maximize your success with a simulation research project, it is important to understand the unique challenges of this educational research modality.

What is your study question?

The first step is formulating a study question that adds to the existing literature and is testable. Oftentimes, educators attempt to answer too many questions at once and leap to conclusions that are not supported by the results. A specific, testable research question and clear methodology allows for understanding, reproducibility, and implementation by its readers. For instance, instead of posing a broad question such as, “Does simulation improve airway skills?,” an investigator  could better phrase this as “Do dedicated airway simulation sessions using an intubating mannequin decrease the rate of missed intubations in emergency medicine residents compared to a conventional bedside teaching?” which provides a much more a specific objective and measurable outcome.

Can these findings be broadly applied?

A single simulation scenario may not be sufficient to answer an overarching question and may lose its applicability. One must create a diverse portfolio of clinical simulation scenarios if the goal is broad application. After careful literature review, the investigator must identify the deficit in the literature and understand the impact of this research on simulation and medical education. The investigator should identify the learner group under study and use scrutiny when suggesting to whom the results can be applied.

Can your research design be accomplished?

The significant resource utilization associated with a high-fidelity simulation project is often a mitigating factor in pursuing this form of research. If you don’t have access to high fidelity mannequins or commercialized trainers, however, it is still possible to conduct a simulation research project. “Do-it-yourself” or home-made simulation models using inexpensive common materials and some creativity have been well-described in the literature and have demonstrated success. Even when fortunate enough to have sufficient financial resources, time is arguably the other most valuable resource we have and it is difficult to dedicate time from one’s own schedule and gather participants to ensure adequate sample size. Be cognizant of financial, time and participant resources required for your project.

Is your assessment tool valid?

The investigator must understand the details regarding the construct validity of his or her assessment instrument and describe these clearly so others can be sure the data collected is reliable enough to make valid judgments. The use of a control group and details regarding the construct validity of the assessment instrument must be addressed during the development phase of the research project. There are many existing tools in the published literature that have already been externally validated and may be applicable to your project. If designing a new tool for your study, manuscript reviewers will expect that you address issues surrounding validity in its development.

What are the limitations of the research design?

Once the author understands the limitations and challenges of simulation research and their study design, targeted adjustments can mitigate these issues. For example, the ability to test cardiopulmonary physical examination skills may be limited by lack of authenticity. To mitigate these issues, high fidelity mannequins may be utilized and specialized simulation stethoscope to improve auscultation and allow the investigators to hear what the participant hears.

By addressing these questions before initiating a simulation research project, it will increase the chance for success in execution of the design as well as ability to publish this work. The ultimate goal is to increase student and trainee exposure to simulation, improve this educational modality and positively impact future patient care.



Leslie A. Bilello, MD
Instructor, Harvard Medical School
Assistant Program Director
Department of Emergency Medicine
Beth Israel Deaconess Medical Center


Nicole M. Dubosh MD
Assistant Professor, Harvard Medical School
Associate Clerkship Director
Department of Emergency Medicine
Beth Israel Deaconess Medical Center


  1. Bond WF, Lammers RL, Spillane LL, et al. The use of simulation in emergency medicine: a research agenda. Acad Emerg Med. 2007;14(4):353-63.
  2. Issenberg SB, Scalese RJ. Five Tips for a Successful Submission on Simulation-Based Medical Education. J Grad Med Educ. 2014;6(4):623-5.
  3. Daly R, Planas JH, Eden MA. Adapting Gel Wax into an Ultrasound-Guided Pericardiocentesis Model at Low Cost. Western J Emerg Med. 2017; 18(1):114-116.
  4. Sullivan GM. A primer on the validity of assessment instruments. J Grad Med Educ. 2011;3(2):119-20.
  5. 5.Warrington SJ, Beeson MS, Fire FL. Are simulation stethoscopes a useful adjunct for emergency residents’ training on high-fidelity mannequins? West J Emerg Med 2013; 14:275.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s