The NCSBN National Simulation Study

The NCSBN National Simulation Study: Results from Year 1 2012 NCSBN Scientific Symposium September 11, 2012 Arlington, VA

Study Collaborators Jennifer Hayden, MSN, RN NCSBN, Project Director

Pamela Jeffries, PhD, RN, FAAN, ANEF Johns Hopkins University

Suzan Kardong-Edgren, PhD, RN, ANEF Boise State University Washington State University

Nancy Spector, PhD, RN NCSBN

Study Phases

Phase I

• National survey of simulation use in prelicensure nursing programs

Phase II

• Multi-site, randomized, controlled study of three levels of simulation use across the nursing curricula

Phase III

• Longitudinal follow-up of nursing graduates into clinical practice

Phase II-Multi-site, randomized study of simulation/clinical experiences Purpose Determine if there are differences in clinical competency among graduating nursing students with 50%, 25% or up to 10% of traditional clinical hours substituted with simulation experiences.

Study Groups

 Control Group (clinical as usual) Up to 10% of clinical time can be in simulation  25% simulation in place of clinical  50% simulation in place of clinical

Research Questions 1. Are there differences in clinical competency among graduating nursing students in the three study groups? 2. Are there differences in knowledge among graduating nursing students in the three study groups? 3. Are their perceived differences in how well learning needs are met in the clinical and simulation environments among the three study groups? 4. Are there differences in clinical competency among the three study groups in each of the core clinical courses?

Null Hypothesis

There are no differences between the study groups in knowledge, clinical competency, and learning needs being met.

Substitution Rate  1 hour of traditional clinical is being replaced with 1 hour of simulation time  Time spent in a simulation lab either as an active participant or active observer of a simulated scenario followed by debriefing  This time in the simulation lab is counted towards required clinical hours for graduation

Core Clinical Courses Randomization will be maintained in the core clinical courses  Foundations  Medical/Surgical Nursing  Advanced Medical/Surgical Nursing  Obstetrical Nursing  Pediatric Nursing  Mental Health/Psychiatric Nursing  Community Health/Public Health Nursing

Study Will Evaluate:  Clinical competency Creighton Competency Evaluation Instrument (CCEI)

 Nursing knowledge ATI Comprehensive Assessment and Review Program

 How well student learning needs were met in the clinical and simulation environments Clinical Learning Environment Comparison Survey (CLECS)

Defining Simulation Operational definition: An activity or event replicating clinical practice. For the purpose of this study, simulation includes multiple types of scenario based simulation using high fidelity manikins, medium fidelity manikins, standardized patients, role playing, and computer based simulations.

NLN/Jeffries Simulation Framework

Study Management at each School  Team Leader  Study Team  Trained in the protocol  Trained in the debriefing technique  Large part in the development of the simulation curriculum

Clinical Faculty Training  Clinical faculty rate students each week of clinical and each day of simulation using the Creighton Competency Evaluation Instrument  Each semester clinical faculty undergo online training in the use of the instrument  Calculate inter-rater reliability and intra-rater reliability each semester  Clinical faculty review objectives and “norm” what are competent/not competent expectations

Simulation Day  Entire clinical group reports to the lab with the instructor  Assigned roles: primary nurse, secondary nurse, family member, documenter, med nurse, observers  15-20 minutes for scenario followed by 20+ minutes debriefing  Clinical instructors rate their students with the CCEI  2-3 clinical groups in the lab at the same time

Stations    

Manikin scenarios Standardized patients/patient actors Skills Stations Group Activities  Layered Learning Critical Thinking Cards  Sensory kits  Group Therapy in a Box

 Computerized scenarios  ATI Real Life  Virtual Clinical Excursions

Scenarios Available Number of Scenarios Available

Course Foundations

26

Medical-Surgical

64

Advanced Medical-Surgical

46

Obstetrical

42

Pediatrics

67

Psychiatric-Mental Health

24

Community Health

23 Total

292

Student “Safety”  Safety data collected each day of simulation and each week of clinical  Oversight Committee at each school Reviews school level safety data, mid-term grades, final grades, ATI scores, feedback from faculty and study team

 Data Safety Monitoring Board established Meets monthly to review school level data and national data to ensure participants’ education is not being jeopardized by being in the study

Participant Eligibility    

18 yo or older Provide informed consent Full-time RN student Complete the nursing program with study cohort Ineligibility  Licensed as LPN/VN or RN  Accelerated RN student

The Students Gender: Age:

Race:

86% Female 56% 18-24 28% 25-34 16% 35+ 83% White 8% Black 6% Asian 1% Native American 1% Hawaiian/PI 2% Multiracial

Ethnicity:

18% Hispanic

Student Demographics Previous Degree: 35% (295) 62% (181) Bachelor’s Degree 41% (120) Associate Degree 3% (9) Master’s Degree Certified Nursing Assistant: 16% Yes Paramedic/EMT: 19 Military Experience: 28 Yes 8 Medical Corps

Study Numbers 847 Students consented and randomized 411 ADN

436 BSN

54 Failed a course 39 ADN

15 BSN

96 Withdrew 40 ADN

56 BSN

697 Continuing on to Year 2 332 ADN

365 BSN

82% rate of continuing in the study

A closer look at attrition

Clinical Instructor Demographics (N=102)

Gender: Race:

95% Female

Ethnicity: Age:

9% Hispanic 20% Less than 35 24% 35-44 18% 45-54 39% 55+

79% White 7% Asian

6% Black 1% Native American

Clinical Instructor Demographics Teaching experience: 94% Yes average 9.0 years (SD 8.7) Current position: 43% Instructor 29% Adjunct faculty 13% Asst or Associate Professor 4% Professor 14% Other

Clinical Instructor Simulation Experience 21% No previous experience 55% Observed simulation 47% Run simulation scenarios 50% Debriefed students after simulations 38% Assessed students who have participated in simulations 23% Written simulation scenarios 33% Participated in formal simulation training

Clinical Hours Range of total clinical hours w/o capstone: 520-776 hours 25%= 130-194 hours of sim 50%= 260-388 hours of sim

What have we learned so far…  Study Model is working      

Use of a trained simulation team Clear objectives 15-20 minute scenarios Having clinical instructors observe Having students observe their peers Debriefing

 Skeptical faculty are now on board

Incidental Findings  Informal training/norming of clinical faculty-more inclusion of adjunct faculty  Bonding of students in their clinical group  Clinical instructors are using DML in their post-conferences  Johns Hopkins University—focus groups of clinical instructors and how this experience is informing their teaching

Study Timeline

Phase I

Phase II

Phase III

• October 2009 survey development began • January 2010 survey mailed • October 2010 results published

• November 2010 sites selected, first training meeting held • March 2011 research protocol finalized • April and June 2011 second and third training meetings held • Summer 2011 students enrolled into study • August 2011 semester 1 of study begins • May 2013 semester 4 of study ends

• April 2013 students consented for longitudinal follow-up study • May 2013 students graduate • December 31, 2014 data collection completed

Current Status of Phase II  Study Students are in the senior year of their programs  Students will graduate in May 2013  We will collect NCLEX pass rates through December 31, 2013  Final results are expected in Spring 2014

Looking Ahead to Phase III  Phase III will follow graduates of the study sites into clinical practice  Purpose of Phase III is to evaluate how well students were prepared for professional practice  Evaluations will include:     

Clinical competencies Safety competencies Job satisfaction Retention Stress

What can regulators take away  Who is conducting the sims? Do they have training, what kind?  What kind of debriefing?  How will students spend their time while in sim?  How long are sims?  Are there clear objectives for each sim?  Evaluations/graded?  Is the sim lab accredited?

www.ncsbn.org/2094.htm [email protected]