A systematic review and meta-analysis of studies on extended reality-based pediatric nursing simulation program development

Article information

Child Health Nurs Res. 2023;29(1):24-36

Publication date (electronic) : 2023 January 31

doi : https://doi.org/10.4094/chnr.2023.29.1.24

Eun Joo Kim ¹

, Ji Young Lim^,²

, Geun Myun Kim ³

¹Associate Professor, Department of Nursing, Gangneung-Wonju National University, Wonju, Korea

²Professor, Department of Nursing, Inha University, Incheon, Korea

³Professor, Department of Nursing, Gangneung-Wonju National University, Wonju, Korea

Corresponding author Ji Young Lim Department of Nursing, Inha University, 100 Inha-ro, Michuhol-gu, Incheon 22212, Korea TEL: +82-32-860-8210 FAX: +82-32-874-5880 E-MAIL: lim20712@inha.ac.kr

Received 2022 November 17; Revised 2023 January 1; Accepted 2023 January 4.

Abstract

Purpose

This systematic literature review and meta-analysis explored extended reality (XR)-based pediatric nursing simulation programs and analyzed their effectiveness.

Methods

A literature search was conducted between May 1 and 30, 2022 in the following electronic databases: MEDLINE, Embase, Cochrane Central Register of Controlled Trials (CENTRAL), and CINAHL. The search period was from 2000 to 2022. In total, 6,095 articles were reviewed according to the inclusion and exclusion criteria, and 14 articles were selected for the final content analysis and 10 for the meta-analysis. Data analysis was performed using descriptive statistics and the Comprehensive Meta- Analysis program.

Results

XR-based pediatric nursing simulation programs have increased since 2019. Studies using virtual reality with manikins or high-fidelity simulators were the most common, with six studies. The total effect size was statistically significant at 0.84 (95% confidence interval=0.50-1.19, z=4.82, p<.001).

Conclusion

Based on the findings, we suggest developing standardized guidelines for the operation of virtual pediatric nursing simulation education and practice. Simultaneously, the application of more sophisticated research designs for effect measurement and the combined applications of various virtual simulation methods are needed to validate the most effective simulation methodology.

Keywords: Systematic review; Pediatric nursing; Simulation training; Virtual reality; Education

INTRODUCTION

As we enter the era of the Fourth Industrial Revolution, the need for education that fosters complex problem-solving skills and convergent and creative thinking, as learners become subjects, has increased in the field of education [1]. In particular, the transition of the educational sphere to the digital generation has accelerated, leading to the rapid emergence of non-face-to-face class arrangements. For this reason and because of advances in information and communication technologies (ICTs), education sites are shifting away from traditional classroom methods and class designs are blending online and in-person learning activities. Furthermore, online platform-based remote lectures are gaining importance as an innovative model of education [2].

Extended reality (XR) is an innovative technology that has changed how humans understand and interact with digital information. The use of XR is expanding significantly through technological development and the accelerated digital transformation. Many attempts have recently been made to utilize the metaverse, which combines virtuality and reality, in education [3]. XR is an interface that connects reality and the virtual world, facilitating the coexistence of reality and virtuality and resolving the physical limitations of the real world [4]. In other words, technologies (virtual reality [VR], augmented reality [AR], mediated reality [MR], and hyper-reality) that enable communication via the realistic coexistence of the virtual world and reality maximize users' immersion, resolve the constraints of actual space, and connect and fuse virtuality and reality. As such, with the rapid development of these medical technologies and ICTs in recent years and the increasing severity and diversity of diseases, changes in nursing education are required to train future nurses who demonstrate these abilities related to ICTs.

Nurses are professionals who start working in the clinical field after graduation from nursing school. For them to acquire practical skills, both theoretical education and a certain amount of clinical practice are mandatory. However, the education of nursing students in the clinical field is limited, because most of it involves indirect rather than direct experiences in situations where patient rights and safety circumscribe the possibility of hands-on involvement. Specifically, because the field of pediatric nursing encompasses patients ranging from newborns to adolescents, the learning goal is to identify their health issues and to communicate and practice nursing appropriate for patients' developmental levels [5]. In 2020, South Korea's fertility rate was 0.84 [6], the lowest among the Organization for Economic Co-operation and Development countries. Clinical practice in the neonatal ward is challenging because of the low birth rate and risk of infection from infectious diseases such as coronavirus disease 2019 (COVID-19). In addition, despite the need to continue education in neonatal wards and involving high-risk newborns, it is difficult for students to have these experiences in person or make indirect observations as more hospitals scale down or close neonatal intensive care units. Regardless, realistic clinical sites are needed to provide educational opportunities for nursing students. An emerging alternative in this regard is simulation practice. Simulation exercises in South Korea have been prevalent since 2011. However, it remains challenging to implement vivid hands-on experiences in the nursing field as simulations despite extensive simulation exercises [7]. It seems necessary to develop a smart laboratory in which actual clinical practice can be conducted by combining virtuality and reality to strengthen the ability to materialize the actual clinical field in a simulation.

Most simulation exercises currently used in nursing programs are operated with scenarios developed by instructors as practice using simulators or standardized patients (SPs). The field of pediatric nursing mainly uses simulators because of the characteristics of its target patients. Recently introduced simulators are contributing to improving the quality of education, but their limitations include their high costs and inadequate realism for monitoring and other related situations [8]. Moreover, it is not possible to use simulations to attempt integrative nursing activities in the clinical field, where integrated thinking is required, because the scenarios are organized around diseases or symptoms. Instructors consider this educational method to be the most similar to the actual environment; however, it is unlike the actual environment from a learner's viewpoint and may lack the effect of learning immersion [9]. The realism of a simulation exercise is related to whether the targets are well-constructed and how faithfully the actual situation is reflected in the scenario [10]. Although some improvements are being made in the recently introduced high-fidelity simulators, the diverse and complex circumstances in the clinical field are not yet being reflected in an integrative manner. Improving these factors necessitates not only well-structured scenarios, but also a simulation environment that closely reflects clinical sites.

Considering these points, a simulation exercise model with a new platform of a smart environment that reflects academic characteristics and a changing environment is needed. To this end, it is necessary to identify the current status of simulations in nursing education, actively utilize the programs available for these purposes, and integrate these efforts. Although attempts are being made to use XR in the field of pediatric nursing [11,12], these attempts have been limited to building a smart environment and creating an integrated environment for nursing exercises based on these results. Therefore, this study aimed to systematically review studies on the development of XR-based pediatric nursing simulation programs, including AR, VR, and MR, assess the current situation in terms of development and effectiveness, and use the findings as a basis for building an environment for a pediatric nursing practicum using XR.

The aim of this study was to systematically review and meta-analyze research on the development of XR-based pediatric nursing simulation programs and assess the current status of their development, effectiveness, and limitations.

METHODS

Ethics statement: The Institutional Review Board (IRB) of the Gangneung- Wonju National University (GWNUIRB-R2022-24) reviewed this study. The committee decided that this study was an exempt research study.

1. Design

The present study is a systematic review and meta-analysis of the development and effectiveness of XR-based pediatric nursing simulation programs for nurses and nursing students. This study followed the criteria of the Preferred Reporting Items for Systematic reviews and Meta-Analyses 2020 (PRISMA 2020) checklist [13].

2. Key Questions and Selection Criteria

The participants, intervention, control, outcomes, and study design (PICO-SD) framework was used for this study, as follows: The participants of this study were nurses or nursing students who took part in studies of XR-based simulations for pediatric nursing. The interventions comprised XR-based simulations for pediatric nursing, and the control group and outcomes were not specified. The eligible study designs included randomized controlled trials, quasi-experimental studies, and similar designs. The inclusion criteria were as follows, and studies that did not meet the selection criteria were excluded:

Inclusion criteria

∙ XR application programs including VR, AR, and MR

∙ Programs applied to nurses or nursing students

∙ Studies that conducted an effectiveness analysis

∙ Studies that included interpretable and analyzable data

∙ Papers published since 2000

Exclusion criteria

∙ Studies that applied simple simulations not based on XR

∙ Studies whose participants were not nurses or nursing students

∙ Paper published in 1999 or earlier

∙ Dissertations or proceedings papers

3. Literature Review and Query Settings

The literature review and selection processes were performed according to the guidelines of the Cochrane Collaboration's PRISMA 2020.

1) Literature review

The primary search sources were electronic databases, namely MEDLINE, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL), and CINAHL. Google Scholar was also searched to procure as much gray literature as possible. The literature search was conducted from May 1 to 30, 2022, and the search period was set from the oldest publication year provided in each database to the date of the search in 2022.

2) Query settings

According to the PICO-SD parameters, the search term was "(nurse or nursing student or nurs*)" and "(reality or holo*) and (simulation) and (pediatric or child or children or baby or newborn or infant or kid*) and (RTC or trial* or experimen* or quasi or effect)".

4. Literature Selection Process

The literature selection was performed according to the PRISMA flow diagram, the results of which are shown in Figure 1.

Figure 1.

PRISMA flow diagram of studies on XR-based pediatric nursing simulation programs. *Reason 1: If a simple simulation was applied, it was not based on extended reality or child nursing. Reason 2: Participants were not nurses or nursing college students. Reason 3: Studies on the evaluation of the process without an effect analysis. Reason 4: Studies that did not include interpretable and analytical data.

5. Literature Quality Assessment

The articles selected for the systematic review of this study were assessed for quality in each area of study design using the Risk or Bias and Risk of Bias in Non-randomized Studies of Interventions quality assessment tools. The areas assessed were as follows: the order of the random assignment, participants, assignment order concealment, levels of blinding of researchers, blinding of outcome assessments, selective reporting of insufficient results, and other sources of bias. Two researchers evaluated the quality assessments independently, and consistency was maintained after further discussion. In case of disagreement, a third researcher separately evaluated the quality assessments.

6. Literature Analysis Tools

The following details were identified and recorded by two researchers: authors, year of publication, country of study, study design, participant characteristics, sample size, type of XR program, scenario content, program operation and methods, outcome variables, measurement tools for the outcome variables, and experimental results. The experimental results correspond to the means, standard deviations, t-values, and p-values before and after the experiments.

7. Meta-analysis and Publication Bias Analysis

A meta-analysis was conducted to identify the effect size of the intervention using Comprehensive Meta-Analysis software. The effect size of variables whose homogeneity was not confirmed was calculated with a random-effects model that reset the weights considering the heterogeneity between the studies. The effect size was calculated by the standard mean difference value of individual studies [14]. The homogeneity of the selected studies was evaluated through the x² and I² tests. I² values of 0% indicate no heterogeneity, 30% to 60% moderate heterogeneity, and 75% high heterogeneity [14]. Publication bias was assessed through an examination of a funnel plot and Duval and Tweedie's trim and fill [15].

RESULTS

1. General Characteristics

Regarding the years of publication, four studies were published in 2020, four in 2021, two in 2019, and one each in 2017 and 2016. The researcher's nationalities were as follows: six studies from the USA, six from South Korea, one from Canada, and one from Hong Kong. For research methods, a nonequivalent control group quasi-experimental study was the most common, used in five studies; followed by a methodological design in three; one-group pretest-posttest study in three; and a one-group posttest study, a mixedmethod study, and a retrospective method in one study each (Table 1).

Table 1.

Systematic Review of Studies on XR-based Pediatric Simulation Programs

2. Specific Contents

The XR type was identified as VR with manikin or high-fidelity simulation utilization in six studies [11,16-20], VR was used in four studies [21-24], AR in one study [25], VR and Hololens study [26] in one, a mixed-reality simulation using VR with SPs was conducted in one study [20], and realistic mock codes and team-based simulation training were implemented in one study [27]. Ng et al. [21] implemented virtual wards using 3D software and that allowed students to acquire injection skills and word dressing. Putnam et al. [26] trained students on pediatric airway management through VR using a Hololens (Table 1).

The target subjects of the programs were nursing students in seven studies [11,16,18,19,21,23,28] and experts, nurse practitioners, reviewers, and transport teams (including nurses) [17,20,24-27] in six studies.

Nine scenarios dealt with respiratory problems, including pediatric asthma, five focused on early neonatal care and neonatal intensive care, and there was one scenario each on pediatric sepsis, seizure, dressing and infection using a pediatric department and ward environment AR, and airway intubation using a VR airway.

The outcome variables that were used to evaluate the effectiveness of the XR programs included knowledge [16,18,19, 26,28], confidence (or self-efficacy) [18,19,25], competency (or performance or skills) [11,17,21,23,26], and critical thinking disposition (Table 1) [11,23].

3. Quality Assessment of Studies

A quality assessment showed that five studies had a low risk of bias, three studies had a moderate risk of bias, two studies had a high risk of bias, and four studies had insufficient information to determine the risk of bias. Reed et al. [27] and Peterson et al. [20] conducted projects among experts working in a hospital, and there was a lack of an experimental design that would exclude confounding factors (Figure 2).

Figure 2.

Traffic light plot of the quality assessment.

4. Effectiveness Analysis of Extended Reality-based Child Simulation Programs

In total, 10 studies were included in the meta-analysis, and the total effect size was statistically significant at 0.84 (95% confidence interval [CI]=0.50-1.19, z=4.82, p<.001) by using random effect. A study by Putnam et al. [26] had the largest effect size and the largest standard difference in means (3.08, 95% CI=2.37-3.78, z=8.49, p<.001). The I² value in the heterogeneity evaluation of 10 studies was 82.0% (Figure 3). In a meta-analysis of nine studies, excluding that of Putnam et al. [26] due to high heterogeneity, the I2 value of heterogeneity was 0.00% and the pooled effect size was 0.60 (95% CI=0.47-0.74, z=8.57, p<.001) (Figure 4).

Figure 3.

Forest plot, funnel plot, and trim and fill of the meta-analysis of extended reality-based pediatric nursing simulation programs (10 studies).

Figure 4.

Forest plot, funnel plot, and trim and fill of the meta-analysis of extended reality-based pediatric nursing simulation programs (9 studies, excluding that of Putnam et al. [26], 2021).

5. Publication Bias

Based on the funnel plots (Figures 3, 4), it was judged that there was no publication bias since the plots demonstrated a symmetrical form around the integrated estimate. Under a random-effects model, the point estimate for the combined studies was 0.84 (95% CI=0.50-1.19). Using Duval and Tweedie's trim and fill, these values are unchanged (Figure 3). Furthermore, under a fixed-effect model, the point estimate for the combined studies was 0.60 (95% CI=0.47-0.74). Using trim and fill, the imputed point estimate was 0.56 (95% CI=0.43-0.69) (Figure 4).

DISCUSSION

The spread of the COVID-19 pandemic has led to a paradigm shift in nursing education worldwide. In particular, the need for simulation-based learning tools has been increasingly emphasized in education on nursing techniques for newborns or children because of the need to protect children who are vulnerable to infection in pediatric nursing. Yu and Mann [29] reported that the use of VR programs, especially for infection control in high-risk neonates, was very effective both for improving practice skills and promoting problem-solving abilities. Our literature analysis showed an increasing trend in applications of VR and AR in pediatric nursing classes and exercises since 2019, along with increasingly many effectiveness analyses, and these trends reflect the social context and needs of educational sites. In particular, while simulations in the laboratory using manikin models were initially deployed in the early days of virtual simulations, the implementation of virtual simulations using online platform-based videos and images has expanded. In recent years, active efforts have been made to enhance realism through three-dimensional stereoscopic images using holograms or mixed VR and AR to provide learners with a realistic learning experience.

The literature analysis identified the following characteristics of simulations for pediatric nursing education and skills enhancement. First, the simulations had similar structures consisting of three components (pre-orientation, simulation run, and debriefing). Kim et al. [11] reported that the development and application of pediatric nursing asthma modules for nursing students with the abovementioned structure improved students' critical thinking, problem-solving, and competency. Goldsworthy et al. [18] also measured the effectiveness of a simulation on the same topic for nursing students, reporting a significant increase in clinical self-efficacy. Furthermore, many studies have induced learners to improve their knowledge through pre- and post-quizzes [21,24,28]. According to the results of these studies, the use of pre- and postquizzes improved learners' knowledge, critical thinking, and competency by reinforcing crucial learning content.

Regarding content, most studies dealt with serious conditions such as pediatric asthma, sepsis, neonatal seizure, infection, and apnea [11,16-18,23]. This is because nursing care for severely ill children at clinical sites requires high technical skills and knowledge in order to make accurate clinical judgments. Considerable risk is posed by the need for nursing students or novice nurses to gain practical experience in providing direct care to severely ill children. Therefore, replacing direct care with simulation nursing could make it possible to acquire the necessary competencies for providing nursing care to ill children while ensuring these children's safety.

In this regard, by applying XR simulations to child nursing education, experiential learning can complement theoretical learning content, enhancing the effectiveness of learning and contributing to the improvement of educational performance. A simulation session can also be organized as a team task for students in a child nursing course as an education support method that induces advanced learning through self-study and team discussion based on simulation cases. In clinical settings, simulations can be used to provide specialized education for nurses in children's wards, pediatric outpatient departments, and neonatal intensive care units. In particular, XR-based simulations developed for rare cases that are difficult to experience in the field or very severe conditions with high nursing difficulty can be used as an educational medium to improve nurses' professionalism and strengthen their practical capabilities.

Wide variation was evident in studies that applied simulations, ranging from research that applied three simulations, such as the studies of Diaz et al. [16] and Goldsworthy et al. [18], to those that applied 18 simulations, such as the study of Reed et al. [27]. The simulation time was mostly around 60 minutes, but there were differences between studies; for instance, the simulation lasted 150 minutes in the study of Rim and Shin [24]. The results of these studies indicate a need to develop standardized guidelines for the operation of virtual simulation education and exercises in pediatric nursing.

The overall quality of the literature included in this study was low, as shown by the fact that four studies had insufficient information for evaluating the risk of bias and two studies had serious concerns. This is similar to the results of the study of Kim and Kim [23], where the quality evaluation score of the literature included in the meta-analysis was found to be low, with an average of 5.61 points. Regarding this, Kim and Kim [30] reported that in a meta-analysis, research in nursing showed a similar level to that in dentistry and pharmacy, but nevertheless, few papers received relatively high quality scores. As a solution to this, it was suggested that academic societies need to update, disseminate, and encourage compliance with reporting guidelines for meta-analyses. Since the number of studies included in the content analysis and meta-analysis was very limited compared to the number of studies that appeared as search results, further analysis according to specific dependent variables or aspects of program operation was difficult. Therefore, it is suggested that a more sophisticated research methodology and research designs should be applied to study individual performance in order to expand the applicability of simulations to child care in the future.

It is expected that the transformation of educational methods that enable students to acquire advanced nursing skills without space and time constraints, especially for newborns and children who are vulnerable to nursing care, will accelerate in the future through virtual online exercises combined with innovative technologies such as VR, AR, and holograms. These advances make it possible to actively cope with sudden changes in the educational environment away from the traditional method of a practicum, which is centered on patient observation in hospitals. A prerequisite for this, however, will be a rigorous analysis and assessment of learning effectiveness. In that regard, several one-group posttest studies and onegroup pretest-posttest studies have been conducted, and there have been many attempts to conduct nonequivalent control group quasi-experimental research. Thus, it is necessary to evaluate the effectiveness of virtual online simulation education in pediatric nursing through the expansion of studies applying more sophisticated research designs, such as randomized controlled trials.

Virtual simulation education has many advantages, such as educational effectiveness, safety, and the absence of spacetime constraints. Investments should be made to cover the initial development costs, and a functional online platform must be developed in addition to simulated learning content. It is necessary to surmount various practical obstacles facing the field of education, such as the relatively short lifecycle of existing simulation programs because the technological development of online learning is evolving rapidly. This is a challenge to overcome for the expansion of virtual simulation education in pediatric nursing in the future. Furthermore, future research should examine the most cost-effective method of applying a mixture of virtual simulation methods, the effectiveness of which has been verified in prior research.

To date, researchers have attempted to develop and apply scenarios appropriate to their experiences and situations and to measure their effectiveness. However, we are moving toward an era where collaborative research is possible through online platforms that allow scholars to move beyond national boundaries, understand the difference between each other's cultures and institutions while interacting in the same space and time despite being physically located anywhere in the world, and create appropriate nursing models. Therefore, international standards for pediatric nursing simulation education should be formulated by conducting international collaborative research to develop, apply, and assess the effectiveness of scenarios. These initiatives will help design the most effective and efficient simulation education tools, contents, and methods for students of pediatric nursing.

CONCLUSION

The global COVID-19 pandemic triggered a dramatic shift in pediatric nursing education because newborns and pediatric patients (i.e., the subjects of pediatric nursing) are particularly vulnerable to infection and require a high level of competency and maturity in nursing education and care. Accordingly, the present study systematically analyzed research on applications of XR-based pediatric nursing simulation programs and assessed the effectiveness of the programs. The results showed that the incorporation of XR technology in the field of pediatric nursing is an advance from traditional patient observation-oriented education that ensures the safety of patients while simultaneously enhancing learners' nursing care capabilities. The development of pediatric nursing simulation programs combined with various virtual simulation technologies should be expanded in the future. We propose searching for pediatric nursing simulation methodologies that enable high effectiveness at a low cost. This search should consider key requirements, including the development of standardized guidelines for the operation of pediatric nursing virtual simulation education and practice, validation of the results of simulation studies through the application of sophisticated study designs for effectiveness measurement, and the combined application of various virtual simulation methods.

Notes

Authors' contribution

Conceptualization: all authors; Data collection, Formal analysis: all authors; Writing-original draft, Writing-review and editing: all authors; Final approval of published version: all authors.

Conflict of interest

No existing or potential conflict of interest relevant to this article was reported.

Funding

This study was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (No. NRF- 2021R1A2C1095530).

Data availability

Please contact the corresponding author for data availability.

Acknowledgements

None.

References

1. Kim S, Cho Y. Proliferation of online learning and the implications for teaching and learning in future university education. Journal of Lifelong Learning Society 2018;14(4):51–78. https://doi.org/10.26857/JLLS.2018.11.14.4.51.

2. Lee Y, Park Y, Yoon J. Exploring the "types" through case analysis on operation of distance education in universities responding to COVID-19. The Journal of Yeolin Education 2020;28(3):211–234. https://doi.org/10.18230/tjye.2020.28.3.211.

3. Han S, Noh Y. Analyzing higher education instructors' perception on metaverse-based education. Journal of Digital Contents Society 2021;22(11):1793–1806. https://doi.org/10.9728/dcs.2021.22.11.1793.

4. Lee YH, O SJ. XR service for non-face-to-face experiment training. Korea Information Processing Society Review 2021;28(1):69–77.

5. Seo JH, Jeong JP, Choi EJ. The effects of smart learning on the academic performance of pediatric nursing education. The Journal of the Korea Contents Association 2016;16(11):784–795. https://doi.org/10.5392/JKCA.2016.16.11.784.

6. Statistics Korea. Number of births and total fertility rate in 2021 [Internet]. Daejeon: Statistics Korea; 2022. [cited 2022 October 19]. Available from: https://www.index.go.kr/unity/potal/main/EachDtlPageDetail .do?idx_cd=1428.

7. Kim JH, Park IH, Shin S. Systematic review of Korean studies on simulation within nursing education. Journal of Korean Academic Society of Nursing Education 2013;19(3):307–319. https://doi.org/10.5977/jkasne.2013.19.3.307.

8. Weaver A. High-fidelity patient simulation in nursing education: an integrative review. Nursing Education Perspectives 2011;32(1):37–40. https://doi.org/10.5480/1536-5026-32.1.37.

9. Ha YK. Systematic review of assessment of learner's flow in simulation education. Journal of Learner-Centered Curriculum and Instruction 2019;19(22):1413–1430. https://doi.org/10.22251/jlcci.2019.19.22.1413.

10. Lee CM, So HS, Kim Y, Kim JE, An M. The effects of high fidelity simulation-based education on clinical competence and confidence in nursing students: a systematic review. The Journal of the Korea Contents Association 2014;14(10):850–861. https://doi.org/10.5392/JKCA.2014.14.10.850.

11. Kim M, Kim S, Lee WS. Effects of a virtual reality simulation and a blended simulation of care for pediatric patient with asthma. Child Health Nursing Research 2019;25(4):496–506. https://doi.org/10.4094/chnr.2019.25.4.496.

12. Kim HD, Joo AR. The study of promoting method for VR contents to reduce pain of child patient -focusing on the case studies of VR contents at home and abroad. The Journal of the Korea Contents Association 2020;20(6):167–176. https://doi.org/10.5392/JKCA.2020.20.06.167.

13. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic Reviews 2021;10(1):89. https://doi.org/10.1186/s13643-021-01626-4.

14. Higgins JP, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society. Series A, (Statistics in Society) 2009;172(1):137–159. https://doi.org/10.1111/j.1467-985X.2008.00552.x.

15. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000;56(2):455–463. https://doi.org/10.1111/j.0006-341x.2000.00455.x.

16. Diaz DA, Anderson M, Hill PP, Quelly SB, Clark K, Lynn M. Comparison of clinical options: high-fidelity manikin-based and virtual simulation. Nurse Educator 2021;46(3):149–153. https://doi.org/10.1097/NNE.0000000000000906.

17. Farre A, Cummins C. Understanding and evaluating the effects of implementing an electronic paediatric prescribing system on care provision and hospital work in paediatric hospital ward settings: a qualitatively driven mixed-method study protocol. BMJ Open 2016;6(2)e010444. https://doi.org/10.1136/bmjopen-2015-010444.

18. Goldsworthy S, Patterson JD, Dobbs M, Afzal A, Deboer S. How does simulation impact building competency and confidence in recognition and response to the adult and paediatric deteriorating patient among undergraduate nursing students? Clinical Simulation in Nursing 2019;28:25–32. https://doi.org/10.1016/j.ecns.2018.12.001.

19. Kang KA, Kim SJ, Lee MN, Kim M, Kim S. Comparison of learning effects of virtual reality simulation on nursing students caring for children with asthma. International Journal of Environmental Research and Public Health 2020;17(22):8417. https://doi.org/10.3390/ijerph17228417.

20. Peterson E, Porter M, Calhoun A. Mixed-reality simulation for a pediatric transport team: a pilot study. Air Medical Journal 2020;39(3):173–177. https://doi.org/10.1016/j.amj.2020.03.001.

21. Ng SC, Lee LK, Lui AKF, Wong KF, Chan WY, Tam HH. Using immersive reality in training nursing students. International Journal of Innovation and Learning 2020;27(3):324–343. https://doi.org/10.1504/IJIL.2020.10028288.

22. Yu M, Yang MR. Effectiveness and utility of virtual reality infection control simulation for children with COVID-19: quasi-experimental study. JMIR Serious Games 2022;10(2)e36707. https://doi.org/10.2196/36707.

23. Kim S, Kim MJ. Effect of learner-centered virtual reality simulation education. Journal of Digital Convergence 2022;209(4):705–713. https://doi.org/10.14400/JDC.2022.20.4.705.

24. Rim D, Shin H. Effective instructional design template for virtual simulations in nursing education. Nurse Education Today 2021;96:104624. https://doi.org/10.1016/j.nedt.2020.104624.

25. Kotcherlakota S, Pelish P, Hoffman K, Kupzyk K, Rejda P. Augmented reality technology as a teaching strategy for learning pediatric asthma management: mixed methods study. JMIR Nursing 2020;3(1)e23963. https://doi.org/10.2196/23963.

26. Putnam EM, Rochlen LR, Alderink E, Auge J, Popov V, Levine R, et al. Virtual reality simulation for critical pediatric airway management training. Journal of Clinical and Translational Research 2021;7(1):93–99.

27. Reed DJW, Hermelin RL, Kennedy CS, Sharma J. Interdisciplinary onsite team-based simulation training in the neonatal intensive care unit: a pilot report. Journal of Perinatology 2017;37(4):461–464. https://doi.org/10.1038/jp.2016.238.

28. Yu M, Yang M, Ku B, Mann JS. Effects of virtual reality simulation program regarding high-risk neonatal infection control on nursing students. Asian Nursing Research 2021;15(3):189–196. https://doi.org/10.1016/j.anr.2021.03.002.

29. Yu M, Mann JS. Development of virtual reality simulation program for high-risk neonatal infection control education. Clinical Simulation in Nursing 2021;50:19–26. https://doi.org/10.1016/j.ecns.2020.10.006.

30. Kim JH, Kim AK. A quality assessment of meta-analyses of nursing in South Korea. Journal of Korean Academy of Nursing 2013;43(6):736–745. https://doi.org/10.4040/jkan.2013.43.6.736.

Article information Continued

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial and No Derivatives License (https://creativecommons.org/licenses/by-nc-nd/4.0/) which permits unrestricted non-commercial use, distribution of the material without any modifications, and reproduction in any medium, provided the original works properly cited.

Table 1.

Systematic Review of Studies on XR-based Pediatric Simulation Programs

Firth author (published year)		Nationality of sample	Study design	Type of simulation	Scenario contents	Process of simulation	Participant characteristics	Outcomes	Results
1	Kim et al. (2019) [11]	South Korea	Non-equivalent control group experimental design	E: blended VR vSim for Nursing (Laerdal Medical) + simulation scenario was developed by NLN using SimJunior (SimJunior, Laerdal Medical)	The scenario included the assessment of respiratory problems, medication, and education for pediatric patients with asthma	VR: suggested reading → Pre-quiz → vSim → Post-quiz → Documentation assignment → Guide for reflection	Nursing students (4th year) E (22) /C (26)	Critical thinking, problem-solving processes, critical competency	The pretest-posttest results of each group showed statistically significant improvements in 3 variables. In a comparison of the 2 groups, critical thinking differed significantly.
						HFS: Pre-orientation → Simulation operation → Debriefing on pediatric nursing asthma modules developed by NLN
				C: only VR
2	Díaz et al. (2021) [16]	USA	Quasi-experiment al design	E1: mannequin-based (HFPS)	Scenario: pediatric respiration, pediatric dehydration, pediatric sickle cell	HFPS: 4 groups, 2 participants, 2 observers total 45 min, scenario 15 min, debriefing 30 min	BSN students E1 (HFS): 119, E2 (VR): 134, C: 120	Knowledge acquisition	Immediately after the simulation, a significant difference was seen in the knowledge of two modules out of seven categories, but only one module showed a significant difference.
				E2: VR (self-development)		VR: 7 hrs until the computer reaches 90% of the set competency score, all simulations are carried out on the same day
				C: SBE
3	Farre and Cummins (2016) [17]	USA	Program development	The simulation was developed to present users with a high-fidelity, immersive training environment to maximize learning (Unity Technologies, San Francisco, CA, USA)	Content from the National Pediatric Readiness Project (2014). VRS scenario: NICU. Standards of Best Practice. The scenarios were built so that there were realistic renderings of the unit, stairwells, halls, equipment, and supplies.	Room doors are activated with an activation key. Sliding doors like those in the J-pod must be manually opened or closed with the action key. Those leaving the neonate room must close the door on the exit. Redirect to close neonate's door if not closed. Redirection: move through a virtual environment using a head-mounted display and controllers.	26 reviewers	Realism and usability during the VRS, ability to complete the VRS	Realism and usability during the VRS; Likert format question responses ranged from 0 to 5 with 5 signifying that participants were highly satisfied.
4	Goldsworthy et al. (2019) [18]	Canada	Quasi-experimental pre/post study	E: HFS + vSim (self-development)	Deteriorating patient: pediatric asthma, sepsis, neonatal seizure	16-hour simulation intervention over 2 days. Adult and pediatric deteriorating patients–HFS: pre-brief 15 min, simulation 20 min, debrief 25 min	Nursing students: E (24) C (39) random assignment	Clinical self-efficacy, knowledge	CSE (9/10) significantly increased in the E group, and the pre-post difference in knowledge was significant in 3 out of 5 modules.
5	Kang et al. (2020) [19]	South Korea	Pretest and posttest quasi-experimental study	VR (vSim for Nursing; Laerdal Medical), HFS (SimJunior, Laerdal Medical) based on vSim	Pediatric patient with asthma	vSim: pre-simulation quiz, simulation, post-quiz HFS, post-simulation debriefing	Nursing students E1 (vSim) 54, E2 (HFS) 69, E3 (vSim+HFS) 69	Knowledge, confidence, performance	Knowledge and confidence were significantly higher in E1 and E3, with the highest performance in E2.
6	Kotcherlakota et al. (2020) [25]	USA	Mixed-methods single-condition study	AR and Interactive Storytelling (ARIS) mobile app.	ARISE clinical scenario: pediatric patient with asthma	ARIS app on iPads	Pediatric nurse practitioner students (21)	Instructional materials motivation survey, SDS, student satisfaction and self-confidence in learning scale	The correlations between IMMS and SSSC were positive. The pediatric asthma test score significantly improved.
7	Ng et al. (2020) [21]	Hong Kong	One group posttest design	Virtual ward environment (Unity3D platform + software). The virtual ward: Autodesk Maya, Blender, Motion Builder, and Adobe Photoshop. The virtual ward can be run with a VR system(e.g., HTCVive) on a PC.	Virtual ward environment (VWE) and AR for skill-based learning-injection methods, wound dressing	Nursing education: ward orientation, fundamental nursing skills training	Junior nursing students	Virtual ward orientation, nursing skills	80% of students agreed that the VWE could help them become familiar with the material. Most students were satisfied with the AR app's development in learning fundamental nursing skills.
						Virtual ward environment	Males (10)
						AR for skill-based learning: injection methods, wound dressing	Females (20)
8	Peterson et al. (2020) [20]	USA	One-group pretest- posttest design	Mixed-reality simulation using SPs portraying family members and high-fidelity manikins	Medical resuscitation of an unstable infant (aged 2 months with bronchiolitis simulated via high-fidelity manikin) and SP, GABHS sepsis and respiratory failure and shock	Short introduction/briefing → running → in-depth debriefing that included team members, faculty, and SPs. Each session lasted 2 hours.	27 clinicians on the transport team: RN (10), RRT (9), EMT (8)	Pre-post self-perceptions of skills and program quality	There was a significant change in preand post-program self-perceptions of skills in communication.
9	Putnam et al. (2021) [26]	USA	One-group pretest- posttest design	VR with Hololens	1. Interactive instructional pediatric airway videos	Interactive instructional videos → VR trainer and a brief orientation to the Hololens. Each participant was allowed up to three attempts to manage the patient. At the conclusion of each module, an interactive quiz tested real-time understanding of the information. Feedback was given after each quiz.	41 (medical students, residents, faculty, and advanced practice nurses)	Knowledge of pediatric airway management, detecting a difference of airway size	Questions on pediatric anatomy, anaphylaxis, Heimlich maneuver, and foreign body removal showed the highest improvement in scores (p＜.05)
9	Putnam et al. (2021) [26]	USA	One-group pretest- posttest design	VR with Hololens	2. VR trainer (Hololens technology), simulating a pediatric critical airway event
10	Reed et al. (2017) [27]	USA	Retrospective report	Realistic mock code + TBST	An in-house pager system was implemented and tested for the NICU resuscitation team based on the NICU Code Blue Communication Pathway.	18 cases: home birth (esophageal intubation, hypothermia, hypoglycemia), acute pulmonary hemorrhage, respiratory failure (sepsis/pneumonia), tension pneumothorax/ cardiogenic shock pneumothorax post-surfactant, ECPR—cardiac, myelomeningocele self-extubation, ECPR—general surgery, premature infant self-extubation	The participants comprised more than 500 NICU staff	Attendance rate Number of attendees	The intervention resulted in 65 completed TBST events and 87% adherence to our schedule. Procedural skills included intubation, needle thoracotomy, intra-osseous needle placement, IV placement, shock management, defibrillation, cardioversion, chest compressions, and medication administration.
11	Yu and Yang (2022) [22]	South Korea	Development of a VR program	VR simulation program addressing neonatal infection control for nursing students	Scenario 1: basic care upon initial admission	Production of VR program → VR program equipment and software	Nursing students (4th year)	PPE knowledge, infection control performance, self-efficacy	The E group showed significantly greater improvement in PPE knowledge, infection control performance, and self-efficacy.
					Scenario 2: feeding management		E (25) / C (25)
					Scenario 3: skin care and environmental management		E (25) / C (25)
12	Yu et al. (2021) [28]	South Korea	Nonequivalent control group design	VR: The program employed a Vive ProFull-Kit Head- Mounted Display and sensor (HTC VIVETM, USA), a Leap Motion Controller™ (Ultraleap, USA) hand-tracking device with a VR developer mount, and a VR kit containing an EliteDesk800 G4 laptop computer	Neonatal intensive care unit practice. The program consisted of three scenarios: basic care, feeding management, and skincare and environmental management for the prevention of neonatal infection.	Virtual reality infection control simulation (VRICS): pre-briefing (20 min) → VR simulation (40 min, 3 scenarios) → Debriefing (20 min)	Nursing students (3rd and 4th years)	Knowledge, performance, satisfaction	There was no significant difference between the E and C groups regarding knowledge. Based on pretest and posttest results, high-risk neonatal infection control performance self-efficacy significantly increased in both the E and C groups. The E group showed significantly higher learner satisfaction.
12	Yu et al. (2021) [28]	South Korea	Nonequivalent control group design				E (25) / C (25)	Knowledge, performance, satisfaction
13	Kim and Kim (2022) [23]	South Korea	One group pretest- posttest test design	VR (vSim for Nursing; Laerdal Medical)	Pediatric vSim for Nursing	Orientation → Briefing → Pre-quiz → vSim operation → Post quiz	Nursing students (4th year): 44	Critical thinking disposition, critical competency, practice satisfaction	The E group showed significantly higher learner critical thinking disposition, critical competency, and practice satisfaction.
					- Gastroenteritis scenario
					- Asthma (mild intermittent asthma, asthma and pneumonia)
14	Rim and Shin (2021) [24]	South Korea	Methodological study	VRS	Hypoglycemia, neonatal apnea, transfusion	Each scenario session comprised an orientation, pre-briefing, simulation, and debriefing, and lasted 130-150 min.	Expert-FGI	CVI and narrative contents	The CVI value was lower than 0.8. For virtual elements, "representation and interaction" and "feedback system" both scored 0.67.
14	Rim and Shin (2021) [24]	South Korea	Methodological study	VRS	Hypoglycemia, neonatal apnea, transfusion		Nursing students	CVI and narrative contents

AR, augmented reality; C, control group; CVI, content validity index; E, experimental group; ECPR, extracorporeal cardiopulmonary resuscitation; EMT, emergency medical technician; FGI, focus group interview; GABHS, group A beta-hemolytic streptococcus aureus; HFS, high-fidelity simulation; HFPS, high-fidelity patient simulation; IMMS, Instructional Materials Motivation Survey; NICU, neonatal intensive care unit; NLN, National League for Nursing; PPE, personal protective equipment; RN, registered nurse; RRT, respiratory therapist; SDS, Simulation Design Scale; SP, standardized patient; SSSC, Student Satisfaction and Self-Confidence in Learning Scale; TBST, team-based simulation training; VR, virtual reality; VRS, virtual reality simulation; vSim, virtual simulation; VWE, virtual ward environment.