Abstract
-
Purpose
This study was conducted to obtain data for the development of an effective fall risk assessment tool for pediatric inpatients through a systematic review and meta-analysis of the diagnostic test accuracy of existing scales.
-
Methods
A literature search using Medline, Science Direct, CINAHL, EMBASE, and the Cochrane Library was performed between March 1 and 31, 2018. Of 890 identified papers, 10 were selected for review. Nine were used in the meta-analysis. Stata version 14.0 was used to create forest plots of sensitivity and specificity. A summary receiver operating characteristic curve was used to compare all diagnostic test accuracies.
-
Results
Four studies used the Humpty Dumpty Falls Scale. The most common items included the patient's diagnoses, use of sedative medications, and mobility. The pooled sensitivity and specificity of the nine studies were .79 and .36, respectively.
-
Conclusion
Considering the low specificity of the pediatric fall risk assessment scales currently available, there is a need to subdivide scoring categories and to minimize items that are evaluated using nurses' subjective judgment alone. Fall risk assessment scales should be incorporated into the electronic medical record system and an automated scoring system should be developed.
-
Key words: Falls; Risk; Sensitivity and specificity; Pediatrics
INTRODUCTION
Patient falls in hospitals are considered to be a major problem associated with morbidity, mortality, and prolonged hospitalizations in the adult and elderly inpatient population [
1]. Falls in hospitalized children are almost equally prevalent and serious. However, little focus has been placed on pediatric inpatient falls [
2].
A survey of pediatric inpatient falls reported that, within a 6-month period, fall rates ranged from .4 to 3.8 cases per 1,000 patients at 26 participating institutions in the US, with 68% of these cases occurring when one or more medical staff or family members were present [
3]. A survey of pediatric inpatient falls reported that 48 patients experienced falls at one institution in Seoul during a 44-month period, with 82.2% of these cases occurring when family members were present [
4]. The findings of that survey also indicated that the majority of medical institutions had a fall prevention program in place. Nevertheless, fall rates had not decreased, and the ineffectiveness of these fall preventive programs has led researchers to question the sensitivity and specificity of fall risk assessment scales. In addition, the problem of inconsistency in reporting and publicly available data on falls has been pointed out by several researchers [
5]. Sensitivity is defined as the ratio of people with a disease to people with a positive test result. It is used to evaluate diagnostic tests, symptoms, or signs for the diagnosis of a disease. If a certain symptom, sign, or test has a high sensitivity, certain diseases can be ruled out when the result is negative. Specificity is defined as the ratio of people without a disease to people with a negative test result. Like sensitivity, it is used to evaluate diagnostic tests, symptoms, or signs for the diagnosis of a disease. If a certain symptom, sign, or test has a high specificity, the probability of having a certain disease is high when the result is positive. The diagnostic odds ratio (DOR) reflects the values of sensitivity and specificity for a diagnostic test. It is defined as the ratio of the odds of the test being positive in subjects with the disease to the odds of the test being positive in subjects without the disease [
6].
In the USA, 89.7% of reviewed institutions used a pediatric fall risk assessment tool to identify patients at risk of falling. It was reported by 81% of these institutions that the tool used was developed internally, with only 22% indicating that a validated tool was used. In summary, fall risk assessment tools and prevention protocols in major pediatric hospitals have not been standardized [
7].
Fall risk screening tools are a very effective intervention strategy for maintaining and improving fall prevention programs. These tools for pediatric inpatients must be evaluated in depth to prevent falls in this population, and the developed tools must be validated for their sensitivity and specificity. The aim of this study was to perform a systematic review and meta-analysis of the published literature on pediatric inpatient fall risk assessment diagnostic test accuracy (DTA) in order to provide useful data for the development of an effective fall risk assessment tool and fall prevention programs for the pediatric inpatient population.
METHODS
1. Study Design
This study was a systematic review and meta-analysis of studies on the DTA of pediatric inpatient fall risk assessment tools.
2. Procedure
1) Literature search
A literature search using PubMed (including Medline), Science Direct, CINAHL, EMBASE, and the Cochrane Library was performed between March 1 and 31, 2018, for articles published.
2) Search terms
Combinations of the following words were used:(child or pediatric or paediatric or children) and (inpatient or hospitalized) and (fall or fall down or accident fall or fall risk) and (scale or tool or assessment) and (validity or sensitivity or specificity). The time limit was set from the earliest year of publication available in each database to the date of the literature search in 2018. The search terms were selected in a way that reflected the query tree structure of each database.
3) Literature selection procedure
(1) Selection and exclusion criteria
All primary studies that evaluated the accuracy of pediatric inpatient fall risk assessment tools, including sensitivity, specificity, odds ratios, true positive values, and true negative values, obtained from the literature search using the search terms were included.
Studies were excluded if they met the following exclusion criteria:
• Duplicates of a study extracted from another database
• Studies that did not assess the effectiveness of fall risk assessment tools
• Studies on pediatric falls in a home or community setting and not an in-hospital setting.
(2) Final selection of studies
According to the above selection and exclusion criteria, two expert-level researchers selected the final sample of studies for analysis by cross-checking. Any disagreements were resolved by consulting a third reviewer until a consensus was reached. All procedures were performed according to the Cochrane guideline [
8]. A more detailed description of the procedures is shown in
Figure 1.
(3) Assessment of study quality
Quality assessment of the selected articles was conducted using questions from the Quality Assessment of Diagnostic Accuracy Studies [
6]. The risk of bias was assessed for domain-patient selection, the index test, the reference standard, and flow and timing. Three studies had a high risk of bias of patient selection and all 10 studies were evaluated as having a low risk of bias in the other domains.
This study was carried out with the approval of the Institutional Review Board (1040782-180228-HR-03-22) for data collection.
4. Statistical Analysis
Data were analyzed using Stata version 14.0 (StataCorp, College Station, TX, USA). First, the metandi command was used to combine the true positive, false positive, true negative, and false negative values of each study, and forest plots and 95% confidence intervals were generated for the sensitivity and specificity of individual studies. Next, summary receiver operating characteristic (SROC) curves of DTA were formulated to increase the statistical power. The SROC model of meta-analysis provides a summary estimate for sensitivity and specificity together. Lastly, bivariate random-effect models from the midas command were used to assess heterogeneity across the studies. The bivariate box plot describes the degree of interdependence, including the central location and identification of any outliers [
9].
RESULTS
1. Characteristics of Included Studies
The included studies were published between 2006 and 2017 (
Table 1). Of the 10 studies reviewed, four used the Humpty Dumpty Fall Scale (HDFS). The sample sizes ranged from 100 to 652 participants, and the studies employed retrospective research methods. The number of items used in each scale ranged from 4 to 10.
The results of the analysis of the items in the fall scales used in the studies are shown in
Table 2. There was a total of 7 scales included mediation items. The detailed contents of each medication item were not consistent across the scales, and various criteria were used for identifying hazardous medications. Four of the seven scales included a diagnosis item. A score was assigned when a certain diagnosis was given. Age, ambulatory aids, and mental status items were included in four of the seven scales. The other items included are shown in
Table 2.
Of the 10 studies reviewed, nine were included in the meta-analysis. A study by Neiman et al. [
2] was excluded due to insufficient data. The meta-analysis showed that the nine studies had a pooled sensitivity of .79 (.59~.91) and a pooled specificity of .36 (.16~.63) (
Figure 2). The pooled DOR was 2.15 (1.19~3.87) and the area under the curve (AUC) was .65 (.60~.69). The SROC curves of the pediatric inpatient fall assessment scales are shown in
Figure 3, with a sensitivity close to 1 and specificity close to 0. The pooled figure also shows scattered points from the center line (
Figure 3-A).
An assessment of between-study heterogeneity showed negative bivariate correlation (logits) of -.89 and a hierarchical summary receiver operating characteristic (HSROC) beta of .14 (
p=.541), indicating that the studies were considered substantial heterogeneity (
Figure 3-A). In addition, there were only two outliers in the bivariate box plot comparing each scale (
Figure 3-B).
DISCUSSION
We performed a systematic review and meta-analysis of pediatric inpatient fall risk assessments. Of the 890 studies identified, 10 studies were selected for review. Of those 10 studies, nine were included in the meta-analysis. Four of the 10 studies used the HDFS, while a different scale was used in the other 6 studies. In these studies, the items used in the scales varied in number and content. The most common items were the patient's diagnosis, use of sedative medications, and mobility. Age was included in 3 studies, and sex was only included in the HDFS. It has been reported that risk factors for pediatric inpatient falls include being between 3 and 5 years of age, male sex, cognitive development, and clinical conditions, such as surgery and use of sedatives [
10]. In addition, some studies have suggested the need to incorporate more risk factors, such as the use of assistive devices, polypharmacy, history of falls, neurological diagnosis, and toileting, that are included in adult fall risk assessment scales [
11]. Of the scales analyzed, the General Risk Assessment for Pediatric Inpatient Falls (GRAF-PIF) is the most commonly used tool, and is considered useful for pediatric patients. However, it requires computerization and the problem of length of stay has not been resolved. Use of the CHAMPS pediatric fall risk assessment tool is recommended in clinical practice, but its sensitivity and specificity are difficult to measure, and its accuracy has been questioned [
12]. The HDFS is also widely used in clinical practice, and four of the 10 included studies used this scale. However, it has been reported that 80% of the children assessed using the HDFS were categorized as having a high risk of falling, indicating a very low specificity [
13]. A similar result was found in our meta-analysis, as the pooled sensitivity of the inpatient fall scales was high (.79), while the pooled specificity was low (.36). Fall prevention programs for pediatric inpatients must effectively screen for children at a high risk of falling. However, the results from this study indicate that the pediatric inpatient fall risk assessment scales currently used in hospitals have failed to do so. As such, fall prevention programs for high-risk children have not been effective [
14], and the fall risk scales currently available are not adequate for screening children at a high risk of falling.
The studies with highest sensitivity were by Pauley et al. [
15] and Zhu, Xu, and Yang [
16], with sensitivity values greater than .95. The HDFS was used in both these studies. The HDFS contains seven items, all of which may be checked from the patient's chart. This scale considers the age and sex of the child, as these two factors are most closely associated with an increased risk of falls. However, the scores given for each age category (4 points for children aged 3 years or younger and 3 points for children aged 3~7 years) do not seem adequate, as falls are reported most frequently in children aged 3~5 years. Furthermore, a history of falls and using beds are given the highest score of 4, while using a crib is given a score of 3 and is classified as an item relating to an environmental factor. As such, some items appear to be grouped without a rational basis. Such items may potentially give rise to confusion among nurses when evaluating a child. There is also a concern regarding giving high scores on ambiguous items. The sensitivity of the studies that used the HDFS was generally high (.11). In contrast, the specificity was very low (.04). This may be partly explained by the tendency of nurses to give high scores for items that are ambiguous and difficult to measure. This is because the likelihood of receiving a neglect of duty evaluation is higher for nurses when falls occur in children classified as low-risk than when no falls occur in children classified as high-risk. Moreover, a study by Myers and Nikoletti [
17] showed that the sensitivity of nurses' clinical judgment was high, while the specificity and positive predictive value were low. This suggests that when using clinical judgment, nurses were not able to distinguish between high-risk and low-risk children. The scale used in the study by Franck et al. [
18] had low sensitivity and contained four items, with a cutoff score of 3. This scale, however, had a very high specificity of .97, and 97% of children who were identified as having a high risk of falling using this scale experienced a fall. The 4 items included in this scale are age, mobility, elimination, and current medication. The simple set of items in this scale and its effectiveness in screening for children at high risk must be considered.
Considering the low sensitivity of the pediatric fall risk assessment scales currently available, there is a need to subdivide scoring categories and to minimize the items evaluated solely by a nurse's individual judgment. It is necessary to incorporate fall risk assessment scales into the electronic medical record system and to develop an automated scoring system.
Our review of pediatric inpatient fall risk assessment scales showed that research in this area is still at an early stage. As pointed out in a number of studies, the accuracy of these scales is poor, requiring more evaluation and validation studies [
5,
7,
12-
14,
19,
20]. Effort should be made to revise and improve the sensitivity and specificity of existing fall scales and to develop a new scale with better predictability through continued research. Moreover, it may be necessary to consider all cases of falls in order to increase predictability, since falls that do not lead to injury have been excluded from reports [
21]. Thus, a more effective fall risk assessment scale must be developed based on the accumulated study results.
CONCLUSION
We aimed to provide useful data through a systematic review and meta-analysis of previous studies on the DTA of pediatric inpatient fall risk assessment scales, as it was evident that there is a need for an effective pediatric inpatient fall risk assessment tool. The findings from the meta-analysis of this study showed that the currently-available inpatient pediatric fall risk assessment scales have high sensitivity and low specificity, suggesting that there is a need to develop a scale with a greater number of subdivided categories and no items that require the subjective judgment of nurses. To this end, continued research is needed on the development of fall risk assessment scales and on the effectiveness of such scales in clinical practice.
ARTICLE INFORMATION
Figure 1.Flow diagram for study selection.
Figure 2.Forest plot of the included studies.
Figure 3-A.
Hierarchical summary receiver operating characteristic (HSROC) curves for pediatric inpatient fall scales.
Figure 3-B.
Bivariate box plot comparing each pediatric inpatient fall risk scale.
Table 1.Characteristics of Included Studies (N=10)
|
Author (year) |
Subjects |
Tool (range) |
Tool items |
Cut-off |
tp |
fp |
fn |
tn |
Sensitivity |
Specificity |
|
Razmus, Wilson, Smith, & Newman (2006) |
Fall: 100 |
Morse (0~125) |
History of falling, secondary diagnosis, ambulatory aid, IV/IV access, gait, mental status (6 items) |
45 |
62 |
54 |
38 |
46 |
62.0 |
46.0 |
|
Control: 100 |
|
Retrospective study |
|
Hill-Rodriguez et al. (2007) |
Fall: 153 |
HDFS (7~23) |
Age, sex, diagnosis, cognitive impairments, environmental factor, response to operation, sedation education (usage) (7 items) |
12 |
128 |
115 |
22 |
37 |
85.0 |
24.0 |
|
Control: 153 |
|
Retrospective study |
|
Graf (2008) |
Fall: 100 |
GRAF-PIF (0~10) |
Length of stay, orthopedic diagnosis, physical therapy, seizure medications, IV (5 items) |
2 |
76 |
24 |
25 |
76 |
75.0 |
76.0 |
|
Control: 100 |
|
Retrospective study |
|
Neiman et al. (2011) |
Fall: 59 |
I'm SAFE (0~8) |
Impairment, medications, sedation, admitting diagnosis, anesthesia within the previous 24 hours, fall history, environment of care (7 items) |
NA |
Odds ratio=1.21 (1.14~1.28) |
|
Control: 177 |
|
|
|
|
|
|
|
|
Retrospective study |
|
|
|
|
|
|
|
|
Zhu et al. (2012) |
Fall: 50 |
HDFS |
|
12 |
73 |
72 |
3 |
9 |
96.3 |
11.1 |
|
Control: 50 |
|
Retrospective study |
|
Ryan-Weinger & Dufek (2012) |
Fall: 326 |
PFRA (0~30) |
Weakness, physical disability, selected medications, risk for seizure, other conditions, age, confusion, previous fall, hazardous medication. equipment, voiding pattern (10 items) |
5 |
81 |
98 |
82 |
65 |
49.7 |
39.9 |
|
Control: 326 (1 month to 21 years) |
|
Retrospective study |
|
Shin, Kim, Kim, Son, & Bang (2014) |
Fall: 48 |
Pediatric fall risk assessment tool |
Age, diagnosis, activity capacity, delay of physical development, number of hazardous medications (above 3) (5 items) |
7 |
44 |
119 |
3 |
23 |
93.6 |
16.2 |
|
Control: 149 (0~18 years) |
|
Retrospective study |
|
Messmer, Williams, & Williams (2013) |
Fall: 74 |
HDFS |
|
12 |
42 |
148 |
17 |
94 |
57.0 |
39.0 |
|
Control 242 |
|
Retrospective study |
|
Pauley et al. (2014) |
Fall: 37 |
HDFS (7~23) |
|
12 |
36 |
71 |
1 |
3 |
97.3 |
4.05 |
|
Control: 74 |
|
Retrospective study |
|
Franck et al. (2017) |
Low-risk: 105 |
Little Shumidy (0~7) |
Age, mobility, elimination, current medications (4 items) |
3 |
19 |
2 |
44 |
61 |
30.0 |
97.0 |
|
High-risk: 21 (＜25 years) |
|
Retrospective study |
Table 2.Items Included in Pediatric Inpatient Fall Scales (N=7)
|
Item |
Detailed contents |
n |
|
Medication |
Sedative medications, seizure medications, hazardous medications, current medications, number of hazardous medications |
7 |
|
Diagnosis |
Secondary diagnosis, diagnosis, orthopedic diagnosis, admission diagnosis |
4 |
|
Age |
|
4 |
|
Ambulatory aids |
Impairment, weakness, physical disability, ambulatory aids |
4 |
|
Mental status |
Mental status, cognitive impairment, confusion, seizure |
4 |
|
Activity capacity |
Gait, activity capacity, mobility |
3 |
|
History of falling |
|
3 |
|
Others |
Voiding pattern, elimination |
2 |
|
Environmental factors |
2 |
|
Response to operation |
2 |
|
Length of stay |
1 |
|
Other condition |
1 |
|
Delay of physical development |
1 |
|
Sex |
1 |
REFERENCES
- 1. Hendrich AL, Bender PS, Nyhuis A. Validation of the Hendrich II Fall Risk Model: A large concurrent case/control study of hospitalized patients. Applied Nursing Research. 2003;16(1):9-21.
- 2. Neiman J, Rannie M, Thrasher J, Terry K, Kahn MG. Development, implementation, and evaluation of a comprehensive fall risk program. Journal for Specialists in Pediatric Nursing. 2011;16(2):130-139. https://doi.org/10.1111/j.1744-6155.2011.00277.x
- 3. Child Health Corporation of America, Nursing Falls Study Task Force. Pediatric falls: State of the science. Pediatric Nursing. 2009;35(4):227-231.
- 4. Shin HJ, Kim YN, Kim JH, Son IS, Bang KS. A pediatric fall-risk assessment tool for hospitalized children. Child Health Nursing Research. 2014;20(3):215-224. https://doi.org/10.4094/chnr.2014.20.3.215
- 5. DiGerolamo K, Davis KF. An integrative review of pediatric fall risk assessment rools. Journal of Pediatric Nursing. 2017;34:23-28. https://doi.org/10.1016/j.pedn.2017.02.036
- 6. National Evidence-based Healthcare Collaborating Agency. Systematic review of diagnostic tests. Manual Development Report. Seoul: National Evidence-based Healthcare Collaborating Agency; 2014 Apr.
- 7. Jamerson PA, Graf E, Messmer PR, Fields HW, Barton S, Berger A, et al. Inpatient falls in freestanding children's hospitals. Pediatric Nursing. 2014;40(3):127-135.
- 8. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560. https://doi.org/10.1136/bmj.327.7414.557
- 9. Dwamena BA. Midas: Retouch regarding diagnostic accuracy meta-analysis [Internet]. Boston: Stata; 2014 [cited 2018 Oct 10]. Available from: http://www.stata.com/meeting/boston14/abstracts/materials/boston14_dwamena.pdf
- 10. Levene S, Bonfield G. Accidents on hospital wards. Archives of Disease in Childhood. 1991;66(9):1047-1049.
- 11. Razmus I, Wilson D, Smith R, Newman E. Falls in hospitalized children. Pediatric Nursing. 2006;32(6):568-572.
- 12. Kramlich DL, Dende D. Development of a pediatric fall risk and injury reduction program. Pediatric Nursing. 2016;42(2):77-82.
- 13. Harvey K, Kramlich D, Chapman J, Parker J, Blades E. Exploring and evaluating five paediatric falls assessment instruments and injury risk indicators: An ambispective study in a tertiary care setting. Journal of Nursing Management. 2010;18(5):531-541. https://doi.org/10.1111/j.1365-2834.2010.01095.x
- 14. Graf E. Magnet children's hospitals: Leading knowledge development and quality standards for inpatient pediatric fall prevention programs. Journal of Pediatric Nursing. 2011;26(2):122-127. https://doi.org/10.1016/j.pedn.2010.12.007
- 15. Pauley BJ, Houston LS, Cheng D, Johnston DM. Clinical relevance of the Humpty Dumpty Falls Scale in a pediatric specialty hospital. Pediatric Nursing. 2014;40(3):137-142.
- 16. Zhu H, Xu H, Yang Y. Preliminary evaluation on Humpty Dumpty pediatric Falls Risk Scale. Chinese Nursing Research. 2012;26(7):1817-1820.
- 17. Myers H, Nikoletti S. Fall risk assessment: A prospective investigation of nurses' clinical judgement and risk assessment tools in predicting patient falls. International Journal of Nursing Practice. 2003;9(3):158-165.
- 18. Franck LS, Gay CL, Cooper B, Ezrre S, Murphy B, Chan JS, et al. The Little Schmidy Pediatric Hospital fall risk assessment index: A diagnostic accuracy study. International Journal of Nursing Studies. 2017;68:51-59. https://doi.org/10.1016/j.ijnurstu.2016.12.011
- 19. Chromá J. Risk of falling in pediatric nursing. Central European Journal of Nursing and Midwifery. 2016;7(4):542-548.
- 20. Schaffer PL, Daraiseh NM, Daum L, Mendez E, Lin L, Huth MM. Pediatric inpatient falls and injuries: A descriptive analysis of risk factors. Journal for Specialists in Pediatric Nursing. 2012;17(1):10-18. https://doi.org/10.1111/j.1744-6155.2011.00315.x
- 21. Cooper CL, Nolt JD. Development of an evidence-based pediatric fall prevention program. Journal of Nursing Care Quality. 2007;22(2):107-112. https://doi.org/10.1097/01.NCQ.0000263098.83439.8c
Appendix
Appendix 1.
- Citations for Studies Included in this Study
1. Hill-Rodriguez D, Messmer PR, Williams PD, Zeller RA, Williams AR, Wood M, et al. The Humpty Dumpty Falls Scale: A case-control study. Journal for Specialists in Pediatric Nursing. 2009;14(1):22-32.
https://doi.org/10.1111/j.1744-6155.2008.00166.x
2. Ryan-Wenger NA, Kimchi-Woods J, Erbaugh MA, LaFollette L, Lathrop J. Challenges and conundrums in the validation of Pediatric Fall Assessment Tools. Pediatric Nursing. 2012;38(3):159-167.
3.Messmer PR, Williams PD, Williams AR. A case-control study of pediatric falls using electronic medical records. Rehabilitation Nursing. 2013;38(2):73-79.
https://doi.org/10.1002/rnj.73
4. Razmus I, Wilson D, Smith R, Newman E. Falls in hospitalized children. Pediatric Nursing. 2006;32(6):568-572.
5. Graf E. Magnet children's hospitals: Leading knowledge development and quality standards for inpatient pediatric fall prevention programs. Journal of Pediatric Nursing. 2011;26(2):122-127.
https://doi.org/10.1016/j.pedn.2010.12.007
6. Neiman J, Rannie M, Thrasher J, Terry K, Kahn MG. Development, implementation, and evaluation of a comprehensive fall risk program. Journal for Specialists in Pediatric Nursing. 2011;16(2):130-139.
https://doi.org/10.1111/j.1744-6155.2011.00277.x
7. Zhu H, Xu H, Yang Y. Preliminary evaluation on Humpty Dumpty pediatric Falls Risk Scale. Chinese Nursing Research. 2012;26(7):1817-1820.
9. Pauley BJ, Houston LS, Cheng D, Johnston DM. Clinical relevance of the Humpty Dumpty Falls Scale in a pediatric specialty hospital. Pediatric Nursing. 2014;40(3):137-142.
10. Franck LS, Gay CL, Cooper B, Ezrre S, Murphy B, Chan JS, et al. The Little Schmidy Pediatric Hospital fall risk assessment index: A diagnostic accuracy study. International Journal of Nursing Studies. 2017;68:51-59.
https://doi.org/10.1016/j.ijnurstu.2016.12.011
