8
abstract OBJECTIVE: The primary objective of this study was to establish the validity and reliability of 2 respiratory scores, the Respiratory Distress Assessment Instrument (RDAI) and the Children’s Hospital of Wisconsin Respiratory Score (CHWRS), in bronchiolitis. A secondary objective was to identify the respiratory score components that most determine overall respiratory status. METHODS: This was a prospective cohort study in infants aged <1 year seen at Children’s Hospital of Wisconsin for bronchiolitis. We evaluated: (1) discriminative validity (the score’s ability to discriminate between 2 different outcomes) of the respiratory scores to identify emergency department (ED) disposition by using receiver operating characteristic curves; and (2) construct validity (the score’s ability to measure what it is thought to measure, overall respiratory status) by using length of stay (LOS) as a proxy for disease severity and comparing correlations between changes in respiratory scores and LOS. Interrater reliability was established by using intraclass correlation. The contribution of individual respiratory score components to determine ED disposition was studied by using multivariate logistic regression. RESULTS: A total of 195 infants were included. The area under the receiver operating characteristic curve was 0.68 for CHWRS versus 0.51 for RDAI in predicting disposition. There was no correlation between initial respiratory scores or change in respiratory scores over the first 24 hours and LOS. Item analysis revealed that oxygen delivery, subcostal retractions, and respiratory rate were independently correlated with ED disposition. The CHWRS was more reliable than the RDAI. CONCLUSIONS: The CHWRS had modest discriminative validity in predicting ED disposition. Neither the CHWRS nor the RDAI had good construct validity. Respiratory rate, oxygen need, and presence of retractions were most useful in predicting ED disposition. RESEARCH ARTICLE 202 | VOLUME 2 • ISSUE 4 www.hospitalpediatrics.org AUTHORS Lauren Destino, MD, 1 Michael C. Weisgerber, MD, MS, 2,3 Paula Soung, MD, 2,3 Deborah Bakalarski, RRT, 4 Ke Yan, PhD, 2,3 Rebecca Rehborg, BS, 2 Duke R. Wagner, DC, 2,3 Marc H. Gorelick, MD, MSCE, 2,3 Pippa Simpson, PhD 2,3 1 Department of Pediatrics, Stanford University, Palo Alto, California 2 Department of Pediatrics, Medical College of Wisconsin, Milwaukee, Wisconsin 3 Children’s Research Institute, Milwaukee, Wisconsin 4 Children’s Hospital of Wisconsin, Milwaukee, Wisconsin KEY WORDS Respiratory Distress Assessment Instrument, respiratory scores, RDAI, RSV ABBREVIATIONS CI: confidence interval CHW: Children’s Hospital of Wisconsin CHWRS: Children’s Hospital of Wisconsin Respiratory Score ED: emergency department LOS: length of stay RACS: Respiratory Assessment Change Score RDAI: Respiratory Distress Assessment Instrument ROC: receiver operating characteristic RT: respiratory therapist This work was presented in part at the annual meeting of the Pediatric Academic Societies; Vancouver Canada, May 4, 2010. Address correspondence to Michael C. Weisgerber, MD, MS, Medical College of Wisconsin, Pediatrics, Section of Hospital Medicine, CCC Suite C560, Milwaukee, WI 53201-1997. E-mail: [email protected] www.hospitalpediatrics.org doi:10.1542/hpeds.2012-0013 HOSPITAL PEDIATRICS (ISSN Numbers: Print, 2154 - 1663; Online, 2154 - 1671). Copyright © 2012 by the American Academy of Pediatrics FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose. FUNDING: Supported by a grant from the Children’s Research Institute and Children’s Hospital and Health System Foundation at Children’s Hospital of Wisconsin. Validity of Respiratory Scores in Bronchiolitis INTRODUCTION Bronchiolitis is a dynamic illness that requires frequent assessment of respiratory status in inpatient and outpatient settings. 1 Respiratory scores are used to assess initial respiratory status, follow trends, and evaluate response to therapeutic interven- tions. There are several different respiratory scores in existence; they vary greatly in composition, from 2 to 5 components on review of current literature. 2 The Respiratory Distress Assessment Instrument (RDAI) is one of the most commonly used respi- ratory scores 3–16 and was developed to assess epinephrine response in wheezing infants and initially used by a small pool of study investigators. 13 When used as a marker of change with respiratory rate, it is called the Respiratory Assessment

Validity of Respiratory Scores in Bronchiolitishosppeds.aappublications.org/content/hosppeds/2/4/202.full.pdf · HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY

Embed Size (px)

Citation preview

Page 1: Validity of Respiratory Scores in Bronchiolitishosppeds.aappublications.org/content/hosppeds/2/4/202.full.pdf · HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY

abstractOBJECTIVE: The primary objective of this study was to establish the validity and reliability of 2 respiratory scores, the Respiratory Distress Assessment Instrument (RDAI) and the Children’s Hospital of Wisconsin Respiratory Score (CHWRS), in bronchiolitis. A secondary objective was to identify the respiratory score components that most determine overall respiratory status.

METHODS: This was a prospective cohort study in infants aged <1 year seen at Children’s Hospital of Wisconsin for bronchiolitis. We evaluated: (1) discriminative validity (the score’s ability to discriminate between 2 different outcomes) of the respiratory scores to identify emergency department (ED) disposition by using receiver operating characteristic curves; and (2) construct validity (the score’s ability to measure what it is thought to measure, overall respiratory status) by using length of stay (LOS) as a proxy for disease severity and comparing correlations between changes in respiratory scores and LOS. Interrater reliability was established by using intraclass correlation. The contribution of individual respiratory score components to determine ED disposition was studied by using multivariate logistic regression.

RESULTS: A total of 195 infants were included. The area under the receiver operating characteristic curve was 0.68 for CHWRS versus 0.51 for RDAI in predicting disposition. There was no correlation between initial respiratory scores or change in respiratory scores over the fi rst 24 hours and LOS. Item analysis revealed that oxygen delivery, subcostal retractions, and respiratory rate were independently correlated with ED disposition. The CHWRS was more reliable than the RDAI.

CONCLUSIONS: The CHWRS had modest discriminative validity in predicting ED disposition. Neither the CHWRS nor the RDAI had good construct validity. Respiratory rate, oxygen need, and presence of retractions were most useful in predicting ED disposition.

RESEARCH ARTICLE

202 | VOLUME 2 • ISSUE 4 www.hospitalpediatrics.org

AUTHORSLauren Destino, MD,1 Michael C. Weisgerber, MD, MS,2,3 Paula Soung, MD,2,3 Deborah Bakalarski, RRT,4 Ke Yan, PhD,2,3 Rebecca Rehborg, BS,2 Duke R. Wagner, DC,2,3 Marc H. Gorelick, MD, MSCE,2,3 Pippa Simpson, PhD2,3

1Department of Pediatrics, Stanford University, Palo Alto, California2Department of Pediatrics, Medical College of Wisconsin, Milwaukee, Wisconsin3Children’s Research Institute, Milwaukee, Wisconsin4Children’s Hospital of Wisconsin, Milwaukee, Wisconsin

KEY WORDSRespiratory Distress Assessment Instrument, respiratory scores, RDAI, RSV

ABBREVIATIONSCI: confi dence intervalCHW: Children’s Hospital of WisconsinCHWRS: Children’s Hospital of Wisconsin Respiratory ScoreED: emergency department LOS: length of stay RACS: Respiratory Assessment Change Score RDAI: Respiratory Distress Assessment Instrument ROC: receiver operating characteristic RT: respiratory therapist

This work was presented in part at the annual meeting of the Pediatric Academic Societies; Vancouver Canada, May 4, 2010.

Address correspondence to Michael C. Weisgerber, MD, MS, Medical College of Wisconsin, Pediatrics, Section of Hospital Medicine, CCC Suite C560, Milwaukee, WI 53201-1997. E-mail: [email protected]

www.hospitalpediatrics.orgdoi:10.1542/hpeds.2012-0013

HOSPITAL PEDIATRICS (ISSN Numbers: Print, 2154 - 1663; Online, 2154 - 1671).

Copyright © 2012 by the American Academy of Pediatrics

FINANCIAL DISCLOSURE: The authors have indicated they have no fi nancial relationships relevant to this article to disclose.

FUNDING: Supported by a grant from the Children’s Research Institute and Children’s Hospital and Health System Foundation at Children’s Hospital of Wisconsin.

Validity of Respiratory Scores in Bronchiolitis

INTRODUCTIONBronchiolitis is a dynamic illness that requires frequent assessment of respiratory status in inpatient and outpatient settings.1 Respiratory scores are used to assess initial respiratory status, follow trends, and evaluate response to therapeutic interven-tions. There are several different respiratory scores in existence; they vary greatly in composition, from 2 to 5 components on review of current literature.2 The Respiratory Distress Assessment Instrument (RDAI) is one of the most commonly used respi-ratory scores3–16 and was developed to assess epinephrine response in wheezing infants and initially used by a small pool of study investigators.13 When used as a marker of change with respiratory rate, it is called the Respiratory Assessment

Page 2: Validity of Respiratory Scores in Bronchiolitishosppeds.aappublications.org/content/hosppeds/2/4/202.full.pdf · HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY

HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY OF PEDIATRICS

203|

Change Score (RACS). The RDAI con-tains fewer markers of respiratory sta-tus than many other scores (wheezing and retractions; respiratory rate is included when measuring change) (Fig 1) and was initially used to deter-mine change in respiratory status as it related to bronchodilator use.13–16 The RDAI validity was established by com-paring retractions with other work of breathing markers such as grunting, nasal fl aring, and breath sounds.13

Although the focus of the RDAI is on wheezing and dyspnea alone, the Children’s Hospital of Wisconsin Res-piratory Score (CHWRS) has a broader focus. It contains 8 markers of respira-tory status, including dyspnea, breath sounds, retractions, heart rate, oxygen-ation, respiratory rate, activity, and cough ability/secretions (Fig 2) and in addition chest x-ray fi ndings and surgical status as it was initially used as a global respi-ratory score. It was created by a panel of local clinicians and respiratory therapists (RTs) after reviewing scores in existence. The CHWRS is a more comprehensive instrument than the RDAI, although its validity has not been established.

Among the many varying respiratory scores for bronchiolitis, reliability and validity have not been adequately eval-uated. To the best of our knowledge, no previous studies have established interrater reliability among the same large pool of raters who use the score in practice. Establishing the validity of respiratory scores in bronchiolitis can be challenging because there is no gold standard for overall respiratory status. Although the RDAI has been validated as an assessment tool for broncho-spasm, its validity constructs may have been too narrow to clearly demonstrate an accurate assessment of respira-tory status in infants with bronchiolitis.

FIGURE 1 RDAI. aRespiratory rate is only used to calculate the RACS, which is the difference in the wheezing and retraction score plus the change in respiratory rate. For example, an increase/decrease in respiratory rate of <5% is counted as a change of 0, an increase/decrease of 6% to 15% is counted as a positive/negative change of 1, and an increase/decrease of 16% to 25% is counted as a positive/negative change of 2.

FIGURE 2 CHWRS. Cath lab, catheterization laboratory; FIO2, fraction of inspired oxygen; HR, heart rate; lpm, liters per minute; RR, respiratory rate; WOB, work of breathing; lpm, liters per minute; RA, room air; Insp, inspiratory; Exp, expiratory.

Bronchiolitis morbidity is more com-plex than bronchospasm alone,1 and it is possible that a more comprehensive score may best measure the overall

respiratory status. Therefore, more thor-ough investigation of the performance of clinical respiratory scores in bron-chiolitis is needed.1,2

Page 3: Validity of Respiratory Scores in Bronchiolitishosppeds.aappublications.org/content/hosppeds/2/4/202.full.pdf · HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY

HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY OF PEDIATRICS

204 | VOLUME 2 • ISSUE 4 www.hospitalpediatrics.org

We sought to study the validity and reli-ability of 2 different types of respira-tory scores, the RDAI and the CHWRS, in the practical setting of a large group of RTs. We further sought to identify which respiratory score components were most important in contributing to overall respiratory status.

METHODSWe performed a prospective cohort study of respiratory score validity in an infant population hospitalized via direct admission or evaluated in the emer-gency department (ED) at Children’s Hospital of Wisconsin (CHW) with bronchiolitis between November 2007 –April 2008 and October 2008–April 2009 (Fig 3). The study was approved by the CHW institutional review board, and written informed consent was obtained from parents or guardians.

Inclusion criteria were as follows: (1) age of 0 to 365 days; (2) clinical evidence of bronchiolitis; and (3) symptoms starting within 7 days of presentation. Research assistants evaluated patient charts for the following terms: upper or lower respiratory tract infection, bron-chiolitis, bronchospasm, or fi rst-time wheezing, and evaluated the examina-tion section for wheeze, crackles, and/or retractions. Infants who had cystic fi brosis, congenital heart disease, croup, or pneumonia were excluded, as were those who had a history of asthma, wheezing, or bronchodilator use during a previous illness. The ED research assistants were not present from midnight until 7 AM, and charts of infants seen in the ED during this time were not reviewed.

RTs participated in a 30-minute RDAI and CHWRS training session before the start of each respiratory season.

It should be noted that CHWRS is the RT “home score” and is used fre-quently throughout the respiratory season. Before the study, the RTs had not used the RDAI. Both scores contain categories for wheezing and retractions and include respira-tory rate; however, the way in which these categories are scored differs (Figs 1 and 2).

For admitted patients, an addi tio n al condition was placement in the bron-chiolitis treatment protocol. This pro-tocol is an RT-driven pathway for the treatment of bronchiolitis that has been effective in reducing length of stay (LOS) at CHW.17 The proto-col uses the CHWRS to determine

need and frequency of respiratory treatments.

Infants in the ED were evaluated by RTs who used both the CHWRS and RDAI. The RTs did not participate in ED patient care, and treatment deci-sions were determined by the ED physician, unaware of the scores. If the patient was admitted and placed in the bronchiolitis protocol, the RTs continued to use the CHWRS and RDAI when performing respiratory assessments. All RT treatment deci-sions were based on the CHWRS. In addition, patients periodically received CHWRS and RDAI assessments by 2 RTs to establish interrater reli-ability. These RTs conducted their

FIGURE 3 Patient fl ow.

Page 4: Validity of Respiratory Scores in Bronchiolitishosppeds.aappublications.org/content/hosppeds/2/4/202.full.pdf · HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY

HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY OF PEDIATRICS

205|

assessments separately but within 5 minutes of each other.

Outcomes and Analyses

Outcomes of this study included an assessment of the validity, reliability, and short-term responsiveness of the RDAI and CHWRS. Criterion validity, the ability to compare a new mea-sure (in our case, a respiratory score) with a known standard, is not possible because there is no gold standard for overall respiratory status in bronchiol-itis. We therefore chose to assess the discriminative and construct validity of the 2 respiratory scores.

Discriminative validity focused on ED disposition because this was believed to be a marker of disease severity. Discriminative validity, the ability of the respiratory score to discriminate between admission and discharge, was determined by constructing receiver operating characteristic (ROC) curves for each respiratory score performed in the ED with admission (yes/no) as the outcome. In addition, an item anal-ysis of the RDAI and CHWRS individual score components was performed. By using multivariate logistic regression, we determined the score components that were more closely related to ED admission.

Construct validity focused on the rela-tionship of hospital LOS and respira-tory scores measured at the time of admission, at ∼24 hours from admission, and at discharge. Construct validity uses related markers to serve as prox-ies for a gold standard. We hypoth-esized that there was a relationship between the pace of improvement, LOS, and the overall respiratory status of the patient. We created the follow-ing validity constructs from these vari-ables. First, we assumed that patients

whose initial respiratory status was most severe would have a longer LOS. Thus, we determined the Spearman’s rank correlation between respiratory score at admission and LOS. Second, we assumed that infants who had more severe respiratory distress on admission would have the highest ini-tial scores and longest LOS and that all infants should have low scores at dis-charge. Thus, we determined the cor-relation between change in respiratory score from admission to discharge and LOS. Third, we assumed that infants whose respiratory status improved more quickly over the fi rst 24 hours of hospitalization would have a shorter LOS. Thus, we determined the correla-tion between the change in respiratory score in the fi rst 24 hours and LOS.

We also assessed the short-term res-ponsiveness of the respiratory scores. Short-term responsiveness refers to the score’s ability to detect a clinical change in respiratory status over a short period of time (eg, 15 minutes’ postintervention). This is important for the purpose of establishing the short-term response to an intervention, which is a common use for respiratory scores in bronchiolitis and recommended by the American Academy of Pediatrics’ bronchiolitis guidelines.1 Establishing short-term responsiveness in bronchi-olitis is challenging because no single intervention reliably changes respi-ratory status in bronchiolitis (unlike bronchodilators in asthma). If both the RDAI and CHWRS were measur-ing clinical change accurately and similarly, their assessment of change should correlate. Thus, we determined the correlation of change between the CHWRS and RDAI (RACS) before and after interventions such as suc-tioning, bronchodilators, and hyper-tonic saline.

Finally, each score’s interrater reli-ability was evaluated. The intraclass correlation coeffi cient was used to determine agreement between 2 RTs performing respiratory scores within 5 minutes of each other.

Sample Size

In our estimates of sample size, we used the construct validity outcomes examining respiratory score correla-tions with LOS. To detect a difference of 0.2 between correlation coeffi -cients of the CHWRS and RDAI, we calculated that 111 patients would be needed. All analyses were performed by using SPSS version 17 (SPSS Inc, Chicago, IL).

RESULTSThree hundred sixteen patients in the ED were eligible for the study and 216 were enrolled by ED research assis-tants. Due to RT time constraints, 151 of the enrolled patients had scores completed in the ED. An additional 44 patients were enrolled on the hospital fl oor (admitted directly to the hospi-tal), creating a study population of 195 patients (Fig 3). From this, 95 patients were discharged from the ED and 100 were admitted and placed in the bron-chiolitis treatment protocol. Ninety-eight patients who were admitted had scores completed from admission to discharge. The baseline study popula-tion characteristics are found in Table 1.

Discriminative Validity

ROC curves were constructed for each score to determine ability in the ED to predict disposition (Fig 4). The area under the curve for the CHWRS was 0.68 with a cutoff point of 7.5 (scores >7.5 predicting admission), giving a sensitivity of 0.65 and a specifi city of 0.65. The area under the curve for

Page 5: Validity of Respiratory Scores in Bronchiolitishosppeds.aappublications.org/content/hosppeds/2/4/202.full.pdf · HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY

HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY OF PEDIATRICS

206 | VOLUME 2 • ISSUE 4 www.hospitalpediatrics.org

the RDAI was 0.51, and no conclu-sion could be made regarding a cutoff point because it was not predictive of disposition.

Item Analysis

We evaluated the association between individual components of the respira-tory score and need for hospital admis-sion from the ED by using multivariate logistic regression. The dependent variable was admission to the hospital (yes/no), and the independent vari-ables were the score components. By using the CHWRS, oxygen need was most signifi cantly correlated with hos-pital admission. The respiratory rate in the CHWRS and subcostal retrac-tions in the RDAI also correlated with admission (Table 2).

Construct Validity

There was no statistically signifi cant correlation between respiratory score on admission and LOS with either

the RDAI or CHWRS (RDAI: r = 0.04, P = .71; CHWRS: r = 0.05, P = .61). Neither score demonstrated correla-tion between change over the fi rst 24

hours and LOS (RACS: r = 0.07, P = .67; CHWRS: r = –0.23, P = .06). There was a modest correlation between change in score from admission to

TABLE 2 Odds Ratios of Individual Respiratory Score Components Predicting Hospital Admission

Score Component Odds Ratio 95% CI

CHWRS Breath sounds 0.95 0.40–2.27

Dyspnea 0.55 0.19–1.56

Retractions 1.84 0.77–4.43

Respiratory rate 1.68 1.00–2.83

Heart rate 1.27 0.86–1.86

Oxygen need 61.12 7.66–487.37

Activity 1.02 0.51–2.02

Cough ability, secretions 1.28 0.91–1.81

Lung sounds 1.18 0.59–2.35

RDAI

Wheezing, expiration 0.84 0.54–1.31

Wheezing, inspiration 1.16 0.48–2.80

Wheezing, location 0.099 0.48–2.03

Retractions, suprasternal 0.61 0.31–1.21

Retractions, intercostal 0.89 0.38–2.08

Retractions, subcostal 2.67 1.41–5.05

TABLE 1 Baseline Study Population Cha rac teristics

Male gender, n (%) 116 (59.5)

Race, n (%) White 80 (41)

African American 72 (36.9)

Hispanic 29 (14.9)

Asian 3 (1.5)

Other 11 (5.6)

Previous treatment, n (%)

Albuterol with this illness 136 (69.7)

Steroids with this illness 10 (5.1)

Prematurity (gestational age <37 wk)

26 (13.3)

Variable, mean ± SD

Age, d 121 ± 99

Gestational age, wk 39 ± 2.2

Initial respiratory rate,a breaths per min

57 ± 14

Initial oxygen saturation on room air, %a

94 ± 7

Hospital LOS, d 3.6 ± 2.3

a Not all 195 patients had an initial charted respiratory rate (n = 194) and/or oxygen saturation on room air (n = 184). FIGURE 4 ROC curves predicting ED disposition.

Page 6: Validity of Respiratory Scores in Bronchiolitishosppeds.aappublications.org/content/hosppeds/2/4/202.full.pdf · HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY

HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY OF PEDIATRICS

207|

discharge and LOS with the RACS (r = 0.27, P = .01), but the correlation was poor with the CHWRS (r = –0.05, P = .62). For short-term responsiveness, there was a mild correlation between the change in the CHWRS and RACS after an intervention (r = 0.39, P = .04).

Interrater Reliability

The intraclass correlation coeffi cient was signifi cantly different between the 2 scores at the 95% confi dence inter-val (CI) level; the CHWRS was 0.73 (95% CI: 0.60–0.82 [n = 72]) and the RDAI was 0.39 (95% CI: 0.17–0.58 [n = 65]) with a group of 46 RTs.

DISCUSSIONTo the best of our knowledge this study is the fi rst to attempt to rigor-ously validate the ability of 2 distinct types of respiratory scores to assess overall respiratory status in infants with bronchiolitis. We determined that the more comprehensive score, the CHWRS, is somewhat discriminative of admission from the ED, although the RDAI is not. In particular, respiratory rate, oxygen need, and the presence of subcostal retractions were most predictive of ED disposition. Neither score performed well using LOS as a validity construct. We established that the RACS and CHWRS showed some agreement when assessing respira-tory status in response to an interven-tion. The interrater reliability of the CHWRS in a large pool of RTs was good, whereas the interrater reliability of the RDAI was relatively poor.

Our study suggests that the RDAI may be more limited than the CHWRS in assessing overall respiratory status at a single point in time because only the CHWRS had modest utility in predicting admission. If a score is appropriately

identifying children who are more ill with bronchiolitis, one would expect it would discriminate ED admission and discharge. Previous studies have enrolled participants based on ini-tial RDAI8–12 but consideration may need to be given to more comprehen-sive tools. Given that oxygen need is a dominant factor in admission and that it had the highest odds ratio in our item analysis, its absence in the RDAI may limit its use for this purpose. Alternatively, no score may perform adequately because disposition could be infl uenced by factors other than disease status alone. Parental prefer-ences, follow-up, dehydration, inabil-ity to take oral fl uids, patient age, and other issues can determine a provid-er’s decision to admit. In addition, in our study, the RTs were completing the score, and their perceptions of illness severity as based on the score could differ from those of the physicians who are deciding to admit.

We sought to identify the most impor-tant respiratory score components in assessing overall status because expert opinion varies on which com-ponents are most important. The CHWRS individual score components of oxygen need, respiratory rate, and the RDAI assessment of subcostal retractions were independently corre-lated with need for hospital admission. Oxygen need is not part of the RDAI but has been shown to be an impor-tant marker of illness severity.18–21 Respiratory rate is part of the RDAI only when used as a marker of change within the RACS. Although both scores contain markers of retractions, it was interesting that only the RDAI’s assessment of subcostal retractions was associated with admission. These results lead us to speculate that per-haps all retractions are not created

equal. Perhaps subcostal retractions have particular value in the respiratory assessment of bronchiolitis because they are either more noticeable or more sensitive of lower airway respi-ratory compromise than intercostal or suprasternal retractions. Similar to our fi ndings, a recent study found low oxy-gen saturation, high accessory muscle use, and respiratory rate predicted need for major intervention in bron-chiolitis.22 Based on our study’s fi nd-ings, the ideal respiratory score should minimally contain: (1) oxygen need; (2) respiratory rate; and (3) retractions.

Although we did not fi nd that the RDAI or CHWRS was valid by using LOS-based constructs, this could be due to problems with the scores or problems with the validity constructs themselves. Our initial assumptions likely underestimated the impact of items outside of respiratory score on LOS. Previous research has identifi ed infl uences on LOS in bronchiolitis, including prematurity, feeding abil-ity, oxygen need, environmental and social factors, practice variation, and others.18,23–28 Furthermore, bronchiolitis is a dynamic illness that may worsen after admission, thus confounding the relationship between initial scores and improvement with LOS. The American Academy of Pediatrics’ 2006 diagnosis and management guidelines specifi -cally state “The physical examination refl ects the variability in the disease state and may require serial obser-vations over time to fully assess the child’s status.”1 Thus, assessments at a few points in time may not be suffi cient.

The CHWRS and RACS correlated mildly in response to an interven-tion, which suggests that at least they could be measuring similar aspects of

Page 7: Validity of Respiratory Scores in Bronchiolitishosppeds.aappublications.org/content/hosppeds/2/4/202.full.pdf · HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY

HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY OF PEDIATRICS

208 | VOLUME 2 • ISSUE 4 www.hospitalpediatrics.org

respiratory status. However, it does not indicate which score is more accurate at measuring change. Although the American Academy of Pediatrics rec-ommends using objective measures of respiratory status to judge response to interventions such as bronchodilators, evidence for the validity of respiratory scores in this outcome is diffi cult to establish.

The CHWRS was the RTs’ “home score,” and its better reliability was therefore not surprising. The RTs had not used the RDAI, but wheezing, retractions, and respiratory rate are part of the CHWRS. Although they are measured differently (quantita-tively and qualitatively), no additional aspects of respiratory status were evaluated. In contrast to previous stud-ies demonstrating good reliability,11,13,15 we found that the RDAI had relatively poor interrater reliability in our insti-tution. The reliability differences may indicate a need for more thorough training and regular use if a score is to be implemented. In addition, although the CHWRS contains more compo-nents and at fi rst glance would seem to take more time to complete, our RTs stated the time required was similar between the 2 scores. Verbal feed-back from the RTs indicated that they struggled with the wheezing compo-nent of the RDAI, resulting in a slightly prolonged time to complete fewer score components. This feedback fur-ther demonstrates the importance of regular score use.

Our study contains some important limitations. First, it is limited to the use of respiratory scores by RTs. Results may be applicable only where respira-tory interventions and pathways are RT driven. Second, the CHWRS is more familiar and routinely used by RTs

at our institution. Third, the CHWRS score determined the patient care decisions and its elements may have been more highly prioritized. Fourth, the item analysis was performed on ED disposition, and caution should be used when generalizing these results to other purposes such as short-term responsiveness. Fifth, LOS and hos-pital admission may be infl uenced by items other than respiratory sta-tus, limiting the validity constructs’ effectiveness. Sixth, some patients with bronchiolitis were missed either due to RT or research assistant avail-ability. Finally, our study contained slightly fewer patients than the goal for the validity construct outcomes and may have been underpowered; however, correlations were not near signifi cance.

CONCLUSIONSAnalysis of the RDAI and CHWRS suggests that the CHWRS more effec-tively identifi ed infants admitted with bronchiolitis. The greater interrater reli-ability of the CHWRS demonstrates the importance of training in establishing a reliable score. Both scores were poorly correlated with LOS-based validity con-structs. Future efforts toward score improvement should consider using the most important elements of the RDAI and CHWRS (ie, oxygen need, respiratory rate, subcostal retractions) to develop a more effective tool. Validity of respiratory scores should also be studied among the individu-als who often make patient care deci-sions such as ED disposition, hospital disposition, and continuation of vari-ous respiratory treatments. Additional study is needed to determine the short-term responsiveness of respi-ratory scores to validate their use in guiding bronchiolitis management.

REFERENCES

1. American Academy of Pediatrics Sub-

committee on Diagnosis and Management

of Bronchiolitis. Diagnosis and manage-

ment of bronchiolitis. Pediatrics. 2006;

118(4):1774–1793.

2. Hartling L, Wiebe N, Russell K, Patel H,

Klassen TP. A meta-analysis of randomized

controlled trials evaluating the effi cacy of

epinephrine for the treatment of acute viral

bronchiolitis. Arch Pediatr Adolesc Med.

2003;157(10):957–964.

3. Kellner JD, Ohlsson A, Gadomski AM, Wang

EE. Effi cacy of bronchodilator therapy in

bronchiolitis. Arch Pediatr Adolesc Med.

1996;150(11):1166–1172.

4. Grewal S, Ali S, McConnell DW,

Vandermeer B, Klassen TP. A randomized

trial of nebulized 3% hypertonic saline

with epinephrine in the treatment of acute

bronchiolitis in the emergency department.

Arch Pediatr Adolesc Med. 2009;163(11):

1007–1012.

5. Kuzik BA, Al-Qadhi SA, Kent S, et al.

Nebulized hypertonic saline in the treat-

ment of viral bronchiolitis in infants. J Pediatr

. 2007;151(3):266–270, 270.e1.

6. Mesquita M, Castro-Rodriguez JA, Heinichen

L, Fariña E, Iramain R. Single oral dose

of dexamethasone in outpatients with

bronchiolitis: a placebo controlled trial.

Allergol Immunopathol (Madr). 2009;37(2):

63–67.

7. Plint AC, Johnson DW, Patel H, et al;

Pediatric Emergency Research Canada

(PERC). Epinephrine and dexamethasone

in children with bronchiolitis. N Engl J Med.

2009;360(20):2079–2089.

8. Klassen TP, Sutcliffe T, Waters LK, Wells

GA, Allen UD, Li MM. Dexamethasone in

salbutamol-treated inpatients with acute

bronchiolitis: a randomized, controlled trial.

J Pediatr. 1997;130(2):191–196.

9. Corneli HM, Zorc JJ, Mahajan P, et al;

Bronchiolitis Study Group of the Pediatric

Emergency Care Applied Research Network

(PECARN). A multicenter, randomized,

controlled trial of dexamethasone for

bronchiolitis [published correction appears

in N Engl J Med. 2008;359[18]:1972). N Engl

J Med. 2007;357(4):331–339.

10. Kuyucu S, Unal S, Kuyucu N, Yilgor E. Additive

effects of dexamethasone in nebulized

Page 8: Validity of Respiratory Scores in Bronchiolitishosppeds.aappublications.org/content/hosppeds/2/4/202.full.pdf · HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY

HOSPITAL Pediatrics® AN OFFICIAL JOURNAL OF THE AMERICAN ACADEMY OF PEDIATRICS

209|

salbutamol or L-epinephrine treated infants

with acute bronchiolitis. Pediatr Int. 2004;

46(5):539–544.

11. Schuh S, Coates AL, Binnie R, et al. Effi cacy of

oral dexamethasone in outpatients with acute

bronchiolitis. J Pediatr. 2002;140(1):27–32.

12. Schuh S, Coates AL, Dick P, et al. A single

versus multiple doses of dexamethasone in

infants wheezing for the fi rst time. Pediatr

Pulmonol. 2008;43(9):844–850.

13. Lowell DI, Lister G, Von Koss H, McCarthy

P. Wheezing in infants: the response to

epinephrine. Pediatrics. 1987;79(6):939–945.

14. Schweich PJ, Hurt TL, Walkley EI, Mullen N,

Archibald LF. The use of nebulized albuterol

in wheezing infants. Pediatr Emerg Care.

1992;8(4):184–188.

15. Klassen TP, Rowe PC, Sutcliffe T, Ropp LJ,

McDowell IW, Li MM. Randomized trial of

salbutamol in acute bronchiolitis. J Pediatr.

1991;118(5):807–811.

16. Menon K, Sutcliffe T, Klassen TP. A

randomized trial comparing the effi cacy

of epinephrine with salbutamol in the

treatment of acute bronchiolitis. J Pediatr.

1995;126(6):1004–1007.

17. Kotagal UR, Robbins JM, Kini NM,

Schoettker PJ, Atherton HD, Kirschbaum

MS. Impact of a bronchiolitis guideline:

a multisite demonstration project. Chest.

2002;121(6):1789–1797.

18. Green M, Brayer AF, Schenkman KA, Wald

ER. Duration of hospitalization in previously

well infants with respiratory syncytial virus in-

fection. Pediatr Infect Dis J. 1989;8(9):601–605.

19. Unger S, Cunningham S. Effect of oxygen

supplementation on length of stay for infants

hospitalized with acute viral bronchiolitis.

Pediatrics. 2008;121(3):470–475.

20. Shaw KN, Bell LM, Sherman NH. Outpatient

assessment of infants with bronchiolitis.

Am J Dis Child. 1991;145(2):151–155.

21. Zorc JJ, Hall CB. Bronchiolitis: recent

evidence on diagnosis and management.

Pediatrics. 2010;125(2):342–349.

22. Parker MJ, Allen U, Stephens D, Lalani A,

Schuh S. Predictors of major intervention in

infants with bronchiolitis. Pediatr Pulmonol.

2009;44(4):358–363.

23. Aujard Y, Fauroux B. Risk factors for severe

respiratory syncytial virus infection in infants.

Respir Med. 2002;96(suppl B):S9–S14.

24. Moler FW, Ohmit SE. Severity of illness

models for respiratory syncytial virus-

associated hospitalization. Am J Respir Crit

Care Med. 1999;159(4 pt 1):1234–1240

25. Wang EE, Law BJ, Stephens D. Pediatric

Investigators Collaborative Network on

Infections in Canada (PICNIC) prospective

study of risk factors and outcomes in patients

hospitalized with respiratory syncytial viral

lower respiratory tract infection. J Pediatr.

1995;126(2):212–219.

26. Horn SD, Smout RJ. Effect of prematurity

on respiratory syncytial virus hospital

resource use and outcomes. J Pediatr.

2003;143(suppl 5):S133–S141.

27. Christakis DA, Cowan CA, Garrison MM,

Molteni R, Marcuse E, Zerr DM. Variation

in inpatient diagnostic testing and mana ge-

ment of bronchiolitis. Pediatrics. 2005;115(4):

878–884.

28. Behrendt CE, Decker MD, Burch DJ,

Watson PH; International RSV Study Group.

International variation in the manage-

ment of infants hospitalized with resp-

iratory syncytial virus. Eur J Pediatr. 1998;

157(3):215–220.