Paediatric Respiratory Reviews 14 (2013) 146–151

Mini-Symposium: Research Methodology

Utilization of Patient-Reported Outcomes as a Step towards CollaborativeMedicine

Laura S. Blackwell *, Kristen K. Marciel, Alexandra L. Quittner

University of Miami, 5665 Ponce De Leon Blvd, Coral Gables, FL 33146

A R T I C L E I N F O

Keywords:

Patient Reported Outcomes

FDA

EMA

Health-Related Quality of Life

Collaborative Medicine

cystic fibrosis

S U M M A R Y

Patient-reported outcomes (PROs) have been successfully developed for a variety of chronic respiratory

diseases, such as asthma and cystic fibrosis (CF). They have recently been used to evaluate the efficacy of

new medications and assess current patient functioning. Although regulatory bodies have favored PROs

that measures symptoms, other domains of functioning, such as treatment burden, should be considered.

This review examines current guidelines for the development and application of PROs in clinical trials,

describes methods for selecting appropriate measures for paediatric populations, and presents a model

incorporating PROs into clinical practice. Guidance on interpretation of these measures and graphic

presentation of results are illustrated. PROs can serve as the link between the health care provider and

patient to foster collaborative and personalized medicine. This model promotes greater patient

responsibility, facilitates communication with providers, encourages shared decision-making, and

enhances adherence.

Published by Elsevier B.V.

EDUCATIONAL AIMS

� To describe the guidelines for developing� To choose appropriate PROs� To describe considerations for use of PROs in clinical trials and clinical use

Contents lists available at SciVerse ScienceDirect

Paediatric Respiratory Reviews

INTRODUCTION

Advances in health outcomes research have highlighted theimportance of measuring the effects of chronic diseases and theirtreatments from the patients’ perspective. A large body of researchhas shown that patients can reliably report on their symptoms,pain, side effects, and daily functioning;1–3 however, patient-reported outcomes (PROs) have only recently been incorporatedinto clinical research and practice. In contrast, traditional medicaloutcomes have focused on physiologic endpoints (e.g., pulmonaryfunction), that often do not capture these critical aspects of livingwith a chronic disease.4 As early as 1946, the World HealthOrganization defined health more holistically as ‘‘completephysical, mental and social well-being, not merely. . . the absenceof disease or infirmity’’.5 Although the WHO definition wasreleased over 60 years ago, significant progress in defining and

* Corresponding author. Tel.: +1 305 284 2097; fax: +1 305 284 4795.

E-mail addresses: l.blackwell@umiami.edu (L.S. Blackwell),

kristen.k.marciel@emory.edu (K.K. Marciel), aquittner@miami.edu (A.L. Quittner).

1526-0542/$ – see front matter . Published by Elsevier B.V.

http://dx.doi.org/10.1016/j.prrv.2013.04.003

implementing PROs has only occurred in the past 20 years.6 Thisreview examines current guidelines for the development of PROs,using a well-validated, internationally recognized instrument, theCystic Fibrosis Questionnaire-Revised.7,8 We also discuss the use ofPROs in clinical trials as primary or secondary endpoints,particularly in paediatric populations. Finally, we outline thebenefits of utilizing PROs in a clinical setting and providerecommendations for their implementation.

DEFINING PROS

PROs form the larger umbrella under which measures, such ashealth-related quality of life (HRQOL), are positioned. A PRO isdefined as ‘‘any report of the status of a patient’s health conditionthat comes directly from the patient, without interpretation of thepatient’s response by a clinician or anyone else’’.9 PROs includeHRQOL instruments that measure four core domains: 1) diseasestate and physical symptoms, 2) functional status (e.g., performingdaily activities), 3) psychological and emotional functioning, and 4)social functioning. Specifically, they include observable (e.g.,coughing, wheezing) or non-observable outcomes (e.g., pain,

Table 1A comparison of the FDA and EMA guidelines for PRO development and utilization in clinical trials

FDA guidelines EMA guidelines

Definition A PRO is any report made by the patient without clinician

influence

HRQOL is different from other PROs and is separate from the

core symptoms of the disease

Review of PROs Have a specific task force dedication to the development of

PROs (SEALD)

Consults with outside academic and clinical research

organizations

ePRO use ePROs must go through same stringent development

guidelines as paper-pencil documents

No formal language addressing ePROs

Recommendationof HRQOL instruments to use

Has strict guidelines for identification of PROs through

desired claims, endpoint model and anticipated therapeutic

benefits

Provides recommendations on validated instruments based

on area of interest

Recall period Daily diary strongly recommended (measurement of the

current state)

No requirements

End point model Accepts PROs as primary endpoints in addition to physio-

logic symptoms

Will only accept PROs as secondary endpoints to physio-

logic symptoms

Content Validity Evidence must be provided to prove response options are

relevant (i.e., saturation grids)

No comprehensive evidence is needed

Previously validatedin population/condition

Instruments must be validated on specific population prior

to being used in trials

Instruments must be validated on specific population prior

to being used in trials

Minimum Important Difference Can be generated by applying a variety of methods, e.g. an

anchor-based or a distribution-based approach or an

empirical rule

Should be based upon a combination of statistical reasoning

and clinical judgment

Translations Must provide evidence that the methods and results of the

translation process were adequate to warrant the validity of

the responses

No formal language addressing translations

L.S. Blackwell et al. / Paediatric Respiratory Reviews 14 (2013) 146–151 147

depressive symptoms) known only to the patient. Thus, theycontribute unique information about a patient’s health status,which promotes collaborative, patient-centered care.10

DEVELOPMENT OF PATIENT-REPORTED OUTCOMES

Efforts to develop reliable and valid PROs have been highlysuccessful, leading to the use of these instruments for severaldifferent purposes: 1) to serve as primary or secondary outcomesin clinical trials,9,11–14 2) to evaluate new pharmaceutical, surgical,and behavioral interventions, 3) to describe the impact of illness onpatient functioning,15 4) to analyze the costs and benefits ofmedical interventions, and 5) to aid in communication and clinicaldecision-making.10 Currently, 14% of registered clinical trialsreport using a PRO; however, this ranges from 3% to 24% based onthe disease.12 Recently, an inhaled antibiotic was approved by theFood and Drug Administration (FDA) using a PRO, the CysticFibrosis Questionnaire-Revised (CFQ-R)7,14 as the primary end-point. Results indicated that respiratory symptoms, as measuredby the CFQ-R Respiratory Symptoms scale, improved significantly,with concomitant increases in pulmonary function.14,16,17

In the US, the FDA has formally recognized the importance andutility of PROs for clinical trials by publishing formal recommen-dations in 2009.9 This Guidance describes both how PROs shouldbe developed and how they can be utilized as endpoints in clinicaltrials. While the FDA has produced a formal guidance, theEuropean Medicines Agency (EMA) has elected not to issue asimilar document and, instead, produced a reflection paperproviding general recommendations on the use of PROs in thecontext of the existing FDA Guidance.18 More recently, the EMAalso launched a Biomarker’s Qualification program to provide aformal mechanism for ratifying clinical trial endpoints, includingnew or existing PROs.19

There are several differences between the EMA and FDAguidelines (see Table 1). The FDA recommendations are extensive,with detailed criteria for their use and stringent restrictions onapproval of specific instruments. In contrast, the EMA providesmore global statements and broad advice.20 There are some keydifferences in the FDA and EMA use of PROs as endpoints for drugregistration trials. The FDA will accept a PRO as a primary endpoint,whereas the EMA only accepts them as a secondary endpoint, withphysiologic measures considered acceptable as primary endpoints.

Each of these documents has provided recommendations on thesteps needed to produce and utilize a reliable and valid instrument.

Instrument Development

The FDA guidance outlines a number of specific steps for PROdevelopment, including: 1) hypothesizing a conceptual framework(e.g., linking concepts to product claim), 2) obtaining patient inputto draft a preliminary instrument (e.g., qualitative interviews), 3)testing and modifying the concepts, 4) confirming the conceptualframework and evaluating psychometric properties (e.g., relia-bility, validity), 5) collecting, analyzing, and interpreting data (e.g.,determining the minimal important difference [MID]), and 6)modifying the instrument.9 Given that this is an extensive,iterative process, the first step in selecting a PRO instrument isto locate and evaluate existing measures (see Table 2).1 Thecreation of a PRO is illustrated using the CFQ-R (see Table 3).7,8,21,22

Mode of Administration

When developing a PRO, the mode of administration should beconsidered. Data collection methods include paper-pencil, elec-tronic, and telephone-administered. Historically, PROs have beencompleted using paper-based versions. However, this adds extratime for data entry and scoring. Electronic methods offer a numberof advantages over paper-pencil.23 First, electronic PROs (ePROs)reduce missing data and errors, which can improve data qualityand integrity. Second, they simplify data management viasimultaneous data entry–reducing errors and personnel time.10

Third, adherence to PRO completion is better with electronic vs.paper-pencil methods.24,25 Fourth, the cost of implementing anePRO is cheaper. For example, the up-front costs for softwaredevelopment and purchase of devices are higher initially thanpaper methods; however, costs for data entry and cleaning aresubstantially less.23

Financial Resources

One potential barrier to the development and/or refinement ofPROs is obtaining adequate funding to complete this process. In arecent ‘‘vision’’ workshop held at the National Institute of ChildHealth and Human Development, the committee determined that

Table 2Literature and internet sources for PRO instruments

Name Link

PRO Literature Sources

PubMed http://www.ncbi.nlm.nih.gov/entrez/query.fcgi

PsycINFO www.psychoinfo.com

International Bibliography of Social Sciences (IBSS) http://www.lse.ac.uk/collections/IBSS/

Exerpta Medica (EMBASE) www.embase.com

Social Science Citation Index (SSCI1) http://scientific.thomson.com/products/ssci/

PRO Instrument Review Sources

QOLID www.qolid.org

On-Line Guided Assessment (OLGA) www.olga-qol.com

Mental Measurement Yearbook Test Reviews On-Line (MMY) www.unl.edu/buros

Health and Psychosocial Instruments (HaPI) www.embase.com

http://www.ovid.com/

Behavioral Tests & Meausures in the Health Sciences Resource Guide http://info.med.yale.edu/library/reference/publications/tests.html

L.S. Blackwell et al. / Paediatric Respiratory Reviews 14 (2013) 146–151148

the development of PROs, particularly for children and adolescentswith chronic illnesses, should be a funding priority.26 Threepotential funding sources are available: academic, government(e.g., NIH), and industry. As PROs are increasingly utilized in bothclinical trials and practice, identifying sources of funding is critical.

USE OF PROS IN CLINICAL TRIALS

There are several important issues to consider when using PROsin clinical trials. First, the PRO should be measured as frequently asthe other endpoints. This ensures adequate power and can identifythe earliest point at which the medication or intervention has itseffect. For example, in the Cayston trials, implementing the CFQ-RRespiratory Symptoms scale at both Baseline and Day 14 showedthat the antibiotic had already produced a significant improve-ment in respiratory symptoms.14 Second, it is critical to administera PRO prior to any assessments of health (e.g., lung function,weight, height), to minimize the possibility that these healthindices will bias the patient’s report. Third, feasibility should beevaluated. This includes length of the measure and the ease withwhich patients can understand and respond to the items. Finally,

Table 3An example of the development of a PRO endpoint, the CFQ and CFQ-R (www.psy.mia

Steps

Literature Review � Reviewed literature databases (e.g.,

� Examined websites with reviews of

� Identified key symptoms for CF (e.g

Conceptual Framework � Performed physician interviews54

� Conducted focus groups with patien

� Identified six generic (e.g., Physical

(e.g., Treatment Burden)

� Hypothesized correlations among co

� Determined mode of administration

Creation of the PRO Instrument � Performed open-ended interviews w

� Transcribed and content-analyzed in

� Utilized Atlas.ti to identify key conc

� Generated items for preliminary ins

� Selected recall period, mode of adm

� Determined scaling; standardized 0

� Conducted cognitive testing of item

Evaluation of Psychometric Properties � Administered draft instrument

� Deleted items with significant floor

� Assessed reliability and construct va

� Evaluated the hypothesized correlat

� Finalized instrument with content, f

Analysis and Interpretation of Data � Identified the minimal important di

� Evaluated treatment response using

� Confirmed conceptual model

� Collected normative data on nationa

Modify Instrument � Added items to the Treatment Burd

� Translated and performed cultural a

� Created a pictorial, preschool-age ve

respondent characteristics, such as age, gender, literacy level,cultural and language identity should be considered whenselecting the appropriate PRO. Whenever possible, the respondentshould be the patient. However, there are populations for whom aproxy respondent is needed, including patients who are too young,too sick, or have significant cognitive disabilities or developmentaldelays that preclude them from responding. In these cases, it maybe helpful to have a proxy respondent, such as a parent or caregiver,complete the measure.1,27

Generic vs. Disease-Specific PROs

Although a number of generic PROs are available, such as theSF-3628 and the KINDL,29 these instruments typically lack theprecision and sensitivity needed to measure concepts for aparticular disease.8,30. Thus, generic PROs may not be responsiveto small, but important changes resulting from a therapeuticintervention. Several studies have used generic HRQOLinstruments in patients with CF, such as the QWB and theChildren’s Health Questionnaire, with disappointing results interms of specificity and sensitivity.31,32 In addition, these

mi.edu/CFQ_QLab)

PubMed, PsychINFO) to locate existing PROs30

PROs

., cough, sputum production, shortness of breath)

ts and parents54

Functioning, Vitality) and six disease-specific HRQOL domains

nstructs and identified potential endpoint model

ith patients and their caregivers

terviews

epts and produce saturation grids demonstrating content validity

trument

inistration

to 100 scores, with higher scores indicating better HRQOL

s with revisions to questions and rating scales21

or ceiling effects

lidity of scales

ions among constructs and health outcomes (e.g., lung function)

ormats, scoring and procedures for administration7,22

fference score (MID)52

cumulative distribution and responder analyses14,16

l CF sample by age, gender and disease severity8

en Scale, expanded Digestive Symptoms Scale, developed Sinus Symptoms scale

daptations in Europe, Latin America and Australasia

rsion1

L.S. Blackwell et al. / Paediatric Respiratory Reviews 14 (2013) 146–151 149

instruments are not accepted by the FDA for approval of newdrugs.

Selecting Endpoints for Clinical Trials

There are several types of endpoints that can be used in clinicaltrials for respiratory conditions, such as lung function, growth,symptom measures, and multidimensional HRQOL instruments.For the EMA, a physiologic endpoint should be selected as primary,but PROs can be used as secondary outcomes. The challenges ofselecting optimal endpoints is best illustrated by CF. First, CF is achronic and deteriorating medical condition, however, patients arenow maintaining a reasonable level of lung function throughoutchildhood and may only exhibit gradual declines. This makes itmore difficult to show improvements in pulmonary function,which has been the focus of prior clinical trials over the past 20years.33 In addition, there is tremendous variability in lungfunction within patients, necessitating larger sample sizes andgreater power to detect group differences. Other endpoints, such asnumber of pulmonary exacerbations, could be useful in clinicaltrials, but standardized definitions and scoring procedures areneeded. In addition, with rates of exacerbation decreasing, thisendpoint requires longer trials which increase the burden onpatients and competition for patients.34 Thus, PROs are apotentially important endpoint.

Currently, both the FDA and EMA support the use of symptommeasures as endpoints for trials in respiratory diseases. Both thePediatric Asthma Quality of Life Questionnaire (PAQLQ)35 and theCFQ-R have been used successfully in clinical trials. Of note, both ofthese instruments are multidimensional, however, only changes insymptoms have been utilized for drug registration trials. As new,disease-modifying drugs, such as VX-77036 are developed andapproved, it is important to measure and assess changes in otherdomains of functioning, such as gastrointestinal symptoms,weight, body image, and physical functioning. In addition, becausethe treatment regimen for CF is so time-intensive, it is important toassess patient perceptions of treatment burden. In a recentepidemiological study of CF patients37 greater treatment complex-ity was associated with worse Treatment Burden on the CFQ-R.Further, a recent study evaluating a dry power formulation oftobramycin, which significantly reduced time and complexity ofadministration, was associated with improvements in treatmentburden on the CFQ-R.38 Thus, it is important to evaluate the effectsof new medications and treatments on a broader range of patient–reported outcomes in both clinical trials and clinical settings.

Figure 1. Graphic depiction of one administration o

CLINICAL UTILITY OF PROS

The International Society for Quality of Life Research (ISOQOL)recently published guidelines for implementing (PROs) in clinicalpractice.10 These guidelines suggest that clinicians: 1) determinethe focus of the PRO assessment, 2) identify the target population,location (e.g., home, clinic or hospital administration) andfrequency of assessments, 3) choose the instrument, 4) decideon mode of administration, 5) develop strategies for providingfeedback to patients on their results, 6) create aids for interpreta-tion of results (e.g., a profile of scores), 7) select appropriateinterventions and provide referrals, and 8) evaluate the impact ofusing the PRO to improve quality of care.

Clinical Applications of PROs

The major advantage of using PROs in clinical settings is tosystematically collect information on symptoms and dailyfunctioning from the patient’s perspective. This is particularlyimportant for patients who have chronic diseases—which tend topersist for long periods of time, affect several domains offunctioning (e.g., school or work absences), and may requireadherence to time-consuming and complex medical regimens.39,40

For example, many physical and emotional symptoms, such aschest congestion, cough, anxiety and depression cannot bemeasured ‘‘objectively.’’

Another advantage of using PROs in clinical practice is tofacilitate communication between the patient and clinician.Review of the patient’s data can serve as a mechanism forpromoting collaborative care and shared decision-making. Forexample, an adolescent with CF may have a busy school and socialschedule, finding it difficult to perform all prescribed treatments. Areview of their PRO scores may reveal increased treatment burden,prompting a discussion about barriers to adherence and thepossibility that the regimen could be altered. This paradigm,termed ‘‘collaborative medicine,’’ includes discussions and jointdecision-making between patient and provider.41,42 Although thephysician is considered most knowledgeable about the diseaseprocess, the patient is viewed as the expert on how his/her diseaseand its treatments affect daily life (i.e, school functioning,treatment burden, side effects).

If the PRO is collected systematically over time, either usingpaper-pencil or computer (or tablet) the patient and his/herclinician can discuss progression of the disease, the efficacy ofnewly prescribed treatments, and steps to maximize the patient’s

f the CFQ-R with normative comparison data.

0

10

20

30

40

50

60

70

80

90

100

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

Respiratory Symptoms

FEV1 % predictedCF

Q-R

Resp

irat

ory

Sym

ptom

s Sc

ale

FEV1

% p

redi

cted

Year 1 Year 2

Figure 2. Graphic depiction of the relationship of the CFQ-R Respiratory Symptoms scale and FEV1 over time.

L.S. Blackwell et al. / Paediatric Respiratory Reviews 14 (2013) 146–151150

functioning. This model emphasizes the patient’s responsibility formanaging his/her condition and empowers the patient tocommunicate more directly with providers, problem-solve issuesrelated to adherence, and participate in medical decision-mak-ing.43–46

This is particularly important for adolescents with chronicillnesses for whom rates of adherence typically decline, leading toworse health outcomes.47–49 Integrating children and adolescentswith chronic respiratory conditions earlier and more fully into thehealth care process is likely to increase their adherence andfacilitate a more successful transition to adult care.50,51

Presentation and Interpretation of Results

In order to facilitate interpretation of results, the scores shouldbe presented visually, possibly in graph form (see Figure 1). Ifnormative data on the illness group are available, the patient’sscores can be compared to this reference group. For example, theCFQ-R has normative data for patients based on age, gender, andlung function.8 In addition, an individual patient’s scores can becollected to indicate changes in functioning over time. Changes inPRO scores can also be correlated with changes in health outcomes,such as lung function and body mass index (BMI) (see Figure 2).

Interventions and Referrals

As part of collaborative care, significant differences between thepatient’s data and normative data, or substantial changes incurrent versus previous functioning, should be discussed with thepatient. In addition, minimal important difference (MID) scoreshave been established for some PROs and can be used to interpretthe clinical significance of changes.52,53 For example, the MID forthe CFQ-R Respiratory Symptoms Scale is 4 points and thus, if anew medication is prescribed (e.g., VX-770) and 4-point differencein respiratory symptoms is observed, the provider can be confidentthat the patient has attained a meaningful improvement.

Effectiveness of Using PROs in Clinical Practice

Evaluating the impact of implementing PROs in clinical practicecan be determined in several ways. First, quality improvementstrategies can be employed to track the consistency of PROadministration within a given time period (e.g., annually,quarterly). Second, plan-do-study-act (PDSA) cycles can be utilizedto ensure that providers review and discuss the PRO data with

clinical team members and patients.10 Outcomes may include: 1)number of interventions or referrals provided to patients, 2)patient satisfaction, 3) ratings of patient-provider communication,and 4) changes in health status.

CONCLUSIONS AND FUTURE DIRECTIONS

PROs have proven to be useful tools for measuring changes inpatients’ symptoms and daily functioning. In fact, it is thought thatPROs may even be more sensitive to change than physiologicmeasures. In cases in which treatment effects are known only tothe patient (e.g., frequency of cough, chest tightness), direct andformal assessment of patient perceptions is the only validmeasurement approach. PROs have been successfully utilized inboth drug registration, device trials and clinical practice, providingmeaningful information to both the patient and clinical team. Inclinical settings, these measures can be used to provide a linkbetween the patient and provider, offering a more collaborativemodel of care. PROs can provide immediate feedback to the patienton how he/she is functioning across a variety of domains, whichultimately personalizes the approach to their care. In the lastdecade, PROs have demonstrated their value and utility acrossrespiratory conditions and in well-controlled clinical trials. Thus,PROs should be incorporated into the evaluation of new medica-tions, devices, and ongoing standard care.

References

1. Quittner AL, Modi A, Cruz I. Systematic review of health-related quality of lifemeasures for children with respiratory conditions. Paediatr Respir Rev 2008Sep;9:220–32.

2. Palermo TM, Long AC, Lewandowski AS, Drotar D, Quittner AL, Walker LS.Evidence based assessment of health-related quality of life and functionalimpairment in pediatric psychology. J Pediatr Psychol 2008 Oct;33:983–96.

3. Goss CH, Quittner AL. Patient-reported outcomes in cystic fibrosis. Proc AmThorac Soc 2007 Aug 1;4:378–86.

4. Cohen SB, Strand V, Aguilar D, Ofman JJ. Patient- versus physician-reportedoutcomes in rheumatoid arthritis patients treated with recombinant interleu-kin-1 receptor antagonist (anakinra) therapy. Rheumatology 2004;43:704–11.

5. Grad FP. The Preamble of the Constitution of the World Health Organization.Bulletin of the World Health Organization 2002;80:981–4.

6. Lohr KN, Zebrack BJ. Using patient-reported outcomes in clinical practice:challenges and opportunities. Qual Life Res 2009 Feb;18:99–107.

7. Quittner AL, Buu A, Messer MA, Modi AC, Watrous M. Development andValidation of the Cystic Fibrosis Questionnaire (CFQ) in the United States: Ahealth-related quality of life measure for cystic fibrosis. Chest 2005Oct;128:2347–54.

8. Quittner AL, Sawicki GS, McMullen A, Rasouliyan L, Pasta DJ, Yegin A, KonstanMW. Erratum to: Psychometric evaluation of the Cystic Fibrosis Questionnaire-Revised in a national. US sample Qual Life Res 2012 Jan 13 [Epub ahead of print].

L.S. Blackwell et al. / Paediatric Respiratory Reviews 14 (2013) 146–151 151

9. U.S. Department of Health and Human Services FDA Center for Drug Evaluationand Research Guidance for industry on patient-reported outcome measures:Use in medical product development to support labeling claims. Federal Reg-ister. 2009; 74: 65132–65133.

10. Snyder CF, Aaronson NK, Choucair AK, Elliott TE, Greenhalgh J, Halyard MY, HessR, Miller DM, Reeve BB, Santana M. Implementing patient-reported outcomesassessment in clinical practice: a review of the options and considerations. QualLife Res 2011 Nov 3 [Epub ahead of print].

11. Spilker B, editor. Quality of Life and Pharmacoeconomics in Clinical Trials. Phi-ladelphia, PA: Lipincott-Raven; 1996.

12. Dinan MA, Compton KL, Dhillon JK, Hammill BG, Dewitt EM, Weinfurt KP,Schulman KA. Use of patient-reported outcomes in randomized, double-blind,placebo-controlled clinical trials. Med Care 2011 Apr;49:415–9.

13. Chassany O, Sagnier P, Marquis P, et al. Patient-reported outcomes: Theexample of health-related quality of life—A European guidance documentfor the improved integration of health-related quality of life assessment inthe drug regulatory process. Drug Info J 2002;36:209–38.

14. Retsch-Bogart GZ, Quittner AL, Gibson RL, Oermann CM, McCoy KS, Montgom-ery AB, Cooper PJ. Efficacy and safety of inhaled aztreonam lysine for airwayPseudomonas in cystic fibrosis. Chest 2009 May;135:1223–32.

15. Guyatt GH, Feeny DH, Patrick DL. Measuring health-related quality of life.Annals of Internal Medicine 1993 Apr 15;118:622–9.

16. McCoy KS, Quittner AL, Oermann CM, Gibson RL, Retsch-Bogart GZ, Montgom-ery AB. Inhaled Aztreonam Lysine for Chronic Airway Pseudomonas aeruginosain Cystic Fibrosis. Am J Respir Crit Care Med 2008 Nov 1;178:921–8.

17. Oermann CM, Retsch-Bogart GZ, Quittner AL, Gibson RL, McCoy KS, Montgom-ery AB, Cooper PJ. An 18-month study of the safety and efficacy of repeatedcourses of inhaled aztreonam lysine in cystic fibrosis. Pediatr Pulmonol 2010Nov;45:1121–34.

18. EMA (Committee for Medicinal Products for Human Use (CHMP)): Reflectionpaper on the regulatory guidance for the use of Health Related Quality of Life(HRQL) measures in the evaluation of medicinal products. 2005, Doc. Ref. EMA/CHMP/EWP/139391/2004. London.

19. EMA (Consultation committee for medicinal products for human use (CHMP)):Draft Biomarkers Qualification: Guidance to applicants. 2008, Doc. Ref. EMA/CHMP/SAWP/72894/2008. London.

20. Bottomley A, Jones D, Claassens L. Patient-reported outcomes: Assessment andcurrent perspectives of the guidelines of the Food and Drug Administration andthe reflection paper of the European Medicines Agency. Eur J Cancer2009;45:347–53.

21. Quittner AL, Sweeny S, Watrous M, Munzenberger P, Bearss K, Gibson NitzaA, Fisher LA, Henry B. Translation and linguistic validation of a disease-specific quality of life measure for cystic fibrosis. J Pediatr Psychol 2000Sep;25:403–14.

22. Modi AC, Quittner AL. Validation of a disease-specific measure of health-relatedquality of life for children with cystic fibrosis. J Pediatr Psychol 2003Dec;28:535–45.

23. Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-andpencil administration of patient-reported outcome measures: A meta-analyticreview. Value Health 2008 Mar-Apr;11:322–33.

24. Palermo TM, Valenzuela D, Stork PP. A randomized trial of electronic versuspaper pain diaries in children: impact on compliance, accuracy, and accept-ability. Pain 2004;107:213–9.

25. Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. Patient non-compliance with paper diaries. BMJ 2002 May 18;324:1193–4.

26. Barkovich, AJ, Szefler, SJ, Olson, E, Rymer, W. Scientific vision workshop ondiagnostics and therapeutics [white paper]; 2011. Retrieved from NationalInstitute of Child Health and Human Development: http://www.nichd.nih.gov/vision/comments/whitepapers/Diag_Therp_White_Paper_040311.pdf.

27. Varni JW, Thissen D, Stucky BD, Liu Y, Gorder H, Irwin DE, Dewitt EM, Lai JS,Amtmann D, Dewalt DA. PROMIS(1) Parent Proxy Report Scales: an itemresponse theory analysis of the parent proxy report item banks. Qual Life Res2011 Oct 5 [Epub ahead of print].

28. Ware JE, Snow KK, Kosinski M, Gandek B. SF-36 Health Survey Manual andInterpretation Guide. Boston, MA: New England Medical Center, The HealthInstitute; 1993.

29. Ravens-Sieberer U, Bullinger M. Assessing health related quality of life inchronically ill children with the German KINDL: first psychometric and contentanalytical results. Qual Life Res 1998;7:399–407.

30. Quittner AL. Measurement of quality of life in cystic fibrosis. Curr Opin Pulm Med1998 Nov;4:326–31.

31. Britto MT, Kotagal UR, Chenier T, Tsevat J, Atherton HD, Wilmott RW. Differ-ences between adolescents’ and parents’ reports of health-related quality of lifein cystic fibrosis. Pediatric Pulmonology 2004;37:165–71.

32. Orenstein DM, Nixon PA, Ross EA, Kaplan RM. The quality of well-being in cysticfibrosis. Chest 1989 Feb;95:344–7.

33. VanDevanter DR, Rasouliyan L, Murphy TM, Morgan WJ, Ren CL, Konstan MW,Wagener JS. Trends in the clinical characteristics of the U.S. Cystic fibrosispatient population from 1995 to 2005. Pediatr Pulmonol 2008;43:739–44.

34. Rosenfeld M. An overview of endpoints for cystic fibrosis clinical trials. Proc AmThorac Soc 2007;4:299–301.

35. Juniper EF, Guyatt GH, Feeny DH, Ferrie PJ, Griffith LE, Townsend M. Measuringquality of life in children with asthma. Qual Life Res 1996;5:35–46.

36. Accurso FJ, Rowe SM, Clancy JP, et al. Effect of VX-770 in persons with cysticfibrosis and the G551D-CFTR mutation. N Engl J Med 2010 November18;363:1991–2003.

37. Sawicki GS, Rasouliyan L, McMullen AH, Wagener JS, McColley SA, Pasta DJ,Quittner AL. Longitudinal assessment of health-related quality of life in anobservational cohort of patients with cystic fibrosis. Pediatr Pulmonol 2011Jan;46:36–44.

38. Quittner AL, Higgins M, Angyalosi G, Brockhaus F, Thomas K, Zhang J. Trends inhealth-related quality of life (HRQoL) in cystic fibrosis with tobramycin inhala-tion powder: the EDIT trial. J Cyst Fibros 2012;11(S1):S73.

39. Modi AC, Quittner AL. Barriers to treatment adherence for children with cysticfibrosis and asthma: What gets in the way? J Pediatr Psychol 2006 Sep;31:846–58.

40. Quittner AL, Modi AC, Roux AL. Psychosocial challenges and clinical interven-tions for children with cystic fibrosis: A developmental approach. In: Brown R,editor. Handbook of Pediatric Psychology in School Settings. News Jersey: Lawr-ence Erlbaum, Associates; 2003. p. 333–61.

41. Berwick DM. Disseminating innovations in health care. JAMA 2003;289:1969–75.

42. Bodenheimer T, Lorig K, Holman H, Grumbach K. Patient self-management ofchronic disease in primary care. JAMA 2002;288:2469–75.

43. Gawande AA. The Checklist Manifesto: How to Get Things Right. New York:Metropolitan Books; 2009.

44. Leape L, Berwick D, Clancy C, et al. Transforming healthcare: A safety impera-tive. Qual Saf Health Care 2009;18:424–8.

45. Safran DG. Defining the future of primary care: what can we learn frompatients? Ann Intern Med 2003;138:248–55.

46. Quittner AL, Alpern A, Blackwell LS. Treatment Adherence in Adolescents withCystic Fibrosis. In: Castellani C, Elborn S, Heijerman H, editors. Health care issuesand challenges in the adolescent with cystic fibrosis. European Cystic FibrosisSociety; 2012.

47. Spencer J, Cooper H, Milton B. Qualitative studies of type 1 diabetes inadolescence: a systematic literature review. Pediatr Diabetes 2009;11:364–75.

48. Zemanick ET, Harris JK, Conway S, Konstan MW, Marshall B, Quittner AL,Retsch-Bogart G, Saiman L, Accurso FJ. Measuring and improving respiratoryoutcomes in cystic fibrosis lung disease: opportunities and challenges totherapy. J Cyst Fibros 2010;9:1–16.

49. Zindani G, Streetman D, Streetman D, Nasr S. Adherence to treatment inchildren and adolescent patients with cystic fibrosis. J Adolesc Health2006;38:13–7.

50. Rosen DS. Transition of young people with respiratory diseases to adult healthcare. Paediatr Respir Rev 2004;5:124–31.

51. Tuchman LK, Schwartz LA, Sawicki GS, Britto MT. Cystic fibrosis and transitionto adult medical care. Pediatrics 2010 Mar;125:566–73.

52. Quittner AL, Modi AC, Wainwright C, Otto K, Kirihara J, Montgomery AB.Determination of the minimal clinically important difference scores for theCystic Fibrosis Questionnaire-Revised respiratory symptom scale in two popu-lations of patients with cystic fibrosis and chronic Pseudomonas aeruginosaairway infection. Chest 2009;135:1610–8.

53. Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal importantchange in a disease-specific quality of life instrument. J Clin Epidemiol1994;47:81–7.

54. Henry B, Aussage P, Grosskopf C, Goehrs J-M. Development of the Cystic FibrosisQuestionnaire (CFQ) for assessing quality of life in pediatric adult patients.Quality of Life Research 2003;12:63–76.