Benchmarking chronic wound healing outcomes

ORIGINAL RESEARCH

Vol. 24, No. 1 January 2012 3

Abstract: Evidence-based medicine (EBM) uses the current best evi-dence to inform decisions about care of individual patients, healthcare procedures, and technologies. The “gold standard” for optimal evidence in the Cochrane system is Level I randomized, controlled trials (RCTs) and meta-analyses of several RCTs. In order to achieve this level of evidence, one of the most important measures is the use of outcomes/endpoints. This article will provide, in short form, recommendations on how to achieve rigorous endpoints or outcomes in studies on wound management. Consistency in measuring endpoints/outcomes improves quality of care. To achieve such consistency it is important to 1) use predefined and robust outcomes; 2) adapt outcomes to the interven-tion under investigation; and 3) use the best evidence available. Also, it is emphasized that the use of complete wound closure or healing as an outcome measure is not always possible or suitable. Remaining patient-focused clarifies which other endpoints are relevant. Finally, “basic care” must be clearly defined and standardized when used as a comparative intervention in a RCT. In conclusion, the use of correct, clinically relevant outcomes or endpoints is of vital importance when establishing optimal evidence in wound healing and care.

WOUNDS 2012;24(1):3–9

From 1Bispebjerg University Hospital, Copenhagen, Denmark; 2Malmö University Hospital, Sweden; 3Cardiff University, United Kingdom

Address correspondence to:Finn Gottrup, MD, DMSciProfessor of SurgeryCopenhagen Wound Healing CenterDepartment of Dermatology, D42Bispebjerg University HospitalDK-2400 Copenhagen NV, [email protected]

Nonhealing wounds are a significant problem for healthcare systems all over the world. In the industrialized world, almost 1%–1.5% of the population has a problem wound at any one time and accounts for

2%–4% of the healthcare budget—a figure that is likely to rise with an increas-ing elderly and diabetic populations.1–3

For these reasons, there is an urgent need to review wound strategies and treatments in order to reduce the burden of care in an efficient and cost-effective way. A primary question is which of the available interventions, tech-nologies, and dressing materials achieve the best outcomes for the best value? Ongoing controversy surrounds the value of various approaches to wound management and care. Evidence-based clinical practice is difficult to achieve due to confusion about the value of the various approaches to wound man-agement. To address this, the European Wound Management Association (EWMA) set up a Patient Outcome Group.4 The first document identifies cri-teria for producing precise outcomes in both RCTs and clinical studies, and

Producing Precise Outcomes in Randomized, Controlled Trials and Clinical Studies

Finn Gottrup;1 Jan Apelqvist;2 Patricia Price3

Gottrup.indd 3 1/10/12 3:10 PM

DO NOT D

UPLICATE

Gottrup et al

4 WOUNDS www.woundsresearch.com

describes how to ensure that studies are consistent and reproducible.5 This review will highlight the principles in the EWMA document.

Systematic reviews have indicated that there are sub-stantial deficiencies in the quality of clinical research (eg, www.cochrane.org, www.nice.org.uk). Wound manage-ment has a paucity of high quality evidence, as the studies are often based on inadequate sample size, short follow up, nonrandom allocation to treatment arms, nonblinded assessment of outcomes, and poor description of control and concurrent intervention(s).

Quality of evidence in wound management is interest-ing from different perspectives:

1. From the clinical perspective the question is which interventions, technologies, and dressing materials are the best from the point of view of a single pa-tient or group of patients, where the primary focus is healing and the absence of complications.

2. From the policy maker and healthcare system per-spectives two issues arise: A) Whether or not a par-ticular product or intervention is safe and effective when used as indicated—this is a question of regula-tory approval, and B) whether or not the product or intervention represents a cost-effective use of funds. Too few high quality clinical or economic studies in wound care have resulted in challenges to the reim-bursement of modern dressings in favour of suppos-edly better value traditional products.

3. From the industry perspective (medical device in-dustry) the challenge is that the standard of care and evidence requirements for reimbursement are different in each country and that the large invest-ments related to RCTs are rarely justified by the pace of innovation and size of markets of most wound care products.

The aim of the background article5 was to provide recommendations on how to achieve rigorous end-points/outcomes in studies on wound management, and to describe an approach that will enable the design of RCTs and clinical studies to be both consistent and re-producible in order to reach a higher quality of evidence in wound management. These principles will be sum-marized with the goal of improving the consistency and quality of wound care outcomes measurement, documen-tation, and management around the world.

MethodsMaterials. The background information, evaluations,

and recommendations of the document are based on a

collection of a large amount of available guidance for evi-dence collection, and an analysis of recent RCTs and com-parative studies. To achieve an updated status on how endpoints/outcome parameters are used, defined, and evaluated, the authors performed a literature search of RCTs on the Cochrane Database, MEDLINE, and Embase from 2003–2009 studying chronic or problem wounds or ulcers, with the objective of examining and registering the use of endpoints, the quality of endpoint definitions, and the robustness of methodologies. Randomized con-trolled or nonrandomized clinical or preclinical studies qualified for inclusion in the analysis if they reported a measured endpoint or outcome of any aspect of wound management. Reviews, opinion papers, and case reports were excluded.

Definition of endpoints/outcomes. Study endpoints or outcomes are the keystone in the evidence discus-sion. To clarify the basic premises of the discussion, the terminology needs to be clarified. An endpoint/outcome parameter is defined as the objective of an evaluation or study.6–8 The objectives should include: 1) a precise state-ment of the degree of benefit expected from the inter-vention, and its duration; 2) clear statements on the time frame of the study (especially in relation to how quickly the benefits might start); and 3) a definition of the pa-tients for whom the benefit is sought.

In the past, the most commonly used outcome in clinical trials was visible reduction in wound size, par-ticularly restoration of intact skin (complete healing). The development of tests and techniques to improve tissue sampling and analysis, imaging technology, and scientific progress in cellular and molecular biology has enabled the development of more objective wound outcome pa-rameters, such as transepidermal water loss (TEWL) for re-epithelization,9 and surrogate outcome parameters that relate to both the wound condition and the treat-ment intervention being assessed. Examples of objective wound condition outcome measures include, exudation rate,10 pain,11 granulation rate,12 resolution of necrosis,12 or infection.13 Another objective measurement of a treat-ment intervention is the operational definition of moist wound healing in terms of the dressing moisture/vapor transmission rate.14

A surrogate endpoint/outcome parameter is defined as a physical sign or a laboratory measurement that can be used as a substitute for a clinically meaningful endpoint, effectively directly measuring how a patient feels, func-tions, or survives.15 The challenge in nonhealing wounds is that endpoints/outcome parameters, such as complete


DO NOT D

UPLICATE

Gottrup et al


wound closure, are difficult to achieve and maintain. If the only gold standard were total wound closure, efficacious therapies capable of hastening partial wound closure or preparing wounds for successful grafting, may never be considered. Alternative clinical or surrogate endpoints are therefore needed.

Methods. To establish a basis for the recommenda-tions and statements related to study designs and use of outcome measures in wound management, data collec-tion issues, concepts, and terms were discussed by the 3 authors, authorized by EWMA and used in the document.

The author panel recommendations addressed: 1) study design to optimize wound care intervention or product safety and efficacy; 2) industry requirements to optimise product development, registration, and reim-bursement; and 3) operational definitions of “standard care” including management of underlying disease.

From the clinical research perspective, there is a need to be aware of the strengths and limitations of different study designs if they are to effectively evaluate which health-care practices are worth considering for different patients in different health contexts. Key issues are, for example, use of a study protocol, problems related to het-erogeneity of the study population, and underlying con-ditions. From an industry perspective, external evidence needs are set by the requirements of national regulatory and reimbursement authorities, and other payers. When developing a new product, there are also internal needs for evidence, which mirror the phases of the development process. When focusing on payment or reimbursement for a new product, the key issue will often be budgetary impact and/or cost-effectiveness, rather than healing.

The document also discusses the importance of a generally accepted definition of “standard care” in con-nection with data collection in wound management. “Standard care” refers to generally accepted wound care procedures and the management of underlying disease, outside of the investigational product/device or drug that will be used in the clinical trial/evaluation. It is essential that the standard care procedures/regimens used are con-sistent, as this will minimize variability and enable assess-ment of the treatment effect.

Results What types of outcomes and endpoint measures have

been used in wound healing studies? Studies published before 2003 are described by Matousek et al7 and 2003–2009 studies by the EWMA Patient Outcome Group.5 In our literature search, abstracts from RCTs and compara-

tive studies were evaluated, identifying 176 articles that qualified for analysis. Many studies measured multiple endpoints—in total, this analysis generated a list of 313 different endpoints. In Table 1 the most often found end-points are mentioned.

The findings of the analysis were used as a basis for discussing and suggesting procedures for the successful measurement of each of the types of endpoint categories defined.

In general, it was found that a substantial number of endpoints (45%) were either not predefined or insuffi-ciently defined. As part of the analysis, the degree of ro-bustness of the measurement techniques used in studies* and the degree of reproducibility‡ were evaluated. In 70% of cases, a standardized or clearly defined measurement technique was used to examine the endpoint (eg, com-puterized planimetry or a standardized evaluation meth-od). These were defined as meeting the criteria for an ac-ceptable degree of robustness. However, 76% of these did not meet the criteria for reproducibility.

The analysis and discussion resulted in a number of

Table 1. List of endpoints was divided into the follow-ing endpoint categories (percentages represent share of 313 registered endpoints).

Reduction rate in wound area (24.1%)

Wound closure (16.9%)

Healing time (9.0%)

Change in wound condition (9.0%)

Biomarkers and bacteriology (4.5%)

Circulation (1.9%)

Infection signs (4.5%)

Symptoms and signs (13.2%)

Dressing performance (7.0%)

Quality of life (5.8%)

Costs and resources used (4.5%)

* Defined as a level of information about the measurement tech-nique, such that others could replicate the data collection in an identical way.

‡ Defined as the inclusion of a verifiable source of data, eg, photos to secure the possibility of validation from an external source by reproducing the study, or the involvement of an external validation source as part of the study design.


DO NOT D

UPLICATE

Gottrup et al


recommendations (“statements”) on use of endpoints in RCTs and comparative studies on nonhealing wounds (Table 2).

What types of biases could influence outcome/end-point evaluation of studies? When evaluating interven-tions in wound management it is a challenge to avoid bias. Designing studies with the aim of obtaining sufficient in-formation regarding outcomes is particularly hazardous.

The document, therefore, discusses and provides recom-mendations regarding some of the considerations that should be made when designing studies on interventions for the treatment of wounds.

Table 3 outlines some of the potential sources of bias that can be introduced into study design, unless great care is taken at the planning stage. In many cases, the is-sues are very similar regardless of intervention and are

Table 2. Statements on outcomes/endpoints*.

Wound closure, defined as total epithelialisation without discharge, is the most important endpoint relating to ulcer healing. It must be confirmed by an independent source (photography) and there must be sufficient follow-up to confirm healing

Wound area reduction is a valid endpoint with regard to wound healing but it must be confirmed by tracing and include a predefined relevant cut-off to ensure that “reduction rate error” (described in section: Reduction Rate) does not occur

There is enough evidence to support the use of a 50% reduction in wound surface area over time as a useful outcome, provided that the initial wound size and the measurement technique are taken into consideration. The time interval used in such assessment will vary depending on the wound type. Any reduction of less than 50% cannot be supported by the current literature; in these instances, more objective measures of size reduction must be used

Time to heal is an important outcome. However, the study protocol must consider the substantial methodological difficulties entailed, particularly confirmation of the exact date of healing for each patient during the specified observation period. To date, the accepted time interval for resource studies is one year

There is an urgent need for a validated scoring system with regard to wound condition

When using changes in the wound condition as an outcome parameter, they must be predefined and measured in such a way that they can be validated independently, wherever possible (for instance, by photograph)

When using biological markers as a primary outcome, they should be clearly predefined, and a clinically relevant unit of change should be specified; reliable and valid quantitative assessment methods should be used

When using wound infection as a primary outcome marker, it should be clearly predefined. At present, this could be either a binary measure of presence/absence or a composite score focusing on clinical signs and symptoms

Regardless of the assessment tool used, when using pain as an outcome measure it is important to predefine the amount of wound pain reduction that is clinically important

When surrogate parameters, such as symptoms and signs or composite endpoints, such as scales, are used as primary end-points, it is essential that both their basic definition and what is considered to be a clinically relevant difference are pre-defined. When used as a primary endpoint, it is favourable for it to be verified by an independent evaluator

When assessing dressing performance in an objective manner, with a focus on a specific aspect of symptom management, a comparative study may not be needed; the relevant data could be better assessed using a cohort study with a standardised, reproducible and validated protocol that includes resource utilisation (when appropriate)

HRQoL assessments must be based on tools with established psychometrics

The type of assessment must fit with the purpose of the data collection: if HRQoL data are to be used for health technology assessment reviews, then generic and/or utility methods must be included

When cost is used as an outcome parameter in wound management, it is essential to measure all the quantities of resources used and then add the value of those resources, according to a predefined protocol. It is recommended that resource use and costs are shown separately.

*A full list of statements can be found in Gottrup et al.5


DO NOT D

UPLICATE

Gottrup et al


not specific to wound management studies; however, the key points related to the wound community are dis-cussed below.

There is always the potential for bias to be introduced in studies, but the RCT design puts the emphasis on re-ducing bias as much as possible. Table 3 highlights the par-ticular difficulties that may apply to wound management studies. There is always a debate about the nature of the design of the study, dependent on the audience for whom the data are prepared: for example, regulatory authorities require the purest form of an RCT based on a restricted population in order to reduce the heterogeneity of the population, to ensure that the study has sufficient internal validity to demonstrate efficacy. However, this restrictive approach to study design will not allow for the gener-alization of the findings to those patients who routinely present at the authors’ clinics, where a comparative effec-tiveness study with the emphasis on whether or not the

treatment works pragmatically in routine practice, may be more appropriate. There are certain situations where the outcomes of an RCT may be very predictable, for example, using healing as an outcome for certain dress-ing trials: this contradicts a basic premise for conducting an RCT which states that the researchers should be in a state of equipoise (ie, uncertain about which intervention works best). In such circumstances, a comparative cohort study may be more appropriate, as the resources used to achieve similar outcomes is a more important question to investigate.

A further level of bias may be introduced if interven-tions are not used appropriately, in line with the manufac-turer’s instructions, or as required by the wound condi-tion. This is particularly difficult when a purist approach to RCT design requires that the same intervention is used throughout the study period, which would directly con-tradict the clinical need to adapt the treatment to the condition of the wound. There is a real tension between the requirement to stick to a purist approach and being pragmatic about the ways in which treatments are used in routine practice.

Table 3. A list of common methodological errors in wound dressing trials that may bias results.

Lack of validation of subjective assessments

Lack of description of objective or subjective measures

Lack of comparable baselines for patient groups

Lack of blinding for the evaluation of primary outcomes

Incorrect randomization methods

Poor definition of primary and secondary objectives

Number of patients not based on a priori sample size cal-culation

Randomization method poorly/not described

Assessment of outcomes not completely objective

Time to wound healing not used as primary outcome

Intention-to-treat analysis not used

No use of single reference wounds

Heterogeneous study population

Number of, and reason for, dropouts not stated

No specification of adjuvant treatments (such as pressure-re-lieving surfaces or offloading devices for neuropathic ulcers)

Small sample size combined with multiple outcome mea-sures

Reporting of multiple outcomes over multiple time points (increases chance of type I error)

Poor overall study reporting

Table 4. Checklist for objectives and outcomes in clinical trials.16

Are the intervention and control (eg, usual care) described in detail?

Has the target patient population been specified?

Has the degree of benefit from the intervention on a particu-lar outcome, and the time frame, been specified?

Has the primary outcome, including how and when it is to be measured, been specified?

Have any secondary outcomes been prespecified in similar detail?

Are the outcomes clinically relevant, objective (wherever feasible), and unambiguous?

Can the outcomes be measured for all patients and, where possible, assessed with researchers blinded to the allocated treatment?

Is the study explicit in the frequency and duration of out-come measurement?

Has the study been specially planned from a statistical viewpoint when multiple outcomes are measured?

If the outcome is a surrogate, will it adequately reflect a main outcome, and is there an indication of how much a benefit observed on the surrogate outcome will translate to a benefit on a main outcome?


DO NOT D

UPLICATE

Gottrup et al


Recommendations for use of outcomes/endpoint. The analysis presented in a previous section resulted in a number of rigorous recommendations for outcomes/endpoints in studies on wound management RCTs and comparative studies on nonhealing wounds (Table 4).

Discussion The main challenge for establishment of a basis for

the recommendations and statements related to study designs and use of outcome measures in wound manage-ment has been to control the heterogeneity of individual patients, concurrent disease states, and confounding fac-tors, as well as variations in the type, site, and condition of wounds and differences in healthcare organizations. Enrolling more subjects into a study cannot solve these problems.

Currently, the majority of wound management studies recruit patients with one wound etiology. However, the development of more targeted strategies specific to dif-ferent phases of treatment (eg, debridement) means that the condition of the wound (eg, exudate rate, pain, and necrosis) may be a better inclusion criterion.

When evaluating interventions in wound management it is also a challenge to avoid performance bias. Designing studies with the aim of obtaining sufficient information regarding outcomes is particularly hazardous. The docu-ment therefore discusses and provides recommenda-tions regarding some of the considerations that should be made when designing studies on interventions for the treatment of wounds.

The definition of nonhealing wounds is still discussed. The authors suggest that the definition “chronic” be re-placed with the definition “nonhealing,” as this better re-flects the clinical problems experienced by such patients. It is also suggested that multicentre trials should imply great efforts to enrol sufficient numbers at each site and a high degree of protocol standardization. Another issue brought forward is the selection of an appropriate study period. Recommendations from various institutions vary from 2 weeks to 12 months. The authors recommend that the type(s) of ulcer and relevant natural outcome are con-sidered when selecting the study duration.

Some common methodological errors in wound dress-ing studies and a checklist for objectives and endpoints/outcomes in clinical trials are included in this summary (Tables 2 and 3). ‡

ConclusionThis article has focused on the role of evidence in

wound healing, especially the problems achieving the highest level of evidence. One of the most important measures in the process of producing evidence is the use of outcomes/endpoints. These measures have been de-scribed along with the importance of choosing rigorous and robust outcomes/endpoints in order to design both consistent and reproducible RCTs and clinical studies in order to reach a higher quality of evidence in wound management.

The key messages of the document can be summa-rized as follows:

1. A substantial number of publications exist, but the quality of evidence needs to rise.

2. Consistency in measuring endpoints/outcomes im-proves quality. To reach consistency it is important to:

•Usepredefinedandrobustoutcomes •Adapt outcomes to the intervention under

investigation •Usethebestevidenceavailable.3. Using intact skin/healing as an outcome measure

is not always possible/suitable – using the patient focus clarifies which other endpoints are relevant.

4. The term “chronic” wound should be replaced with “nonhealing” wound.

5. Basic care must be standardized.6. The Patient Outcome Group does not reject RCTs in

wound management, but advocates that clinical tri-als using alternative endpoints to healing are evalu-ated as being equally suitable for the evaluation of various wound interventions.

References1. Dale J. Chronic ulcers of the leg: a study of the prev-

alence in a Scottish community. Health Bulletin.

1983;41(6):310–314.

2. Liu PY, Eriksson E, Mustoe TA. Wound healing: practical

aspects. In: Russels RC, ed. PSEF Instruction Courses. St.

Louis, MO: Mosby;1991.

3. Gottrup F. A specialized wound-healing center concepts:

importance of a multidisciplinary department structure

and surgical treatment facilities in the treatment of chron-

ic wounds. Am J Surg. 2004;187(Suppl):38S–43S.

4. Gottrup F. The EWMA patient outcome group. J Wound

Care. 2009;18(11):460.

5. Gottrup F, Apelqvist J, Price P; European Wound Manage-

ment Association Patient Outcome Group. Outcomes ‡ Tables 15 and 16 in the full document.


DO NOT D

UPLICATE

Gottrup et al


in controlled and comparative studies on non-heal-

ing wounds: recommendations to improve the qual-

ity of evidence in wound management. J Wound Care.

2010;19(6):237–268.

6. Biomarkers Definitions Working Group. Biomarkers and

surrogate endpoints: preferred definitions and conceptual

framework. Clinical Pharmacol Ther. 2001;69(3):89–95.

7. Matousek S, Deva AK, Mani R. Outcome measurements

in wound healing are not inclusive: a way forward. Int J

Lower Extrem Wounds. 2007;6(4):284–290.

8. US Department of Health and Human Services, Food and

Drug Administration. Guidance for Industry: Chronic Cu-

taneous Ulcer and Burn Wounds: Developing Products

for Treatment. June 2000.

9. Hinman CD, Maibach H. Effect of air exposure and oc-

clusion on experimental human skin wounds. Nature.

1963;200:377–378.

10. Gelfand JM, Hoffstad O, Margolis DJ. Surrogate endpoints

for the treatment of venous leg ulcers. J Invest Dermatol.

2002;119(6):1420–1425.

11. Arnold TE, Stanley JC, Fellows EP, et al. Prospective, multi-

center study of managing lower extremity venous ulcers.

Ann Vasc Surg. 1994;8(4):356–362.

12. Romanelli M. Objective measurement of venous ulcer de-

bridement and granulation with a skin color reflectance

analyzer. WOUNDS. 1997;9(4):122–126.

13. Gardner SE, Frantz RA, Doebbling BN. The validity of

the clinical signs and symptoms used to identify local-

ized chronic wound infection. Wound Repair Regen.

2001;9(3):178–186.

14. Bolton LL. Evidence-based report card: operational defini-

tion of moist wound healing. J WOCN. 2007;34(1):23–29.

15. Cohn JN. Markers of malign across the cardiovascular

continuum: interpretation and application. Introduction

to surrogate markers. Circulation. 2004;109(25 Suppl

1):IV20–IV21.

16. ECRI Institute. Appendix B, Quality of Literature. ECRI

Institute Study Quality Assessment Instrument, Negative

Pressure Wound Therapy Devices. Available at: www.ecri.

org.

Other recommended literature can be found in the background article5 and at www.ewma.org/english/patient-outcome-group.


DO NOT D

UPLICATE

Documents

Benchmarking chronic wound healing outcomes