Purpose of Network Evaluation

• Increase understanding of the relationship between the network design, objectives and functions and the outcomes achieved

• Provide data to examine trends, identify strengths and weaknesses and assist in leadership decisions

Standing HPTN committees

Ongoing, internal peer review

• Protocol Review Committee• Study Monitoring Committee• Manuscript Review Committee

Network Evaluation Approach in 2000

• Qualitative rather than quantitativeWhy? Breadth of science Initial focus on infrastructure

development

• Comprehensive evaluation Site and protocol performance Working group functioning Central resources: CL, CORE, SDMC Standing committees

2001 Evaluation Report

• Survey results Science working groups Central resources Standing committees

• Quantitative data Protocol implementation Site performance

HPTN Evaluation -How did we measure up?

Slow overall progress – complex reasons

The large number of contributors (NIAID, FDA, CABS) that are an intrinsic part of a network has a direct relationship to protocol progress.

Many steps in review process – most seem to be unavoidable

Protocols – Development and Progress

Monitor protocol progress and take corrective action when needed to move slow developers along.

Shorten review process whenever possible; clear communication of the need for each review step; combine steps whenever possible.

Define realistic expectations for fielding a protocol.

Protocols – Development and Progress

- Suggested Corrective Measures

CWG – Survey Comments

Concern that CWG was not properly integrated into the HPTN research agenda

CWG/Network leadership not proactive enough in integrating the community into network activities

CWG should develop a clearer mission for the group and a master plan for achieving mission goals

Need clear definition of the roles of the CWG and CORE community program staff

Central Laboratory – Survey Comments

CL contributions to protocols were applauded

Site laboratory training – Good quality, more quantity

More and earlier involvement More site visits by CL Work on timing of training to

coincide more closely with study initiation.

CORE – Survey Comments

Protocol meetings and trainings are well managed

Receptive staff members More involvement by CORE staff

More site visits Closer monitoring More frequent and focused

trainings More vocal involvement in

protocol development

CORE – Survey Comments (cont.)

More scientific/technical training for CORE staff

More technical information on the web site and in the newsletter

Better definition of the roles of the CORE’s community program and the CWG

SDMC – Survey Comments

“Knowledgeable,” “Professional”

Accurate and prompt

More flexibility in approach to protocol development

Increase number of site and training visits to HPTUs

Increase interface with site staff in dealing with data management issues

Standing Committees – Survey Comments

SMC General satisfaction with decisions Earlier involvement in protocols during

implementation and site visitsPRC Reviews were timely, thorough and

helpful Concern that expertise of some

reviewers may not be adequate, suggest use of additional outside reviewers

Standing Committees – Survey Comments (cont.)

MRC Improve response time to authors Reviews need to be helpful, provide

guidance for improvement Provide written comments with

review decision

Prevention Leadership Group – Survey Comments

Conflict resolution is handled well by the PLG

Good decisions on making the best use of available resources

Clarify and communicate the role of PLG to the Network

Need to focus on tough issues and make “tough” decisions

Summary and ConclusionKey Evaluation Results to Consider• Increase communications to entire

network Clarify the review process from

concept to study initiation Clarify leadership roles and activities

• Increase network partner involvement with HPTUs More visits by CORE, SDMC and CL to

improve HPTU performance• Increased focus on ethics and

community issues in regard to conducting international research