View
3
Download
0
Category
Preview:
Citation preview
2016 Metrics on Human
Research Protection
Program Performance
Updated April 24, 2017
About the Metrics____________________________ Improving the quality of human research protection programs (HRPP) is a top priority of AAHRPP. Effective and efficient systems of oversight with organizations provide better protections for research participants and produce higher quality research. And collectively, they raise the bar globally to ensure research participants are safe and respected. AAHRPP is pleased to present the 2016 metrics for HRPP performance.
The metrics are collected from annual reports and new applications from our current clients. From data supplied by our client organizations in 2016, AAHRPP has compiled an information database to help research organizations, researchers, sponsors, government agencies, and participants identify and support high-performing practices for HRPPs. The data range from types of research and conformance with regulations and guidance to financial and personnel resources and IRB review times. For the 2016 data, the metrics are represented as the median, except where indicated as the mean.
Table of Contents
General Description of Organizations Slide 1. All Accredited Organizations in 2016 4
Slide 2. Responding Organizations in 2016 4
General Description of the Research Conducted or Overseen by Organizations Figure 1. Where Organizations Conduct Research 5 Figure 2. Type of Research Organizations Conduct or Review 5
Selected Types of Research Conducted or Overseen by Organizations Figure 3: Selected Types of Research that Organizations Conduct or Review 6
Figure 4: Organizations that Conduct or Review Research Involving Vulnerable Populations 6
Sponsors and Regulators of Research Figure 5: Sponsors of Research 7
Figure 6: Federal Sponsors of Research 7
Figure 7: Regulations and Guidance Followed by Organizations 8
Regulatory Oversight of Research Figure 8: Checking the Boxes on the Federalwide Assurance 9
Figure 9: Comparison of AAHRPP Organizations and Those Registered with OHRP on
Checking the Boxes on the Federalwide Assurance 9
Reliance on the IRB Figure 10: Use of IRBs 10
Figure 11: Use of External IRBs 10
Figure 12: Number of IRBs Per Organizations 11
Compensation of IRB Members Figure 13: Compensation of IRB Members by Organizations 12
Figure 14: Financial Compensation of IRB Members 12
Figure 15: Percentage of Organizations that Compensate IRB Chairs and Vice Chairs Over Time 13
Characteristics of IRBs Figure 16: Median Number of Active Protocols Organizations Oversee 14
Figure 17: Mean Number of Active Protocols Organizations Oversee 15
Figure 18: Median Number of Active Protocols Overseen by an IRB Based on the Number of
IRBs Per Organization 16 Figure 19: Mean Number of Active Protocols Overseen by an IRB Based on the Number of IRBs Per Organizations 17
IRB Review Times Figure 20: IRB Review Times by Type of Review 18
Figure 21: Four-Year Trends of Median IRB Review Times from Submission to Approval 19
Use of Technology Figure 22: Technology Use by Organizations with an IRB 20
Resources for the IRB Table 1: IRB Staffing and Funding Levels 21
Figure 23: Four-Year Trends in Median IRB Budgets 21
Audits of the HRPP Conducted by Organizations Table 2: Number of Internal Audits Organizations Conducted within the Past Year 22
Figure 21: Four Trends in Mean Number of Audits Organizations Conducted 22
Complaints Reported to the IRB Figure 22: Five-Year Trends in Mean Numbers of Complaints Reported 23
Non-Compliance Reported to the IRB Table 3: Number of Cases of Non-Compliance Reported to the IRB in the Past Year 24
Figure 26: Five-Year Trends in Mean Number of Reported Cases of Non-Compliance 24
AAHRPP Survey Results Figure 27: Survey Results Regarding AAHRPP Accreditation 25
General Description of Organizations
Metrics on HRPP Performance 4
All Accredited Organizations 2016
05
101520253035404550
Perc
ent
of O
rgan
izat
ions
Slide 1: 43% of all accredited organizations are academic institutions, 38% are hospitals, 9% are independent IRBs, 4% are VA facilities, 3% are dedicated research sites or research institutes, 2% are government organizations, 1% are contract research organizations, and 0.5% are sponsors.
Responding Organizations 2016
05
101520253035404550
Perc
ent
of O
rgan
izat
ions
Slide 2: 43% of organizations that responded in 2016 were academic institutions, 39% were hospitals, 10% were independent IRBs, 4% were dedicated research sites or research institutes, 2% were VA facilities, and 2% were government organizations.
General Description of the Research Conducted or
Overseen by Organizations
Metrics on HRPP Performance 5
Figure 1: Where Organizations Conduct Research
0 20 40 60 80 100
In home state/province/region pluscountries other than home country
In home state/province/region only
In home state/province/region plusother state/province/regions within
home country
In home state/province/region, otherstate/province/regions, and othercountries outside of home country
Percent
Figure 1: 50% of responding organizations conduct research home state/province/region, other states/provinces/regions within their home country, and countries other than their home country, 24% conduct research in their home state/province/region and other states/provinces/regions within their home country, 23% of responding organizations conduct research in their home state/province/region only, and 4% conduct research in their home state/province/region and countries other than their home countries.
Figure 2: Type of Research Organizations Conduct or Review
0 20 40 60 80 100
Other
Social Science/Behavioral
Biomedical
Percent
Figure 2: 100% of responding organizations conduct biomedical research, 91% conduct social/behavioral research, and 30% conduct research falling into other categories.
Selected Types of Research Conducted or Overseen
by Organizations
Metrics on HRPP Performance 6
Figure 3: Selected Types of Research that Organizations Conduct or Review
0 20 40 60 80 100
Planned Emergency Research withoutConsent
Investigational Devices
Investigational Drugs
Percent
Figure 3: 90% of organizations conduct research involving investigational drugs, 84% conduct research involving investigational devices, and 16% of organizations conduct planned emergency research without consent.
Figure 4: Organizations that Conduct or Review Research Involving Vulnerable Populations
0 20 40 60 80 100
Other
Prisoners
Pregnant Women
Students
Adults with Diminished Capacity
Employees
Children
Percent
Figure 4: 98% of all organizations conduct research with children, 95% conduct research with employees, 95% conduct research with adults with diminished capacity, 94% conduct research with students, 83% conduct research with pregnant women, 50% conduct research with prisoners, and 15% conduct research with other vulnerable populations.
Sponsors and Regulators of Research
Metrics on HRPP Performance 7
Figure 5: Sponsors of Research
0
5
10
15
20
25
30
35
40
45
50
Internal Industry Federal External
Perc
ent
Figure 5: For all organizations, a median of 43% of research is internally sponsored, a median of 20% of research is industry sponsored, a median of 19% of research portfolios is federally sponsored, and a median of 7% of research portfolios is externally sponsored.
Figure 6: Federal Sponsors of Research
0
5
10
15
20
25
30
35
40
US Government Only Both US and Non-USGovernment
Non-US GovernmentOnly
Perc
ent
Figure 6: For all organizations receiving federal funds, a median of 25% of research is sponsored by the US Government only, a median of 22% of research portfolios is sponsored by both US and Non-US governments, and a median of 12% of research is sponsored by only Non-US Governments.
Sponsors and Regulators of Research
Metrics on HRPP Performance 8
Figure 7: Regulations and Guidance Followed by Organizations
0 20 40 60 80 100
Deparment of Energy
Enviornmental Protection Agency
Department of Veterans Affairs
Department of Justice
Country-specific Law
Department of Education
ICH-GCP when requested by the sponsor
ICH-Good Clinical Practice
Department of Defense
State
Department of Health & Human Services
Food & Drug Administration
Percent
Figure 7: 83% of all organizations follow FDA regulations, 80% follow Department of Health and Human Services regulations, 77% follow regulations or guidelines of their state, 57% follow Department of Defense requirements, 38% follow the ICH- Good Clinical Practice guideline, 29% follow the ICH- Good Clinical Practice guideline when requested by the sponsor, 28% follow Department of Education requirements, 22% follow Country-specific regulations or guidelines, 12% follow Department of Justice requirements, 13% follow Department of Veterans Affairs requirements, 11% follow Environmental Protection Agency requirements, and 10% follow Department of Energy requirements.
Metrics on HRPP Performance 9
Regulatory Oversight of Research
Figure 8: Checking the Boxes on the Federalwide Assurance
7%
22%
71%
Checked Box ApplyingSubpart A
Checked Boxes ApplyingSubparts A, B, C, D
Did Not Check BoxesApplying Subparts A, B, C,or D
Figure 8: 71% of all organizations did not check boxes applying Subparts A, B, C, or D, 22% of all organizations check boxes applying all Subparts (A, B, C, D), and 7% checked the box only applying Subpart A.
29%
34%
37%
Checked BoxApplyingSubpart A
Checked BoxApplyingSubparts A,B, C, D
Did NotCheck BoxesApplying toSubparts A,B, C, or D
Figure 9: Comparison of AAHRPP Organizations and Those Registered with OHRP on Checking the Boxes on the
Federalwide Assurance
From Office for Human Research Protections (2015)
Accredited Organizations
7%
22%
71%
Checked BoxApplyingSubpart A
Checked BoxesApplyingSubparts A, B,C, D
Did Not CheckBoxesApplyingSubparts A, B,C, or D
Figure 9: Compared to percentages provided by OHRP, more organizations accredited by AAHRPP or applying for AAHRPP accreditation unchecked the box.
Reliance on the IRB
Metrics on HRPP Performance 10
Figure 10. Use of IRBs
Does not have own IRB, 4%
Have own IRB, 96%
Figure 10: 96% of all organizations have their own IRB; 4% do not have their own IRB.
Figure 11. Use of External IRBs
Rely on External IRB for no more than 10%, 71%
Rely on External IRB for more
than 10%, 26%
Rely on External IRB for 100%,
3%
Figure 11: 71% of all organizations use external IRBs for a maximum of 10% of total protocols, 26% of organizations rely on external IRBs for more than 10% of total protocols, and 3% of all organizations rely on external IRBs for 100% of total protocols.
Reliance on the IRB
Metrics on HRPP Performance 11
Figure 12: Number of IRBs per Organization
1 IRB, 44%
2 IRBs, 16%
3 IRBs, 18%
4 IRBs, 9%
5 or more IRBs, 13%
Figure 12: 44% of all organizations have one IRB, 16% have two IRBs, 18% have three IRBs, 9% have four IRBs, and 13% have five or more IRBS.
Compensation of IRB Members
Metrics on HRPP Performance 12
Figure 13: IRB Member Compensation
0
10
20
30
40
50
60
70
80
90
100
Compensate IRBChairs
Compensate Non-AffiliatedMembers
Compensate IRBVice Chairs
CompensateAffiliatedMembers
Perc
ent
of O
rgan
izat
ions
Figure 13: 83% of all organizations compensate IRB chairs, 61% compensate non-affiliated members, 56% compensate IRB vice chairs, and 38% compensate affiliated members.
Figure 14: Financial Compensation of IRB Members
75
80
85
90
95
100
Compensate Non-AffiliatedMembers
Compensate IRBVice Chairs
Compensate IRBChairs
CompensateAffiliatedMembers
Perc
ent
of O
rgan
izat
ions
Figure 14: Of organizations that compensate IRB members, 94% compensate non-affiliated members financially, 93% compensate IRB vice chairs financially, 89% compensate IRB chairs financially, and 85% compensate affiliated members financially.
Compensation of IRB Members
Metrics on HRPP Performance 13
Figure 15: Percentage of Organizations that Compensate IRB Chairs and Vice Chairs Over Time
0
10
20
30
40
50
60
70
80
90
100
2012 2013 2014 2015 2016
Pe
rce
nt
of
Org
an
iza
tio
ns
Chairs
Vice Chairs
Figure 15: The percentage of organizations that compensate IRB chairs and vice-chairs has remained relatively consistent since 2012.
Characteristics of IRBs
Metrics on HRPP Performance 14
Figure 16: Median Number of Active Protocols Organizations Oversee
41
366
254
780
0
200
400
600
800
1000
1200
ExemptProtocols
Reviewed bythe Expedited
Procedure
Reviewed bythe Convened
IRB
Total
Nu
mb
er
of
Act
ive
Pro
toco
ls
95% CI Upper Bound
95% CI Lower Bound
Median
Figure 16: A median of 41 protocols were deemed exempt by all organizations*, a median of 366 protocols were reviewed by the expedited procedure, a median of 254 protocols were reviewed by the convened IRB, and a median of 780 total protocols were overseen by all organizations. *Exempt determinations made within 12 months of an organization’s submission to AAHRPP.
Characteristics of IRBs
Metrics on HRPP Performance 15
Figure 17: Mean Number of Active Protocols Organizations Oversee
143
787
526
1401
0
200
400
600
800
1000
1200
1400
1600
1800
ExemptProtocols
Reviewed bythe Expedited
Procedure
Reviewed bythe Convened
IRB
Total
Nu
mb
er
of
Act
ive
Pro
toco
ls
95% CI Upper Bound
95% CI Lower Bound
Mean
Figure 17: A mean of 142 protocols were deemed exempt by all organizations*, a mean of 787 protocols were reviewed by the expedited procedure, a mean of 526 protocols were reviewed by the convened IRB, and a mean of 1401 total protocols were overseen by all organizations. *Exempt determinations made within 12 months of an organization’s submission to AAHRPP.
Characteristics of IRBs
Metrics on HRPP Performance 16
Figure 18: Median Number of Active Protocols Overseen by an IRB Based on the Number of IRBs Per Organization
0
500
1000
1500
2000
2500
3000
3500
4000
All One IRB Two IRBs Three IRBs Four IRBs Five orMore IRBs
Nu
mb
er
of
Act
ive
Pro
toco
ls
Figure 18: A median of 509 protocols were overseen by all organizations, a median of 254 protocols were overseen by organizations with one IRB, a median of 438 protocols were overseen by organizations with two IRBs, a median of 1352 protocols were overseen by organizations with three IRBs, a median of 1179 protocols were overseen by organizations with four IRBs, and a median of 3708 were overseen by organizations with five or more IRBs.
Characteristics of IRBs
Metrics on HRPP Performance 17
Figure 19: Mean Number of Active Protocols Overseen by an IRB Based on the Number of IRBs Per Organization
0
500
1000
1500
2000
2500
3000
3500
4000
All One IRB Two IRBs Three IRBs Four IRBs Five orMore IRBs
Nu
mb
er
of
Act
ive
Pro
toco
ls
Figure 19: A mean of 1054 protocols were overseen by all organizations, a mean of 384 protocols were overseen by organizations with one IRB, a mean of 648 protocols were overseen two IRBs, a mean of 1468 protocols were overseen by organizations with three IRBs, a mean of 1131 protocols were overseen by organizations with four IRBs, and a mean of 3593 protocols were overseen by organizations with five or more IRBs.
IRB Review Times
Metrics on HRPP Performance 18
Figure 20: IRB Review Times by Type of Review
16
37
8
19
10
0
5
10
15
20
25
30
35
40
45
Time fromSubmission toReview by theConvened IRB
Time fromSubmission to
Approval by theConvened IRB
Time fromSubmission toReview by the
ExpeditedProcedure
Time fromSubmission to
Approval by theExpeditedProcedure
Time fromSubmission to
ExemptDetermination
Ca
len
da
r D
ay
s
Figure 20: The time from submission to review by the convened IRB is a median of 16 calendar days, the time from submission to approval by the convened IRB is a median of 37 calendar days, the time from submission to review by the expedited procedure is a median of 8 calendar days, the time from submission to approval by the expedited procedure is a median of 19 calendar days, and the time from submission to exempt determination is a median of 10 calendar days.
IRB Review Times
Metrics on HRPP Performance 19
Figure 21: Four-Year Trends of Median IRB Review Times from Submission to Approval
0
10
20
30
40
50
60
2013 2014 2015 2016
Ca
len
da
r D
ay
s Reviewed by the Convened IRB
Reviewed by the Expedited Procedure
Exempt Determinations
Figure 21: Median review times have remained consistent since 2014.
Use of Technology
Metrics on HRPP Performance 20
Figure 22: Technology Use by Organizations with an IRB
0 20 40 60 80 100
Use of an Online System for IRB Review
Use of an Online IRB Application
Use of an Electronic System for theDistribution of Materials
Use of a Database to Track IRB Protocols
Percent
Figure 22: 94% of all organizations use a database to track IRB protocols, 91% use an electronic system for the distribution of materials, 74% use an online IRB application, and 74% use an online system for IRB review functions.
Resources for the IRB
Metrics on HRPP Performance 21
Table 1: IRB Staffing and Funding Levels
Protocol Category
MedianNumber of Staff
MedianNumber of Protocols
Median Protocols per FTE
Median Dollars Budgeted for IRB
All 7 865 123.6 $485,436
1-100 2 37.5 18.75 $100,800
101-500 4 271.5 67.9 $352,288
501-1000 5.5 729.5 132.6 $457,500
1001-2000 9 1379.5 156.8 $596,013
2001-4000 18 2839 157.7 $1,063,264
4000+ 27 6344 235 $2,215,500
Figure 23: Four-Year Trends in Median IRB Budgets
$300,000
$400,000
$500,000
$600,000
$700,000
$800,000
2013 2014 2015 2016
Ave
rage
Dol
lars
Figure 23: Median IRB budgets have slightly increased since 2015. .
Audits of the HRPP Conducted by Organizations
Metrics on HRPP Performance 22
Table 2. Number of Internal Audits Organizations Conducted within the Past Year
For-Cause Audits of
Researchers
Random Audits of Researchers
For-Cause Audits of IRBs
Random Audits of IRBs
Mean 6 44 3 22
Median 1 15 0 4
Min 0 0 0 0
Max 149 860 162 506
Figure 24: Five-Year Trends in Mean Number of Audits Organizations Conducted
0
5
10
15
20
25
30
35
40
45
50
For-Cause Auditsof Researchers
Random Audits ofResearchers
For-Cause Auditsof IRBs
Random Audits ofIRBs
Mea
n N
umbe
r of
Aud
its
2012 2013 2014 2015 2016
Figure 24: For-cause audits of researchers and random audits of researchers have reached an all-time high in 2016. Random audits of IRBs and for-cause audits of IRBs have remained consistent since 2014.
Complaints Reported to the IRB
Metrics on HRPP Performance 23
Figure 25: Five-Year Trends in Mean Numbers of Complaints Reported
0
2
4
6
8
10
12
Complaints
Nu
mb
er
of
Re
po
rts
2012 2013 2014 2015 2016
Figure 25: The number of complaints from research participants reached its highest total since 2012. .
Non-Compliance Reported to the IRB
Metrics on HRPP Performance 24 Metrics on HRPP Performance 24
Table 3. Number of Cases of Non-Compliance Investigated in the Past Year
Protocol Category
Median Number of Allegations of Non-
Compliance
Median Number of Determinations of Serious Non-
Compliance
Median Number of Determinationsof Continuing Non-Compliance
All 3 1 0
1-500 1 0 0
501-1,000 1 1 1
1,001-2,000 4 2.5 0.5
2,001-4,000 11 4 1
>4,000 16 4 2.5
Figure 26: Five-Year Trends in Mean Number of Reported Cases of Non-Compliance
0
5
10
15
20
25
30
35
40
Non-Compliance Allegations Serious Non-Compliance Determinations Continuing Non-Compliance Determinations
Num
ber
of R
epor
ts
2012 2013 2014 2015 2016
Figure 26: Allegations of non-compliance, determinations of serious non-compliance, and determinations of continuing non-compliance have slightly increased since 2015.
AAHRPP Survey Results
Metrics on HRPP Performance 25
Figure 27: Survey Results Regarding AAHRPP Accreditation
0
10
20
30
40
50
60
70
80
90
100
Improves Human ParticipantProtection
Imrpoves Quality of ResearchReview
Improves RegulatoryCompliance
Provides Value
Strongly Agreed Agreed Neutral Opinion Disagreed
Figure 27: 58% of responding organizations strongly agreed that AAHRPP Accreditation improves human participant protection, 35% agreed, and 7% had a neutral opinion. 63% of responding organizations strongly agreed that AAHRPP Accreditation improves the quality of research review, 30% agreed, and 7% had a neutral opinion. 60% of responding organizations strongly agreed that AAHRPP Accreditation improves regulatory compliance, 33% agreed, 6% had a neutral opinion, and less than 1% disagreed. 69% of responding organizations strongly agreed that AAHRPP Accreditation provides value to their organization, 25% agreed, 5% had a neutral opinion, and less than 1% disagreed.
2301 M Street, NW (202) 783-1112 phone
Suite 500 (202) 783-1113 fax
Washington, DC 20037 www.aahrpp.org
Recommended