2019 Metrics on Human Research Protection Program Performance Documents/Metrics 2019.pdf · Slide 7. Where Organizations Conduct Research. Slide 8. Type of Research Organizations

2019 Metrics on Human Research Protection Programs Performance

July 2020

1

About the Metrics

Improving the quality of human research protection programs (HRPP) is a top priority of AAHRPP. Effective and efficient systems of oversight by organizations provide better protections for research participants and produce higher quality research. And collectively, they raise the global bar to ensure research participants are safe and respected. AAHRPP is pleased to present the 2019 metrics for HRPP performance.

These metrics are collected from annual reports, as well as step 1 and step 2 applications, submitted by accredited organizations. All the quantitative data were based off the most recent data reported by AAHRPP organizations as of December 31, 2019. Qualitative data was selected based on what most accurately reflected AAHRPP organizations in 2019. AAHRPP has compiled an information database to help research organizations, researchers, sponsors, government agencies, and participants identify and support high-performing practices for HRPPs. The data range from types of research, IRB review times, to conformance with regulations and guidance, to financial and personnel resources.

2

Table of ContentsGeneral Description of Organizations

Slide 6. All Accredited Organizations

General Description of Research

Slide 7. Where Organizations Conduct Research

Slide 8. Type of Research Organizations Conduct or Review

Slide 9. Specific Types of Research

Selected Types of Research

Slide 10. Organizations That Conduct Research With Vulnerable Populations

Sponsors of Research

Slide 11. Sponsors of Research

Slide 12. Federal Sponsors of Research

Slide 13. Regulations and Guidance Followed by Organizations

Reliance on the IRB

Slide 14. Use of IRBs

Slide 15. Use of External IRBs

Slide 16. Number of IRBs per Organization3

Compensation of IRB Members

Slide 17. Compensation of IRB Members by Organizations

Slide 18. Financial Compensation of IRB Members

Characteristics of IRBs

Slide 19. Median Number of Active Protocols Organizations Oversee by Quartile

Slide 20. Median Number of Active Protocols Overseen by all Organizations

Slide 21. Mean Number of Active Protocols Organizations Oversee by Quartile

Slide 22. Mean Number of Active Protocols Overseen by all Organizations

Slide 23. Median Number of Active Protocols Overseen by an IRB Based on the Number of IRBs per Organization

Slide 24. Mean Number of Active Protocols Overseen by an IRB Based on the Number of IRBs per Organization

IRB Review Times

Slide 25. IRB Review Times by Type of Review

Slide 26. Five Year Trend in IRB Review Times

Use of Technology

Slide 27. Technology Use by Organizations with an IRB

4

Resources for the IRB

Slide 28. IRB Staffing and Funding Levels

Slide 29. Five Year Trends in Median IRB Budgets

Audits of the HRPP Conducted by Organizations

Slide 30. Number of Audits Conducted within the Past Year

Slide 31 Five Year Trends in Mean Number of Audits Conducted

Complaints Reported to the IRB

Slide 32. Five Year Trends in Mean Number of Complaints Reported

Non-Compliance Reported to the IRB

Slide 33. Number of Cases of Non-Compliance Reported to the IRB in the Past Year

Slide 34. Five Year Trends in Mean Number of Reported Cases of Non-Compliance

5

Of the 258 accredited organizations in 2019, 46% are academic institutions; 36.8% are hospitals; 7.8% are independent IRBs; 2.3% are VA Facilities, 3.9% are dedicated research facilities or research institutes; 2.3% are governmental organizations; 0.4% are contract research facilities; 0.4% are sponsors.

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Academic Institutions Hospitals Independent IRBs VA Facility ResearchInstitute/Dedicated

Research Site

Government Facility Contract Research Sponsor of Research

Perc

enta

ge o

f Org

aniz

atio

ns

Type of Organization

All Accredited Organizations 2019

6

Of the 258 accredited organizations, 58.5% conduct research in their home state/province/region, other states/provinces/regions, and countries outside their home country. 21.3% conduct research in their home state/province/region only. 17.4% conduct research in their home state/province/region and other states/provinces/regions within their home country. 2.7% conduct research in their home state/province/region and countries outside their own country.

7

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Conduct Research In Their HomeState/Province/Region, OtherStates/Provinces/Regions, And

Countries Outside Of Their HomeCountry

Conduct Research In Their HomeState/Province/Region Only

Conduct Research In Their HomeState/Province/Region And OtherStates/Provinces/Regions Within

Their Home Country

Conduct Research In Their HomeState/Province/Region And Countries

Outside Of Their Home Country

Perc

enta

ge o

f Org

aniz

atio

nsWhere Organizations Conduct Research

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Biomedical Social Science/Behavioral Other

Perc

enta

ge o

f Org

aniz

atio

nsTypes of Research Conducted

Of the 258 accredited organizations, 99% conduct biomedical research; 91% conduct social science/behavioral research; 26% conduct other types of research.

8

Nine organizations conduct biomedical research but did not specify the type. Of the remaining 249 organizations, 94% conduct research on drugs, while 92.8% conduct research on biomedical devices. Seven organizations did not specify if they had or had not conducted emergency research without consent in the last 12 months of their most recent report. Of those who did, 22% did conduct emergency research without consent in the last 12 months.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Drugs Devices Without Consent

Perc

enta

ge o

f Org

aniz

atio

nsSpecific Types of Research

9

Of all 258 organizations, 95.3% conduct research involving children; 94.2% conduct research involving adults unable to consent; 93% conduct research involving employees; 89.9% conduct research involving students; 87.2% conduct research involving pregnant women; 60.1% conduct research involving prisoners; 14.3% conduct research involving other vulnerable groups.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Children Adults Unable toConsent

Employees Students Pregnant Women Prisoners Other VulnerableGroups

Perc

enta

ge o

f Org

aniza

tions

Research on Vulnerable Populations

10

For all organizations, a median of 15.4% of research is federally funded; a median of 16.3% is industry funded; a median of 7.5% is externally funded; a median of 45% is internally funded.

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Federal Industry External Internal

Med

ian

Perc

enta

geMedian Sources of Funding

11

215 Organizations revealed the source of their federal funding. Of those 215 organizations, 70.2% receive funding from the United States government only. 17.2% receive funding from both US and non-US governments. 12.6% receive funding from only non-US governments.

0%

10%

20%

30%

40%

50%

60%

70%

80%

US Govt Only Both US and non-US Govt Non-US Govt Only

Perc

enta

ge o

f Org

aniz

atio

ns W

hich

Rec

eive

Fed

eral

Fun

ding

Sources of Federal Funding

12

Of 258 organizations, 85.7% follow FDA regulations; 84.9% follow DHHS regulations; 78.2% follow state and local laws; 64.0% follow ICH regulations in some form; 58.1 follow DOD regulations; 34.4% follow country-specific laws outside the US; 33.7% follow DoEd regulations; 27.5% follow DOJ guidelines; 16.7% follow VA regulations; 16.7% follow EPA regulations; and 14.3% follow DoE regulations.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

FDA DHHS State and LocalLaws

ICH DOD Country-SpecificLaw

Dept ofEducation

DOJ VA EPA Dept of Energy

Perc

enta

ge o

f Org

aniz

atio

nsRegulations and Guidelines Followed

13

Of the 239 non-IRB organizations, 95% have their own IRB while 5% do not.

95%

5%

Use of IRBs

Have Own IRB Do not have own IRB

14

Does Not Rely on External IRB18%

Rely on External IRB for less than 10% of Protocols

48%

Rely on External IRB for more than 10% of Protocols

29%

Rely on External IRB for all Protocols

5%

Use of External IRBs

Does Not Rely on External IRB Rely on External IRB for less than 10% of Protocols

Rely on External IRB for more than 10% of Protocols Rely on External IRB for all Protocols

For all non-IRB Organizations, 18% do not rely on external IRBs for any protocols, 48% rely on external IRBs for less than 10% of protocols, 29% rely on external IRBs for more than 10% of protocols, 5% rely on external IRBs for all protocols.

15

0 IRBs5%

1 IRB37%

2 IRBs23%

3 IRBs13%

4 IRBs8%

5 IRBs4%

6 IRBs 3%

7 IRBs3%

8 or more IRBs4%

Number of IRBs

0 IRBs 1 IRB 2 IRBs 3 IRBs 4 IRBs 5 IRBs 6 IRBs 7 IRBs 8 or more IRBs

For all accredited organizations, 5% have zero IRBs; 37% have one IRB; 23% have two IRBs; 13% have three IRBs; 8% have four IRBs;4% have five IRBs; 3% have six; 3% have seven; and 4% have eight or more. 16

85% of organizations with an IRB (excluding private IRBs) compensate IRB chairs; 62% compensate IRB vice chairs; 44% compensate affiliated members; 63% compensate non-affiliated members.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

IRB Chairs Vice Chairs Affiliated Members Non-Affiliated Members

Perc

enta

ge o

f Org

aniz

atio

nsCompensation of IRB Members

17

Within AAHRPP organizations that compensate IRB chairs, 95% provide financial compensation for chairs; within AAHRPP organizations that compensate IRB vice chairs, 96% provide financial compensation for vice chairs; within AAHRPP organizations that compensate affiliated members, 84% provide financial compensation for affiliated members; within AAHRPP organizations that compensate non-affiliated members, 93% provide financial compensation for non-affiliated members.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

IRB Chairs Vice Chairs Affiliated Members Non-Affiliated Members

Perc

enta

ge o

f Org

aniz

atio

nsFinancial Compensation of IRB Members

18

Median Number of Protocols Overseen organized by Quartiles of AAHRPP accredited Orgs

39 38 5

138

Reviewed By aConvened IRB

Reviewed by theExpeditedProcedure

Exemptions Total

Median Number of Active Protocols for Organizations (0-310 Protocols)

180259

27

518



Exemptions Total

Median Number of Active Protocols Overseen (311-876 Protocols)

312

740

130

1294



Exemptions Total

Median Number of Active Protocols Overseen (879-2028 Protocols)

10851916

359

3501



Exemptions Total

Median Number of Active Protocols Overseen (2049+ Protocols)

19

227

435

50

878

0

100

200

300

400

500

600

700

800

900

1000

Reviewed By a Convened IRB Reviewed by the Expedited Procedure Exemptions Total

Prot

ocol

sMedian Number of Active Protocols Overseen by all Organizations

20

Mean Number of Active Protocols Organizations Oversee by Quartile

67 67

16

139


Reviewed by theExpedited Procedure

Exemptions Total

Mean Number of Active Protocols(0-310 Protocols)

218264

64

552



Exemptions Total

Mean Numbers of Active Protocols(311-876) Protocols

418

764

192

1409



Exemptions Total

Mean Numbers of Active Protocols(879-2028 Protocols)

1304

2519

402

4306



Exemptions Total

Mean Number of Active Protocols(2049+ Protocols)

21

500

900

168

1595

0

200

400

600

800

1000

1200

1400

1600

1800

Reviewed By a Convened IRB Reviewed by the Expedited Procedure Exemptions Total

Prot

ocol

sMean Number of Protocols Overseen by all Organizations

22

A median of 878 protocols were overseen by all organizations; a median of 130 protocols were overseen by organizations without an IRB; a median of 345 protocols were overseen by organizations with one IRB; a median of 1009 protocols were overseen by organizations with two IRBs; a median of 1255 protocols were overseen by organizations with three IRBs; a median of 2105 protocols were overseen by organizations with four IRBs; a median of 3617 IRBs were overseen by organizations with five or more IRBs.

0

500

1000

1500

2000

2500

3000

3500

4000

All Zero IRBs 1 IRB 2 IRBs 3 IRBs 4 IRBs 5+ IRBs

Prot

ocol

sMedian Number of Active Protocols overseen by IRBs per Organization

23

A mean of 1595 protocols were overseen by all organizations; a mean of 179 protocols were overseen by organizations without an IRB; a mean of 649 protocols were overseen by organizations with one IRB; a mean of 1239 protocols were overseen by organizations with two IRBs; a mean of 1857 were overseen by organizations with three IRBs; a mean of 2780 protocols were overseen by organizations with four IRBs; a mean of 4162 protocols were overseen by organizations with five or more IRBs.

0

500

1000

1500

2000

2500

3000

3500

4000

4500

All Zero IRBs 1 IRB 2 IRBs 3 IRBs 4 IRBs 5+ IRBs

Prot

ocol

sMean Number of Active Protocols Overseen by IRBs per Organization

24

15

37

8

18

9.5

0

5

10

15

20

25

30

35

40

Time From Submission toReview by Convened IRB

Time From Submission toApproval by Convened IRB

Time From Submission toReview by the Epedited

Procedure

Time From Submission toApproval by the Expedited

Procedure

Time From Submission toExempt Determination

Cale

ndar

Days

Median IRB Review Times by Type of Review

25

0

5

10

15

20

25

30

35

40

2015 2016 2017 2018 2019

Cale

ndar

Day

sFive Year Trends in Median IRB Approval Times

Approved by the Convened IRB Approved by The Expedited Procedure Exempt Determinations

26

26 organizations did not answer this question. Of those who did, 97% use a database to track IRB protocols; 84% use an online IRB application; 93% use an electronic system for distribution of materials; 84% use an online system for IRB review.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Use an Online Database to Track IRBProtocols

Use an Online IRB Application Use an Electronic System forDistribution of Materials

Use an Online System for IRB Review

Perc

enta

ge o

f Org

aniz

atio

nsTechnology Use by Organizations

27

Number of Protocols

Median Number of Protocols

Median Number of Staff

Median Number of Protocols per Staff Member Median IRB Budget

All 878 6.9 128.1 $ 492,042.00

0-100 57 2 28.3 $ 127,000.00

101-500 271 3.0 90.3 $ 232,781.00

501-1000 688 4.9 141.0 $ 375,000.00

1001-2000 1298 8.75 148.3 $ 657,272.00

2001-4000 2667 16.0 166.7 $ 1,293,559.00

4001+ 5652 25.5 221.6 $ 2,757,331.50 28

$300,000

$350,000

$400,000

$450,000

$500,000

$550,000

2015 2016 2017 2018 2019

IRB

Budg

ets

Five Year Trend in Median IRB Budgets

29

For Cause Audits of Researchers

Random Audits of Researchers

For Cause Audits of IRBs

Random Audits of IRBs

Mean 5.0 44.5 1.9 23.8

Median 2 15 0 430

0

10

20

30

40

50

60

For Cause Audits of Researchers Random Audits of Researchers For Cause Audits of IRBs Random Audits of IRBs

Mea

n N

umbe

r of A

udits

Five Year Trend in Mean Number of Audits Conducted

2015 2016 2017 2018 2019 31

0

2

4

6

8

10

12

2015 2016 2017 2018 2019

Num

ber o

f Com

plai

nts

Five Year Trend in Mean Number of Complaints per Organization

32

Protocols

Median Number of Allegations of Non-

ComplianceMedian Number of Determinations of

Serious Non-ComplianceMedian Number of Determinations of

Continuing Non-Compliance

0-100 0.5 0 0

101-500 1 0 0

501-1000 5.5 1 0

1001-2000 10 2 1

2001-4000 10 4 1

4000+ 33 8 233

0

5

10

15

20

25

30

35

40

45

50

Allegations of Non-Compliance Determinations of Serious Non-Compliance Determinations of Continuing Non-Compliance

Five Year Trend in Non-Compliance

2015 2016 2017 2018 2019 34