A Responsible C onduct of R esearch (RCR) T raining Program: Summary and Evidence

A Responsible Conduct of Research (RCR) Training Program:

Summary and EvidenceZhanna Bagdasarov

University of Oklahoma

Training rationale and background

Overview and summary of the training

Implementing the training at your

institution: some basics and lessons learned

Evidence for training effectiveness

Overview

Training Background &

Summary

Major cases of research misconduct

◦ Academic and professional misconduct a growing

concern across fields

Research institutions developing own courses

◦ Training effectiveness

◦ Training evaluation

NIH and NSF mandates

◦ COMPETES act-Creating Opportunities to Meaningfully

Promote Excellence in Technology, Education, & Science

The Broader Context

Two-day course

◦ Approximately 14 hours of instruction

10 “blocks” or training modules

◦ 2 out-of-class blocks, 8 in-class blocks

Course Summary

Knowledge of guidelines

Awareness of own biases and common errors

Model of ethical decision-making (EDM)

Situational analysis and interpretation

Strategies, or tools, for decision-making

Field differences

Topics covered

◦ Mentor-mentee relationships, collaboration, interdisciplinary

research, management practices, whistle-blowing, handling

data, publication practices, conflict of interest, authorship

Training Content

Develop students’ understanding of the ambiguous, complex nature of the problems that they might encounter in their work

Teach strategies that help students to identify and think through complex problems to make ethical decisions

Overarching Objectives

Decision-making strategies facilitate EDM

Decision-making errors and personal biases hinder

EDM

Decisions involve social-emotional considerations

and implications

Case-based reasoning facilitates development of

knowledge to serve as foundation

Cooperative learning fosters social awareness and

social reinforcement

Conceptual Basis for Training

Cases

Self-reflection questions

Role-play

Viewpoint activity

Generation of strategies

Generation of constraints

Group discussion

Activities

Emphasis on decision-making and its complexity

rather than rule-based guidelines

Recognition of the “gray areas”

Discussion of field differences

Emphasis on generation, practice, and application

Highly interactive (limited lecture)

Based on research and evaluation

Not a “check-the-box” training

Uniqueness of the Course

Mini-Tutorial: Block-by-Block Synopsis

Interpret &

Apply

Rules&

Principles

Decision &

Action

Constraints

Strategies

The EDM Model

Government regulations◦ Policies and legislation regulating research

◦ Examples: 1966 Animal Welfare Act; 1974 National Research Act

Professional society guidelines

◦ Length and specificity vary widely across fields

◦ Example: American Psychological Association

Institutional guidelines

◦ NIH, ORI, universities

◦ Examples: Protection of Human Subjects; Data Management;

Conflicts of Interest

Regulations & Guidelines

Making closed-ended decisions

Neglecting to consider hidden motives or agendas

Engaging in black-and-white thinking

Making hasty decisions

Deceiving one-self and/or others

Avoiding personal responsibility

Overly simplistic application of research guidelines

Being overly subjective

Failure to consider other’s perspectives; being self-focused

Neglecting to consider long-term consequences

Failure to consider time and resource constraints

Potential Constraints in EDM

Recognize your circumstances

Seek help

Question your judgment

Anticipate consequences

Manage emotions

Look within/consider personal motivations

Consider others’ perspectives

Decision-Making Strategies

Integrate all relevant information

Understand and integrate multiple perspectives

Develop an overall understanding of the nature

of the ethical problem

Basis of the EDM Model

◦ An expanded, more complex version of EDM model

◦ Sensemaking culminates in “Interpret & Apply” Stage

SensemakingInterpre

t

&

Apply

Rules&

Principles

Decision &

Action

Constraints

Strategies

Approaches to problems generally

What is valued and rewarded

Guidelines

Established norms

“Appropriate” or “inappropriate” behavior

Field Differences

Different persons view the problem from

different perspectives

◦ Grad student vs. faculty vs. university vice-

president

Different persons may use different

approaches to decision-making

Differing Viewpoints

Training Implementation

Initial Planning

Determine initial scale

Determine budget

Determine timeline

Gain approval from university

administration

Major Questions Who will be required to take it?

• How will it be required?

Who will provide the salary for instructors?

Who will provide ongoing training & training for new

instructors?

How will updates be made to content?

Ongoing administration

• Who will handle recruitment and enrollment?

• Who will oversee updates?

• Who will provide new and ongoing training to instructors?

Major Obstacles/Challenges Some people/departments think they are

exempt

Choosing good trainers

Organizational logistics

Instructors

University faculty or graduate students

• 2 or more years experience in field

Training Instructors

• 2 days

• Instructor manual

• Practice delivery of training modules

• Feedback

Funding

OU Office of the Senior Vice President and

Provost & Graduate College Funding

• Administrative coordination

• Training Coordinator & Assistant (tuition & stipend)

• Trainer pay

• Trainee certificates

• Materials

Funding Funded projects for development & refinement of RCR program

NIH: Organizational Influences on Scientific Integrity (1R01NS042397-

01)

NIH: Environmental and Educational Influences on Scientists

(5R01NS049535-02)

NIH: Development of Strategies for Improving Ethical Decision-Making in

the Sciences (5R01NR010341-02)

NSF: Development and Evaluation of a Work Practices Approach for

Ethics Education in Science and Engineering (SES 0529910)

CGS: Analysis of graduate students understanding of ethical issues

(LTR090506)

NSF: Case Based Reasoning and Ethics Instruction: Content and

Processing Exercises for Effective Education (SES 090436)

Ongoing Issues

Field-specific trainingTrainees feel that the training does not apply to them

Refreshments for training

Training being mandatory• A requirement for all incoming, funded graduate

students

• Maintaining the same feel and not making it feel

“required”

Lessons Learned

Organizational Logistics

• Need to find appropriate office to handle training

• Not all teachers can be trainers

Some departments are resistant

• Want to be exempt, already “teach” ethics

Most students & instructors want this training

Strong evaluation data is critical

Lessons Learned

Systematic reports are necessary

• Instructors

• Funding office

Ongoing changes are necessary

• Adapt to other fields

• Must monitor training content

Key people in charge of ongoing planning

Instructors must have ongoing refreshers

• Feedback from students

• Prevent delivery drift

Resource intensive process

• Research

• Organizing and planning

• Instructors, administrators, researchers, TAs

Lessons Learned

Training Evidence and Future Directions

Pre-test and post-test EDM measures

◦ Multiple answer selection

“Pick two”

Multiple “high-ethicality” responses

◦ Real-world cases of ethical misconduct

Not transparent

Trainee Reactions

◦ Ratings of training effectiveness

Evaluation of Training

Evaluation Results

Effect Size (Cohen’s d) Scientist Sample

Engineer Sample

Undergrad Sample

Decision-Making Ethicality Data Management .66** .53* 1.79** Study Conduct 1.46** .66* .34 Professional Practices .61** 1.06** 1.68** Business Practices .49* 1.82** 1.09** Strategies Recognizing one’s circumstances 1.24** .16 .35 Seeking help .84** −0.65* −0.01 Questioning one’s judgment 1.27** 1.92** 2.45** Dealing with emotions .90** 1.51** 2.56** Anticipating consequences .77** .33 .79** Analyzing personal motivations 1.36** .60* 1.05** Considering others’ perspectives .45** .08 .27

Training effects held over 6-month period

Training changed mental models

◦ Trained individuals: represented ethical problems

as complex – based on in-depth analysis

◦ Untrained individuals: focused on outcomes

Trainee reactions to training favorable (M=6

out of 7)

Evaluation Results

High ratings of effectiveness (M = 5.24 on 7-point scale)

Trainee Comments:

◦ “It was a great workshop; I loved it. It helped me a lot!”

◦ “Very useful and more interesting than I anticipated”

◦ “Overall, good information and the discussions with students

outside of my field were great”

◦ “The presenters did a fine job. I want to thank them for

putting their efforts into this”

◦ “This training made me think of a couple of misconducts I

have done in the past and certainly won’t do again”

◦ “Great workshop!”

Results for Student Reactions 2010-2011 Academic Year

Tailoring to specific disciplines

◦ Engineering, Physical Sciences, Humanities, Arts

◦ Evaluation instruments already tailored

Training adaptions: remedial ethical training

◦ Decision-making focus

◦ Mental-model focus

Faculty Instruction

◦ No explicit regulations yet, but may be coming

◦ In-coming & junior faculty

Future Directions

Thank You!