EXPLORING THE EFFECTIVENESS OF RCR

EDUCATION IN THE SOCIAL AND BEHAVIORAL SCIENCES

Jim Vander Putten Department of Educational Leadership

Amanda L. NolenDepartment of Teacher Education

University of Arkansas-Little Rock

Paper presented at the 2009 Research Conference on Research Integrity, May 15 - 17, 2009 Niagara Falls, NY.

Purpose and Background

Critical issue analysis of problems and issues in conducting research on responsible conduct of research education, instruction and training (RCR EIT) in the Social and Behavioral Sciences

From descriptive institutional data collected over two years, we recommend strategies for future research in this area

Importance of studying less prestigious segments of professions can yield perspectives very different from elite cultures that receive the

majority of media attention University of Arkansas-Little Rock (UALR) is a metropolitan

commuter institution in the Doctoral/Research-Intensive Carnegie institutional classification

Context

• August 2003: UALR implemented the CITI online social and behavioral sciences RCR EIT program

• Between 2004 and 2006, approximately 875 UALR faculty, staff, and students participated in the CITI training program:

• 88.5% (n=770) completion rate: - Students (n=551)- Principal Investigators (n=197), or - IRB Members (n=22)

• Non-completers (11.5%, n=97) identified themselves in one of several research roles:, Student (74), Principal Investigator (18), or IRB Member (5).

Institutional Data

Between 2004 and 2006, approximately 625 IRB proposals were submitted for review

Roughly 25% were unsuccessful (defined as being disapproved on first submission) for inappropriate research practices. Most common reasons for IRB protocol rejection: incomplete IRB protocols, inappropriate consent processes, missing elements in informed consent documents, insufficient data security measures coercion in the data collection process.

Inspection of CITI training completion dates on unsuccessful IRB proposals indicated that the vast majority had been completed within 2 weeks prior to IRB proposal submission and review.

Analysis of Issues

• Evaluation surveys of CITI basic and refresher courses “have been very positive”

• In the Social and Behavioral Sciences, the least useful testing is indirect measurements polling students and asking them how they feel about their education.

• At UALR, the short timelines between CITI training completion and IRB proposal rejections don’t yield any insights into what researchers learn.

• Braunschweiger (2007) noted “the very fact that only 2% of survey responders reported that they preferred traditional classroom instruction to the Web-based approach that CITI provides clearly indicates that old paradigms must be reviewed and new ways to effectively deliver ethics education tested.”

• Question: “Although students are the best judges of what they want, are they the best judges of what they need?”

Critical Issues in ‘Learning’

LEARNING – A relatively permanent change in the capacity of an organism to make a response (adaptation) that cannot be explained by maturation or the passage of time.

ISSUES: CITI Training Validity Problem Solving – Algorithm vs Heuristic Knowledge Transfer

CITI Training Validity Format

Presents content, case studies, scenarios Researcher answers a series of multiple

choice and true/false items based on content

Multiple choice and true false items assess the researcher’s understanding at the knowledge level.

TAXONOMY OF LEARNING KNOW – Recall or Recognition UNDERSTAND – Internalize APPLY – Apply to new similar problem ANALYZE– Reduce problem into parts EVALUATE– Identify criteria and judge

quality CREATE– Novel problemAnderson, et al. 2001; Bloom, et al. 1958

TAXONOMY OF LEARNING

KNOW – Recall or Recognition UNDERSTAND – Internalize APPLY – Apply to new similar problem ANALYZE– Reduce problem into parts EVALUATE– Identify criteria and judge

quality CREATE– Novel problem

CITI Training Assessment

RCR Behavior

Problem Solving Algorithm – Well defined problems

If X, then apply Y, the result will be Z A recipe, a math problem, a science

experiment Heuristic – Poorly defined problems

Social sciences research involving human subjects with free will!

Ormrod, 2008

Knowledge Transfer Applying knowledge, skills, and abilities

from the classroom/hypothetical setting to real situations with real consequences

Multiple models/examples Relevant models New knowledge built on prior knowledge

Bandura, 1986

Conclusions and Recommendations Create a more appropriate assessment of

researchers’ knowledge, skill, and ability before completing RCR education modules;

Teach researchers appropriate heuristic problem solving techniques;

Provide examples and scenarios from across the disciplines that more closely resemble the research being conducted at that university;

Provide a breadth of examples of ethical issues involving human participants (Breadth & Depth)