www.unialliance.ac.uk/dta/ www.tees.ac.uk/sections/research/training.cfm https://shdoctoralschool.wordpress.com/ https://shardprogramme.wordpress.com/

Reputation, Risk and Reward - The Challenges and Opportunities of Research Integrity

A Doctoral Training Alliance Workshop

14 April 2016, 10:00-16:00 in The Workstation, 15 Paternoster Row, Sheffield S1 2BX #DTARI

This workshop aims to promote good research practices and show how excellent research can only be produced when resolute rigor, respect and responsibility are applied. In particular it will:

Frame the research integrity agenda by looking at the risk, rewards and reputational concerns involved

Explore three different complex and interesting research integrity areas and why they matter to researchers

Look at different types of conflicts of interest that can affect researchers, and think about why and how these can be prioritised

Consider the moral and practical issue around in vivo testing

Reflect on data-related integrity and the line between processing and questionable manipulation practices

Outline the underlying policies that govern and guide in these areas

Who should attend? The workshop is provided as an elective for the Doctoral Training Alliance in Applied Bioscience for Health researchers. It is also open to all doctoral and staff researchers from Sheffield Hallam and Teesside universities (and other DTA partner institutions), and is suitable for those from all disciplines and all levels of experience.

The workshop’s programme is outlined overleaf

Excellent research can only exist where intellect and integrity combine. There are fundamental moral, cultural and procedural responsibilities of being a researcher, reflecting both collective values and regulation. This session will explore some of the research integrity dilemmas which the current generation of researchers is being required to tackle.

www.unialliance.ac.uk/dta/ www.tees.ac.uk/sections/research/training.cfm https://shdoctoralschool.wordpress.com/ https://shardprogramme.wordpress.com/

PROGRAMME

The programme may be subject to change

About the speakers Dr Andrew C. Rawnsley is Research Governance & Training Manager at Teesside University; an Advisor for the UK Research Integrity Office (UKRIO); and member of Council for the Association for Research Ethics (AfRE) Dr Nikki Osborne is Director of Responsible Research in Practice and former Senior Scientific Officer for the RSPCA www.responsibleresearchinpractice.co.uk/ Dr Keith Fildes is Research Development Manager (Policy and Performance) at Sheffield Hallam University

Booking Reserve a place at https://dtari.eventbrite.co.uk Venue This event will take place in the Creative Lounge of The Workstation, 15 Paternoster Row, Sheffield (www.showroomworkstation.org.uk/workstation/) Dietary Requirements Lunch will be provided. If you have special dietary requirements, beyond standard provision for vegetarians, please contact kal@unialliance.ac.uk as soon as possible.

Time Item Lead

9.45 Networking and coffee

10.00 Introduction - Reputation, Risk and Reward

10.20 Conflicts of Interest Andrew Rawnsley

11.15 Coffee

11.30 Case Study 1 Andrew Rawnsley

12:00 Case Study 2 Nikki Osborne

12.30 Lunch

13.15

In Vivo Testing Nikki Osborne

14:30 Coffee

14:45

Data-Related Integrity Keith Fildes

15:45 Policy Links

16:00 Close

14 April 2016

Reputation, Risk and Reward - The

Challenges and Opportunities of

Research Integrity

Research Integrity

Research Integrity

Aims and objectives of a particular research project

Methods of a particular research project

Objectives of research areas/programmes broadly

Types of methods used in research

The conducting of a project

The conduct of researchers

Research infrastructure and funding

The Scope of an Ethics of Research

Research Integrity

Schedule

Research Integrity

The Concordat to Support Research Integrity

Culture of Scientific Research in the UK

Darling Review – COI review (URL to follow)

Concordat on Openness on Animal Research in the UK

The ARRIVE (Animal Research Reporting of In Vivo Experiments)

Guidelines

Responsibility in the Use of Animals in Bioscience Research

Reproducibility and Reliability of Biomedical Research: Improving

Research Practice

Guiding Principles for Behavioural Laboratory Animal Science

Concordat On Open Research Data (Draft)

Policy Links

Reputation Risk & Reward Session One:

Conflicts of Interest

Doctoral Training Alliance - Elective

Sheffield Hallam University

14 April 2016

Dr Andrew C. Rawnsley

UKRIO; AfRE; Teesside University

Outline

• Definitions

• Types of conflict

• Conflict prioritising exercise

• Disclosure

‘Interest’

The fact or relation of having a share or concern in, or a right to, something A thing which is to the advantage of someone; (a) benefit, (an) advantage The relation of being involved or concerned as regards potential detriment or (especially) advantage Personal influence (with a person etc.) A thing that is of some importance to a person, company, state A business, cause, or principle that is of some importance to a number of people; a party or group having such a thing in common

OED

Committee on Publication Ethics

A conflict of interest is a situation in which a person or organization is involved in multiple interests, financial interest, or otherwise, one of which could possibly corrupt the motivation of the individual or organization. The presence of a conflict of interest is independent of the occurrence of impropriety.

International Committee of Medical Journal Editors

A conflict of interest exists when professional judgment concerning a primary interest (such as patients' welfare or the validity of research) may be influenced by a secondary interest (such as financial gain). Perceptions of conflict of interest are as important as actual conflicts of interest.

UKRIO Code of Practice: 3.6 Conflicts of interest

3.6.1 Organisations and researchers must recognise that conflicts of interest (i.e. personal or institutional considerations, including but not limited to financial matters) can inappropriately affect research. Conflicts of interest must be identified, declared and addressed in order to avoid poor practice in research or potential misconduct

3.6.2 When addressing a conflict of interest, it must be decided whether it is of a type and severity that poses a risk of fatally compromising the validity or integrity of the research, in which case researchers and organisations should not proceed with the research, or whether it can be adequately addressed through declarations and/or special safeguards relating to the conduct and reporting of the research.

UKRIO Code of Practice: 3.6 Conflicts of interest

3.6.5 Researchers should comply with their organisation’s policy for addressing conflicts of interest, as well as any external requirements relating to conflicts of interest, such as those of funding bodies. This should include declaring any potential or actual conflicts of interest relating to their research to their manager or other appropriate person as identified by their organisation; any ethics committee which reviews their research; and when reporting their findings at meetings or in publications. Conflicts of interest should be disclosed as soon as researchers become aware of them.

Types of interest

• Personal

• Professional

• Scholarly

• Financial

Personal interests

• work-life balance, holidays, time-off

• family, friends, social life

• ‘the truth’ (epistemological commitments)

• ‘the good’ (moral commitments)

• curiosity, job satisfaction

Professional interests

• successful publications

• winning grant income

• prestige and recognition in the field

• promotion (Reader, Professor etc)

Scholarly interests

• proving that a technique or method achieves successful or noteworthy outcomes

• demonstrating the validity of a theory

• making important new discoveries

• ‘the truth’

Financial interests

• consultancy or speaker fees • patents • spin-out companies • contract research income • higher salary

Who has interests?

• researcher level of seniority

• manager local, institutional

• institution type, reputation

• funder size, influence, agendas

• journal, publisher size, influence, agendas

• research user specialist, non-specialist

‘the public’

A problematic conflict of interest

Three conditions:

• two or more interests are in conflict with one another

• this conflict might compromise the integrity or trustworthiness of the research

• this conflict goes unresolved or undisclosed

Financial interests

a researcher conducts research benefitting them financially, and that benefit depends on a particular research outcome

Financial interests

conflict between the financial interests and the scholarly interests of the researcher

Financial interests

a funder requests that a researcher suppresses or misreports an outcome because a particular outcome has commercial value

Financial interests

conflict between the funder’s interests and the scholarly interests of the researcher

conflict between the financial interests of the funder and the scholarly interests of the researcher, the discipline and the interests of research users and the interests of the public in the dissemination of trustworthy research

Interest complexes

a funder requests suppression or misreporting of outcomes in order to provide evidence for a particular policy agenda which aims to affect public opinion

a researcher’s promotion depends on particular research outcome

close personal or professional relationship between (eg) an author and peer reviewer/editor

Interest prioritisation exercise

• Revisit the interests you identified earlier on the ‘Type Exercise’ sheet

• Identify the SIX most important interests TO YOU

• Rank these six interests in an order of priority (1 = most important)

• Could any of these six interests come into conflict?

• If so, then how would you remove, reduce, or declare the conflict?

Making declarations

DECLARATIONS OF INTEREST

The reviewers publish papers in peer-reviewed biomedical journals and some of their work involves technical editing. One of the review authors (PM) was a co-investigator on one of the studies (Silagy 1998) included in this review

COMPETING INTERESTS

The authors have declared that no competing interests exist.

Report shines light on ex-researcher’s misconduct

Carolyn Y. Johnson – Boston Globe May 30, 2014

When former Harvard psychology professor Marc Hauser was found solely responsible in a series of six scientific misconduct cases in 2012, he distanced himself from the problems, portraying them as an unfortunate consequence of his heavy workload. He said he took responsibility, “whether or not I was directly involved.” But a copy of an internal Harvard report released to the Globe under the Freedom of Information Act now paints a vivid picture of what actually happened in the Hauser lab and suggests it was not mere negligence that led to the problems. The 85-page report details instances in which Hauser changed data so that it would show a desired effect. It shows that he more than once rebuffed or downplayed questions and concerns from people in his laboratory about how a result was obtained. The report also describes “a disturbing pattern of misrepresentation of results and shading of truth” and a “reckless disregard for basic scientific standards.”

The report details the lengths to which the committee went to check Hauser’s defenses. They reviewed his written responses, read seven letters of support from scientific colleagues, and met with him and his lawyer for nine hours. When Hauser suggested that someone had doctored a videotape of raw data showing monkeys responding to sounds, for example, two external firms were commissioned to do a forensic examination. They found no signs of tampering. When possible, the committee consulted original videotape of monkeys responding to cues to figure out how the problems arose and who was responsible. “We did not find evidence that [professor] Hauser has been inventing findings out of whole cloth,” the committee wrote. “. . . Hauser’s shortcomings in respect to research integrity have in the main consisted instead of repeated instances of cutting corners, of pushing analyses of data further in the direction of significance than the actual findings warranted, and of reporting results as he may have wished them to have been, rather than as they actually were.” What was found:

• In a 2002 paper published in the journal Cognition that has since been retracted, videotape of monkeys being exposed to two different patterns of syllables never showed the animals being exposed to one of the specific set of syllable patterns that were reported in the paper. Hauser suggested a number of alternative explanations for the problem, including the possibility the tape had been doctored, which were carefully considered and rejected by the committee.

• In 2005, Hauser and colleagues did a statistical analysis of an experiment in which monkeys responded to two artificial languages. In a later statistical analysis, an unnamed individual using the raw data got very different results. Hauser had changed

values, causing the result to be statistically significant, an important criterion showing that findings are probably not due to chance. For example, after the data from one experiment were analyzed in 2005, the results initially were not statistically significant. After Hauser informed a member of his lab of this by e-mail, he wrote a second e-mail: “Hold the horses. I think I [expletive] something up on the coding. Let me get back to you.” After correcting for that problem, he concluded that the result was statistically significant. According to the Harvard report, five data points had changed from the original file, and four of the five changes were in the direction of making the result statistically significant. In a second, related experiment, a collaborator asked to be walked through the analysis because he or she had obtained very different results when analyzing the raw data. Hauser sent back a spreadsheet that he said was simply a reformatted version, but then his collaborator made a spreadsheet highlighting which values had apparently been altered. Hauser then wrote an e-mail suggesting the entire experiment needed to be recoded from scratch. “Well, at this point I give up. There have been so many errors, I don’t know what to say. . . . I have never seen so many errors, and this is really disappointing,” he wrote. In defending himself during the investigation, Hauser quoted from that e-mail, suggesting it was evidence that he was not trying to alter data. The committee disagreed. “These may not be the words of someone trying to alter data, but they could certainly be the words of someone who had previously altered data: having been confronted with a red highlighted spreadsheet showing previous alterations, it made more sense to proclaim disappointment about ‘errors’ and suggest recoding everything than, for example, sitting down to compare data sets to see how the ‘errors’ occurred,” the report states.

• In 2007, a member of the laboratory wanted to recode an experiment involving rhesus monkey behavior, due to “inconsistencies” in the coding. “I am getting a bit pissed here. There were no inconsistencies!” Hauser responded, explaining how an analysis was done. Later that day, the person resigned from the lab. “It has been increasingly clear for a long time now that my interests have been diverging sharply from what the lab does, and it seems like an increasingly inappropriate and uncomfortable place for me,” the person wrote. The committee said it carefully considered Hauser’s allegation that people in his laboratory conspired against him, due to academic rivalry and disgruntlement, but did not find evidence to support the idea.

The committee also acknowledged that many of Hauser’s overall findings about the cognitive abilities of animals may stand. His results that showed that animals may have some of the same cognitive abilities as people have been important for the field. But science depends on the data.

“Skepticism above all toward the veracity of one’s own hypotheses is, of course, an essential virtue for scientists,” the committee wrote, “and one that must be modeled for the benefit of trainees.”

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

ANIMAL RESEARCH REPORTING: Issues to Consider

Nikki Osborne BSc. PhD

DTA Research Integrity/Values Elective

Sheffield Hallam University - 14th April 2016

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Activity Session Overview

• Introduction

• Initiatives to improve reporting standards

• Case study…..

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Introduction

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Introduction

Human Clinical Research

• Approved by a research ethics committee.

• Monitored by the MHRA to ensure good clinical practice.

• Must be based upon a review of current evidence i.e. Systematic Review

Animal Research

• Approved by an animal welfare and ethical review body.

• ASRU responsible for ensuring legislative compliance.

• Research quality assessed by peer review.

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Reporting Initiatives

CONSORT

CONsolidated

Standards

Of

Reporting

Trials

ARRIVE

Animal

Research

Reporting

In

Vivo

Experiments

Enhancing the QUAlity and Transparency Of health Research

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Activity

1. Split into 3 groups.

2. Using the ARRIVE guidelines read and discuss:

– Title, Abstract and Introduction, OR

– Methods, OR

– Results and Discussion.

3. Feedback to the whole group.

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Don’t believe everything that you read…..

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

ANIMAL RESEARCH: What you need to know

Nikki Osborne BSc. PhD

DTA Research Integrity/Values Elective

Sheffield Hallam University - 14th April 2016

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Session Overview • Introduction to animal research within the UK

• Research framework – Legislation

– Policies

– Guidance

– Other

• Openness & Transparency – Concordat on Openness on Animal Research in the UK

• Research Outputs – Hot topics & current initiatives

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Animal Welfare Legislation

1822 Cruel Treatment of Cattle Act

1835 Cruelty to Animals Act

1849 Cruelty to Animals Act

1876 Cruelty to Animals Act

1986 Animals (Scientific Procedures) Act

(EU Directive 86/609)

2012 ASPA Amendment Regulation(SI2012/3039) : (EU Directive 2010/63)

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

ASPA 1986 Amendment Legislation 2012

• Is an enabling act • Provides a three tier licensing system:

1. Place: Establishment licence (PEL) 2. Project licence (PPL) 3. Person: personal licence (PIL)

• Requires establishments to have

named persons and an AWERB.

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

AWERB

• Defined membership

• Function is to advise the PEL holder

• Enables a local perspective on the ethical issues and harm/benefit analysis.

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Animal research in the UK

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

31 cases – procedures conducted without licence authority. 9 cases – a failure to provide food and/or water. 20 cases describe animals found dead or dying. 37 cases describe avoidable suffering or animal welfare issues.

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Animal research in the UK

• In 2014:

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Animal research in the UK

• In 2014:

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Animal research in the UK • In 2014:

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Animal research in the UK • In 2014:

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest ? ?

? ? ?

? Any questions ?

? ? ? ? ?

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research - definition

•creative work undertaken in a systematic way to increase

the stock of knowledge and use it to devise new

applications

1. asking a question or proposing a hypothesis

2. generating or collating data/information

3. analysing and interpreting the data/information

4. reflecting on the research question or hypothesis and what the data/information tells you

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Integrity - Definition

Research Framework Openness &

Transparency

Research Outputs

•a concept of consistency of

actions, values, methods,

measures, principles,

expectations and outcomes

•acting in an honest, accurate

and truthful way

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Framework

• Legislation - ASPA 1986

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Nuffield Council on Bioethics

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Framework

• Legislation - ASPA 1986

• Policies – funding body T&Cs

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Framework

• Legislation - ASPA 1986

• Policies – funding body T&Cs

• Guidance – signatory organisations

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Framework

• Legislation - ASPA 1986

• Policies – funding body T&Cs

• Guidance – signatory organisations

• Others e.g. 3Rs

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

The 3Rs Principles of Humane Experimental Technique

Russell and Burch, 1959

1. Replacement:

– Relative - use animal cells, tissues, organs

– Absolute - use non-animal alternative

2. Reduction: use fewer animals

3. Refinement: minimise pain etc. and enhance welfare

Consider welfare throughout the animal’s life − husbandry, transport and death, as well as during procedures

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Replacement Methods which avoid or replace the use of animals

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Alternative methods may be used to replace animals in:

• a whole research programme

• a project within a programme

• an individual experiment

• one type of procedure

Alternatives can also be used as part of a structured approach e.g. in vitro screening to select candidate drugs with the desired properties.

Replacement Not an ‘ALL OR NOTHING’ Concept

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Reduction

Minimising the numbers of animals used - for example by improving the experimental design and statistical analysis used in a study.

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Refinement

Improving experimental procedures, and other factors affecting animals such as their housing and care, to reduce suffering and improve welfare throughout the animals’ lives

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Framework Openness &

Transparency

Research Outputs

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Openness & Transparency Why is this important?

“I agree with animal

experimentation for all

types of medical

research, where there is

no alternative”

0

10

20

30

40

50

60

70

80

Support

Don't know

Opposition

Ipsos MORI ‘Public attitudes towards animal research’ survey conducted for the Department for Business, Innovation and Skills (BIS) – June 2014

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Commitment 1: We will be clear about when, how and why we use animals in research Commitment 2: We will enhance our communications with the media and the public about our research using animals Commitment 3: We will be proactive in providing opportunities for the public to find out about research using animals Commitment 4: We will report on progress annually and share our experiences

Openness & Transparency

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Framework Openness &

Transparency

Research Outputs

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Outputs Houston, we have a problem…….

Key issues: • lack of reproducibility

• bias • competition

Conclusions: • most research findings are

false for most research designs and for most fields.

• research findings may often be simply accurate measures of prevailing bias.

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Outputs Reflecting current practice….

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Outputs Problems and possible solutions

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Outputs Poor reporting standards….

• Only 59% stated the hypothesis or objective of the study and the number and characteristics of the animals used.

• 87% did not use randomisation. • 86% did not use blinding. • Of those that did use a

statistical method only 70% reported what it was and presented results with a measure of error or variability.

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Research Outputs

Journal publication policies……

• Only 53% had an editorial policy on the animal research they publish.

• Of the journals with a policy the majority either:

– just included the word ‘animal’

– just required that the research met local legal standards

NOTE: very few journals required adherence to their policies as a condition of publication

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

• Literature search for studies of EAE in all Nature and PLoS journals pre and post-ARRIVE.

• Similar numbers reporting ‘ethical statements’, ‘blinding’, ‘randomisation’.

• Increase in incidence of reporting ‘species’, ‘sex’, and ‘age of animals’.

Change takes time……

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Risk of Bias Tool……

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Experimental Design Tools……

• InVivoStat: free statistical software program specifically

for scientists conducting animal experiments

• The Experimental Design Assistant:

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

Systematic Reviews……

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

3Rs information

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

the quality of

animal research reporting

access to unpublished animal research data

the impact of publication bias

the ability to systematically review the data

Disseminate good practice, improve animal welfare & scientific quality

Openness & transparency

the quality of animal research funded & conducted

Inform future research

wastage of money, animals, other resources

Why does any of this matter?

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

THANK YOU FOR LISTENING… nikki@responsibleresearchinpractice.co.uk

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest

RESPONSIBLE

Ethical

Science that is

Evidence based

AND above all

Reproducible

Challenging and

Honest ? ?

? ? ?

? Any questions ?

? ? ? ? ?

Dr Keith E. Fildes 14 April 2016

Data-Related Integrity

Data-Related Integrity

Research integrity is about rigor, respect and responsibility

Will look at how this applies to the collection, processing,

presentation and sharing of data

In both datasets and images

And concerning both research misconduct and questionable

research practices

Data and Research Integrity

Data-Related Integrity

Woo-Suk Hwang stem cell and cloning fraud case (2005) - all stem

cell lines for two Science and one Nature paper were fabricated

Duplications of microscopic photographs in different panels

designated as different lines + DNA comparison showed all lines

were performed on the same fingerprint profile

National scandal, unconditional retractions

and stripped of positions

Uncovered as postdoc bloggers raised

questions about the images

Consequences

Data-Related Integrity

It is a researcher's responsibility to ensure the integrity - the truth

and accuracy - of their data

By planning the data collection, analysis and sharing beforehand,

can identify risks to integrity and mitigate against them

E.g. backing-up raw data and saving an unprocessed version

Documenting is key - an audit trail so each manipulation is

replicable. Lab book, or similar process recording for non-STEM

Although manipulations may improve the data, you never know

when might need to go back to the original - when you need to

investigate an issue, or when your editor or your audience do

Start with a Plan

Data-Related Integrity

Can now generate huge amounts of data with relatively little effort

Established guidelines on data analysis, integrity, accessibility and

archiving have not kept pace with the data explosion

Interdisciplinarity means navigating many areas each with their own

standards to safeguard data accuracy

Open data movement places even more emphasis on investing time

and energy into data preservation and annotation

Datasets

Data-Related Integrity

Manipulations 1

Data-Related Integrity

Manipulations 2

Data-Related Integrity

Manipulations 3

Data-Related Integrity

Manipulation of images is now easy. Imaging software provides a

valuable means of showing imagery data more clearly

However there is a fine line between processing and questionable

manipulation practices

The scientific community has become concerned about

inappropriate image manipulation. 20-25% of journal papers

accepted contain at least one figure that does not comply with the

journal's instructions to authors. The scientific press continues to

report a small, but steady stream of cases of fraudulent image

manipulation.

Following examples are from biosciences, but the principle applies

to other disciplines

Images

Data-Related Integrity

Images

Data-Related Integrity

Example 1

Data-Related Integrity

A = a band deleted from the original data (lane 3). Deleting a band

from a blot, even if you believe it to be an irrelevant background

band, is a misrepresentation of your data

B = a band added to the original data (lane 3). Adding a band to a

blot, even if you are only covering the fact that you loaded the wrong

sample, and you know for sure that such a protein or DNA fragment

or RNA is present in your sample, is a misrepresentation of your

data. The additional band in lane 3 has been generated by simply

duplicating the band in lane 2.

Gross Misrepresentation

Data-Related Integrity

Example 2

Data-Related Integrity

A whole single panel has been replicated (arrows) and presented as

the loading controls for two separate experiments

Gross Misrepresentation

Data-Related Integrity

Example 3

Data-Related Integrity

A = Adjusting the intensity of a single band (arrow) in a blot. This constitutes a

violation of the widely accepted guideline that “no specific feature within an image

may be enhanced, obscured, moved, removed, or introduced.” In the manipulated

image the arrow indicates a single band whose intensity was reduced to produce an

impression of more regular fractionation. Although this manipulation may not alter the

overall interpretation of the data, it still constitutes research misconduct.

B = Adjustments of contrast. Images 1, 2, and 3 show sequentially more severe

adjustments of contrast. Although the adjustment from 1 to 2 is acceptable because

it does not obscure any of the bands, the adjustment from 2 to 3 is unacceptable

because several bands are eliminated. Cutting out a strip of a blot with the contrast

adjusted provides the false impression of a very clean result (image 4 was derived

from a heavily adjusted version of the left lane of image 1). While it is acceptable

practice to adjust the overall brightness and contrast of a whole image, such

adjustments should “not obscure or eliminate any information present in the original”.

When you scan a blot, no matter how strong the bands, there will invariably be some

grey background. While it is technically within the guidelines to adjust the brightness

and contrast of a whole image, if you over-adjust the contrast so that the background

completely drops out (part 2 vs. part 3), this should raise suspicions among reviewers

and editors that other information (especially faint bands) may have dropped out as

well.

Brightness/Contrast Adjustments

Data-Related Integrity

Example 4

Data-Related Integrity

The Photoshop 'rubber stamp' or 'clone stamp' tool has been used in the manipulated

image to clean up unwanted background in the original data. Close inspection of the

image reveals a repeating pattern in the left lane of the manipulated image, indicating

that such a tool has been used.

This kind of manipulation can usually be detected by someone looking carefully at the

image file because it leaves tell-tale signs.

What may seem to be a background band or contamination may actually be real and

biologically important and could be recognised as such by another scientist.

Cleaning Up Background

Data-Related Integrity

Example 5

Data-Related Integrity

The particles, which were actually present in the original (left), have been enhanced

in the manipulated image (right). The background dot in the original data has also

been removed in the manipulated image.

The intensity of the particles has been enhanced by manually filling them in with

black colour using Photoshop. This type of manipulation misrepresents your original

data and is thus misconduct. There are acceptable ways to highlight such a feature,

which include arrows or pseudo-colouring. If pseudo-colouring is done with the

'colourise' function of Photoshop, it does not alter the brightness of individual pixels.

Pseudo-colouring should though always be disclosed in the figure legend.

Enhancing a Specific Feature

Data-Related Integrity

Example 6

Data-Related Integrity

Cells from various fields have been juxtaposed in a single image, giving the

impression that they were present in the same microscope field. A manipulated

panel is shown at the top. The same panel, with the contrast adjusted to reveal the

manipulation, is shown at the bottom.

The reader assumes that a single micrograph presented in a figure represents a

single microscope field. Combining images from separate microscope fields into a

single micrograph constitutes a misrepresentation of your original data. In the

manipulated image, cells have been combined from several microscope fields into a

single micrograph. This manipulation becomes visible when the contrast of the image

is adjusted, so that the inserted images become visible (bottom panel). You may want

to combine images from several fields into a single micrograph to save space, but

this assembly should be clearly indicated by thin lines between the different pieces.

Misrepresentation

Data-Related Integrity

Example 7

Data-Related Integrity

Square boxes show acceptable manipulation (if properly described), as it is equally

applied across the whole image.

However the rectangular image shows that 'tidying' has taken place - i.e. non-

concurrent portions are not indicated, which is misrepresentation

Acceptable Manipulation?

Data-Related Integrity

Example 8

Data-Related Integrity

Acceptable manipulation between the first and second images. However the third

includes misrepresentation

Enhancement?

Data-Related Integrity

It is essential that the manipulation carried out is described when presenting the data.

This enables the reader to fully understand the processes being used and be able to

reproduce the data

'Raw' pre-enhanced data should also be available (to reviewers, examiners and/or

readers)

Good Practices

Data-Related Integrity

Nature

Images should be minimally processed, for instance to add arrows

Authors should retain unprocessed data and metadata files, as editors may request

them to aid manuscript evaluation

A certain degree of image process is acceptable, but the final image must represent

the original data and confirm to community standards

Authors should list all image acquisition tools and image processing software

packages use, and should document key image-gather settings and processing

manipulations in their methods

Images gathered at different times or from different locations should not be combined

into a single image

Touch-up tools, such as cloning and healing tools in Photoshop, or any feature that

deliberately obscures manipulations, are to be avoided

Processing, such as changing brightness and contrast, is appropriate only when it is

applied equally across the entire image and is applied equally to controls. Contrasts

should not be adjusted so that data disappears. Excessive manipulations, such as

processing to emphasise one region in the image at the expense of other, is

inappropriate, as is emphasising experimental data relative to the control

Publisher Guidelines 1

Data-Related Integrity

Science

Does not allow certain electronic enhancements or manipulations of micrographs,

gels, or other digital images.

Figures assembled from multiple photographs or images, or non-concurrent portions

of the same image, must indicate the separate parts with lines between them.

Linear adjustment of contrast, brightness, or colour must be applied to an entire

image or plate equally.

Nonlinear adjustments must be specified in the figure legend.

Selective enhancement or alteration of one part of an image is not acceptable.

May ask authors of papers to provide additional documentation of their primary data.

Journal of Cell Biology

No specific feature within an image may be enhanced, obscured, moved, removed or

introduced.

Publisher Guidelines 2

Data-Related Integrity

Microscopy Today

Always keep unaltered originals on permanent un-rewritable media

Make adjustments to the whole image

Avoid lossy compression files (JPEG) and use lossless formats (TIFF)

Any adjustments which are unusual should be documented and reported on

Publisher Guidelines 3

Data-Related Integrity

In software such as Photoshop, ImageJ, and ImagePro, an audit trail log can be

enabled to track each step in a manipulation process. It must be enabled in order to

record replicable steps used to process images

Audit Trail

Data-Related Integrity

Audit Trail Logs

Data-Related Integrity

Rossner, M and Yamada, K. M, 'What’s in a Picture? The Temptation of Image Manipulation',

Journal of Cell Biology, clxvi (2004), pp.11-15.

Cromey, D. W, 'Avoiding Twisted Pixels: Ethical Guidelines for the Appropriate Use and Manipulation

of Scientific Digital Images', Science and Engineering Ethics, xvi (2010), pp.639-67.

Cromey, D. W, 'Digital Images Are Data: And Should Be Treated as Such', Methods in Molecular

Biology, cmxxxi (2013), pp.1-27.

Jordan, S. R, 'Research Integrity, Image Manipulation and Anonymizing Photographs in Visual

Social Science Research, International Journal of Social Research Methodology, xvii (2014),

pp.441-54.

Lan, T. A, Talerico, C and Siontis, G. C. M, 'Documenting Clinical and Laboratory Images in

Publications - The CLIP Principles', CHEST, cxli (2012), pp.1626-32.

Miles, K, 'Integrity of Science Image Data: Issues and Emerging Standards'

www.indiana.edu/~lmic/Workshops/Image%20Processing%20Workshop%20copy/IU-

Bloomington2013.pdf

South-West Environmental Health Sciences Centre (University of Arizona), 'Digital Image Ethics:

Introduction to Image Editing Ethics'

http://swehsc.pharmacy.arizona.edu/micro/digital-image-ethics

References

Data-Related Integrity

k.fildes@shu.ac.uk

@SHDocSchool

@SHaRD_Programme

https://shdoctoralschool.wordpress.com

https://shardprogramme.wordpress.com

Contact