Teaching Medical Students, Interns, and

Residents to read Medical Literature:

The STATS Method

Session Objectives

1) Examine the current trends in physician use of medical literature.

2) Explain why teaching medical students, interns, and residents how to read medical literature is important.3) Explore the STATS Method to research evaluation.

Current Trends in Physician use of Medical

Literature

The Good, the Bad, and the Lack of Probability

The Good

Growth in Clinical Trials

According to the Centre for Evidence Based Medicine, there are now around 1 million clinical trials that have been completed.

Trial Classification(NIH Standards)

Phase I Trials

• 20-80 Participants• Examine drug/therapy safety, dosage recommendations, and possible side effects

Phase II Trials

• 100-300 Participants• Purely a Safety Evaluation

Phase III Trials

• 1,000-3,000 Participants• Confirms effectiveness, monitor’s side effects, compares to other treatments, collects info to allow for safe usage of drugs/treatments

Phase IV Trials

• Post Marketing Examine • Collects additional information on risks, benefits and optimal use

Growth in Medical Research

In 2002 alone, there were 522,943 articles published in some 4,600 medical journals catalogued by PubMed.

There is an abundance of medical research being conducted!

The Bad

Last 5 Years

• 1999 – 473,044• 2000 – 512,226• 2001 – 518,741• 2002 – 522,943 • 2003 – 100,001

–As of March 24, 2003

That’s a total of 2,126,955

Articles in just the last 5 years

According to the PubMed Website1, they have over 12 million citations dating back to the 1960s.

So the last five years accounts for 18% of all of those citations.

To stay up-to-date with the current medical literature, you would have to read about 6,000 research articles a day.

How many minutes did you spend last week reading about your patients?2

Stage of CareerRange of median reading times

% who reported NO reading in the last week

Medical students 60-120 min 0%

House officers 0-20 min up to 75%

Senior house officers 10-30 min up to 15%

Attendings 10-90 min up to 40%

Senior Attendings 10-45 min up to 15%

Consultants graduating since 1975

15-60 min up to 30%

Consultants graduating pre-1975

10-45 min up to 40%

Medical Knowledge with Years Since Graduation3

The Probability

Statistics is simply a matter of mathematical probability.

Most modern medical research is based in statistics.

It is estimated that nearly 50% of all medical research articles contain serious methodological and/or statistical errors4.

Why teaching medical students, interns, and

residents about research is so

important

Paradigm Shift5

EBM

Evidence

Based

Medicine

EBM Definedis the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of EBM means integrating individual clinical expertise with the best available external clinical evidence from systematic research.

Traditional vs. EBM Training

Sackett et al. (1977) Study on EBM and Hypertension Treatment

EBM in the USAIn the early 1990s, JAMA established the Evidence-Based Medicine Working Group5.

This group, released a series of articles about reading medical literature throughout the 1990s and early 2000s7.

The STATS Method

ST

AT

S

Sample

Treatment

Argument

Testing

Significance

SAMPLE

Questions to ask about the sample.1. How were participants selected for inclusion in the study?

2. How were they approached for inclusion?

3. Do study participants have special characteristics?

Questions to ask about the sample.4. Do the demographics of the study bias the study in any direction?

5. Does the location of the participants bias the study in any direction?

6. What could you do as a researcher to prevent sampling bias?

TREATMENT

Treatment Defined

“Treatment” refers to those variables actively manipulated or measured in a scientific study.

Variables

Any entity that can take on different values.

Dependent Variables

The variable that is effected or not effected by another variable in a research study.

Independent Variables

The variable that is being manipulated or examined in a study to see if it effects the dependent variable.

Ultimately We want Causation

We want the IV to cause a difference in the DV.

FOR EXAMPLE

Is there a difference between females and males and their use of physical aggression?

The treatment is how we go about measuring and explaining this process.

Questions to ask about the treatment.

1. What is the primary dependent variable studied in the article?

2. How did the researchers go about measuring this variable?

3. Did the researchers attempt to manipulate the target variable? If so, how?

ARGUMENT

Logical argumentation should be the basis for all scientific research.

A logical argument in science is called a hypothesis.

Questions to ask about the arguments.

1. Was the argument for the study made clear in the article?

2. Is the article’s argument supported by current medical research and practice?

3. Is the argument in the article based on science or conjecture?

TESTNG

Testing is the process a researcher goes through to either prove or not prove a hypothesis.

Two Types of Tests

1. Difference Tests

2. Relationship Tests

Difference TestsA difference test attempts to see if there is a difference between the means of two or more groups in relation to one or more dependent variable(s).

FOR EXAMPLE

In this case, we wanted to see if there was a difference in the mean scores of female and male physical aggression.

Two Types of Difference Tests

1. Non-Parametric Tests – Involve Nominal and Ordinal Variables (x2 or rank order test)

2. Parametric Tests – Involve interval or ratio variables with each other or with nominal and ordinal variables(t-tests or ANOVAs)

Relationship TestsA relationship test attempts to see if two (or more) variables can be plotted in a linear fashion.

Relationships

Two Types of Relationship Tests

1. Symmetric – Unable to designate cause and effect(Correlation)

2. Asymmetric – Existence of explanatory variables or predictors(Regression)

BIG NOTECorrelation does not mean causation

However, Causation does mean Correlation

Significance

What is the actual significance of this study?

If research doesn’t say anything new, why should we care?

Activity

Concluding Thoughts1. EBM is a new direction for medical practitioners.

2. With the abundance of research available we have to be more cynical.

3. As people involved in post-graduate medical education, we need to teach critical analysis of research.

Concluding Thoughts cont…4. Ask questions about research based on the STATS method.

5. Learn more about how research is actually conducted. Don’t be a sucker!!!

6. One study does not equal science.

Prepared by:Jason S. Wrench, Ed. D.

Medical Educational SpecialistWest Virginia School of Osteopathic Medicine