Pitfalls in Studies

Models from Literature

Clinical Expertis

e

Best Researc

h Evidenc

e

Patient

Values

EBPEBP

now You know

Now you know where to search for evidence using the study design hierarchy of evidence.

systematic reviews

Prospective controlled trial

Cohort trial

Case series studies

Expert opinion

RCT

So you can decide

►the proper terminology of the research design you perform to avoid others’ mistakes

Example I

►A non-randomized controlled trial was described as prospective randomized study (Alhelou et al, MEFS 2004: 37-41)

Example II

► A case series was described as prospective cohort study (Bigelow et al,

Human Reprod 2004; 889-92)

Example III

►Semen sample collection in medium enhances the implantation rate following ICSI in patients with severe oligoasthenoteratozoospermia Zollner , Human

Reproduction, , 1110-1114, June 2001 ►However, on re-analysis of the

data, non-significant P value of 0.37 which is quite different from the authors' P value of < 0.001 Van Royen, and J.

Gerris

Thus

►Clinical as well as statistical knowledge is needed in the field of subfertility Dickey, 2003

Miracle Trial

►Infertile women who were prayed for by prayer groups became pregnant twice as often as those who did not have people praying for them.

►Later, accused of being fabricated

Getting Started

► Read to learn; read to analyze● About research methodology

● Studies on similar topics

● Interesting studies

Then perform

►Methodologically sound studies with appropriate follow up

►To improve outcome (conception)

►To prevent Adverse event (OHSS)

Keep In Mind That

►No study is perfect

►“All data is contaminated some way or another; research is what you do with these data”

►Data collection involves agreement & consent

►Partnership job description

Basic steps of a research project

►Find a topicWhat►Formulate questionsWhat, Why►Define populationWho, When►Select design & measurementHow►Gather dataHow►Interpret resultsWhy►Tell about what you did and found out

Common Pitfalls

►Problems with population●Sampling?

Representativeness?

Common Pitfalls

►Problems with operationalization●Defining of what is measured

Common Pitfalls

►Problems with generalizability ●False conclusions

How to avoid research pitfalls

►Treatment efficacy is most reliably assessed by undertaking a randomized, controlled trial.

►allocation to the experimental and control interventions occurs by chance

30% in the last 5 ys

►acceptance of the randomized controlled trial (RCT) in the field of reproductive medicine is evident by the increasing numbers of such trials being published

►From 1966 -2005 = 864

►From 2000-2005 = 258

But even in well designed studies

►Certain pitfalls could happen and can be avoided

Intention-to-treat analysis:

►Including and analysing all randomised patients according to their original treatment allocation, irrespective of whether they actually received that treatment. This preserves the unbiased comparison of treatment groups afforded by randomization.

Loss to follow-up:

►Where patients stop contributing outcome data. This may be because they can no longer be contacted,

►for example, having moved away or because they actively want to drop-out of further participation in the trial. The latter may be related to clinician withdrawal or patient compliance.

cross-over trial

►Women will have the opportunity to receive the experimental treatment, if not in the first cycle (or period) then in the second cycle (or period).

►when pregnancy is the outcome of interest, it is an inappropriate methodology and should be avoided

Why

►the subject who conceives with onetreatment in the first period will be classified as a dropout in the secondperiod.

►The effect of treatment in the first period could extend to the second period

Bias

primary outcome indicator ►It needs to be stressed that in

RCTs in which women undergoing assisted reproduction treatment are randomized to receive an experimental or control intervention, the unit of analysis is the randomized woman

For example

►The use of implantation rates (which requires calculating the proportion of all embryos that implant) uses the embryo as the unit of analysis.

►This is methodologically incorrect and inflates the denominator because each randomized woman may contribute several embryos to the analysis.

Example II

►evaluating outcomes on a per-cycle of treatment basis rather than a per-patient basis.

Clinical heterogeneity

►Down regulation protocol long, short, agonist or antagonist

►Day of ET

►Luteal phase support regimen

The CONSORT statement

checklist and flow diagram for reporting RCTs

associated with an improvement in the quality of reports of RCTs (Moher et al., 2001)

Gaps: Example

►Currently, there is no randomized study addressing the effect of metformin on the rate of early miscarriage

►PCOS are well known cause of miscarriage

Example II

►Effect of fibroids on fertility in patients undergoing assisted reproduction

Be Critical About Numbers►How was the choice for the

measurement made?

►What type of sample was gathered & how does that affect result?

►Is the statistical result interpreted correctly?

►If comparisons are made, are they appropriate?

Estimate of effectEstimate of effect►The observed relationship between

an intervention and an outcome is statistically expressed as an “estimate of effect” e.g. an ● Odds ratio (OR) or a ● Relative risk (RR)

Odds ratio (OR)Odds ratio (OR)►If the OR = 1:

●Intervention has no effect●The ratio of the number of people

in a group with an event to the number without an event =1

Relative risk (RR)Relative risk (RR)► If the RR = 1:

●there is no difference between the risk of the event occurring in the intervention group or the control group.

●The risk of the event in both intervention and control groups is equal.

Confidence Interval (CI)Confidence Interval (CI)►The range within which the “true”

value (e.g. size of the effect of the intervention) is expected to lie with a given degree of certainty (e.g. 95% or 99%).

Estimate of effect is graphically displayed as the midline of the blob or square

Confidence interval (CI)shows the range within which the true size ofeffect of intervention is likely to lie

Overall effect sizeThis denotes the overall statistical result.

Number needed to treat (NNT)Number needed to treat (NNT)

►The NNT reflects the number of patients who need to be treated to prevent one bad outcome.

Meta-analysisMeta-analysis►A meta-analysis is a statistical

technique used to combine or pool the results numerically of several independent studies addressing the same question.

Thank You