A Study on the Relative Efficiency Of Colorectal Endoscopists

A Study on the Relative Efficiency Of Colorectal

Endoscopists

Barbara LumNathan Hedges

Ruwan KiringodaSarah Hong

Colonoscopy

Colonoscopy is the best technique for examination of the large intestine, and for the biopsy and treatment of mucosal lesions

Data Source for the Project

The Cleveland Clinic in Cleveland, Ohio

Conducted by a group member at the Cleveland Clinic, this study examined the frequency and impact of problems that interfere with

smooth colonoscopy

Goals of the Study To create an objective, quantifiable method of

measuring the efficiency with which a doctor operates

To determine whether there are mathematically significant differences in efficiency levels between doctors

Determine what independent factors may be responsible for varying efficiency levels and to what degree each factor is responsible.

Background

Colonoscopy is the visual examination of the large intestine (colon) using a lighted, flexible fiber-optic or video endoscope.

Background (con’t)

The colon begins in the right-lower abdomen and looks like a big question mark as it moves through the abdomen, ending in the rectum. It is 5 to 6 feet long.

Equipment The flexible colonoscope can be

directed and moved around the many bends of the colon

The scope uses a small, optically-sensitive computer chip at the end.

Equipment (con’t)

Electronic signals are transmitted to the computer.

An open channel in the scope allows other instruments to be passed through in order to perform biopsies, remove polyps, etc.

Colonoscopies are not fun!

Benefits

Identification and/or correction of a problem in the colon.

Allows for diagnosis and specific treatment.

The Study (some numbers to keep in mind)

Five staff surgeons were observed as they performed

colonoscopies.

A total of 203 colonoscopies were observed.

The study began in June 2002 and concluded the following September.

104 men, 99 women.

Mean age of 61; standard deviation of 12 years.

Efficiency Errors The study focused on the incidence of potential efficiency

problems that occur during the implementation of the procedure.

The phrase ‘efficiency error’ was used to define any error in setup, procedure or scope function that results

in the avoidable loss of time.

The researcher quantified how much time was lost by using a stopwatch.

Four Types of Errors

Setup - valves, caps, hoses not fastened securely

Scope function - scope clogged

Procedural - Switching colonoscope midway through the exam, errors by trainees.

Torque - Scope hose overly twisted, needs to be re-set.

Efficiency error results

Church Hull Senagore Strong Remzi Total

“n” (total) 77 32 9 56 28 202

Inefficiency Error21 5 1 14 7 47

Inefficiency Errors7 2 0 1 1 11

Inefficiency Errors2 0 0 0 0 2

otalfficiency Errors

41 9 1 16 9 76

rojected Efficiencyrrors (p=76/202)

28.97 12.04 3.39 21.07 10.53 76

Statistical Analysis, Chi- Squared Test 1

Graph 1: Ho: p = .376Ha: P .376

ChiTest([28.97,12.04,3.39,21.07,10.53 ],[41,32,9,56,28]);

"The chi-squared value is", 134.2603333, "where there are", 5, "classes, and a probability of about ", 0., "percent that the null hypothesis is true and we see such chi squared value.""In particular this test is highly significant and we may reject the null hypothesis based on it"

Chi- Squared Test, Graph 1

Revised Efficiency Error Results

Hull Senagore Strong Remzi Total

“n” (total) 32 9 56 28 125

1 Inefficiency Error5 1 14 7 47

2 Inefficiency Errors2 0 1 1 11

3 Inefficiency Errors0 0 0 0 2

Total Efficiency Errors 9 1 16 9 35

Projected EfficiencyErrors (p=76/202)

8.96 2.52 15.68 7.84 35

Statistical Analysis, Chi- Squared Test 2

Graph 2: Ho: p = .280Ha: P .280

ChiTest([8.96,2.52,15.68,7.84 ],[9,1,16,9]);

"The chi-squared value is", 1.095167234, "where there are", 4, "classes, and a probability of about ", 77.82408589, "percent that the null hypothesis is true and we see such chi squared value.”

"In particular this test is not significant and we should not reject the null hypothesis based on it"

Chi- Squared Test, Graph 2

Part II

What factors could potentially affect the colonoscopists’ incidence of error?

Possible factors: • Years of Practice at the Cleveland Clinic • Level of Specialty Education

• Ex: Residencies, fellowships, graduate degrees• Number of American Board Certifications

Doctor

% of TotalProcedures with

EfficiencyErrors

YearsPracticing

Years ofSpecialty

Education

Number ofBoard

Certifications

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

D r . Jame s Chu r ch

57 . 14% 13 7 0

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

D r . T r acy Hu ll

28 . 13% 8 8 2

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

D r . An t hony S enago r e

11 . 11% 3 12 3

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

D r . Sco tt S t rong

28 . 57% 10 10 1

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

D r . Fe z a R em z i

32 . 14% 5 7 2

The Doctors

Practice

Errors Fitted values

5 10

27.8326

32.14

Remzi

Hull

Strong

Regression of Percentage Errors on Years Practicing

= 0.05 with 3 d.fT = 2.77

Critical value = 2.353

Regression of Percentage Errors on

Specialty

Errors Fitted values

7 12

11.11

57.14

Church

Remzi

HullStrong

Senagore

Years of Specialty Education

= 0.05 with 3 d.fT = -2.24

Critical value = 2.353

Regression of Percentage Errors on

Number of Board Certifications

Certifications

Errors Fitted values

0 3

11.11

57.14

Church

Strong

Remzi

Hull

Senagore

= 0.05 with 3 d.fT = -3.90

Critical value = 2.353

Multivariate Regression of Percentage Errors on

Variable Coefficient Standard Error t Beta

Practice -0.810 3.616 -0.22 -0.194

Specialty -2.970 2.165 -1.37 -0.389

Certifications -12.762 13.200 -0.97 -0.880

Constant 84.288 47.368 1.78 .

Years Practicing, Years of Specialty Education and Number of Board Certifications

Regression Analysis

All three variables appear to be negatively related to percentage of efficiency errors

Relation is not strong enough to be statistically conclusive

Directions of correlations “appear” right, but more doctors need to be sampled to obtain medical significance.

Conclusions

By a medically-acceptable level of significance, one of the doctors was found to be operating at a far inferior efficiency level than the other endoscopists.

“Efficiency errors,” however, refer only to small errors that result in a loss of time for the procedure. They do not, necessarily have an impact on patient safety or welfare.

Implications

If a doctor can be established in a court of law as being inferior, then potential litigation could be brought by unknowing patients who experience deleterious effects as a result of inferior treatment.

Patients should know of a doctor’s track record before undergoing the procedure

Limitations of this study

Number of trials Only 202. Perforated colons, the most serious of operational

errors, only occur in 1/1000 operations. Number of doctors

Only 5. What would a study of 500 doctors yield?

Summary

The overall scope of the study is limited Does not take into account other factors

Patient experience in terms of suffering Risk

As outlined by the abstract, we have met the goals set forth in this presentation.

Overall, this study provides a helpful and informative framework for future studies.