STATISTISTICAL METHODS FOR RANKING

Sture HolmFMS Jubileumskonferens på Utö

25 oktober 2012

A COMMON TYPE OF PRESENTATIONComparison between care centers

• Here about 200 care centers • 40 000 involved patients, 56 percent answers• In central part often 1 unit differences • Weighting of 6 ”items” med 50-steps (from 0

to 600 ”points” for the individuals)• NO MEASURES OF DISPERSION GIVEN !!• ”Manufactured” by consultants ”Indikator”

A typical exempleSuccess rates in units

A B C D E F G H I J

Tot 157 100 245 199 107 299 479 305 442 207

Count 78 61 182 146 70 210 327 158 214 150

Estim 0.497 0.610 0.743 0.734 0.654 0.702 0.683 0.518 0.484 0.725

How can we make correct statistical statements on the ranks ?

• As I see it there are two steps:1.A suitable statistical method for pairwise

comparison between cases. This is standard technique i statistics.

2.To use results in this first step for making a proper statistical statement on the rank of a unit with a confidence interval.

Step 1. Comparison within pairs

• Different problems requires different basic methods, for example:

• Wilcoxon test methods• Binomial comparison• Normal approximation• OBSERVE: MOSTLY THERE ARE DIFFERENT

SAMPLE SIZES IN THE UNITS

Step 2:

• Principle: To use one-sided p values for the pairs in order to get the number of pairs, that can be considered ”significant” on each side. (within the half over all risk).

• But how ?• To use mutiple test methods• Will give choosen confidence degree

Simplest: Holm-Bonferroni

• Bonferroni: Devide the risk (e.g. 2.5 %) with the number of cases, and use it individually.

• + Holm (SJS 1979): Continue with new steps where the number is the remaining non-rejected. Go on as long as there are new rejections.

• ”Translate” to intervals for rank.

Illustration of multiple test

The simple binomial example

• Consider for instance unit E. Test hypothesis that all other units are at least as good with multiple level 2.5 % and find the ones which can be declared worse than E.

• Repeat on other side

A B C D E F G H I J

Tot 157 100 245 199 107 299 479 305 442 207

Count 78 61 182 146 70 210 327 158 214 150

Estim 0.497 0.610 0.743 0.734 0.654 0.702 0.683 0.518 0.484 0.725

The estimatesA [1;4] (0.497 rank 2)

B [1;10] (0.610 rank 4)

C [4;10] (0.743 rank 10)

D [4;10] (0.734 rank 9)

E [2;10] (0.654 rank 5)

F [4;10] (0.702 rank 7)

G [4;10] (0.683 rank 6)

H [1;4] (0.518 rank 3)

I [1;4] (0.484 rank 1)

J [4;10] (0.725 rank 8)

• Estimated order:• I, A, H, B, E, G, F, J, D, C

• The ”central” B and E have no or almost no information on rank

• I, H, A are positioned low [1;4]• G, F, J, D, C are positioned high [4;10]

An exemple with scales

• A: 35, 44, 51, 88, 46 • B: 19, 29, 34, 41, 30 • C: 43, 52, 54, 99, 56 • D: 42, 50, 70, 117, 61• E: 13, 37, 49, 85, 45 • F: 15, 21, 28, 71, 35 • G: 16, 40, 46, 86, 34 • H: 19, 36, 39, 73, 31• I: 17, 37, 55, 126, 65 • J: 16, 15, 18, 50, 20

Confidence intervals for ranks

A [1;8]

B [1;7]

C [1;7]

D [1;8]

E [6;9]

F [7;10]

G [1;9]

H [1;8]

I [8;10]

J [1;9]

Estimated order

• B, C, A, H, D, G, J, E, F, I• For the extremes B and I (153 resp. 300 obs.):

• Correct general method for different situations with chosen confidence degree.

• A little inefficient since it is based on the Boole inequality, which is very good for negatively dependent and independent cases, but not so good for positively dependent cases. We have positive dependence here since there is comparison with the same case.

Advantages and disadvantages

Is there a big loss ?

• Here follows a number of cases with 50 % correlation exact normal bound correspoding Bonferroni bound. Total risk 1 %.

• Is it worthwhile to try to be exact ?

Units 2 4 8 12 20

Exact 2.56 2.77 2.97 3.08 3.21

Bonf. 2.58 2.81 3.02 3.14 3.29