week2 1

Describing distributions with numbers

• A large number or numerical methods are available for describing quantitative data sets. Most of these methods measure one of two data characteristics:

The central tendency of the set of observations – it is the tendency of the data to cluster, or center, about certain numerical values.

The variability of the set of observation – it is the spread of the data.

week2 2

Measuring Center

• The Mode is the observation that occurs most frequently.

• The mode for categorical variable will be the label of thecategory with the highest number of counts.

• Measuring center

Two common measures of center are the mean and the median.These two measures behave differently.The mean is the “average value” and the median is the “middle value”.

week2 3

Measuring center: the median

• The median M is the midpoint of the distribution, the number such that half the observations are smaller then it and the other half are larger.

• To find the median of a distribution:1. Arrange the observations in order of size, from smallest

to largest.

2. If the number of observations n is odd, the median is the center observation in the ordered list.

3. If the number of observations n is even, the median is the average of the two center observations in the ordered list.

week2 4

Example

The annual salaries (in thousands of $) of a random sample of five employees of a company are: 40, 30, 25, 200, 28

Arranging the values in increasing order:25 28 30 40 200

median = 30Excluding 200 median = (28+30)/2=29.

week2 5

• MINITAB commands Stat > Basic Statistics > Display Descriptive Statistics

• MINITAB output for the data in the example above is given bellow:

Variable N Median

salary 5 30.0

week2 6

Measuring center: mean

• To find the mean of a set of observations, add their values and divide by the number of observations. If the nobservations are x1,x2,…xn, their mean is given by

• ExampleFind the mean of the following observations: 4, 5, 9, 3, 5.Solution:

nx

nxxxxmean in ∑=+⋅⋅⋅++

== 21

x

week2 7

Example• The annual salaries (in thousands of $) of a random sample of

five employees of a company are: 40, 30, 25, 200, 28.

If we exclude 200 as an outlier,

• Mean is sensitive to the influence of a few extreme observations. Because the mean cannot resist the influence of extreme values, we say that it is NOT a resistant measure of center.

week2 8

Mean versus median

• The median and mean are the most common measures of the center of a distribution.

• If the distribution is exactly symmetric, the mean and median are exactly the same.

• Median is less influenced by extreme values.• A data set is said to be skewed if one tail of the distribution

had more extreme observations than the other tail.• If the distribution is skewed to the right, then

mode < median < mean• If the distribution is skewed to the left, then

mean < median < mode.

week2 9

Questions1. You are asked to recommend a measure of center to

characterize the following data:0.6, 0.2, 0.1, 0.2, 0.2, 0.3, 0.7, 0.1, 0.0, 22.5, 0.4.What is your recommendation and why?

2. The mean is ____ sensitive to extreme values than the median.(a) more(b) less(c) equally(d) can’t say without data

3. Changing the value of a single score in a data set will necessarily cause the mean to change. (T/F)

4. Changing the value of a single score in a data set will necessarily cause the median to change. (T/F)

week2 10

Percentiles

• The simplest useful numerical description of a distribution consists of both a measure of center and a measure of spread.

• We can describe the spread or variability of a distribution by giving several percentiles.

• The pth percentile of a distribution is the value such that ppercent of the observations are smaller or equal to it.

• The median is the 50th percentile.

• If a data set contains n observations, then the pth percentile is the value in the ordered data set.( 1) 100

thpn + ×

week2 11

Example• Find the 20th percentile of the data represented by the

following stem-and-leaf plot.Stem-and-leaf of Rural N = 29Leaf Unit = 1.0 N* = 71 2 15 3 3589

(12) 4 12233345678812 5 1124676 6 75 7 043 8 481 9 1 10 8

week2 12

Solution

week2 13

Quartiles

• The 25th percentile is called the first quartile (Q1).

• The first quartile (Q1) is the median of the observations whose position in the ordered list is to the left of the location of the overall median.

• The 75th percentile is called the third quartile (Q3).

• The third quartile (Q3) is the median of the observations whose position in the ordered list is to the right of the location of the overall median.

NOTE: The median is the second quartile Q2 .

week2 14

ExampleThe highway mileages of 20 cars, arranged in increasing order are:

13 15 16 16 17 19 20 22 23 23 | 23 24 25 25 26 28 28 2829 32.

The median is …

The first quartile Q1 is …

The third quartile Q3 is…

• Exercise: Find (a) the 10th percentile.(b) the 90th percentile of the above data set.

week2 15

Measuring Spread

• The range (max-min) is a measure of spread but it is very sensitive to the influence of extreme values.

• The distance between the first and third quartiles is called theInterquartile range (IQR) i.e. IQR =Q3 – Q1 .

• The IQR is another measure of spread that is less sensitive to the influence of extreme values.

week2 16

The five-number summary

• The five-number summary of a set of observations consists of the smallest observation, the first quartile, the median, the third quartile and the largest observation.

• These five numbers give a reasonably complete description of both the center and the spread of the distribution.

• MINITAB commands: Stat > Basic Statistics > Display Descriptive Statistics

week2 17

Example• The highway mileages of 20 cars, arranged in increasing order are:13 15 16 16 17 19 20 22 23 23 23 24 25 25 26 28 28 28 29 32.

Give the five number summary.

• AnswerWe have, min = 13, first quartile = 18, median = 23, third quartile = 27 , max. = 32.

The MINITAB output using the above commands is as follows:

Variable N Minimum Q1 Median Q3 Maximummileage 20 13.00 17.50 23.00 27.50 32.00

week2 18

Box-plot

• A box-plot is a graph of the five-number summary.• Example:

Make a box-plot for the data in the above example.

• MINITAB commands: Graph > Boxplot

Mile

ages

30

25

20

15

Boxplot of Mileages

week2 19

Exercise

The stem-and-leaf plot for a set of 50 observations is given below:Draw a box-plot for the data.Stem-and-leaf of Fees N = 50Leaf Unit = 1.0

2 0 897 1 00234

15 1 55558899(28) 2 0000000000000111112222222223

7 2 595 3 04 3 53 4 001 4 1 5 0

week2 20

Exercise• The box-plot, histogram and stem-and-leaf plot for a data set

are given below. Describe the distribution.Stem-and-leaf of C2 N = 50Leaf Unit = 1.0(29) 0 0001111111112222222223344444421 0 5555566678810 1 02346 1 664 2 13 2 881 3 1 3 8

403020100

C2

4036322824201612840

20

10

0

C2

Freq

uenc

y

week2 21

Exercise• Consider the following Minitab generated box-plots of coagulation times in

seconds for samples of blood drawn from animals receiving three different diets denoted 1, 2, and 3 :

• State whether the following statements are true or false a) The animal that had the longest coagulation time was given diet 3. b) The greatest variability occurs with diet 2. c) Diet 1 shows evidence of right (positive) skewness but diet 2 shows evidence

of left (negative) skewness. d) Approximately 25% of animals on diet 2 had coagulation times less then 63. e) The smallest upper (third) quartile is for diet 3. f) We can see that the mean for diet 1 is less than 62 seconds.

321

70

65

60

Dietco

agtim

es

week2 22

Measuring spread: Standard deviation• The variance (s2) of a set of n observations is

• The standard deviation (s) is the square root of the variance (s2). i.e.

• It can be shown that,

This formula is usually quicker.

nxxx ,...,, 21

2 2 2 2( ) ( ) ( ) ( )2 1 21 1

x x x x x x x xn isn n

− + − + − −∑= =− −

L

2 2 2 2( ) ( ) ( ) ( )1 21 1

x x x x x x x xn isn n

− + − + − −∑= =− −

L

2 21

x nxis n−∑=−

week2 23

• The deviations display the spread of the values xi about their mean. Some of these deviations will be positive and some negative because the observations fall on each side of the mean.

• The sum of the deviations of the observations from their mean will always be zero.

• Squaring the deviations makes them all positive, so that observations far from the mean in either direction have large positive squared deviations.

• The variance is the average of the squared deviations.• The variance, s2, and the standard deviation, s, will be large if

the observations are widely spread about their mean, and small if the observations are all close to the mean.

xxi −

week2 24

Example• Find the standard deviation of the following data set: 4, 8, 2, 9, 7.

• Solution: n=5 ,

• Using the second formula we have

week2 25

• MINITAB commands Stat > Basic Statistics > Display Descriptive Statistics

• MINITAB output for the above data is given below:

Variable N StDevC1 5 2.92

• Exercise: Find the standard deviation of the following data set:5, 8, 7, 9, 7, 11.

week2 26

Properties of standard deviation (s)

• s measures the spread about the mean and should be used only when the mean is chosen as the measure of center.

• s = 0 only when there is no spread. This happens only when all observations have the same value. Otherwise, s > 0.

• s, like the mean , is not resistant to extreme values. A few outliers can make s very large.

week2 27

Ballpark approximation for s

• The ballpark approximation for the standard deviation s is the Range/4 (divide by 3 if there are less then 10 observations, divide by 5 if there are more then 100 observations).

• For the data set 4, 8, 2, 9, 7, range = 9 – 2 = 7 and so .7 2.333s≈ =

week2 28

The empirical (68-95-99.7) rule

• With a bell shaped distribution,about 68% of the data fall within a distance of 1 standarddeviation from the mean.95% fall within 2 standard deviations of the mean.99.7% fall within 3 standard deviations of the mean.

• What if the distribution is not bell-shaped?There is another rule, named Chebyshev's Rule, that tells us that there must be at least 75% of the data within 2 standard deviations of the mean, regardless of the shape, and at least 89% within 3 standard deviations.

week2 29

Outliers

An outlier is an observation that is usually large or small relative to the other values in a data set. Outliers are typically attributable to one of the following causes:

1. The observation is observed, recorded, or entered incorrectly. 2. The observation comes from a different population.3. The observation is correct but represents a rare event.

week2 30

The 1.5×IQR Criterion for outliers

• Call an observation a suspected outlier if it falls more than 1.5×IQR above the 3rd quartile or below the 1st quartile.

• Example Consider the data given in exercise 1.13 on page 43 in IPS (mileage data with an extra observation of 66).

Variable N Mean Min Q1 Median Q3 MaxMileages 21 24.67 13 18 23 28 66

The IQR = 28-18 = 10 and the largest observation, 66, falls more than 1.5×IQR above Q3 and therefore is an outlier.

week2 31

Choosing a summary

• The five-number summary is usually better than the mean and the standard deviation for describing skewed distributions or distributions with strong outliers.

• Use mean and standard deviation for reasonably symmetric distributions that are free of outliers.

week2 32

Questions

1. How do the mean, median, and mode compare, usually, when a distribution is positively skewed? negatively skewed? Draw a picture and try to estimate the locations of these measures.

2. Which type of display is the most useful type for clear direct comparisons of the key characteristics of several data sets (e.g. blood cholesterol changes for several different treatments) ?

3. In a frequency table of 300 scores, the mean is reported as 80 and the median as 65. One would expect this distribution to bea. positively skewed. b. negatively skewed. c. symmetrical d. rectangular.

week2 33

4. Find the median of the following frequency distribution.

5. On sta220 term test, John scored at the 78th percentile, and Jack scored at the63rd. State whether the following statements are true of false

a. John is 15 times better than jack.b. John scored 15 more points than Jack.c. 15% of those taking the test got scores ranging between

John's and Jack's scores. d. 62 students scored less than John.

Score Frequency1 22 93 54 35 1

week2 34

6. Estimate the mean and standard deviation of the distribution represented by the following histogram.

1 3 5 7 9 11 13 15

0

5

10

Rate

Freq

uenc

y