Upload
eric-ellis
View
223
Download
0
Tags:
Embed Size (px)
Citation preview
Looking at data: distributions- Displaying distributions with graphsIPS section 1.1
© 2006 W.H. Freeman and Company (authored by Brigitte Baldi, University of California-Irvine; adapted by Jim Brumbaugh-Smith, Manchester College)
ObjectivesDisplaying distributions with graphs
Recognize numerical vs. categorical data
Construct graphs representing a distribution of numerical data Histograms
Stemplots
Describe overall patterns in numerical data
Identify exceptions to overall patterns
Discuss pros and cons of histograms vs. stemplots
Terminology Individual (or “observation”)
Variable numerical (or “quantitative”)
categorical (or “qualitative)
Value
Frequency absolute
relative
Frequency table
Distribution
Terminology (cont’d) Graphs for quantitative data
Histogram
Stemplot (or “stem-and-leaf diagram”)
Boxplot (section 1.2)
Symmetric
Skewed right (or “positively skewed”)
Skewed left (or “negatively skewed”)
Peak (or “mode”)
Unimodal vs. Bimodal
Outlier
Variables
In a study, we collect data from individuals, more formally known as
observations. Observations can be people, animals, plants, or any
object or process of interest.
A variable is a characteristic that varies among individuals in a
population or in a sample (a subset of a population).
Example: age, height, blood pressure, ethnicity, leaf length, first language
The distribution of a variable tells us what values the variable takes
and how often it takes on these values. A distribution can also be
thought of as the pattern of variation seen in the data.
Two types of variables Variables can be either numerical (a/k/a quantitative) …
Something that can be counted or measured for each individual and then
added, subtracted, averaged, etc. across individuals in the population.
Example: How tall you are, your age, your blood cholesterol level, the
number of credit cards you own
… or categorical (a/k/a qualitative).
Something that falls into one of several categories. What can be
computed is the count or proportion of individuals in each category.
Example: Your blood type (A, B, AB, O), your hair color, your ethnicity,
whether you paid income tax last tax year or not
How do you know if a variable is categorical or quantitative?Ask: What are the individuals in the sample? What is being recorded about those individuals? Is that a number (“quantitative”) or a statement (“categorical”)?
Individualsin sample
DIAGNOSIS AGE AT DEATH
Patient A Heart disease 56
Patient B Stroke 70
Patient C Stroke 75
Patient D Lung cancer 60
Patient E Heart disease 80
Patient F Accident 73
Patient G Diabetes 69
QuantitativeEach individual is
attributed a numerical value.
CategoricalEach individual is assigned to one of several categories.
Ways to chart categorical dataBecause the variable is categorical, the data in the graph can be ordered any way we want (alphabetical, by increasing value, by year, by personal preference, etc.)
Bar graphsEach category isrepresented by a bar.
Pie chartsPeculiarity: The slices must represent the parts of one whole.
Example: Top 10 causes of death in the United States 2001
Rank Causes of death Counts% of top
10s% of total
deaths
1 Heart disease 700,142 37% 29%
2 Cancer 553,768 29% 23%
3 Cerebrovascular 163,538 9% 7%
4 Chronic respiratory 123,013 6% 5%
5 Accidents 101,537 5% 4%
6 Diabetes mellitus 71,372 4% 3%
7 Flu and pneumonia 62,034 3% 3%
8 Alzheimer’s disease 53,852 3% 2%
9 Kidney disorders 39,480 2% 2%
10 Septicemia 32,238 2% 1%
All other causes 629,967 26%
For each individual who died in the United States in 2001, we record what was
the cause of death. The table above is a summary of that information.
Top 10 causes of deaths in the United States 2001
Bar graphsEach category is represented by one bar. The bar’s height shows the count (or
sometimes the percentage) for that particular category.
The number of individuals who died of an accident in 2001 is
approximately 100,000.
0100200300400500600700800
Counts
(x1000)
Bar graph sorted by rank Easy to analyze
Top 10 causes of deaths in the United States 2001
0100200300400500600700800
Cou
nts
(x10
00)
Sorted alphabetically Much less useful
Percent of people dying fromtop 10 causes of death in the United States in 2000
Pie chartsEach slice represents a piece of one whole. The size of a slice depends on what
percent of the whole this category represents.
Percent of deaths from top 10 causes
Percent of deaths from
all causes
Make sure your labels match
the data.
Make sure all percents
add up to 100.
Child poverty before and after government intervention—UNICEF, 1996
What does this chart tell you?
•The United States has the highest rate of child
poverty among developed nations (22% of under 18).
•Its government does the least—through taxes and
subsidies—to remedy the problem (size of orange
bars and percent difference between orange/blue
bars).
Could you transform this bar graph to fit in 1 pie chart? In two pie charts? Why?
The poverty line is defined as 50% of national median income.
Histograms
Vertical bar chart where horizontal axis is a numerical scale corresponding to the data. Vertical axis represents frequency (how many) or relative frequency (what proportion) “Peaks” correspond to commonly occurring data values. “Valleys” and “tails” correspond to values which do not occur as frequently.
Most states have between 0 and 10 percent Hispanic residents
A very small number have between 25 and 45 percent.
Histograms
The range of values that a variable can take is divided into equal size intervals.
The histogram shows the number (i.e., frequency) of individual data points that fall in each interval.
The first column represents all states with a percent Hispanic in their
population between 0% and 4.99%. The height of the column shows how
many states (27) have the percent Hispanic residents in this range.
The last column represents all states with a percent Hispanic between 40%
and 44.99%. There is only one such state: New Mexico, at 42.1% Hispanics.
Creating a histogram
What “class size” should you use?
Use an appropriate number of classes − usually between 5 and 15 work well, depending on number of data values being represented.
Either too few or too many classes will obscure the pattern in the data.
Not so detailed that it is no longer summary
Avoid using many classes having frequency of only 0 or 1
Not overly summarized so that you lose all the information
rule of thumb: start with 5 to 10 classes
Look at the distribution and refine your classes.
(There isn’t a unique or “perfect” histogram.)
Guidelines for histograms
Label the horizontal axis with a consistent numerical scale. (Don’t leave any gaps in the scale or compress the scale toward the left or right side of the graph)
Use vertical bars of equal width.
Label the horizontal scale on the class boundaries.
An exception is when each bar corresponds to a single whole number. Then it is reasonable to label each bar at its midpoint.
Not summarized enough
Too summarized
Same data set
Interpreting histograms
When describing the distribution of a quantitative variable, we look for the
overall pattern and for striking deviations from that pattern. We can describe the
overall pattern of a histogram by its shape, center, and spread.
Histogram with a line connecting
each column too detailed
Histogram with a smoothed curve
highlighting the overall pattern of
the distribution
Most common distribution shapes
A distribution is symmetric if the right and left sides
of the histogram are approximately mirror images of
each other.
Symmetric distribution
Complex, multimodal distribution
Not all distributions have a simple overall shape,
especially when there are few observations.
Skewed right distribution
A distribution is skewed to the right if the right
side of the histogram (side with larger values)
extends much farther out than the left side. It is
skewed to the left if the left side of the extends
much farther out than the right.
IMPORTANT NOTE:
Your data are the way they are.
Do not try to force them into a
particular shape.
It is a common misconception
that if you have a large enough
data set, the data will eventually
turn out nice and symmetrical.
Histogram of Drydays in 1995
Alaska Florida
Outliers
An important kind of deviation is an outlier. Outliers are observations
that lie outside the overall pattern of a distribution. Always look for
outliers and try to explain them.
The overall pattern is fairly
symmetrical except for two
states clearly not belonging
to the main trend.
Alaska and Florida have
unusual representation of
the elderly in their
population.
A large gap in the
distribution is typically a
sign of an outlier.
More on Outliers
If outlier is incorrect data (e.g., data entry error, false response) Correct it if possible. Discard if absolutely sure it is wrong. Discarding data might introduce a bias if uncorrectable errors
tend to be mainly high (or mainly low) values.
If outlier is correct data consider effect on analysis Which statistical technique is most appropriate? How are conclusions affected by the outlier?
Some Other Strategies: Refine population definition if unusual responses are not part
of intended study group. If sample size is quite small collect more data to see if any
gaps fill in.
Stemplots (or “stem-and-leaf” diagrams)How to make a stemplot:
1) Truncate (or “trim”) the data to appropriate level of
accuracy.
2) Separate each observation into a stem, consisting of
all but the final (rightmost) digit, and a leaf, which is
the remaining final digit. Stems may have as many
digits as needed, but each leaf contains a single digit.
3) Write the stems in a column with the smallest value
at the top, and draw a vertical line at the right of this
column.
4) Write each leaf in the row to the right of its stem, in
increasing order out from the stem. Leaves should
be aligned in vertical columns. (Why?)
STEM LEAVES
State PercentAlabama 1.5Alaska 4.1Arizona 25.3Arkansas 2.8California 32.4Colorado 17.1Connecticut 9.4Delaware 4.8Florida 16.8Georgia 5.3Hawaii 7.2Idaho 7.9Illinois 10.7Indiana 3.5Iowa 2.8Kansas 7Kentucky 1.5Louisiana 2.4Maine 0.7Maryland 4.3Massachusetts 6.8Michigan 3.3Minnesota 2.9Mississippi 1.3Missouri 2.1Montana 2Nebraska 5.5Nevada 19.7NewHampshire 1.7NewJ ersey 13.3NewMexico 42.1NewYork 15.1NorthCarolina 4.7NorthDakota 1.2Ohio 1.9Oklahoma 5.2Oregon 8Pennsylvania 3.2RhodeIsland 8.7SouthCarolina 2.4SouthDakota 1.4Tennessee 2Texas 32Utah 9Vermont 0.9Virginia 4.7Washington 7.2WestVirginia 0.7Wisconsin 3.6Wyoming 6.4
Percent of Hispanic residents
in each of the 50 states
Step 2:
Assign the values to
stems and leaves
Step 1:
Sort the data
State PercentMaine 0.7WestVirginia 0.7Vermont 0.9NorthDakota 1.2Mississippi 1.3SouthDakota 1.4Alabama 1.5Kentucky 1.5NewHampshire 1.7Ohio 1.9Montana 2Tennessee 2Missouri 2.1Louisiana 2.4SouthCarolina 2.4Arkansas 2.8Iowa 2.8Minnesota 2.9Pennsylvania 3.2Michigan 3.3Indiana 3.5Wisconsin 3.6Alaska 4.1Maryland 4.3NorthCarolina 4.7Virginia 4.7Delaware 4.8Oklahoma 5.2Georgia 5.3Nebraska 5.5Wyoming 6.4Massachusetts 6.8Kansas 7Hawaii 7.2Washington 7.2Idaho 7.9Oregon 8RhodeIsland 8.7Utah 9Connecticut 9.4Illinois 10.7NewJ ersey 13.3NewYork 15.1Florida 16.8Colorado 17.1Nevada 19.7Arizona 25.3Texas 32California 32.4NewMexico 42.1
Stemplots are quick and dirty histograms that can easily be done by
hand, therefore very convenient for back of the envelope calculations.
However, they are rarely found in scientific publications.
Stemplots versus histograms
1) Advantages of Stemplots
Quick to do by hand (no frequency table needed)
Maintains all the numerical data
Good for comparing two distributions (using back-to-back plots)
2) Disadvantages
Not as crisp visually (compared to a graphics-quality histogram)
Most people like numerical scale on horizontal axis.
Stemplots versus histograms
1) Stem-splitting
Use to increase the number of stems
Create one stem for leaves 0-4
Second stem for leaves 5-9
2) Back-to-back Stemplots
Use to compare two data sets measured on same scale (e.g., male vs.
female heights).
Use single stem with leaves on opposite sides for different data sets.
Variations on stemplots
Ways to chart quantitative data
Line graphs: time plots
Use when there is a meaningful sequence, like time. The line connecting the
points helps emphasize any change over time.
Histograms and stemplots
These are summary graphs for a single variable. They are very useful to
understand the pattern of variability in the data.
Other graphs to reflect numerical summaries (see Chapter 1.2)
Line graphs: time plots
A trend is a rise or fall that persist over time, despite small irregularities.
In a time plot, time always goes on the horizontal, x axis.
We describe time series by looking for an overall pattern and for striking
deviations from that pattern. In a time series:
A pattern that repeats itself at regular intervals of time is
called seasonal variation.
Retail price of fresh oranges over time
This time plot shows a regular pattern of yearly variations. These are seasonal
variations in fresh orange pricing most likely due to similar seasonal variations in
the production of fresh oranges.
There is also an overall upward trend in pricing over time. It could simply be
reflecting inflation trends or a more fundamental change in this industry.
Time is on the horizontal, x axis.
The variable of interest—here
“retail price of fresh oranges”—
goes on the vertical, y axis.
1918 influenza epidemicDate # Cases # Deaths
week 1 36 0week 2 531 0week 3 4233 130week 4 8682 552week 5 7164 738week 6 2229 414week 7 600 198week 8 164 90week 9 57 56week 10 722 50week 11 1517 71week 12 1828 137week 13 1539 178week 14 2416 194week 15 3148 290week 16 3465 310week 17 1440 149
0100020003000400050006000700080009000
10000
0100200300400500600700800
0100020003000400050006000700080009000
10000
# c
ase
s d
iag
no
sed
0
100
200
300
400
500
600
700
800
# d
ea
ths
rep
ort
ed
# Cases # Deaths
A time plot can be used to compare two or more
data sets covering the same time period.
The pattern over time for the number of flu diagnoses closely resembles that for the
number of deaths from the flu, indicating that about 8% to 10% of the people
diagnosed that year died shortly afterward from complications of the flu.
Death rates from cancer (US, 1945-95)
0
50
100
150
200
250
1940 1950 1960 1970 1980 1990 2000
Years
Death
rate
(per
thousand)
Death rates from cancer (US, 1945-95)
0
50
100
150
200
250
1940 1960 1980 2000
Years
Dea
th r
ate
(per
thou
sand
)
Death rates from cancer (US, 1945-95)
0
50
100
150
200
250
1940 1960 1980 2000
Years
Death
rate
(per
thousand)
A picture is worth a thousand words,
BUT
There is nothing like hard numbers.
Look at the scales.
Scales matterHow you stretch the axes and choose your scales can give a different impression.
Death rates from cancer (US, 1945-95)
120
140
160
180
200
220
1940 1960 1980 2000
Years
Death
rate
(pe
r th
ousan
d)
Why does it matter?
What's wrong with these graphs?
Careful reading reveals that:
1. The ranking graph covers an 11-year period, the tuition graph 35 years, yet they are shown comparatively on the cover and without a horizontal time scale.
2. Ranking and tuition have very different units, yet both graphs are placed on the same page without a vertical axis to show the units.
3. The impression of a recent sharp “drop” in the ranking graph actually shows that Cornell’s rank has IMPROVED from 15th to 6th ...
Cornell’s tuition over time
Cornell’s ranking over time