1

Introduction to Policy Introduction to Policy ProcessesProcesses

Dan Laitsch

2

Overview (Class meeting 4)Overview (Class meeting 4)

Sign in Agenda

– Cohort break outs– Review last class– Mid term assessment– PBL Groups– Significance [dismiss]– Policy and unifying content– T-tests– PBL groups– Action research– PBL and dismiss

3

Class : Review Class : Review

Stats– Hypothesis testing– Z scores

PBL– Topic determined

Policy– Role Play

4

Cohort Break OutCohort Break Out

Courses and dates (summer session)– EDUC 813: organizational Theory (Drescher)

April24/25, May 8/9, May 22/23, June 6/7, June 19/20, and June 26/27

Summer Institute– EDUC 822: Evaluation of Educational Programs

July 2, 3, 6, 7, 8, 9, 10, 13, 14, 15, 16. (Mornings 8:30 to 1:30 or Evenings 4:30 to 9:30). SI public lecture times included as part of class hours (July 6, Evening; 7,9,14 and 16, 1:00 pm to 3:00 pm).

Action Research Time Frame Comprehensive exams

5

Midterm AssessmentMidterm Assessment

Data drive decision making– What do the following data “tell” you?– What questions do they leave unanswered?

Analysis and response

6

7

8

ResponseResponse

Heavy workload– Addressing past student concerns

– Creating balance

– Unifying vision Possible solutions

– Goals: meet course description (policy processes)

– Prepare students for Action Research

– Continued tomorrow

9

PBL groupsPBL groups

Touch base Status check

– Group functioning? Forming, storming, norming, performing?

– Topic identified?– Action plan?– Turn in report (handout)

Plan for tomorrow– 2-3 hours of group time (2 break out 1 hour to 1.5

hours each)

10

Part IV: Significantly DifferentPart IV: Significantly DifferentUsing Inferential StatisticsUsing Inferential Statistics

Chapter 9 Significantly SignificantWhat it Means for You and Me

11

What you learned in Chapter 9What you learned in Chapter 9

What significance is and why it is important– Significance vs. Meaningfulness

Type I ErrorType II ErrorHow inferential statistics worksHow to determine the right statistical test

for your purposes

12

The Concept of SignificanceThe Concept of Significance

Any difference between groups that is due to a systematic influence rather than chance– Must assume that all other factors that might

contribute to differences are controlled

13

If Only We Were Perfect…If Only We Were Perfect…

Significance level – The risk associated with not being 100% positive that what

occurred in the experiment is a result of what you did or what is being tested

The goal is to eliminate competing reasons for differences as much as possible.

Statistical Significance– The degree of risk you are willing to take that you will

reject a null hypothesis when it is actually true.

14

The World’s Most Important The World’s Most Important TableTable

15

Type I Errors Type I Errors (Level of Significance)(Level of Significance)

The probability of rejecting a null hypothesis when it is true

Conventional levels are set between .01 and .05

Usually represented in a report as

p < .05

16

Type II ErrorsType II Errors

The probability of rejecting a null hypothesis when it is false

As your sample characteristics become closer to the population, the probability that you will accept a false null hypothesis decreases

17

Significance Versus Significance Versus MeaningfulnessMeaningfulness A study can be statistically significant but not

very meaningful Statistical significance can only be interpreted

for the context in which it occurred Statistical significance should not be the only

goal of scientific research

– Significance is influenced by sample size…we’ll talk more about this later.

18

How Inference WorksHow Inference Works

A representative sample of the population is chosen.

A test is given, means are computed and compared

A conclusion is reached as to whether the scores are statistically significant

Based on the results of the sample, an inference is made about the population.

19

Deciding What Test to UseDeciding What Test to Use

20

Test of SignificanceTest of Significance1. A statement of null hypothesis.2. Set the level of risk associated with the null hypothesis.3. Select the appropriate test statistic.4. Compute the test statistic (obtained) value5. Determine the value needed to reject the null hypothesis

using appropriate table of critical values6. Compare the obtained value to the critical value7. If obtained value is more extreme, reject null hypothesis8. If obtained value is not more extreme, accept null

hypothesis

21

The Picture Worth a Thousand The Picture Worth a Thousand WordsWords

22

Glossary Terms to KnowGlossary Terms to Know

Significance levelStatistical significanceType I errorType II errorObtained value

– Test statistic valueCritical value

23

End of ClassEnd of Class

PBL Work if time allowsClarifying grades

Journal, portfolio, stats notebook

Homework:– Thinking about research

What areas are you thinking about?What questions do you have?Prepare to chat with colleagues tomorrow

24

AgendaAgenda

– Policy and unifying content– T-tests– PBL groups– Action research– PBL and dismiss

25

Unifying themesUnifying themes

Diffusion models– Communication networks

Diffusion of innovation Adoption

– Internal (policy window)• Severity (crisis)• Opportunity

– External (policy borrowing)• National• Regional• Leader-Laggard• Isomorphism (similar states)• Vertical

Unifying themes

Internal (policy window)– Severity (crisis)

– Opportunity Evidence/Data (my insert)

– Research

– Statistics External (policy borrowing)

– Governments (CMEC)

– Organizations (CTF, JCSH, CERC-CA)

Problem

Solution

Policy Study

Unifying themes

Problems– Identification (what is the problem)– Analysis (what is the cause)

Solutions– Research (what has been done)– Context (how does it fit here)

Policies– Action (what are the rules and procedures)

Evaluation– Analysis (what happened)– Refinement (what might we change)

PBL

Research ReviewsAction Research

Policy Analysis

Unifying themes

Leadership– Identifying context (observation and data gathering)

Data gathering and synthesis (problem identification) Identifying parameters (policy analysis)

– Setting direction (goals and outcomes) Research (identify interventions) Policy (identify rules and procedures for action) Analysis (identify consequences)

– Achieving Goals (problem solving) Implementation of actions and activities Application of rules and procedures (policy)

– Evaluation (refining context)

Part IV: Significantly DifferentUsing Inferential Statistics

Chapter 10 t(ea) for Two

Tests Between the Means of Different Groups

What you learned in Chapter 10

When to use a t testHow to compute the observed t valueInterpreting the t value and what it means

t Tests for Independent Samples

Determining the correct statistic

Computing the Test Statistic

Numerator is the difference between the means

Denominator is the amount of variation within and between each of the two groups

Degrees of Freedom

Degrees of freedom approximate the sample size

Degrees of freedom can vary based on the test statistic selected

For this procedure…n1 – 1 + n2 – 1

So How Do I Interpret…

t (58) = -.14, p > .05– t represents the test statistic used– 58 is the number of degrees of freedom– -.14 is the obtained value (from the formula)

–p > .05 indicates the probability (n.s.)p = n.s.

–p < .05 indicates the probability (sig.)

Special Effects…

Effect size is a measure of how different two groups are from one another

Standardized difference between to group means

Jacob Cohen

Computing Effect Size

Small = 0.0 - .20 Medium = .20 - .50 Large = .50 and above

1 2 ,X X

ESSD

−=

Effect Size Calculator

http://web.uccs.edu/lbecker/Psy590/escalc3.htm

Glossary Terms to Know

Degrees of freedom t Test

– Independent t Test– Obtained value– Critical value

Effect size

Part IV: Significantly DifferentUsing Inferential Statistics

Chapter 11 t(ea) for Two (Again)

Tests Between the Means of Related Groups

What you learned in Chapter 11

When to use a t test for dependent meansHow to compute the observed t valueInterpreting the t value and what it means

t Tests for Dependent SamplesDetermining the correct statistic

Computing the Test Statistic

Numerator reflects the sum of the differences between two groups

Degrees of Freedom

Degrees of freedom approximate the sample size

Degrees of freedom can vary based on the test statistic selected

For this procedure…– n – 1 (where n is the number of observations)

So How Do I Interpret…

t (24) = 2.45, p > .05

– t represents the test statistic used

– 24 is the number of degrees of freedom– 2.45 is the obtained value (from the formula)

–p > .05 indicates the probability (n.s.)p = n.s.

–p < .05 indicates the probability (sig.)

45

PBL GroupsPBL Groups

Break into groupsLunch

46

Action ResearchAction Research

Pair share

Model and paper process– Observations– Questions– Data– Methods– Analysis

Discuss

47

PBL GroupsPBL Groups

PBL Work if time allows