Crafting a Performance Culture:

Anxiety, Climate and School District Reform

Crafting a Performance Culture:

Anxiety, Climate and School District Reform

Katharine Neem Destler

University of Washington

“Improving Education Through Accountability And Evaluation: Lessons From Around the World”

APPAM and INVALSI Rome, Italy

October 3, 2012

Sponsored by the US Department of Education, Institute of Education Sciences (#R305B090012)

Katharine Neem Destler

University of Washington

“Improving Education Through Accountability And Evaluation: Lessons From Around the World”

APPAM and INVALSI Rome, Italy

October 3, 2012

Sponsored by the US Department of Education, Institute of Education Sciences (#R305B090012)

Shifting Tides in Education Reform

Shifting Tides in Education Reform

Standardized Practices--> Site-based control Instructional Practice --> Managerial Behavior Increased Funding --> Accountability for Results

Standardized Practices--> Site-based control Instructional Practice --> Managerial Behavior Increased Funding --> Accountability for Results

England

New Zealand

Victoria, AustraliaU.S. Charter Schools

A turn towards “Portfolio” School District Reform

A turn towards “Portfolio” School District Reform

Parental School Choice Site-Based Decision Making Performance Monitoring Closure of Chronically Underperforming Schools Aggressive Development/Recruitment of New

Schools, including those run by private providers (Bulkey and Henig 2010, Hill et al. 2012)

Parental School Choice Site-Based Decision Making Performance Monitoring Closure of Chronically Underperforming Schools Aggressive Development/Recruitment of New

Schools, including those run by private providers (Bulkey and Henig 2010, Hill et al. 2012)

Moving too quickly?Moving too quickly?

Despite the attention to high stakes accountability reforms, research suggests that the results have been mixed: District-wide improvement Teaching to the test Narrowing of the Curriculum Focus on “Bubble” Kids

(Corcoran and Beverage 2010; Jacob 2005; Kemple 2010)

Despite the attention to high stakes accountability reforms, research suggests that the results have been mixed: District-wide improvement Teaching to the test Narrowing of the Curriculum Focus on “Bubble” Kids

(Corcoran and Beverage 2010; Jacob 2005; Kemple 2010)

Reforms in Education aren’t “new” everywhere

Reforms in Education aren’t “new” everywhere

Performance Management Reform:An ongoing, systematic approach to improving results through evidence-

baseddecision making, continuous organizational learning, and a focus onaccountability for performance”(National Performance Management

Advisory Commission, 2010).

Performance Management Reform:An ongoing, systematic approach to improving results through evidence-

baseddecision making, continuous organizational learning, and a focus onaccountability for performance”(National Performance Management

Advisory Commission, 2010).

Managerial Discretion Publication of Performance Data Accountability for Outcomes

Education reflects findings in other sectors

Education reflects findings in other sectors

Performance management is well-established in other sectors

The provision of data does not ensure that public leaders or policymakers will act on that data

Incentives have perverse, as well as positive, results

(Heinrich and Marschke 2010; Jennings and Haist 2004; Moynihan

2008; Radin 2006)

Performance management is well-established in other sectors

The provision of data does not ensure that public leaders or policymakers will act on that data

Incentives have perverse, as well as positive, results

(Heinrich and Marschke 2010; Jennings and Haist 2004; Moynihan

2008; Radin 2006)

Formal Systems Change is Not Enough…

Formal Systems Change is Not Enough…

Need to understand the conditions under which schools and other organizations experience a change in front line culture.

Under what conditions do front-line workers and managers espouse performance management values and enact performance management values in their daily behavior?

Need to understand the conditions under which schools and other organizations experience a change in front line culture.

Under what conditions do front-line workers and managers espouse performance management values and enact performance management values in their daily behavior?

Understanding Performance Management Behavior

Understanding Performance Management Behavior

Formal Performance Management Reform:

Relevant and Reliable Data

Performance Accountability

Managerial Discretion

Learning Forums

Performance- Oriented

Front-Line Behavior

Understanding Performance Management Behavior

Understanding Performance Management Behavior

Formal Performance Management Reform:

Relevant and Reliable Data

Performance Accountability

Managerial Discretion

Learning Forums

Performance- Oriented

Front-Line Behavior

Performance Management Values

Cultural Characteristics

Organizational Climate

Survival Anxiety

Performance- Oriented

Front-Line Behavior

Two Alternate Paths to Organizational ChangeTwo Alternate Paths to Organizational Change

The “Incentive” Approach Increased performance leads to change in culture and

behavior. “Survival Anxiety” will provoke greatest need for change

(Khademian and Harttman 2010; Kotter 1990; Schein 2006)

The “Internal Dynamics” Approach Not every organization is equally “ready” to change A positive Organizational Climate (trust, support) is an

important prerequisite for cultural shift. (Bryk and Schneider 2005; Childress et al., 2011; Hou, Moynihan, & Ingraham, 2003; Schein 2006)

The “Incentive” Approach Increased performance leads to change in culture and

behavior. “Survival Anxiety” will provoke greatest need for change

(Khademian and Harttman 2010; Kotter 1990; Schein 2006)

The “Internal Dynamics” Approach Not every organization is equally “ready” to change A positive Organizational Climate (trust, support) is an

important prerequisite for cultural shift. (Bryk and Schneider 2005; Childress et al., 2011; Hou, Moynihan, & Ingraham, 2003; Schein 2006)

Research QuestionResearch Question

What is the role of external incentives (in the form of organizational survival anxiety) and organizational climate in the espousal of performance management values and the adoption of performance management values-in-use in schools?

What is the role of external incentives (in the form of organizational survival anxiety) and organizational climate in the espousal of performance management values and the adoption of performance management values-in-use in schools?

Data and MethodsData and Methods

Sequential, Mixed-Method Study Cross-Sectional School System Analysis

NYC Learning Environment Survey, Progress Report, Quality Review 2007-2011

n : 115-1,153 depending on model Analysis conducted using OLS with schools as the unit of

analysis

Comparative Case Study of Four Schools

Sequential, Mixed-Method Study Cross-Sectional School System Analysis

NYC Learning Environment Survey, Progress Report, Quality Review 2007-2011

n : 115-1,153 depending on model Analysis conducted using OLS with schools as the unit of

analysis

Comparative Case Study of Four Schools

New York City: A “Critical Case” for Performance Management Reform in Education

New York City: A “Critical Case” for Performance Management Reform in Education

Mayoral Control, dating from 2005 Abolishment of the internal school district

structure “Empowerment” strategy:

Devolution of decision-making power down to school principals

School-level accountability for performance

Mayoral Control, dating from 2005 Abolishment of the internal school district

structure “Empowerment” strategy:

Devolution of decision-making power down to school principals

School-level accountability for performance

Key VariablesKey VariablesDEPENDENT VARIABLESEspoused Values: explicitly embraced by organizational

members (Argyris and Schon 1974,1977; Schein 2006)

Values-in-Use: tacitly revealed through organizational behavior. (Argyris and Schon 1974,1977; Senge 1990; Schein 2006.

INDEPENDENT VARIABLES OF INTERESTOrganizational Climate: level of organizational trust;

perceptions of support and norms of open/honest dialogue (Childress et al 2011; Moynihan & Pandey, 2010; Schein, 2006; Senge, 1990)

Survival Anxiety: perceived risk of inaction to organizational wellbeing (Kotter, 1996,Schein, 2006)

DEPENDENT VARIABLESEspoused Values: explicitly embraced by organizational

members (Argyris and Schon 1974,1977; Schein 2006)

Values-in-Use: tacitly revealed through organizational behavior. (Argyris and Schon 1974,1977; Senge 1990; Schein 2006.

INDEPENDENT VARIABLES OF INTERESTOrganizational Climate: level of organizational trust;

perceptions of support and norms of open/honest dialogue (Childress et al 2011; Moynihan & Pandey, 2010; Schein, 2006; Senge, 1990)

Survival Anxiety: perceived risk of inaction to organizational wellbeing (Kotter, 1996,Schein, 2006)

Measurement ChallengesMeasurement Challenges

High Correlation between Organizational Climate and Performance Values Construct Validity Common Source Bias

Endogeneity Risk

High Correlation between Organizational Climate and Performance Values Construct Validity Common Source Bias

Endogeneity Risk

High standards for student learning

Satisfaction with governing regime

Focus on innovation/autonomy

Survival Anxiety

Values-in-Use

OrganizationalClimate

Espoused Values

PerformanceCulture

FormalPerformanceManagement

ReformManagerial Discretion

Accountability for Results

Valid/Reliable Data

Learning Forums

Psychological Safety

Norms of Collaborationand Open Dialogue

Perceived School Quality

Parental Engagement

Parental Support for Reform

Organizational strategizingand goal setting

Performance data use

Continuous improvementprocesses

Perceptions of Support

Leadership Behavior

Conceptual Model

Socio-Political Context

HypothesesHypothesesH1: Survival anxiety will have a positive and

statistically-significant effect on the espousal of performance values and the adoption of performance values-in-use.

H2: Organizational Climate will have a positive

and statistically-significant effect on the espousal of performance values and the adoption of performance values-in-use

H1: Survival anxiety will have a positive and statistically-significant effect on the espousal of performance values and the adoption of performance values-in-use.

H2: Organizational Climate will have a positive

and statistically-significant effect on the espousal of performance values and the adoption of performance values-in-use

Findings--Espoused ValuesFindings--Espoused Values

• Organizational climate = strong predictor

• Survival anxiety= weak but significant predictor

• Limited interaction effect

• Few statistically-significant formal controls

n -0.13* (0.08)

-0.33* (0.18)

R2 1,240 300

All

Schools Low 2007

Value Schools

(A) (B)

Organizational Climate 0.80*** (0.02)

0.86*** (0.03)

Survival Anxiety 0.04*** (0.01)

0.07*** (0.03)

Survival Anx*Org Climate 0.05*** (0.01)

0.01 (0.02)

Peer Index 0.01*** (0.00)

<0.001 (0.00)

Part of Empowerment Zone

-0.16*** (0.03)

-0.08 (0.07)

Affiliated with Partner Support Org

0.04 (0.05)

0.22** (0.09)

Proportion of <4 yr teachers

-0.004*** (0.002)

0.002 (0.004)

Proportion of 15+ yr teachers

0.01** (0.00)

0.01** (0.01)

Stable leadership (dummy) 0.003** (0.001)

0.01** (0.002)

Number of full-time teachers

-0.25*** (0.06)

-0.11 (0.13)

Values in 2006-2007 -0.05** (0.02)

-0.02 (0.08)

Intercept 0.00*** (0.001)

0.00** (0.001)

Findings: Values-in-UseFindings: Values-in-Use

• Organizational climate = strong predictor

• Survival anxiety= negative and significant predictor

• No evidence of interaction effect

• Even fewer statistically-significant formal controls

All Schools Schools

with low

values 2007

(A) (B)

Organizational Climate 0.25*** (0.05)

0.35*** (0.11)

Survival Anxiety -0.25*** (0.04)

-0.23** (0.09)

Survival Anx*Org Climate

0.05 (0.04)

-0.03 (0.08)

Stable leadership -0.19* (0.11)

-0.04 (0.23)

High School (dummy) -0.81*** (0.18)

-0.91** (0.38)

Number of full-time teachers

0.18*** (0.05)

0.02 (0.13)

Values in 2006-2007 0.001** (0.002)

0.004 (0.01)

Intercept 0.47** (0.22)

0.41 (0.50)

n 426 108 R2 0.388 0.359

Summary of FindingsSummary of Findings

Internal Dynamics Perspective Strong support for both espoused values and values-in-use

External Dynamics Perspective Weak support for espoused values Contrary findings for values-in-use

Lack of support in formal controls

No evidence that incentives and climate shape behavior together

Internal Dynamics Perspective Strong support for both espoused values and values-in-use

External Dynamics Perspective Weak support for espoused values Contrary findings for values-in-use

Lack of support in formal controls

No evidence that incentives and climate shape behavior together

Why the Limited Impact of Survival Anxiety?

Why the Limited Impact of Survival Anxiety?

Inadequate Time Frame for change High-anxiety lack capacity and control:

Policy Complexity Political Uncertainty

Low-Anxiety schools go beyond incentives Stewards of organizational mission Efforts to define district mandates in line with

organizational mission

Inadequate Time Frame for change High-anxiety lack capacity and control:

Policy Complexity Political Uncertainty

Low-Anxiety schools go beyond incentives Stewards of organizational mission Efforts to define district mandates in line with

organizational mission

ConclusionConclusion Performance incentives are, on their own,

insufficient to change organizational behavior. Organizational change--even the development of

performance behaviors-- takes time. Policymakers and school leaders need to focus

less on perfecting accountability instruments and more on how to create the internal organizational conditions for change.

Performance incentives are, on their own, insufficient to change organizational behavior.

Organizational change--even the development of performance behaviors-- takes time.

Policymakers and school leaders need to focus less on perfecting accountability instruments and more on how to create the internal organizational conditions for change.