35
Gained in Translation: An International Look at Behavior Beth Karlin, Ph.D. [email protected] 1

Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

Gained in Translation:An International Look at Behavior

Beth Karlin, Ph.D.

[email protected]

Page 2: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

Our Team

2

Dr. Sea Rotmann

Operating Agent

Kira Ashby

Program ManagerDr. Beth Karlin

National Expert

Page 3: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

3

Agenda

➢ What is Behavior?

➢ About IEA-DSM Task 24

➢ Lessons Learned

Page 4: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

4

Page 5: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:
Page 6: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

We’re making a lot of progress in energy efficiency

Page 7: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

…will they come?

But . . . if you build it,

Page 8: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:
Page 9: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

What is behavior?

Program

Process

Everything

➢ The response of an individual,

group, or species to its environment

➢ The way in which something

functions or operates

Page 10: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

10

What is behavior?

1. Comparative Usage2. Randomized Control Trials3. Ex Post measurement

Page 11: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

11

Behavior

HERs

Ignelzi et al. (2013). Paving the Way for a Richer Mix of

Residential Behavior Programs.Prepared for the

California Investor-Owned Utilities.

Social Norms

Feedback

Framing

Follow

Through

Rewards or

Gifts

Commitment In Person

Interaction

Page 12: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

About IEA-DSM Task 24: Behavior

DSM

Task 22: EE Portfolio

Standards

Task 23: Customers

and the Smart Grid

Task 24: Behavior

Change in DSM

Task 25: Business

Models for EE Market

Uptake

Energy Storage

Smart Grid

Solar

Wind

38

30

Page 13: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

About IEA-DSM Task 24: Behavior

Why?Subtask 6

‘The Issues’

Who?Subtask 7

‘The People’

How?Subtask 8

‘The Tools’

What?Subtask 9

‘The Measure’

So what?Subtask 10

‘The Story’

Subtask 1

Subtask 2

Subtask 4 Subtask 5

Subtask 1

Subtask 4 Subtask 3

Subtask 11 – Real-life pilots13

Page 14: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

About IEA-DSM Task 24: Behavior

Formal Participants• Austria

• Belgium

• Ireland

• Italy

• New Zealand

• The Netherlands

• Norway

• Sweden

• SwitzerlandIn-Kind Contributors• Australia

• Japan

• Portugal

• South Africa

• United Kingdom

• United States

Page 15: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

The Opportunity

CEE was invited to partner with the U.S. Department of Energy to

represent US and Canadian program administrators in IEA DSM Task 24.

• Leverages collaborative approach for unique and high impact work product

• Untapped behavioral techniques from other countries’ programs

• Rigorous evaluation findings from abroad as evidence of behavior

credibility for US regulators

Page 16: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

The Work

• Two domestic behavioral workshops

• Program resources

• US-specific project

• Workshop 1: overview of other countries’ programs, discuss focus area

• Workshop 2: synthesize findings, develop resources

• Toolbox of interventions

• Evaluation framework

Page 17: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

Workshops

• Workshop 1: overview of other countries’ programs, discuss focus area

April 11th in San Francisco

Page 18: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

Resources

Page 19: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

Project

For more information, visit www.ieadsm.org

Challenges Driving Focus

• Cost-effectiveness

• Scrutiny of evaluation methodologies

• Savings data (measurement, privacy, associated costs)

• Regulatory acceptance

• Need to move beyond HERs Identified Focal Areas

● Evaluation methodologies

● NEBs

● Methods, credibility,

persistence

● Survey questions/design

● Connected Home

● SMBs

Page 20: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

Project

● Semi-structured interviews with CEE member sponsors

● Outreach to other global behavioral evaluation experts

● Synthesis of recent behavioral evaluation findings:

⇒ Database of primary literature

⇒ Database of CEE program inventory

⇒ Database of Task 24 case studies

● Dissemination of learnings

⇒ CEE Workshop 2: October 10-11, Washington DC

⇒ Final report and policy brief on IEA DSM website (end 2018)

Page 21: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

21

There are building blocks to behavior change.

Following a process helps.

You may need new capacities.

1

2

3

Three Lessons Learned

Page 22: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

22

Behavior

HERs

Ignelzi et al. (2013). Paving the Way for a Richer Mix of

Residential Behavior Programs.Prepared for the

California Investor-Owned Utilities.

Social Norms

Feedback

Framing

Follow

Through

Rewards or

Gifts

Commitment In Person

Interaction

Ignelzi et al. (2013). Paving the Way for a Richer Mix of

Residential Behavior Programs.Prepared for the

California Investor-Owned Utilities.

Page 23: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

Framing

Rewards

or Gifts

Feedback

Social

Norms

In Person

Interaction

Commitment

Goal

Setting

Rebates

Schools

And More?

And More?

And More?

And More?

And More?

Energy

Kits

HEMS

Follow

ThroughBehavior

Audits

Smart AC

Karlin et al. (2017). From Categorizing to Characterizing: A Landscape Analysis of Behavior-Based

Energy Programs. San Francisco, CA: Pacific Gas and Electric.

Page 24: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:
Page 25: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

What’s the alternative?

Page 26: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

26

Building Blocks of Behavior Change

Page 27: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

Your

Program

Desired

Result

Does it Work?

Page 28: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

Does it Work?HOW

Ingredients Activities/Processes Metrics

Your

Program

Desired

Result

Page 29: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

A Process for Behavior Change

A L I G N D E F I N E D E S I G N D E P L O Ytarget audience and

behavior goals

ethographic

research

customer

models

stakeholder

analysis

landscape

analysis

stakeholders on “why”

and the ecosystem

user

testing

A/B

testing

and test content and

delivery strategies

process

evaluation

outcome

evaluation

a pilot and evaluate

process and outcomes

Page 30: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

You May Need New Capacities…

Information

Capacity

Staff

Capacity

Collaborative

Capacity

Page 31: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

31

Do we have the right people (with

the right skills) on the team?1

2

3

4

Do we have (or can we get) the

data we need?

Is everybody talking to each other?

Are we leveraging opportunities

at every touch point?

Questions to Ask...

Page 32: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

32

If you walk away with anything,

let it be this…

Page 33: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

There are benefits to simplicity…

33

Page 34: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

… but it’s not always that simple

34

Page 35: Gained in Translation · • Workshop 1: overview of other countries’ programs, discuss focus area ... Ruhi Shaikh -X (ruhshaik - LIONBRIDGE TECHNOLOGIES INC at Cisco) Created Date:

Thank you!

Beth Karlin, Ph.D.

[email protected]