12

Click here to load reader

Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Embed Size (px)

Citation preview

Page 1: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Impact Assessment Strategies: Key Challenges

IPTS Workshop “The Paradoxes of ICTs

and Social Inclusion: Do ICTs

Increase Opportunities for Young People at Risk?”

Alexandra Haché

External expertBruxelles-

November

2010

Page 2: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Contents

* Aims study

* Methodological approach

* Facts and figures

* Key findings & main challenges

Page 3: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Aims of

the

study

Young people:

Access, uses and

aspirations

YAR:Access, uses &

aspirations+ Using

ICT

To

support

them

ICTIN: What

works?

+For

whom? +Under

which

circunstance s?

Monitoring and

IA methodologies

Page 4: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Aims of

the

study

* Better understand which are the profiles of YAR and risk scenarios putting them at risk

* Improvement of the understanding of policies and

stakeholders targeting YAR

* Providing a systematic map of current ICT driven

initiatives

* Assessing the success of ICT driven initiatives in

reengaging youth at risk of social exclusion

* Understanding the factors explaining their impact

Page 5: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Boundaries and

definitions

* Young people: “the passage from a dependant childhood to independent adulthood”

(EC Communication "Youth

Investing and Empowering)

* E-inclusion

-

‘access’

(to ICT infrastructure and tools); ‘usability’

(acquiring the digital

literacy skills to use ICTs); ‘quality of use’ (opening up opportunities for life chances)

* ICTIN –

i) using ICT in a primary role to support YP inclusion ii) using ICT in a secondary role to support YP inclusion iii) using ICT to engage YP in the knowledge society

* ICTs

not merely technological artefacts but ‘cultural practices’

(Jenkins, 2006)

Page 6: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Methodological approach

1

* Theory of change: Explores the explicit or implicit expected changes behind a policy, programme or initiative (causal effects)* Cultural

Logic

Analysis: Explores

‘vision’ of policies, programmes and

initiatives (aspects of innovation)

* Triangulation: Evidence-based Synthesis

Page 7: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Methodological approach

2

* Scoping and data audit

* Conceptual overview on youth at risk and

ICT

* Policies Review

* Database/survey of 61 current ICT-

based initiatives

* Impact assessment of ICT based

initiatives

Page 8: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Facts & figures ICT-based

initiatives for

YAR

* 250 initiatives identified (EU level, UK, Spain, the Netherlands)+ 61 surveyed (http://www.delicious.com/ICT_YAR/)* Key stakeholders: private sector, EU agencies, regional and local government and TSO. Most initiatives have multi-stakeholder involvement* Dimensions are structural (management structure; funding base; target groups; size of user base); inclusion focus (objectives, approaches and activities); ICT focus ( tools and applications deployed) and change orientation (intended outcomes and impacts)

Page 9: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Typologies of

ICTIN for

YAR

Page 10: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Methods and

expected

outcomes

05

101520253035

Web

dat

a an

alys

is

User

sur

veys

Imte

rvie

ws

Perfo

rman

ce s

tats

Utilis

atio

n m

onito

ring

Pre-

test

/pos

t

Obs

erva

tion

%

0

5

10

15

20

25

30

35

Increasetech skills

Increasesocial skills

Increaseactive cit

Reduceisolation

Labourmarket re-insertion

ICT access Dissem

%

05

101520253035

Web

dat

a an

alys

is

User

sur

veys

Imte

rvie

ws

Perfo

rman

ce s

tats

Utilis

atio

n m

onito

ring

Pre-

test

/pos

t

Obs

erva

tion

%

05

101520253035

Web

dat

a an

alys

is

User

sur

veys

Imte

rvie

ws

Perfo

rman

ce s

tats

Utilis

atio

n m

onito

ring

Pre-

test

/pos

t

Obs

erva

tion

%

Page 11: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Challenges of

IA

* The initiatives indicate positive outcomes on engagement in education and training, social interaction, digital literacy, employment and participation in society/local community.

* The most successful initiatives were those focusing on learning as a catalyst for improving social inclusion.

* BUT generally self-reported information + Very little data on actual impacts

* There is a lack of an ‘evaluation culture’

in the field. No systematic reviews or meta-analysis were identified and little is known about ‘what works for whom under which circumstances’.

Page 12: Impact Assessment Strategies: Key Challenges - …is.jrc.ec.europa.eu/pages/EAP/documents/HACHE... · Impact Assessment Strategies: Key Challenges IPTS Workshop “The Paradoxes of

Challenges of

IA

* Methodologies of ICT usage are most effective when they combine ICT tools with appropriate and relevant social inclusion and learning strategies that show a close goodness of fit with the needs and situation of target groups.