Overview: Evaluation in Out of School Arts Programs

8/9/2019 Overview: Evaluation in Out of School Arts Programs

1/11

BYAEP : BRIEF OVERVIEWBoston Youth Arts Evaluation Projec t

1

Thursday, June 3, 2010


2/11

Introduction to BYAEP

2

The challenge for the arts eld is that we may not have caused the evaluation and accountability problem, but we can and must nd our own solution...what is missing from the growing body of research,

however, is a commitment on the part of the arts eld to gathering information on an individual basis and

sharing it with others.--Suzanne Callahan, Singing Our Praises: Case Studies in the Art of Evaluation, 2004



3/11

Why is it so hard to show what we know?

Are the arts tongue tied? Why cant we as artists use the arts to make our case?--former Chairman of the NEA, John Frohnmayer



4/11

Measure what you valueand others will value

what you measure.

First step in evaluation:Figure out your own sacred bundle.

**Evaluation and the Sacred Bundle by John Bare, Summer 2005 issue of the Evaluation Exchange

4



5/11

YEAR ONE: OUTCOME 1

A youth art evaluation language and methodology is developed, based on the existing research andliterature, and the experiences of Boston youth art programs: Hyde Square Task Force, ZUMIX, MedicineWheel, and The Theater Offensive.

YEAR OF RESEARCH : Lots of reading, talking, meeting, and wrestling with models, tools, and words.

YEAR TWO: OUTCOME 2

This methodology is used to design, pilot, and disseminate evaluation systems for RAW and the fourBoston youth arts programs.

YEAR OF USING NEW TOOLS : Implementing a Self-Evaluation, Program Evaluation, ArtisticResponse Evaluation, Teacher Evaluation, and Alumni Evaluation.

YEAR THREE: OUTCOME 3

Programmatic ndings of the project and pilot are documented and shared. A publication is designed tohelp others implement evaluation systems for youth arts programs.

YEAR OF SHARING OUR KNOWLEDGE and TOOLS : Passing onto others what we have learnedand engaging further opportunities for sharing our experience and expertise.

5

BYAEP: BRIEF OVERVIEW



6/11

* Adapted from The National Research Council and Institute of Medicine. (2002). Community Programs to Promote Youth Development .**Ada ted from The Communit Action Framework for Youth Develo ment. 2002 .

BYAEPs Framework for Outcomes in Youth Arts Programs

* Adapted from The National Research Council and Institute of Medicine. (2002). Community Programs to Promote Youth Development .**Adapted from The Community Action Framework for Youth Development, (2002)

I CREATE:

Builds Artistic,Problem Solving, andExpressive Skills

Youth develop skills inartistic engagement,problem solving, criticalthinking, and expression.

I AM:StrengthensIdentity

Youth developcondence,knowledge of self ! an informed culturalidentity, and apositive view oftheir future.

WE CONNECT

:

Develops Community Youth develop meaningful relationshipsand civic engagement as they contributeto and are recognized by an inclusivecommunity.

Quality Youth Arts Programs provideOpportunities, a Positive Climate, andConnections* to create change in thelives of youth.

THEORY OF CHANGEIf youth participate in high-quality arts programs, then theywill develop specic skills andcompetencies (I Create, I Am,

We Connect), which lead to aset of intermediate outcomes**(able to engage and beproductive, to navigate, and tomake connections with others),which in turn lead to a set oflong-term outcomes (resiliency,self-efcacy and personalfulllment, and communityengagement) that togetherconstitute life success.

6



7/11

* Adapted from The National Research Council and Institute of Medicine. (2002). Community Programs to Promote Youth Development .**Adapted from The Community Action Framework for Youth Development. (2002).

BYAEPs Logic Model for High Quality Youth Arts ProgramsINPUTS OFPROGRAM QUALITY*

SHORT TERM OUTCOMES INTERMEDIATEOUTCOMES**

LONG TERM OUTCOMES

Opportunities: Challenges andexperiences thatincrease innovation,

expressive skills,self-efcacy and funin the lives of youth.

I CREATE:Builds Artistic, Problem Solving, and ExpressiveSkills

Increases Artistic Engagement in focus, skills, and in ones

authenticity and passion in the arts. Uses Problem Solving/Critical Thinking to be reective, analyticand creative in nding solutions to challenges.

Develops Expressive Skills and the ability to convey feelings andthoughts artistically and verbally.

Able to Navigate Takes responsibility indiverse settings Navigates risk-taking

Responds effectively tochallenges and opportunities

Resiliency Adapts and learns to thrivewith change, challenges, andeven failure

Is respectful of laws and/orworks to change those thatare unfair Strives to be physically andmentally healthy Engages in positive activitiesthat brings one joy, pleasure,and captivation

Positive Climate:Meaningful structurethat is youth-

centered, safe,inclusive, engaging,and challenges youthto see, reveal, andstrengthen who theyare.

I AM: Strengthens Identity Builds Condence with self-assurance arising from one's belief in

one's own abilities or qualities. Increases Knowledge of Self through: self-awareness ofcharacteristics, strengths, and weaknesses; honest self-reectioninto ones history, cultural inuence, and ones current thoughts,feelings, and actions; and self-regulation of behavior to increase thelikelihood of a desired end goal. Understands how ones identity is informed by Cultural Identity (place, gender, race, history, nationality, abilities, language, sexualorientation, religious beliefs, ethnicity, class and aesthetics). Develops a Positive View of the Future by internalizing optimismabout the outcomes for ones life (possible selves) and increasingones ability to set short and long-range goals (especially ineducation and/or employment).

Able to Engage and beProductive Displays commitment(dedication shown in school/ employment) Strives for achievement(effort, courage, skills inprogress toward goals) Possesses a positive senseof ones own uniqueness andpotential (differentiation) Displays a character that isgenuine, empathic, and isconnect to ones culturalidentity (integration)

Self-Efcacy andPersonal Fulllment Education level Economic self-sufciency Job satisfaction Self-actualization (the desireand efforts that lead torealizing one's capabilities) Continues to take stepstowards dreams

Connections:Opportunities tobelong, contribute,and build supportiverelationships withpeers, adults, andcommunity.

WE CONNECT: Develops Community Increases Support and Belonging where one develops positivebonds, empathy, respect for others, and an increased ability tocommunicate and work with a diverse set of people including thosewith cultural identities and experiences different than ones own. !! Builds Contribution by nding opportunities, exchanging ideas,and working together to create something in the community. Gains Recognition, appreciation and/or acknowledgement for anachievement, service, or ability in the eyes of others/community.

Able to MakeConnections Connects with family Connects with peers/ signicant other(s) Connects with adultmentors Joins groups/organizations Respects and is respectedby others

CommunityEngagement Has dependable networks Has signicant relationship(s) Involves oneself in socialgroups (PTA, unions, supportgroups, religious groups, etc.) Votes, volunteers, works tocreate social change

S h o r t T e r m

O u t c o m e s c o m b

i n e a n d l e a d t o I n t e r m e d i a t e a n d L o n g T e r m

O u t c o m e s

7



8/11

Year 2: Tools for 2009-2010BYAEP Pilot in 5 sites

8



9/11

5 BYAEP PILOT EVALUATIONS 09-10! Self Evaluation: Beginning

and Final -Youth evaluate self inOctober and in May.

! Program Evaluation - Youthevaluate the programs in which they areinvolved in May/June. !

! Artistic Response -Youth respondin a drawing to what they feel haschanged due to their experience here. !

! Teacher Evaluation - Teachersevaluate the youth in November and May/ June.

! Alumni Evaluation - Graduates llout an Alumni Survey from 1 to over 15years after graduation to assess ourintermediate and long-term outcomes.

9



10/11

So What? : BYAEPs NEW INDICATORS--21st Century Workforce SkillsTeens were asked to mark the 21st Century Workforce Skills that they have now andto pick one they really would like to improve in. In the Final Self Eval we will ask which skillsthe organization has helped them imporove in. Below is an example of ZUMIX. Sample sizeincluded 47 youth.

10



11/11

WE CONNECT: Develops CommunityDRAWING EVALUATIONS: Example from RAW

Increases Support and Belonging !!

Before coming to RAW I never went out of my circle of

friends, now I am more open to different types of people. -- C.C., age 16

Before I used to keep all to myself, like my problems and worries. I thought people would

judge me and label me weak, but now I know itsnot always like that. THANK YOU. -- S.K., age 17

Thursday June 3 2010

Documents

Overview: Evaluation in Out of School Arts Programs