1

Evaluating Success: HIVE Playable Technology ProjectJanuary 13, 2016

Our Vision is for All Children to Excel

• MISSION: Transform the academic achievements, self-confidence, and life trajectories of children living in under-resourced communities.

• APPROACH: BELL partners with schools and community organizations to combine resources and leverages time in the summer and after school to improve student achievement.

15School Partners

3,500Scholars Served (2015)

Summer Learning Loss66% of the academic achievement gap between disadvantaged students and their higher-income peers in 8th grade can be traced to unequalaccess to summer learning opportunities.

+1 MonthMiddle-Income Students Gain Academic Skills in the Summer

-2 MonthsDisadvantaged Students Lose Academic Skills in the Summer

Reading Achievement, Middle - Income Student

Reading Achievement, Disadvantaged StudentSource: National Summer Learning Association

Evaluation at BELL

Why Measure?Short term impact on student’s basic math and literacy skillsEffectiveness of curriculumEffectiveness of teachers and program staffEngagement of parents/guardians and students with learningWhat students like and don't likeHow to improve

Assessment & EvaluationQUANTITATIVE TOOLS

STAR Enterprise Assessments

Scholar Management System (Salesforce.com)Independent Evidence

QUALITATIVE TOOLSParent Survey

Teacher Survey

Quality Monitoring System

A rigorous, independent evaluation of BELL Summer found strong evidence that the program increases scholars’ academic skills and parental engagement.

Challenges: Measuring Success of Playable Technology Program

1. How to test students for improvement in specific curricular concepts? For example, problem solving or implementing hypothesis.

2. How to gauge student’s engagement with STEM enrichment program?

Solutions: • CONTROL GROUP COMPARISON

• ENHANCE CURRENT MEASURING TOOLS

• ARRANGE CIRRICULUM GOALS AROUND PRE-DETERMINED CONCEPTS

Sample Updated Survey:1. How much did you enjoy the Playable Tech enrichment class? Not at all A little bit Some Quite a bit A tremendous

amount

2. How much did you learn about building wearable interfaces this summer?

Almost Nothing A little bit Some Quite a bit A tremendous

amount

3. Did you find the playable tech course easy or difficult? Very easy Somewhat easy

Both easy and difficult

Somewhat difficult Very difficult

Think about each sentence and put an X in the box that matches how you feel about it. Yes Maybe No

4. Would you take another class involving coding?

5. Would you tell your friends to take a class on wearable technology?

6. Would you want to do coding as a job when you grow up? 7. What was your favorite part about the Playable Technology class? __________________________________________________

8. What are some projects you worked on during the Playable Technology class? __________________________________________________

Enhanced Curriculum :The Scientific Method

We plan to adapt the curriculum to reflect the scientific method. The scientific method provides clear guidelines for instructors to gauge student comprehension of basic curricular concepts that are not captured through the STAR assessment tool.

11

Thank You!