Capstone Final Paper

1 Running head: THE ASSOCIATION BETWEEN YOUTH PARTICIPATION

The association between youth participation in the El Sistema orchestra program at

he Rainey Institute and academic outcomes

Allison Silverman

Cleveland State University


Allison Silverman

Introduction

Having the ability to develop and grow in a suitable environment is important for any child. For some

children, this may be more difficult to achieve because of a lack of opportunity to develop skills in

vulnerable environments due to their circumstantial socio-economic status. According to the adolescent

health topic in Healthy People 2020, racial and ethnic minorities who live in poverty experience worse

outcomes for educational achievement compared to adolescents of similar ages that are white

(Adolescent Health, 2016). An over-whelming amount of students in the Cleveland Metropolitan School

District (CMSD) and the Rainey Institute identify as low-income racial or ethnic minorities. A potential

method of providing more positive prospects and outcomes for these low-income racial and ethnic

minority children is through after-school programs. For some children, there is a gap between the time

that their school day ends and when their parents or guardians end their work day (Zief, Lauver, &

Maynard, 2006). This gap in the day has impacted more children due to the fact that more than 80% of

mothers with school-aged children work away from home, in addition to more than two-thirds of those

classified in low- and moderate- income homes not being supervised by parents after school (Zief,

Lauver, & Maynard, 2006). Around three million children between six and twelve years old in the United

States do not have supervision often after school, equating to about 15% of children in that age range

(Zief, Lauver, & Maynard, 2006). According to multiple research articles, when a child is left

unsupervised, there is a chance of increased risk-taking behavior, victimization, and poor academic

results (Zief, Lauver, & Maynard, 2006). An increase in funding, from around $40 million in 1998 to over

$1 billion by 2004, and demand for after-school programs has led to an increase in after-school

programs throughout the last two decades (Kremer, Maynard, Polanin, Vaughn, & Sarteschi, 2015).

A literature reviews of past studies examined the association between participation in after-school

activities, such as arts-based programs, and academic outcomes. A regression study conducted by

Grogan, Henrich & Malikina, (2014) showed that there was a statistically significant relationship

between almost daily engagement in after-school activities and academic skills during the 2010-2011

school year, but infrequent engagement did not lead to statistically significant effects. Another study by

Johnson & Memmott (2006) looked at the quality of music programs on academic success. This study

found that students with excellent music programs, in which excellence was based on the instructional

quality of the program, generally performed better on standardized tests compared to poorer quality

music programs, and students in poorer music programs had better test scores than those not

participating in music at all (Johnson & Memmott, 2006). Another study looked at the level of

participation of the arts and academic achievement using standardized test scores (Catterall, 1997). This

study showed that about 71% of high-arts students scored in the top-half of performance distribution of

test scores compared to 46% of low-arts students. There was also a 20.4% difference between high-arts

and low-arts youth of low SES status on standardized test scores (Catterall, 1997). An association study

between instrumental music participation and academic achievement has already been conducted in

Ohio (Fitzpatrick, K. R., 2006). Fitzpatrick (2006), looked at the comparison of the Ohio Proficiency Test

results between instrumental and non-instrumental students for the entire Columbus Public School

District, but this study also classified the students based on high- or low- socioeconomic status (SES) The

results of the study showed that instrumental music students at both SES levels scored higher on the


Ohio Proficiency Test compared to non-instrumental students at both SES levels by the 9th grade

(Fitzpatrick, 2006). The instrumental students with the higher SES scored the highest compared to the

other groups (Fitzpatrick, 2006).

An after-school program option for children who come from vulnerable backgrounds in the Cleveland

area is participation in an arts-based program at the Rainey Institute. The Rainey Institute has provided

arts-based educational programing for children in the Cleveland area since the 1960s.Their most popular

program is the El Sistema Orchestra Program, which is a three hour a day and five days per week

internationally-recognized program that started in Venezuela. One of the major goals listed in the

Rainey Institute’s 2013-2016 Strategic Plan is to strengthen the impact of their programs, with one of its

strategies being “to measure and evaluate the progress and development of children and teens in our

programs”. This preliminary study hopes to create the beginning foundation of evaluating the El Sistema

Orchestra Program by looking at the participant’s academic outcomes in association to program

involvement. In this study it is hypothesized that grade-school level standardized test outcomes are

positively affected by a student’s participation in the El Sistema Orchestra program. Past research from

several studies has shown an association between high quality arts programs and better academic,

personal and social outcomes for students that are at-risk (Executive Summary of the Pilot Evaluation of

an El-Sistema-Inspired). The concern with these past studies is that their design needs to be more

rigorous in order to show a cause-and-effect relationship (Executive Summary of the Pilot Evaluation of

an El-Sistema-Inspired). This preliminary study will not be rigorous enough to show a cause-and-effect

relationship, but will be a beginning foundation to establish and create an evaluation that will better

express a relationship between participation in the El Sistema Orchestra program and academic

achievement.

Methods

Design

This preliminary study was strictly a records review of data that was previously collected by the

Rainey Institute. The review used existing, secondary records to analyze the initial academic

performance of the El Sistema participants at the Rainey Institute. Existing records are of limited use

since they may not provide specific information that is beneficial for outcome evaluations. Records from

the Rainey Institute included standardized test score results from the 2014-2015 school year for math

and reading, students’ attendance rate and number of days missed at school during the 2014-2015

school year, and the subject’s demographics, which included their age, grade, and gender. Socio-

economic status (SES) was not an included variable for the study because according to the 2014-2015

Ohio Department of Education’s Report Card for the Cleveland Municipal School District all 38,555

enrolled students in the school district classified as economically disadvantaged (Pages - District-Report,

n.d). Statistical differences could not be made based on SES status if every student in the CMSD, which

included all of the El Sistema subjects, was identified with a lower SES status. Data for the comparison

groups, which includes all of the students in the CMSD and the state of Ohio, were collected from the

Ohio Department of Education’s Ohio State Report Cards. Data on topics such as academic achievement,

attendance, and enrollment for each school district and individual school is accessible to the public

through the Ohio State Report Card, located on the Ohio Department of Education’s website.

Comparison data with the El Sistema Orchestra subjects was obtained for the CMSD and the state of

Ohio for the 2014-2015 school year using this website. Since the study was only using retrospective


data, and the subjects of the study sample were attempting to generalize conclusions for the entire

target population (all of the participants in the El Sistema Orchestra program), this was classified as a

cross-sectional study.

The model used to frame this study was the social cognitive theory, which is based on three

concepts: reinforcement, behavior, and cognition. For the purposes of this study, the goal was to show

that regularly participating (reinforcement) in the El Sistema Orchestra program (behavior) would have

an impact on academic achievement (cognition) measured by higher scores compared to non-

participating students in the standardized test for their grade. The exposed environment was the

participation in the El Sistema Orchestra program at the Rainey Institute, which is a structured, five day-

per-week after school orchestra program lasting three hours per day. The intended behavioral outcomes

were the ability to pass the standardized test scores for math and reading, however his model

accounted for personal factors, such as gender, grade, and age towards the outcome of the study. The

social cognitive theory was also a more appropriate model for this study because this theory cannot

prove causation since this study format did not have the proper elements, such as a large sample size

and randomization of subjects. This theory, however, can help show if one element may be mutually

influenced by another, known as reciprocal determinism. Based on the variables of the study design, this

would better be categorized as a preliminary association study between participation in the El Sistema

program and academic achievement. The concepts of the social cognitive theory better compliment an

association study that does not have the capabilities of proving causation.

Sample

The targeted sample for the study was any CMSD student that participated in the El Sistema Orchestra

Program at the Rainey Institute. This included a sample estimate of n = 65 participants. The objective for

this study was to obtain 85% of the standardized test scores for the El Sistema Orchestra participants by

April 1st, however the objective was not met after the April 1st deadline. The actual sample used in the

study was n = 13 participants, with only 20% of the intended sample. The actual sample only

represented one of the CMSD schools, out of 96 total schools in the entire district. 4th, 5th and 6th grade

student participants made up the El Sistema Orchestra sample, with more specifically five 4th grade

participants, three 5th grade participants, and five 6th grade participants. The El Sistema sample became

more specific as it was stratified by the type of standardized test. There were five 4th grade, three 5th

grade, and five 6th grade participants for the math standardized test of the El Sistema sample. The

reading standardized test sample included one 4th grade, three 5th grade, and five 6th grade participants.

Data was missing for four out of the five 4th grade El Sistema participants for the reading standardized

test. The El Sistema Orchestra study participants were also categorized by gender, with four (30.8%)

being female, and nine (69.2%) being male.

The CMSD comparison sample also varied based on grade level and type of standardized test score.

There was an overall total of 15,480 subjects from the CMSD. The number of subjects for the math

standardized test from the CMSD totaled 7,736, while the number of reading/English standardized test

subjects totaled 7,744. The number of subjects from both standardized tests was further stratified by

grade level. For the math standardized tests, the number of 4th grade subjects were 2,502, the number

of 5th grade subjects 2,511, and the number of 6th grade subjects 2,723. The number of reading/English

standardized test subjects was 2,500 from the 4th grade, 2,515 from the 5th grade, and 2,729 from the 6th

grade.


Measures

Descriptive Statistics

Using Excel and the R Statistical Software, descriptive graphical analyses were conducted to provide a

quick visual aide for communication of data results, and to show the differences between the El Sistema

sample group at the Rainey Institute, the CMSD and the State of Ohio comparison groups. A bar chart

was created to depict the weighted percentage averages for proficient or better results for math

standardized tests for Rainey and CMSD, and for reading standardized tests for Rainey and CMSD.

Cluster bar charts were created to show the percentage for proficient or above test scores for the

reading and math standardized tests. Both cluster bar charts were first stratified by group: Ohio, CMSD,

and El Sistema Orchestra Rainey Institute subjects. The charts were further stratified by grade level,

which included grades four, five and six. Figure (2) and (3) are the cluster bar charts used to depict the

raw percentages of the proficient or above standardized test scores for reading and math. Pie charts

were used to visually show the breakdown for the number of participants in each grade for the different

standardized tests, which were the math standardized tests for the CMSD and El Sistema Orchestra

Rainey subjects, and the reading standardized tests for the CMSD and El Sistema Orchestra Rainey

subjects. A total of four pie charts were created to depict the number of subjects for each test subject

categorized by grade, (figures (4) through (7)). To better understand the central tendency, or the

measurement that best represents the categorized group, a weighted average was computed. A

weighted average was more statistically rigorous because the sample was further stratified by 4th, 5th,

and 6th grade for each participant category. There were a total of six weighted averages calculated; the

math standardized test scores of the 2014-2015 school year for the El Sistema Orchestra participants,

the CMSD, and the State of Ohio, and the reading standardized test of the 2014-2015 school year for the

El Sistema Orchestra participants, the CMSD and the State of Ohio. These weighted averages could help

descriptively aide in understanding the overall results of the study.

Attendance data for the thirteen El Sistema Orchestra subjects was received on June, 1st 2016. This data

included the percentage of days present at school during the 2014-2015 school year, and the number of

days absent during the 2014-2015 school year. The objective for this project was to obtain attendance

data for 60% of the El Sistema Orchestra study participants by April, 1st 2016. That part of the objective

was achieved, as attendance data was obtained for 100% of the El Sistema Orchestra study participants.

However, the data was not received until June 1st 2016, which was exactly two months after the

intended date of April 1st, 2016.

The overall percentage of days present at school during the 2014-2015 school year for the El Sistema

Orchestra subject participants stratified by grade was 96.9% for 4th grade, 98.0% for 5th grade, and

98.9% for 6th grade. The overall percentage of days present at school during the 2014-2015 school year

for the all CMSD students stratified by grade was 92.5% for 4th grade, 91.9% for 5th grade, and 91.7% for

6th grade. Figure (8) displays these attendance data percentages in cluster bar chart (another descriptive

statistic) for 4th, 5th, and 6th grade El Sistema Orchestra subjects at the Rainey Institute and all CMSD

subjects. The overall percentage of days present at school for the combined 4th, 5th, and 6th grade

subjects from 2014-2015 school year was 92.0% for subjects from the CMSD subjects compared to

97.9% for the subjects from the El Sistema Orchestra program.


Analysis

This study should be treated as preliminary because there was a lack of target subjects to conduct a

feasible comparative statistical analysis. An ideal sample size, which should account for a 5% margin of

error, would have been no less than n = 56 subjects. This study had n = 13 subjects, leading to a

response rate of only 20%. When separating the number of El Sistema Orchestra subjects by type of

standardized test, the response rate for the math test was 20%, with a total of n = 13 subjects, and the

response rate for the reading test was 14%, with only a total of n = 9 subjects. Most statistical tests are

dependent on sample size, therefore when a small sample is used in statistical tests results may lead to

conclusive results, but those results would not have enough data to support significant results towards

the target population. In general, a larger sample size would be needed to detect a smaller difference

(Principle of Biostatistics, Ch. 10, p.g. 248). Studies should include a large enough sample size in order to

show an appropriate statistical power to identify if an association exists (Epidemiology in Public Health,

2014, p.g. 213). Results from chi-square tests, which is a statistical significance test, relies greatly on

sample size, therefore it was not feasible to conduct the test with the study’s small sample size.

Weighted averages of the PARCC standardized test score for the reading and math tests were one of the

descriptive statistical measurements of the study, and the weight was placed on the three different

grade levels computed in the average; 4th, 5th and 6th grade. The weighted averages were specifically for

the percentage of subjects for each group that passed the standardized test scores, and at least met the

state standards for academic achievement. 69.2% of 4th, 5th, and 6th grade El Sistema Orchestra Study

Participants at the Rainey Institute scored “Proficient” or better on the 2014 school year PARCC

Standardized Assessment Test in math compared with 31.9% for CMSD 4th, 5th, and 6th grade students.

The State of Ohio PARCC test score average for the 2014 School Year in math for 4th, 5th, and 6th grade

students combined was 68.1%. 88.9% of 4th, 5th, and 6th grade El Sistema Orchestra Study Participants at

the Rainey Institute scored “Proficient” or better on the 2014 school year PARCC Standardized

Assessment Test in reading/English compared with 45.3% for CMSD 4th, 5th, and 6th grade students. The

State of Ohio PARCC test score average for the 2014 School Year in reading for 4th, 5th, and 6th grade

students combined was 74.1%. The cluster bar chart in Figure (1) shows the weighted averages for

proficient or better standardized test scores for the math and reading test for both the El Sistema Rainey

participants and all of the CMSD participants. The objective was for at least 75% of the El Sistema study

participants to have at least met the standards for the test. When stratified by type of test, the study

participants did not meet the objective for the math standardized test, with only 69.2% of the El Sistema

study participants meeting the state minimum standards during the 2014-2015 school year.

Nevertheless, El Sistema study participants met the objective for the reading/English test, with 88.9% of

the participants meeting the state minimum standards during the 2014-2015 school year.

Discussion

Explanation of Results

The results from the provided data showed a difference with the passing rate of the math and

reading/English standardized test scores from the 2014-2015 school year between all CMSD subjects

and the target subject participants from the El Sistema Orchestra Program at the Rainey Institute.

Weighted averages of the 2014-2015 math and reading/English standardized test scores based on grade


showed the percentages of the subjects that at least had proficient test scores were almost doubled for

the El Sistema Orchestra study subjects compared to all of the CMSD subjects. Chi-square tests are

useful in calculating statistical significance; however, these tests are sensitive to sample size (Limitations

of the Chi-Square Test, (n.d.)). The size of the chi-square test is dependent of size of the sample, and

independent of the strength in relationship between the variables (Limitations of the Chi-Square Test,

(n.d.)). Frankfort-Nachmias and Leon-Guerrero (2011) stated that most researchers limited the use of

chi-square tables when a cell was less than five variables or have more than 20% of the cells with less

than five (Limitations of the Chi-Square Test, (n.d.)). After attempting to complete chi-square tests for

either meeting the minimum state standards or not meeting the minimum state standards for 2014-

2015 math standardized test scores for El Sistema Orchestra subjects compared with all CMSD subjects,

the test led to inconclusive results since one of the cells had fewer than five subjects (i.e. four subjects).

The same chi-square test was attempted for the 2014-2015 reading/English standardized test, and this

chi-square also led to inconclusive results because one of the cells also had less than five subjects (i.e.

one subject). Creating a generalized linear model for the El Sistema Orchestra study participants was

another possible statistical test to show the observed differences in variability for the test scores based

on the included explanatory variables in the model. Gender and missing five days of school or more

during the 2014-2015 school year were the explanatory variables used to help predict the observed

variability for receiving at least proficient test scores for the math and reading/English standardized

tests during the 2014-2015 school year. Because the sample size of the El Sistema Orchestra participants

was so small, a generalized linear model could not show any specific conclusive results, and at best it

could only detect major statistical differences between groups.

Limitations

There were many limitations with the study that impacted the ability to make any conclusive results and

conclusions from the data. The most significant issue was the sample size of El Sistema Orchestra

subjects at the Rainey Institute, with only thirteen in the sample out of a possible sixty-five subjects.

Only 20% of the potential El Sistema Orchestra subjects were included in the study, meaning that non-

response bias was a major source of error. The rationale behind the response bias, such as not willing to

provide the information requested or not reaching the necessary individuals to receive the data, is

unknown, but regardless of the rationale a study with non-response bias cannot lead to feasible results.

Ideally, the number of El Sistema Orchestra subjects should have totaled 56 participants if the study was

to account for a 5% margin of error, improving the consistency, or reliability, of the study (Sharma &

Petosa, 2014). Reliability refers to the degree to which the study measures what it was intended to

measure (Sharma & Petosa, 2014), and data that is more reliable may also yield results that can better

be generalized to the target population. Unfortunately, the sizes of the sample population, which

consisted of the thirteen 4th, 5th, and 6th study participants in the El Sistema Orchestra program at the

Rainey Institute, did not accurately represent the size of the target population, which should have

consisted of all the 4th, 5th, and 6th grade subjects in the El Sistema Orchestra Program. Due to the many

limitations of this preliminary cross-sectional study, any inferential statistical test measuring statistical

significance between variables would lead to inconclusive results. Some issues could have led to a small

number of sample population subjects, and these issues could have included the amount of time to

obtain the subjects, the lack of incentives for schools and students to participate in the study, and the

voluntary nature of the study (Sharma & Petosa, 2014).


Deficiency of research protocols

Arts-based program research, which includes this research study on academic outcomes of the El

Sistema Orchestra participants, is currently lacking certain research protocols (Symposium Final Report,

2015). There is a lack of large-scale longitudinal studies that follow the same population over time

(Symposium Final Report, 2015), which could help track the outcomes of students who receive intensive

exposure to a treatment, such as participation in an after-school program, compared to students who

are not exposed to the treatment (Symposium Final Report, 2015). If the El Sistema Program at the

Rainey Institute created a longitudinal study for their participants, then it would be advisable to find

willing participants of the in the sample population group that will be attending the El Sistema Program,

and to compare their academic outcomes before, during and after participation in the El Sistema

Program with non-orchestra participating students of similar demographic backgrounds in the CMSD.

Subjects for the comparison group of non-El Sistema orchestra students in the CMSD would also need to

be identified before the study commenced. This preliminary study, along with most other academic

achievement comparison studies between participation in organized music or not, has used a cross-

sectional approach. (Kinney, 2008). Cross-sectional studies cannot examine the potential group

differences between students that are involved in music classes compared to those that are not involved

in those classes (Kinney, 2008). When studying academic achievement, researchers have tried to protect

against confounding variables that may have a significant effect on academic achievement, such as

socioeconomic status (SES), school environment, and mobility (Kinney, 2008). When identifying these

variables after the fact, it may be much harder to clarify the association between variables.

Both longitudinal and cross-sectional studies are observational studies, therefore the researchers are

not interfering (Vu (Ed.), 2015). Longitudinal studies have the scope to more likely establish a cause-and-

effect relationship compared to cross-sectional studies (Vu (Ed.), 2015). A cross-sectional study, which is

completed more quickly than a longitudinal study, is usually created first by researchers to show a

potential link or association between variables before a cause-and-effect relationship is studied using a

longitudinal study (Vu (Ed.), 2015). There is a lack of rigorous activity establishing cause and effect

relationships between participants in high-quality arts programs and improved academic, personal, and

social outcomes for at-risk students. (Symposium Final Report, 2015). This study has recognized a

potential association between participation in the El Sistema Orchestra program and academic

achievement using a cross-sectional approach. The goal for a future academic outcome study for the El

Sistema Orchestra program is to create a longitudinal study to help satisfy the lack of cause and effect

studies in art-based educational studies.

Protocols for a future academic research study with the El Sistema Program

If a future academic outcome study for El Sistema Orchestra participants were to be conducted again,

then several factors would need to be addressed to help establish a more causal relationship. Academic

achievement cannot be measure strictly by standardized test scores. Standardized test scores can be a

high-stakes, summative evaluation, in which the results can lead to significant consequences for the

participating students, teachers, schools, and/or school districts (Mitchell, 2006). These consequences

can include school accountability measures, as well as school district ratings. The problem with using

standardized test scores as a high-stakes evaluation is that many factors can weaken the validity of

these test results (Mitchell, 2006). These factors can include cheating, inability to read test materials


accurately, non-response to questions, motivation levels of the participants, instruction protocol, test

preparation, and test development (Haladyna, 2006). Test scores may also be influenced by a number of

other factors, such as family resources, the student’s health, family mobility, and influential

neighborhood peers (Haladyna, 2006). Test scores should only be used modestly in a larger set of

evidence to measure academic achievement for students. In high-need areas, test-based accountability

results have been associated in surveys to influence the teacher’s attrition and discouragement

(Haladyna, 2006). The No Child Left Behind (NCLB) Act has evaluated schools using test scores, and

negative authorizations has been placed on schools who do not meet the expected standards (Haladyna,

2006). The goal of the NCLB for utilizing a test-based accountability approach was to close the

achievement gap, especially for minority students (Haladyna, 2006). Standardized test scores do not

provide absolute student achievement measurements (Haladyna, 2006). When standardized tests rely

on multiple-choice responses, it cannot measure the student’s communication skills, knowledge-depth

and understanding, or critical thinking skills, which all attribute to student achievement (Haladyna,

2006).

Other past studies on El Sistema after-school orchestra programs have used a mixed-method evaluation,

in which the evaluator collects quantitative and qualitative measurements. An urban school community

in Canada that has an El Sistema inspired orchestra program created a pilot evaluation of their program

(Morin, 2014). The evaluation used a mixed-method approach of data collection; interviews, focus

groups, student assessments, surveys, observations and institutional reviews (Morin, 2014). This study

specifically looked at the attendance rates of their El Sistema participants in school compared to the

non-El Sistema Orchestra participants in the CMSD (Morin, 2014). An examination of detailed school and

program attendance records showed that school attendance did not improve for participants in the El

Sistema Orchestra Program (Morin, 2014). The rationale behind the absenteeism was unknown,

however there was a low instance of suspensions found in the participants of the El-Sistema program

(Morin, 20014). Using a mixed-method study approach for future studies at the Rainey Institute could

potentially create more rigor in the evaluation, and may also provide more insight into the hypothesis-

forming process using qualitative conclusions, and also the hypothesis-conclusion process using

quantitative statistical analysis.

A future study would need to identify and control for more of the identified extraneous factors that can

impact standardized test score outcomes in order to help improve the reliability and validity of the

association study between participation in the El Sistema Orchestra program and academic outcomes.

These extraneous factors may include parental involvement and the support of the student’s education

(Thornton, 2013). Other measures of academic outcomes should be obtained outside of standardized

test scores. These other measures can include report cards and homework completion rates. Schools

with participants in the study can also be asked to disclose the participant’s behavioral infraction

information and the number of tardies per month and/or year for the participants.

If the Rainey Institute would like to continue this academic outcomes study of their participants, it

would be best to work closely with the CMSD. The Baltimore El Sistema Program called OrchKid’s has

been able to create a proficient annual report with the assistance from the Baltimore City Public Schools

and the University of Maryland Baltimore County (Baltimore Symphony Orchestra, 2015). A data sharing

agreement should be established between the Rainey Institute and the CMSD before completing future

studies of academic outcomes and overall effectiveness of the El Sistema program at the Rainey

Institute. If a future partnership would be established between the Rainey Institute and the CMSD, then


it would be advisable to have a data sharing agreement for the following items: the Rainey Institute

should be given the standardized test scores for all of their El Sistema Orchestra participants in the

Cleveland Metropolitan School District (CMSD), as well as the test score averages for the all of the CMSD

students not enrolled in the El Sistema Orchestra Program at the Rainey Institute non-participating El

Sistema orchestra students in the CMSD. School year attendance data should also be provided by the

CMSD for all of the El Sistema Orchestra student participants, in addition to average attendance data for

all non-participating El Sistema orchestra students in the CMSD. Another item in the data sharing

agreement should include that all CMSD students, including the El-Sistema orchestra participants and

non-orchestra participants, be asked by their parental guardians to participate in a socio-emotional

questionnaire. Healthy social, emotional, and behavioral modifications in young children are more likely

to lead to positive academic performances during elementary school (Cohen et al., 2005). The results

from the socio-economic questionnaire would then be shared with the Rainey Institute for further

evaluation. In sum, a strong data sharing agreement between the Rainey Institute and the CMSD would

include these items: attendance data, standardized test scores and/or school classroom test results in

math and reading, socio-emotional health status questionnaire results, and college outcome data if the

study would become a longitudinal one.

Correlations have been identified between socio-emotional behavioral and academic outcomes.

Researchers have discovered positive findings for classroom prosocial behavior and intellectual

outcomes, and further have predicted performances on standardized test achievement outcomes (Zins,

Weissberg, Wang, & Walberg, 2004). On the contrary, antisocial behavior has often been correlated with

poor academic outcomes (Zins, Weissberg, Wang, & Walberg, 2004). Adelman and Taylor (2004) have

argued that three components are necessary to achieve academic success: academic instruction and

school management, which are the more traditional components, and an enabling component, which

focuses on the social and emotional contexts in the classroom to promote a better conductivity to

learning and achieving academically. School success can be categorized into several different variables:

school attitudes (i.e. motivation), school behavior (i.e. attendance data and engagement), and school

performance (i.e. grades, test performance) (Zins, Weissberg, Wang, & Walberg, 2004). Engagement

should also be measured at the Rainey Institute because past studies have shown that students who

were highly engaged in after-school programs, and especially arts-based activities, are more related to

academic skills (Grogan, Henrich, & Malikina, 2014). This preliminary study has attempted to address the

school behavior and school performance component using attendance data and standardized test

scores, but has not addressed school attitudes. If a future data sharing agreement is created between

the CMSD and the Rainey Institute to continue further studies on this academic subject question, the

agreement should also include subjects filling out a socio-emotional questionnaire. This socio-emotional

questionnaire may help clarify the school attitudes of its participants by addressing school motivations,

management of emotions, decision-making processes, and ethical behaviors (Zins, Weissberg, Wang, &

Walberg, 2004). The practice of measuring academic achievement using standardized test scores and

attendance data is more widely recognized in research. Social and emotional questionnaires are not

easily understood, but research has acknowledged an association between social, emotional, and

academic outcomes.

This study used a status model to measure the academic outcomes for the El Sistema Orchestra program

participants, a method for measuring how participants perform at one point in time (Hull, 2007). For this

study, the status model was used to help show an academic achievement comparison between the non-


El Sistema Orchestra participating students in the CMSD and the participants of the El Sistema Orchestra

program. Such a model, according to educators, may not be the most accurate way to measure school

and student effectiveness for schools in high-impoverished urban and rural locations (Hull, 2007). At

these schools, a large proportion of the students are already academically behind students from more

advantageous communities (Hull, 2007). The argument is that the growth model more accurately

showcases a school’s academic performance (Wallis and Steptoe, 2007). The growth model looks at

academic progress between two points in time, instead of one point for the status model, to illustrate

potential student growth (Hull, 2007). A growth target can hypothesize the positive change in student

achievement rates for previous and upcoming years. This growth target model can help establish if non-

proficient students from the previous year met the growth target in the next year (Hull, 2007).

Comparisons of El Sistema study with previous research studies

Studying the potential association between arts-based programming and academic achievement may

have never been conducted at the Rainey Institute, but has been previously researched. Although there

have been many limitations discussed for this preliminary study, it is possible to show the similarities

and differences between this study and past studies on arts-based programming and academic

achievement. In 2003, a school district in a Midwestern metropolitan area was chosen for the study,

including two middle schools in the district that were identified by the state’s department of education

as “in need of improvement” (Kinney, 2008). A school that is “in need of improvement” could not show

yearly progress of state proficiency test scores for two consecutive years (Kinney, 2008). The results of

this study showed that band students scored significantly higher than non-participants on all subsets of

the 6th grade proficiency tests and all subsets of the 8th grade cohort, an exception being the Social

Studies proficiency test (Kinney, 2008). During the 2014-2015 school year for CMSD, the performance

index measure for the test score results of every student in the school district was the letter “D” (Pages -

District-Report, n.d.). A performance test letter “D” indicates that the passing rate for every student in

the school district was between 50.0-69.9%, and that the CMSD specifically had a performance index

percentage of 55.8% (Pages - District-Report, n.d.). This low performance index score and percentage

showed that the CMSD was in need of improvement for standardized test scores after the 2014-2015

school year (Pages - District-Report, n.d.). When understanding the comparison of test score results

between the El Sistema Orchestra program at the Rainey Institute and the non- El Sistema participating

subjects in the CMSD, it is important to keep this performance index measurement in mind.

Along with the 2003 study of a Midwestern metropolitan school district, the preliminary El Sistema

study at the Rainey Institute also showed better standardized test scores results for the 2014-2015 math

and reading/English test scores with the El Sistema Orchestra subjects compared to all CMSD students.

Statistical significance could not be demonstrated between the two groups, however, due to the small

sample size (only thirteen total subjects). The 2003 study by Kinney, D.W. was also superior because it

was longitudinal. The baseline measure of academic achievement for the 6th and 8th grade cohorts was

at 4th grade, a point that is prior to participants’ enrollment in band. The baseline data also showed

statistically higher test score results in both the 6th and 8th grade band cohorts for the 4th grade

proficiency subtests (Kinney, 2008). The Kinney (2008) study is consistent with past research findings

asserting that higher achieving students may be more attracted to instrumental music participation

(Fitzpatrick, (2006), Klinedinst, (1990), and Young, (1971)), as well as the academic results related to

participation in music (Kvet, (1985), McCrary, & Ruffin, (2006), and Wallick, (1998)).


Babo (2004) was another association study, but more specifically investigated instrumental music

participation instead of arts-based programming as a whole. This study observed the relationship

between instrumental music participation and academic achievement and used a multiple regression

analysis (Babo, 2004). The instrumental music participation was called IMUSIC, and the variables that

were identified to measure the variance in standardized test score results, specifically the CAT-NCE,

were socio-economic status (SES) and gender (Babo, 2004). When controlling for the variables SES and

gender, participation in the IMUSIC program showed significant results for the reading standardized

CAT-NCE scores (Babo, 2004). The regression analysis led to conclusive significant responses due to a

large enough sample size of n = 132 (Babo, 2004). The El Sistema Orchestra study attempted to create a

regression analysis that controlled for the variables gender and attendance rate to show a potential

significance between participation in the El Sistema Orchestra program and standardized test results.

Without a large enough sample size there was not enough data to show significant results, and there

was no scientific rationale to completing the regression analysis for this El Sistema Orchestra study.

Other studies that were measuring the relationship between participation in music and standardized

test score results also had small sample sizes. The Pennsylvania Department of Education Art Area and

the Pennsylvania Music Educators Association conducted a state-wide comparison of state assessment

scores between music and non-music students (Thornton, 2013). This study attempted to receive

standardized test score data for the math and reading tests, as well as student music participation data,

from every school district in Pennsylvania (Thornton, 2013). Of the 187 contacted school districts, only

36 responded to the data request (Thornton, 2013). 21 out of the 26 districts declined to participate in

the study for various reasons, and only 11 districts out of 187 contacted districts participated (Thornton,

2013). Having a small sample size of only 11 districts meant that the results may not have been

generalizable to all of the school districts in Pennsylvania (Thornton, 2013). Statistical testing of the

available data did show significantly higher test score differences for students that participated in music

compared to non-participating students (Thornton, 2013). These results were asked to be interpreted

cautiously because the sample sizes were very different between the music and non-music participants,

exemplified by the number of non-music 11th grade subjects being nearly four times greater than the

11th grade music participants (Thornton, 2013). These differences were accounted for in the statistical

analysis, but the results of the analysis should still be cautiously interpreted (Thornton, 2013). This study

had fewer sample size limitations compared to the El Sistema study because it was at least possible to

conduct and interpret statistical tests. Both studies had issues with sample size and the generalizability

of the results to the entire target population. The state-wide Pennsylvania study could not properly

generalize its results to every school district in Pennsylvania, and the El Sistema study could not

generalize its results to every El Sistema participant. Although there were significant sample size

limitations with both studies, it is important to further discuss other study limitations to learn how

future studies can become more scientifically rigorous.

Research on the best practices for afterschool programs has discussed the limitations of the available

research on the evaluation of afterschool programs. The literature searches for past research studies

explained that afterschool programs showed limited quality program evaluation, even though

afterschool programs have been available for a long time, but the research included the need for these

afterschool programs to continue (Fashola, 1998, Harvard Family Research Project [HFRP], 2003, The

Forum for Youth Investment [FYI], 2002). The limitations of afterschool program research included

knowing what features of such programs led to what outcomes, the level of optimal participation


needed for participants of a program, and the most effective activities based on the circumstances of

the participants (FYI, 2002, p.g. 1). Even if this El Sistema Orchestra program study had enough

participants to show significant results, these questions could not be answered because the preliminary

study design was not sophisticated enough. In addition, this study design lacked the variables needed to

answer these detailed limitation questions. When assessing the effectiveness of a program, a

continuous and quality evaluation should be created to help establish clear goals and objectives to help

gauge the success of the program (Beckett et al., 2001; USDOJ/USED, 2000; Fashola, 1998). Fashola

(1998) argued that the most valuable assessments for the quality of a program should compare the

measurements of the after-school program subjects with a similar group of non-participating students

as a control or comparison group. The El Sistema study had a comparison group of all students in the

CMSD, however the group did include data from the thirteen El Sistema Orchestra subjects. Luckily their

sample size from the El Sistema participants was so small that including their data in the comparison

data was most likely insignificant for a comparison sample size of over 15,000 CMSD subjects. In future

studies, research best practices should include a random control group of non-orchestra participating

subjects with similar demographic backgrounds, or a well-controlled comparison group of non-orchestra

participating subjects of similar demographic backgrounds that were established before the study

began. The control or comparison group should strive to have a proportionate balance of subjects as the

El Sistema Orchestra participating subjects.

A team of researchers from the SERVE Center at The University of North Carolina at Greensboro

conducted a small version of a meta-analysis for evaluations of after-school programs. The research

findings from the meta-analysis showed many associations between youth development and behavior

outcomes when participating in an after-school program (Brown et al., 2003). The associations for

participating in an after-school program include (a) improved school attendance (Brown et al., 2003; Hall

et al., 2003; Miller, 2003; Vandell in FYI, 2002), (b) better grades/achievement (Brown et al, 2003; Hall et

al, 2003; Miller, 2003; OJJDP, 2005; Vandell in FYI, 2002), (c) more positive attitudes toward school

(Brown et al, 2003), (d) better work and interpersonal skills (Brown et al, 2003; Hall, Yohalem, Tolman &

Wilson, 2003; Miller, 2003; Vandell in FYI, 2002), among other important associations. Based on the

comparisons for weighted averages by grade level between the subjects from the El Sistema Orchestra

program at the Rainey Institute and all of the subjects from the CMSD, the preliminary study showed a

likely association between participating in the El Sistema Orchestra Program, and better standardized

test score achievement. When breaking down the attendance rate data from the 2014-2015 school year

by grade level, the subjects of the El Sistema Orchestra program also had higher raw attendance rate

percentages for each grade compared to all CMSD subjects. This also presented a potential association

between participation in the El Sistema Orchestra program and improved school attendance. Those

were the only potential associations that could be recognized in the study, but luckily those associations

complemented the associations found in the SERVE Center’s meta-analysis for achievement and

attendance outcomes.


Overall Conclusions

Although this study could have been considerably more scientifically rigorous, some positive aspects

were yielded. The Rainey Institute has not often used statistical analysis to measure the effectiveness of

their programs, and such analyses may better show the impact of a program by describing and possibly

giving conclusions about a variable using quantitative and qualitative methods. This study used various

descriptive statistics to show how the El Sistema Orchestra study participants may have a greater

likelihood of passing standardized tests for math and reading compared to all CMSD students. The study

should be recognized as a preliminary study that has the ability to expand as the El Sistema Orchestra

program gains more participants, as well as become more scientifically rigorous after well-thought-out

adjustments are made to the study design. A small sample size, poor response rate, and an overall

deficiency of important data limited the feasibility of drawing significant and meaningful conclusions

from this cross-sectional study. Future studies on this subject have the potential to draw more tangible

and concrete associations between participation in the El Sistema Orchestra program and academic

achievement if the future recommendations herein are followed.


Figure (1)

*Weighted averages need percentage of proficient or better test scores and number of test subjects for both the

reading and math tests stratified by grade

*CMSD number of subjects tested data were found in the advanced reports section of the Ohio State Report Cards

website under “Tested Students Counts (District)” at

http://bireports.education.ohio.gov/PublicDW/asp/Main.aspx. The school year “2014-2015 School Year” and the

school district “Cleveland Municipal City” were selected.

*CMSD number percentage of proficient of better test scores were found in the advanced reports section of the

Ohio State Report Cards website under “Proficiency Levels (District)” at



31.9

69.2

45.3

88.9

0 10 20 30 40 50 60 70 80 90 100

CMSD - Math

El Sistema - Math

CMSD - Reading

El Sistema - Reading

Weighted Average Percentage

Weighted Averages for Proficient or Better (Combining 4th, 5th and 6th Grades)

http://bireports.education.ohio.gov/PublicDW/asp/Main.aspx



Figure (2)

*Ohio percentage data was found on Download Data tab in the Ohio State Report Cards website. The 2014-2015 school year, disaggregated district data, and district gender disaggregation were selected. An average data was generated for each grade level and test type in the excel file. * CMSD percentage data was found in the advanced reports section of the Ohio State Report Cards website under “Test Results (District)” at http://bireports.education.ohio.gov/PublicDW/asp/Main.aspx?evt=3014&src=Main.aspx.3014&Main.aspx=-*-NaHRYK3*_crZeze6l9sGCFf6ddkM%3D.ReportCard.*-nXgfWhY8Ub2ALtal_&SaveReportProperties=*-1.*-1.0.0.0&rb=0.0.D03BB6F74D98D381BCAB0E9420D82F75.Test%2BResults%2B*-28District*-29.*-1.16875904.1.0.1.0.0.0.1.0*.0*.0*.0*.800*.35*.800*.35*.0*.1*.1.1-F44384AE4336F565A746179135D6A985.1.0.1.1.1.*-1.1.167772192.0.100.2000.0.0.0..1.800.1.35..*-1..*-1...*0.0_*0.*-1.0.*0.*0.1.0.*0.2.1.1.0.0.*0. The school year “2014-2015 School Year” and the school district “Cleveland Municipal City” were selected.

47.6

100

75.2

43.2

100

73.1

45

80

74

0 20 40 60 80 100 120

CMSD - Percentage of Passing Reading Test Scores for 4th,5th & 6th Grade

El Sistema at the Rainey Institute - Passing Rate forReading Test Scores for 4th, 5th & 6th Grade

Ohio- Reading for 4th, 5th & 6th Grade

Percentage for Proficient or Above in Reading Test Scores

4th Grade 5th Grade 6th Grade

http://bireports.education.ohio.gov/PublicDW/asp/Main.aspx?evt=3014&src=Main.aspx.3014&Main.aspx=-*-NaHRYK3*_crZeze6l9sGCFf6ddkM%3D.ReportCard.*-nXgfWhY8Ub2ALtal_&SaveReportProperties=*-1.*-1.0.0.0&rb=0.0.D03BB6F74D98D381BCAB0E9420D82F75.Test%2BResults%2B*-28District*-29.*-1.16875904.1.0.1.0.0.0.1.0*.0*.0*.0*.800*.35*.800*.35*.0*.1*.1.1-F44384AE4336F565A746179135D6A985.1.0.1.1.1.*-1.1.167772192.0.100.2000.0.0.0..1.800.1.35..*-1..*-1...*0.0_*0.*-1.0.*0.*0.1.0.*0.2.1.1.0.0.*0







Figure (3)

*Ohio percentage data was found on Download Data tab in the Ohio State Report Cards website. The 2014-2015 school year, disaggregated district data, and district gender disaggregation were selected. An average data was generated for each grade level and test type in the excel file. * CMSD percentage data was found in the advanced reports section of the Ohio State Report Cards website under “Test Results (District)” at http://bireports.education.ohio.gov/PublicDW/asp/Main.aspx?evt=3014&src=Main.aspx.3014&Main.aspx=-*-NaHRYK3*_crZeze6l9sGCFf6ddkM%3D.ReportCard.*-nXgfWhY8Ub2ALtal_&SaveReportProperties=*-1.*-1.0.0.0&rb=0.0.D03BB6F74D98D381BCAB0E9420D82F75.Test%2BResults%2B*-28District*-29.*-1.16875904.1.0.1.0.0.0.1.0*.0*.0*.0*.800*.35*.800*.35*.0*.1*.1.1-F44384AE4336F565A746179135D6A985.1.0.1.1.1.*-1.1.167772192.0.100.2000.0.0.0..1.800.1.35..*-1..*-1...*0.0_*0.*-1.0.*0.*0.1.0.*0.2.1.1.0.0.*0. The school year “2014-2015 School Year” and the school district “Cleveland Municipal City” were selected.

32.9

80

62.5

33.9

0

71.2

29.3

100

70.6

0 20 40 60 80 100 120

CMSD - Percentage of Passing Math Test Scores for 4th, 5th& 6th Grade

El Sistema at the Rainey Institute - Percentage of PassingMath Test Scores for 4th, 5th & 6th Grade

Ohio - Math for 4th, 5th & 6th Grade

Percentage for Proficient or Above in Math Test Scores

4th Grade 5th Grade 6th Grade








Figure (4)





Figure (5)

5

3

5

El Sistema - Number of Sujects for Math Standardized Test

4th 5th 6th

2502

2511

2723

CMSD - Number of Subjects for Math Standardized Test

4th 5th 6th



Figure (6)





Figure (7)

2500

2515

2729

CMSD - Number of Subjects for Reading Standardized Test

4th 5th 6th

1

35

El Sistema - Number of Subjects for Reading Standardized Test

4th 5th 6th



Figure (8) Cluster Bar Chart for Attendance Data Stratified by Grade for El Sistema Subjects and CMSD

Subjects

*CMSD attendance data were found in the advanced reports section of the Ohio State Report Cards website under

“Attendance Rate with Student Dissagg (District)” at

http://bireports.education.ohio.gov/PublicDW/asp/Main.aspx. The pick student disaggregation topic “grade level”

and the school year “2014-2015 School Year” were selected.

92.50%

91.90%

91.70%

96.90%

98.00%

98.90%

88.0% 90.0% 92.0% 94.0% 96.0% 98.0% 100.0%

4th Grade

5th Grade

6th Grade

Attendance Rate Data for the 2014-2015 school year

Attendance Rate (El Sistema Subjects at the Rainey Institute) Attendance Rate (CMSD Subjects)



References

1. Adolescent Health. (2016). Retrieved May 23, 2016, from

https://www.healthypeople.gov/2020/topics-objectives/topic/Adolescent-Health.

2. Babo, G. D. (2004). The Relationship Between Instrumental Music Participation And

Standardized Assessment Achievement Of Middle School Students. Research Studies in Music

Education, 22(1), 14-27. doi:10.1177/1321103x040220010301

3. Baltimore Symphony Orchestra: OrchKids. (2015). Retrieved May 23, 2016.

4. Catterall, J. S. (1997). Involvement in the arts and success in secondary school. Americans for the Arts.

5. Beckett, M., Hawken, A., & Jacknowitz, A. (2001). Accountability for after-school care: devising standards and measuring adherence to them. RAND Corporation. Available online at www.rand.org/publications/MR/MR1411/.

6. Brown, E., McComb, E., & Scott-Little, C. (2003). Expanded learning opportunities programs: A review of the research and evaluations on participant outcomes in school readiness and after-school programs. Greensboro, NC: SERVE. Available online at http://www.serve.org/_downloads/REL/ELO/AfterSchool.pdf

7. Cohen, J., and others. 2005. Helping Young Children Succeed: Strategies to Promote Early Childhood Social and Emotional Development (accessed on December 7, 2006) Washington, DC: National Conference of State Legislatures and Zero to Three.

8. Fashola, O. (1998). Review of extended-day and after-school programs and their effectiveness.

Center for Research on the Education of Students Placed at Risk (CRESPAR), Johns

Hopkins University and the Office of Educational Research and Improvement (OERI),

U.S. Department of Education. Available online at

http://www.csos.jhu.edu/crespar/techreports/report24.pdf. 9. Fitzpatrick, K. R. (2006). The Effect of Instrumental Music Participation and Socioeconomic

Status on Ohio Fourth-, Sixth-, and Ninth-Grade Proficiency Test Performance. Journal of Research in Music Education, 54(1), 73-84.

10. Grogan, K. E., Henrich, C. C., & Malikina, M. V. (2014). Student Engagement in After-School Programs, Academic Skills, and Social Competence among Elementary School Students. Child Development Research, 1-9. doi:10.1155/2014/498506

11. Hall, G., Yohalem, N., Tolman, J. & Wilson, A. (2003). How afterschool programs can most effectively promote positive youth development as a support to academic achievement: A report commissioned by the Boston after-school for all partnership. National Institute on Out-of-School Time, Center for Research on Women at Wellesley Centers for Women, Wellesley College. Available online at http://www.niost.org/wcw3.pdf.

12. Haladyna, T. (2006). Perils of Standardized Achievement Testing. Educational Horizons, 85(1),

30-43. Retrieved May 23, 2016.

13. Harvard Family Research Project (HFRP). (2003, August). A review of activity implementation

in out-of-school time programs. Out-of-School Time Evaluation Snapshot, 2. Cambridge,

MA: Author.

14. Harvard Family Research Project (HFRP). (2003, July). Out-of-school time evaluation snapshot:

A review of out-of-school time program quasi-experimental and experimental evaluation

results. Harvard Graduate School of Education. Available online at

http://www.gse.harvard.edu/hfrp/pubs/pubsbytitle.html.

15. Hull, J. (2007). Measuring student growth: A guide to informed decision making. Retrieved May

23, 2016, from Measuring student growth: A guide to informed decision making - See more at:

https://www.healthypeople.gov/2020/topics-objectives/topic/Adolescent-Health

http://policy.db.zerotothree.org/policyp/view.aspx?InitiativeID=686&origin=results&QS=%27&union=AND&viewby=50&startrec=1&tbl_Public_InitiativeYMGHFRECategory=Child+Welfare&top_parent=164

http://policy.db.zerotothree.org/policyp/view.aspx?InitiativeID=686&origin=results&QS=%27&union=AND&viewby=50&startrec=1&tbl_Public_InitiativeYMGHFRECategory=Child+Welfare&top_parent=164


http://www.centerforpubliceducation.org/Main-Menu/Policies/Measuring-student-growth-At-

a-glance/Measuring-student-growth-A-guide-to-informed-decision-

making.html#sthash.ZHpB18ex.dpuf

16. Johnson, C. M., & Memmott, J. E. (2006). Examination of Relationships between Participation in

School Music Programs of Differing Quality and Standardized Test Results. Journal of Research in

Music Education, 54(4), 293-307

17. Kinney, D. W. (2008). Selected Demographic Variables, School Music Participation, and

Achievement Test Scores of Urban Middle School Students. Journal of Research In Music

Education, 56(2), 145-161.

18. Klinedinst, R. (1990). Predicting performance achievement and retention of fifth-grade

instrumental students. Journal of Research in Music Education, 39, 225–238.

19. Kremer, K., Maynard, B., Polanin, J., Vaughn, M., & Sarteschi, C. (2015). Effects of After-School

Programs with At-Risk Youth on Attendance and Externalizing Behaviors: A Systematic Review

and Meta-Analysis. Journal of Youth & Adolescence, 44(3), 616-636. doi:10.1007/s10964-014-

0226-4

20. Kvet, E. J. (1985). Excusing elementary school students from regular classroom activities for the

study of instrumental music: The effect on sixth-grade reading, language, and mathematics

achievement. Journal of Research in Music Education, 32, 45–54.

21. Limitations of the Chi-Square Test. (n.d.). Retrieved June 17, 2016, from

https://learn.bu.edu/bbcswebdav/pid-826908-dt-content-rid-

2073693_1/courses/13sprgmetcj702_ol/week05/metcj702_W05S02T05_limitations.html

22. McCrary, J., & Ruffin, M. (2006). Seventh-grade music ensemble participants’ and non-ensemble

participants’ academic achievement and school attendance. Paper presented at MENC’s 60th

National Biennial In-Service Conference, Salt Lake City, UT.

23. Miller, B. (2003). Critical hours: Afterschool programs and educational success. Quincy, MA:

Nellie Mae Education Foundation. Available online at

http://www.nmefdn.org/CriticalHours.htm.

24. Mitchell, R. (2006). A guide to standardized testing: The nature of assessment. Retrieved May

23, 2016, from http://www.centerforpubliceducation.org/Main-Menu/Evaluating-

performance/A-guide-to-standardized-testing-The-nature-of-assessment

25. Morin, F. (2014). From Caracas to the Canadian Prairies: Executive Summary of the Pilot

Evaluation of an El-Sistema-Inspired After-School Orchestral Program. Canadian Music Educator

/ Musicien Educateur Au Canada, 56(1), 20-26.

26. Office of Juvenile Justice & Delinquency Prevention (OJJDP) (2005). Model programs guide

for afterschool /recreation. Retrieved December 13, 2005 from

http://www.dsgonline.com/mpg2.5/afterschool_recreation_prevention.htm.

27. Pages - District-Report. (n.d.). Retrieved June 22, 2016, from

http://reportcard.education.ohio.gov/Pages/District-Report.aspx?DistrictIRN=043786

28. Sharma, M., & Petosa, R. L. (2014). Measurement and evaluation for health educators.

Burlington, MA: Jones & Bartlett Learning.

29. Symposium Final Report Addendum: Case Studies. (2015). Retrieved May 23, 2016, from

http://www.elsistemausa.org/userfiles/file/Symposium2015-CaseStudies.pdf.

30. The Forum for Youth Investment (FYI). (2002). Out-of-school-time policy commentary #1: Outof-

school research meets after-school policy. Washington, DC: The Forum for Youth

https://learn.bu.edu/bbcswebdav/pid-826908-dt-content-rid-2073693_1/courses/13sprgmetcj702_ol/week05/metcj702_W05S02T05_limitations.html

https://learn.bu.edu/bbcswebdav/pid-826908-dt-content-rid-2073693_1/courses/13sprgmetcj702_ol/week05/metcj702_W05S02T05_limitations.html

http://www.dsgonline.com/mpg2.5/afterschool_recreation_prevention.htm

http://www.elsistemausa.org/userfiles/file/Symposium2015-CaseStudies.pdf


National Dropout Prevention Center 71 Investment, Impact Strategies, Inc. Available online at

http://www.forumfyi.org/files//ostpc1.pdf.

31. Thornton, L. (2013). A Comparison of State Assessment Scores Between Music and Nonmusic

Students. UPDATE: Applications Of Research In Music Education, 32(1), 5-11.

doi:10.1177/8755123313502339

32. U.S. Department of Justice and U.S. Department of Education (USDOJ/USED). (2000). Working

for children and families: Safe and smart after-school programs. Washington, DC: Authors.

Available online at www.ed.gov/offices/OESE/archives/pubs/parents/SafeSmart/index.html.

33. Vu, U. (Ed.). (2015). Cross-sectional and longitudinal are two types of studies that let scientists

study links without changing subject behaviour. Retrieved June 13, 2016, from

https://www.iwh.on.ca/system/files/at-work/at_work_81.pdf

34. Wallick, M. D. (1998). A comparison study of Ohio Proficiency Test results between fourth grade

string pullout students and those of matched ability. Journal of Research in Music Education, 46,

239–247.

35. Young, W. (1971). The role of musical aptitude, intelligence, and academic achievement in

predicting the musical attainment of elementary instrumental music students. Journal of

Research in Music Education, 19, 385–398.

36. Zief, S. G., Lauver, S., & Maynard, R. (2006). Impacts of after-school programs on student

outcomes. Campbell Systematic Reviews. doi:10.4073/csr.2006.3.

37. Zins, J. E., Weissberg, R. P., Wang, M. C., & Walberg, H. J. (2004). Building academic success on

social and emotional learning: What does the research say? New York: Teachers College Press.

http://www.forumfyi.org/files/ostpc1.pdf

Documents

Capstone Final Paper