34
Energizing Career Opportunities through Local Education and Development (ECO LED) Evaluation Final Report California State University Consortium Funding Provided by Southern California Edison Submitted by Cobblestone Applied Research & Evaluation, Inc. Rebecca M. Eddy, Ph.D., H. Todd Ruitman, M.A., & Nancy Hankel, M.A. July 17, 2012

Energizing Career Opportunities through Local Education ... · of the Energizing Career Opportunities through Local Education and Development (ECO LED) Program. The evaluation was

  • Upload
    others

  • View
    10

  • Download
    0

Embed Size (px)

Citation preview

Energizing Career Opportunities through Local Education and Development (ECO LED)

Evaluation Final Report

California State University Consortium

Funding Provided by

Southern California Edison

Submitted by

Cobblestone Applied Research & Evaluation, Inc.

Rebecca M. Eddy, Ph.D., H. Todd Ruitman, M.A., & Nancy Hankel, M.A.

July 17, 2012

CSU ECO LED Program Final Evaluation Report 2 Cobblestone Applied Research & Evaluation, Inc.

Table of Contents

Table of Contents .......................................................................................................................................... 2

Executive Summary ....................................................................................................................................... 4

Introduction .................................................................................................................................................. 6

Energizing Career Opportunities through Local Education and Development Program .............................. 8

Evaluation of the ECO LED Program ............................................................................................................. 8

Evaluation Methods .................................................................................................................................. 9

Evaluation Instruments ............................................................................................................................. 9

Student Survey ........................................................................................................................................... 9

Faculty Written Reports ........................................................................................................................... 10

Community Partners Survey .................................................................................................................... 10

Key Evaluation Questions ....................................................................................................................... 11

Participating Sample ............................................................................................................................... 11

Service-Learning Course Descriptions ......................................................................................................... 12

Course 1: Aquatic Ecology (BIOL 172S) at Fresno State .......................................................................... 13

Course 2: Mine Reclamation (EES 250T) at Fresno State ........................................................................ 13

Course 3: Biogeography of Southern California (GEO 380) at CSU Dominguez Hills .............................. 14

Course 4: Water and Conflict in the West (UNIV 298) at CSU Channel Islands ...................................... 14

Evaluation Results ....................................................................................................................................... 15

Section 1: Education in STEM.................................................................................................................. 15

STEM Career Interest ............................................................................................................................... 16

Hands-on Learning ................................................................................................................................... 18

Education Summary ................................................................................................................................. 19

Section 2: Environment ........................................................................................................................... 19

Environment Summary ........................................................................................................................... 22

Section 3: Community Building ............................................................................................................... 22

Community Education ............................................................................................................................. 22

Student Community Engagement............................................................................................................ 23

Developing Community Partnerships ...................................................................................................... 25

Community Building Summary ............................................................................................................... 27

Successes and Challenges of the ECO LED Program ................................................................................... 27

Successes of the ECO LED Program ........................................................................................................ 27

CSU ECO LED Program Final Evaluation Report 3 Cobblestone Applied Research & Evaluation, Inc.

Challenges of the ECO LED Program ....................................................................................................... 28

Conclusions ............................................................................................................................................. 29

References .................................................................................................................................................. 31

Appendix A: Student Survey Factor Analysis .............................................................................................. 32

Appendix B: Community Partner Survey Items .......................................................................................... 33

Tables

Table 1. Key Evaluation Questions and Corresponding Data Sources ........................................................ 11

Table 2. Course Influence on Degree and Career Posttest Individual Item Means ..................................... 18

Table 3. Influence of Service Learning on Student Learning Items Posttest Mean Scores ......................... 19

Table 4. Summary of ECO LED Program Activities ...................................................................................... 20

Table 5. Challenges and Possible Solutions for the ECO LED Program ....................................................... 29

Figures Figure 1. Student Posttest Mean Ratings of Interest in Pursuing a STEM Career ....................................... 17

Figure 2. Student Posttest Mean Ratings of Community Engagement ...................................................... 24

Figure 3. Posttest Student Mean Rating of Course Influence on Community Engagement........................ 25

Figure 4. Community Partners Satisfaction, Frequency of Survey Responses ........................................... 25

CSU ECO LED Program Final Evaluation Report 4 Cobblestone Applied Research & Evaluation, Inc.

Executive Summary

Cobblestone Applied Research & Evaluation, Inc. was hired to evaluate the effectiveness of the Energizing Career Opportunities through Local Education and Development (ECO LED) Program. The evaluation was designed to capture anecdotal evidence on how service-learning STEM courses in the California State University (CSU) system might create unique opportunities for students, faculty, and their local communities. Data collected in the evaluation provides some evidence of the potential for impact on education, environment, and community building through service-learning courses. Evaluation Question 1: How were ECO LED Program activities implemented? A total of four service-learning courses were part of the ECO LED Program at three southern California CSU campuses. These courses included the Mine Reclamation and Aquatic Ecology courses at Fresno State, Biogeography of Southern California at CSU Dominguez Hills, and Water Conflict in the West at CSU Channel Islands. These courses included service-learning activities that reinforced knowledge obtained in the classroom as well as benefited the community. For example, students at Fresno State developed an erosion control plan report after studying the abandoned Tip Top Mine in San Luis Obispo County. Other projects included a community garden, outreach and education for middle school students, and restoration and protection of an endangered species habitat. Evaluation Question 2: What was the effect of the ECO LED Program on students’ education in STEM? An analysis of student survey data revealed two main themes that emerged with regard to students’ experiences in the service-learning course related to their education in STEM; specifically, STEM career interest and the possible impact of hands-on learning. Results indicated that the service-learning activities provided students with a positive, hands-on learning experience that, according to students, influenced their degree and career goals. Evaluation Question 3: What was the effect of the ECO LED Program on the environment? Results showed that the four courses participated in fifteen separate activities that made a positive impact on the environment. Some examples of activities include: students monitored, analyzed, and reported on water quality to the community partner leading to identified areas of concern where future efforts might be focused; students monitored habitat for a river salmon restoration project; and students conducted an erosion control project on the campus through the installation of erosion control matting, wattles, and silt fencing. Evaluation Question 4: What was the effect of the ECO LED Program on promoting community building?

We investigated how the efforts of faculty, students, and community partners combined to address community needs and create opportunities for the future. The analysis revealed that the ECO LED Program had an effect on: community education, student community engagement, and the development of community partnerships. Specifically, multiple

CSU ECO LED Program Final Evaluation Report 5 Cobblestone Applied Research & Evaluation, Inc.

opportunities were found to educate the community regarding course content. Also, students, already willing to help the community, were given the information and skills needed to engage in community service on their own. Lastly, the community partners reported their satisfaction of working with the faculty and students. All community partners indicated that they would be willing to partner with faculty and students for a future service-learning course. Evaluation Question 5: What were the major successes and challenges of the ECO LED Program? Success of ECO LED

It was clear that faculty, community partners, and most students felt that the ECO LED Program was a positive experience. Beyond the programs’ impacts to education, environment, and community, another success was that students’ showed interest in pursuing careers that were directly related to the service-learning projects. These courses demonstrated that service learning can be used in higher education to provide students with experiences that both reinforce classroom learning and theory and provide students with opportunities to utilize what they have learned in possible workplace environments. Challenges of ECO LED The challenges that were encountered in the courses were logistical in nature, and most could be avoided in future course offerings with adequate planning and coordination. Lessons from these challenges can be used to aid in the creation of new service-learning courses or adding the service-learning component to existing courses. Conclusions Faculty, students, and community partners were able to accomplish much during the scope of the four service-learning courses that comprised the ECO LED Program in spring 2012. The positive effects of these courses on students’ education, the environment, and community building were evident given the analysis of student surveys, faculty reports, photographic and video evidence, and community partners’ surveys. Additional time and study is needed to better understand how service-learning courses such as these have longer-terms impacts such as increasing interest in STEM disciplines, graduation in STEM degrees, and desire for STEM careers.

CSU ECO LED Program Final Evaluation Report 6 Cobblestone Applied Research & Evaluation, Inc.

Introduction

Academic, public, and private institutions have a vested interest in promoting the

development of students’ knowledge and skills that are necessary in the current global

economy. Often referred to as 21st century skills, these include a combination of academic or

core content areas skills; learning and innovation; information, media, and technology; and life

and career skills (Partnership for 21st Century Skills). Topics such as global awareness,

environmental literacy, flexibility and adaptability, leadership, and responsibility have joined

core content areas such as language arts, mathematics, and science as key learning outcomes

for all students. In addition, “students must also learn the essential skills for success in today’s

world, such as critical thinking, problem solving, communication, and collaboration.”

(Partnership for 21st Century Skills).

One critical area in the development of these core skills relies on educating students

about the importance of conservation and the environment as a way of promoting the health

and life of future generations. Research suggests that high levels of environmental knowledge

(i.e., knowledge of current environmental issues) predict students’ concern, attitudes, and

feelings of personal responsibility toward environmental protection (Teksoz, Sahin, & Tekkaya-

Oztekin, 2012). In other words, environmental education helps to promote - “environmental

literacy” – awareness, sensitivity, knowledge, concern, responsibility, and action related to

environmental conservation (Roth, 1992). Providing the opportunity for young people to both

participate in and become educated about environmental needs can serve as an effective

platform for building these types of 21st century skills. Further, providing opportunities for

them to interact with their communities may help to decrease civic disengagement that is

prevalent.

One strategy that can address the problem of civic disengagement is community

engagement or community-based education. Community engagement refers to the

“collaboration between educational institutions and their larger communities for the mutually

beneficial exchange of knowledge and resources in a context of partnership and reciprocity”1

(Carnegie Foundation for the Advancement of Teaching and Learning). Community engagement

1 http://www.calstate.edu/cce/about_us/vision.shtml

CSU ECO LED Program Final Evaluation Report 7 Cobblestone Applied Research & Evaluation, Inc.

includes a number of activities such as service learning, voter registration drives, community-

based research and community service.

Service learning is one type of community engagement activity that specifically focuses

on civic involvement, active learning, and application of knowledge. Service learning is an

educational methodology that combines community service with classroom learning to engage

students in the educational process, teach civic responsibility, and strengthen communities

(Bringle & Hatcher, 1995). Service learning helps students gain socially responsive knowledge,

that is, education obtained through direct academic-based problem solving of social issues

(Altman, 1996). Students participating in service learning apply the skills and knowledge they

learn in their academic coursework to identify and solve real-world community problems and

also access the expertise of community partners in addressing these problems. Additionally,

students become contributing citizens and active community members through the service they

perform.

Research indicates that service learning has an overall positive impact on students’

social, personal, and cognitive outcomes (Giles & Eyler, 1994). Students who participated in

service-learning education had greater gains in perspective-taking, complex problem solving,

and critical thinking than students who did not participate in service-learning education

(Batchelder & Root, 1994; Markus, Howard, & King, 1993). Additionally, service learning

increases students’ awareness of contemporary social issues (Driscoll, Holland, Gelmon &

Kerrigan, 1996), enhances students’ self-efficacy relating to community service (Reeb,

Katsuyama, Sammon, & Yoder, 1998), and increases positive attitudes about civic engagement

and social responsibility (Markus et al., 1993).

Given the need to inform the next generation of critical issues in the environment and

conservation, service learning can serve as a functional strategy to promote active learning and

advocacy while providing students with a rewarding educational opportunity. This opportunity

can have reciprocal, positive impacts on students and their local community. Service-learning

experiences are often developed through a collaboration of community organizations and

higher education institutions through academic departments or programs. The Energizing

Career Opportunities through Local Education and Development (ECO LED) Program was

CSU ECO LED Program Final Evaluation Report 8 Cobblestone Applied Research & Evaluation, Inc.

developed to address educational needs of southern California students to promote 21st

century skills while serving the local community related to specific environmental issues.

Energizing Career Opportunities through Local Education and Development Program

The California State University (CSU) system is the largest system of higher education in

the United States and recognizes the responsibility to provide its students with knowledge and

opportunity to fully participate in civil society. For more than a decade, the CSU system has

been committed to increasing opportunities for student community engagement, and has been

expanding partnerships to do so. Southern California Edison (SCE) is committed to supporting

the development and education of the workforce for the 21st century as well. In partnership

with the CSU, SCE is committed to ensuring that students in southern California gain both

academic and technical skills to succeed in the workforce while supporting efforts to sustain a

quality environment. Therefore, SCE has provided financial support for CSU’s ECO LED Program.

All eight CSU campuses based in southern California were eligible to submit proposals for

serving-learning courses during spring 2012. Courses were focused on the disciplines of science,

technology, engineering, and mathematics (STEM) with a particular emphasis on environmental

support. Faculty formally applied to participate in the program and received funding sub-grants

to design and implement a service-learning STEM course in one of four priority areas. This

report summarizes all ECO LED Program activities and describes the evaluation of the ECO LED

Program.

Evaluation of the ECO LED Program

Cobblestone Applied Research & Evaluation, Inc. (Cobblestone) was hired to evaluate

the effectiveness of the ECO LED Program and to determine the effectiveness of program

activities on students, faculty and the community. The small number of courses and relatively

few faculty and students involved in the overall program was considered in designing the

evaluation. A rigorous, experimental study was not possible given the limited participation in

the program; however, the evaluation was designed to capture anecdotal evidence on how

service-learning STEM courses in the CSU system might create unique opportunities for

students, faculty, and their local communities in the future. The ultimate goal of these courses

CSU ECO LED Program Final Evaluation Report 9 Cobblestone Applied Research & Evaluation, Inc.

in the CSU is to increase the number of CSU students interested in, and successfully graduating

from STEM disciplines thus developing and educating the workforce of the 21st century through

service learning. While the current evaluation was not intended to address this longer-term

vision, data collected in the evaluation provides some evidence of the potential for impact on

education, environment, and community building through service-learning courses.

Evaluation Methods

The evaluation relied on data collection activities such as administration of student

pretest and posttest surveys and community partner surveys. Course faculty also provided

reports that included written descriptions of course activities, successes, and challenges of each

project. In addition, photographic evidence and videos were provided as evidence of program

implementation and the effect on the environment. The evaluation team at Cobblestone

designed the evaluation to facilitate minimally intrusive data collection, which relied heavily on

course faculty to facilitate.

Evaluation Instruments

Evaluators from Cobblestone designed the instruments such as the surveys and faculty

reporting protocols in cooperation with the Project Director of the ECO LED Program at the CSU

Center for Community Engagement. The following section describes these evaluation

instruments in greater detail, with references to appendices, where appropriate.

Student Survey

Student surveys were designed to capture participating students’ attitudes and behavior

on topics such as community engagement, interest in STEM, and the like. Students were

administered a pretest survey in class by the faculty member at the beginning of their

respective courses. Survey items had a response set of a 5 point scale that ranged from 1 =

Strong Disagree to 5 = Strongly Agree and also included a Not Applicable or Don’t Know option.

The pretest survey contained 11 survey items to measure students’ community engagement (6

items) and interest in pursuing a STEM career (5 items). At the end of the course, students were

asked to complete the posttest survey containing the original 11 items in addition to questions

related to their experience in the course. Additional posttest items inquired about topics such

as specific attitudes and behaviors around service learning, how the course influenced their

CSU ECO LED Program Final Evaluation Report 10 Cobblestone Applied Research & Evaluation, Inc.

community involvement, and influence of the course on reaffirming or clarifying their future

major and/or career. Items for the student survey were modified from Gelmon, et al. (2001)

and additional items were created specifically to evaluate the impact and success of the ECO

LED Program.

Only responses from three of the four courses were able to be matched between the

student pretest and posttest data.2 In addition, overall descriptive data were obtained by

calculating means on the entire sample of non-matched pretest and posttest data. To analyze

these data, a factor analysis was conducted to test items for reliability and develop scales for

future studies (Fabrigar, Wegener, MacCallum & Strahan, 1999). The results of the factor

analysis indicate good, reliable survey items that can serve as the basis for future student

surveys in service-learning courses. In total, four factors were obtained from the student survey

data, including the following categories: Community Engagement; Interest in STEM Careers;

Course Influence on Community Engagement; and Course Influence on Degree and Career

Selection. Appendix A contains the results of the reliability analyses of the survey factors.

Faculty Written Reports

As part of the ECO LED Program sub-grants to faculty, recipients were required to

submit a final report describing the service-learning course activities. The guidelines for the

final report included a description of the course, identification of community partners, and

implementation of service-learning projects. In addition, faculty members were asked to

provide a written reflection of the successes, challenges, and impacts on the community and

students’ learning. Finally, the faculty members provided digital photos and video that

documented the interaction of their projects with their local community and participating

students.

Community Partners Survey

An online survey was created to ascertain feedback from community partners involved

with the ECO LED Program. Course faculty were asked to email the survey link to participating

community partners and responses were captured and analyzed by the evaluation team. The

survey measured the general satisfaction of the community partners in working with students

2 One course required students to complete the posttest survey online. While individual student responses were

obtained, matching with pretest scores was not possible.

CSU ECO LED Program Final Evaluation Report 11 Cobblestone Applied Research & Evaluation, Inc.

and the faculty, the influence of the partnership on the capacity of the community partners to

fulfill their organizations’ mission, any challenges that were experienced, any economic effects

that extended above and beyond what the organization would have normally done, and any

negative impacts to the organization. In total, four community partners responded to the

survey requests and hence these data were considered a limited view of community partners’

experiences. The community partners’ survey items can be found in Appendix B.

Key Evaluation Questions

The purpose of the evaluation was to answer key evaluation questions developed by the

ECO LED Program and the evaluation team to understand the effect of service-learning courses

on students, faculty and the community. Evaluation questions were related to each major

program area (i.e., Education, Environment and Community Building). Key evaluation questions

are listed in Table 1. Data sources that addressed each question are also listed in the table.

Table 1. Key Evaluation Questions and Corresponding Data Sources

Evaluation Question Data Sources

1. How were ECO LED Program activities implemented?

Faculty written reports Photographs Video

2. What was the effect of the ECO LED Program on students’ education in STEM?

Student surveys Faculty written reports

3. What was the effect of the ECO LED Program on the environment?

Faculty written reports Community partner surveys

4. What was the effect of the ECO LED Program on promoting community building?

Faculty written reports Student surveys Community partner surveys

5. What were the major successes and challenges of the ECO LED Program?

Faculty written reports Student surveys Community partner surveys

Participating Sample

A total of four service-learning courses were part of the ECO LED Program at three

southern California CSU campuses. Five faculty members (two co-taught one course) led the

courses. While only four separate community partners (from two courses) responded to the

online survey, a total of 15 community partners participated in one of the four courses. A total

of 54 students completed posttest surveys, while 35 of these surveys were matched with

pretest responses. A number of K-12 students and their teachers also participated in aspects of

CSU ECO LED Program Final Evaluation Report 12 Cobblestone Applied Research & Evaluation, Inc.

the program; however, no data were collected from these individuals as part of the ECO LED

Program evaluation.

Service-Learning Course Descriptions

The ECO LED Program was developed to support southern California CSU campuses in

service-learning courses. Courses were designed to promote SCE’s priority in “sustaining a

quality environment” in one of four areas:

Protect and restore the habitats of endangered and threatened plants and animals;

Provide environmental education to various communities, especially to youth and

families;

Address water quality and storm water management in ways that use electricity wisely;

and/or

Renewable energies.

In all, the funding provided support for four separate service-learning courses delivered

at three CSU campuses (Fresno, Dominguez Hills, and Channel Islands) in spring 2012. While the

original intent was to support eight separate service-learning courses throughout the CSU

system, the planning time required to develop courses conflicted with the funding award

timeline and therefore only four of eight possible projects were supported by program funds.

The following section provides more detailed information about the courses that were

supported in the ECO LED Program.

To answer the first evaluation question, the next section includes a description of the

ECO LED-supported service-learning courses from spring 2012. The majority of course

descriptions were generated from applications to the ECO LED Program as well as faculty final

reports, photographic evidence and videos.

Evaluation Question 1: How were ECO LED Program activities implemented?

CSU ECO LED Program Final Evaluation Report 13 Cobblestone Applied Research & Evaluation, Inc.

Course 1: Aquatic Ecology (BIOL 172S) at Fresno State

Aquatic Ecology at Fresno State was offered in

spring 2012 and taken primarily by graduating seniors.

The course focused on physical and chemical features

of inland waters as well as other elements such as

community structure and function, ecological

interactions, adaptations, and identification of aquatic

organisms. The course was also intended to foster

students’ career development in aquatic ecology.

Students worked with multiple community partners in

the course and therefore had a variety of projects

as part of their experience. For example, one group

worked with a community partner, Friends of Lost

Lake Park, to develop inventories of flora and fauna

in the park. Another group worked on Cottonwood

Creek with the San Joaquin River Stewardship

Program to discover how cattle-grazing affects the

quality of downstream recreational waters. Through the course, service-learning projects were

documented with videos and a Facebook page was established to highlight the projects.

Course 2: Mine Reclamation (EES 250T) at Fresno State

The Mine Reclamation course at Fresno

State was developed to educate students about

abandoned mines in the region and to

separately investigate erosion control

methods—a service-learning project that was

completed on the Fresno State campus. Course

activities included the installation of erosion

control matting, wattles and silt fencing in a

storm water retention basin on campus that was

CSU ECO LED Program Final Evaluation Report 14 Cobblestone Applied Research & Evaluation, Inc.

subject to erosion. This project provided the added ecological and water quality benefit of

minimizing suspended sediment through work on the storm drain. A second course component

was working with the community partner, the US Forest Service, to develop an erosion control

plan report after studying the abandoned Tip Top Mine in San Luis Obispo County. This course

was the first service-learning based course in the Earth & Environmental Science Department

on campus and included participation of twelve graduate students.

Course 3: Biogeography of Southern California (GEO 380) at CSU Dominguez Hills

The course offered at CSU Dominguez Hills,

Biogeography of Southern California, was designed

to educate students about regional Mediterranean

ecosystems. The Earth Science club on campus

designed a service-learning project to build a native

plant garden on the Dominguez Hills campus. Also,

students in the course assisted in restoring a coastal

sage scrub habitat with their community partner,

the Palos Verdes Land Conservancy. In addition to classroom components that included

research and presentations, the project also focused on community education through

participation in Earth Day events to provide information on topics such as water use,

sustainable gardening, native plants and insects through information signs, handouts and

conversations. The garden established through the course will continue as a permanent part of

the campus and additional projects with the community partner are already underway,

including efforts to assist the endangered Blue Butterfly at the Chandler Preserve.

Course 4: Water and Conflict in the West (UNIV 298) at CSU Channel Islands

Water Conflict in the West was offered as an interdisciplinary course at CSU Channel

Islands to provide students with a research-based and community-based experience related to

water restoration, watershed management and environmental education. Undergraduate

students mentored local middle school students in a variety of activities including data

collection, analysis and interpretation of water quality data related to pollution as well as the

facilitation of a water symposium for these students. Service-learning course students

CSU ECO LED Program Final Evaluation Report 15 Cobblestone Applied Research & Evaluation, Inc.

monitored water quality in the Callegues Creek watershed and worked with the community

partner Ventura Coastkeeper. Undergraduate students also learned key science concepts

related to water quality monitoring protocols. The course focused on critical thinking in real

world problems where field lessons for middle school students were reinforced by classroom

activities such as:

Building water impoundments on watershed models

Taste testing water from various sources

Water jeopardy (quiz show format)

Role playing to resolve water conflicts

In addition to these course activities, students also took a three day tour of the Colorado

River Aqueduct. Students in the course were also engaged in stream restoration by replanting

native habitat near the campus to improve nesting conditions for an endangered bird species.

Evaluation Results

The following section addresses the influence of ECO LED on education, environment,

and community building. The information used to report on the evaluation results was obtained

through student surveys, faculty written reports, and community partner surveys. Each of these

data sources were used to answer the remaining key evaluation questions.

Section 1: Education in STEM

An analysis of student survey data revealed two main themes that emerged related to

students’ experiences in the service-learning course related to their education in STEM:

specifically, STEM career interest and the impact of hands-on learning. These themes will be

discussed in further detail in the following section in which we describe how the ECO LED

Program influenced students’ learning and the potential for this learning to impact their future

education and career choices.

Evaluation Question 2: What was the effect of the ECO LED Program on students’

education in STEM?

CSU ECO LED Program Final Evaluation Report 16 Cobblestone Applied Research & Evaluation, Inc.

STEM Career Interest

Students’ STEM careers aspiration and influence was specifically used to measure the

success of the ECO LED Program given the long-term goal of increasing the number of CSU

students interested in, and successfully graduating from STEM disciplines. Of course, due to the

limited scope and timing of the program, this goal was not fully measured; however, many of

the evaluation results indicated that continued and increased use of service-learning courses

may lead to the achievement of this goal in the CSU system. Our findings indicate that students

who participate in these service-learning STEM courses are likely to pursue STEM degrees and

thus pursue STEM careers.

Interest in pursuing a career in the STEM disciplines was captured by two composite

items developed in the analysis of student survey results. The two composite factors include a)

Interest in STEM Careers; and b) Course Influence on Degree and Career (posttest only).

Students were administered a pretest and posttest survey containing several items measuring

their attitudes towards STEM careers (see Appendix A). The pretest-posttest survey method

was utilized by the evaluation team to measure if students’ attitudes changed over the course

of the service-learning experience. As a reminder, students responded on a scale of 1 = Strongly

Disagree to 5 = Strongly Agree, indicating that higher scores are generally more positive or have

higher agreement. Negative items were reverse-coded to correspond to the same scale.

Survey results indicated that for the first composite, Interest in STEM Careers, students

entered the course already highly interested in a STEM career (M = 4.1, SD = .87, n = 64) so

there was no overall growth from pretest to posttest (t(32) = -.282, ns). Considering students

who are initially interested in course topics such as these, this finding is not surprising. That is,

these courses were created for upper-level undergraduate students that were primarily STEM

majors with an established strong interest in STEM careers.

One group that might have demonstrated growth during the course, unfortunately, was

the one course for which matched pretest-posttest data were not available. For these students,

we were unable to measure the change in attitude scores from the beginning to the end of the

course. This course was a graduate-level course that included primarily political-science

students (non-STEM majors). However, given the available data, the pretest survey showed that

CSU ECO LED Program Final Evaluation Report 17 Cobblestone Applied Research & Evaluation, Inc.

the graduate students were neutral in the interest for a STEM career at pretest (M = 3.2, SD =

.66, n = 15) but made slight gains at the posttest survey (M = 3.4, SD = .68, n = 12) regarding

STEM career interest. Although this growth was minimal, with larger samples and matched data

it might have been possible to observe attitude changes over the course of the program. Figure

1 shows posttest mean ratings for individual questions included in the Interest in Pursuing a

STEM Career composite factor.

Figure 1. Student Posttest Mean Ratings of Interest in Pursuing a STEM Career

(R) = Reverse-coded item

Despite the lack of evidence to support the growth of interest in STEM careers during

the service-learning experience, students and faculty reported on the success of service

learning in confirming students’ interests in pursuing a STEM career. For example, one faculty

member mentioned, “It is interesting to see this degree of influence among the students.

Some are already set on a career path, others are not and are influenced by the experience.

But positive or negative, it’s rewarding to see that the [service-learning] experience at least

provided them with an idea of the workplace and tasks within this sub-discipline of Biology.”

Additionally, most students responded that they Agreed or Strongly Agreed with statements

that inquired about the degree to which the community work influenced or reaffirmed the

students’ choice of degree and profession. The composite variable, Course Influence on Degree

and Career yielded fairly positive results on the posttest (M = 4.1, SD = .64, n = 52). The

4.3 4.5

4.3

3.9 4.0

1

1.5

2

2.5

3

3.5

4

4.5

5

A career in the STEMfields would be

enjoyable.

STEM fields areinteresting to me.

I am confident that Ican succeed in a

STEM-based career.

I have alwayswanted a career in

the STEM fields.

I have no interest inpursuing a career inthe STEM fields.(R)

CSU ECO LED Program Final Evaluation Report 18 Cobblestone Applied Research & Evaluation, Inc.

following Table 2 indicates the overall mean scores for the three items that comprised this

score.

Table 2. Course Influence on Degree and Career Posttest Individual Item Means

Survey Item Mean (SD)

Performing work in the community helped me clarify or reaffirm which major I will pursue.

4.2 (.72)

The community work in this course assisted me in defining which profession I want to enter.

4.0 (.77)

The work I accomplished in this course has made me more marketable in my chosen profession when I graduate.

4.2 (.72)

Given the student survey results, the service-learning activities provided students with a

positive, hands-on learning experience that, according to students and faculty, influenced their

degree and career goals.

Hands-on Learning

While influencing students toward a STEM career may be a long-term impact of ECO

LED, more immediate impacts were suggested by the results of the student survey. Specifically,

the survey contains items that addressed attitudes of students regarding, for example, learning

the course material. Table 3 shows these specific survey items and their results. Most students

reported that the community service activities of the courses helped with applying course work

to everyday life and to better understand the lectures and course readings. Students had the

most favorable agreement to the idea that service learning be instituted more widely on their

campuses. This also speaks to how much students liked their service-learning courses

considering that this was the first time these courses had the service-learning component and

will, in all likelihood, improve given what faculty learned during the program. The faculty

members also reported their views on how service learning impacted student learning. One

faculty member wrote, “The activities of the course, were very much enhanced by the ECO LED

grant. The opportunity to conduct the erosion control project, in the words of one student,

‘made sure that I remembered how to design these sorts of things’… Students have

mentioned how they notice erosion control measures all around them now along highways

and in construction sites. The experience opened their eyes. When the time came for students

to design their own erosion control measures for the Tip Top Mine project, students reported

CSU ECO LED Program Final Evaluation Report 19 Cobblestone Applied Research & Evaluation, Inc.

that it was straightforward because they had already designed and installed one project

already.” This quote reinforces the view held by these faculty members regarding the

importance of students being able to work on location to put theory into practice.

Table 3. Influence of Service Learning on Student Learning Items Posttest Mean Scores

Survey Item Number of Students

Mean (SD)

The community participation aspect of this course helped me to see how the subject matter I learned can be used in everyday life.

53 4.3 (.61)

The community work I did through this course helped me to better understand the lectures and readings in this course.

53 4.3 (.78)

The idea of combining work in the community with university coursework should be practiced in more classes in the university.

53 4.5 (.58)

I feel I would have learned more from this course if more time was spent in the classroom instead of doing community work. (R)

52 3.65 (1.10)

(R) = Reverse-coded item

Education Summary

Both faculty and student data indicated that the ECO LED program had a positive impact

on student education. Students in these courses had, on average, a high interest in a STEM

career both before and following the course. Further, these service-learning courses influenced

students’ decision to continue working toward a STEM degree and eventually a STEM career.

Students and faculty also agreed on the benefit of service learning on helping students’ learning

through community involvement and hands-on projects and activities. Faculty reported that

students showed positive behaviors toward learning that were not found as often in traditional

courses.

Section 2: Environment

The second focus area of the ECO LED Program was to provide specific improvement in

the environment through service-learning students’ hands-on projects. The following section

Evaluation Question 3: What was the effect of the ECO LED Program on the environment?

CSU ECO LED Program Final Evaluation Report 20 Cobblestone Applied Research & Evaluation, Inc.

relates to Evaluation Question 3. Table 4 shows all of the environment-focused activities

students engaged in through their service-learning courses on each campus.

Table 4. Summary of ECO LED Program Activities

Campus Course Environment Activities

Channel Islands

Water and Conflict in the West

Engaged middle school students with environmental education possibly influencing the future academic and careers paths of students

Restored and protected the habitat of a local California endangered species, the Least Bell’s Vireo

Monitored, analyzed, and reported on water quality to the community partner leading to identified areas of concern where future efforts might be focused

Dominguez Hills

Biogeography of Southern California

Planned, prepared, and installed a 20’ by 20’ native plant garden on campus which continues to serve as an environmental education showplace for the community

Worked with the Palos Verdes Land Conservancy - students individually volunteered at restoration days, both clearing land and planting as well as volunteering in the Conservancy’s greenhouse

Fresno Aquatic Ecology

Developed inventories of flora and fauna in a park

Studied how the Western Pond Turtles’ upland migration could be linked to how weather and climate affect the turtles’ pond habitats

Investigated the effect of land use, specifically cattle grazing, on water quality downstream of recreational waters

Helped with a middle school class and wildlife club

Determined fish species that survived following the draining of a lake for dam repairs

Monitored habitat for a river salmon restoration project

Studied effects of net pens on salmonid survivorship and growth

Studied rainbow trout natural spawning habitat

Fresno Mine Reclamation

Developed a mine reclamation report for the US Forest Service to improve wildlife habitat, reduce environmental and safety hazards, and improve water quality in the Morro Creek, CA watershed

Conducted erosion control project on the campus through the installation of erosion control matting, wattles, and silt fencing

Students engaged in various projects through their service learning courses, but all

projects focused on improving the environment in some capacity. Examples of project goals

CSU ECO LED Program Final Evaluation Report 21 Cobblestone Applied Research & Evaluation, Inc.

included: habitat restoration and protection of endangered and threatened animals;

environmental education for youth; improving water quality; and developing a campus garden.

The local environments have already been impacted by these courses and their accompanying

service-learning projects.

Students at Fresno in the Mine Reclamation course reduced the need for repairs to the

campus storm water retention basin facility due to their installation of erosion control

matting, wattles, and silt fencing. This project showed an immediate impact by reducing

erosion of sediment into the storm water retention basins caused by winter storms in

2012. Additionally, students in the Mine Reclamation course produced a mine

reclamation plan for a local abandoned mine. This plan has the potential impact of

improving wildlife habitat, reducing environmental and safety hazards, and improving

water quality in the Morro Creek, CA watershed.

Students in the Water and Conflict in the West course at Channel Islands learned how to

properly test for water quality. They were able to analyze results and report findings to

local agencies as well as present possible solutions for water quality problem areas.

Additionally, these same students improved nesting habitat conditions for the Least

Bell’s Vireo.

Students in the Aquatic Ecology course, also at Fresno, worked with multiple community

partners and engaged in multiple activities that impacted their local environment. They

reported water quality levels in a nearby creek, determined if a local pond could serve

as a habitat for a particular species of turtle, and determined densities of juvenile trout,

which can lead to management decisions that will manage the habitat to foster

additional natural spawning of trout in a local river.

Students in the Biogeography of Southern California course planted a native, sustainable

garden that will be a permanent part of the campus beyond the time of the service-

learning course. A “mailbox” installed will hold information sheets about the garden’s

plants, the insects and animals they attract as a continued support for community

education. In addition, through work started with the community partner, Palos Verdes

Land Conservancy, additional projects started include a 7-month commitment to the

CSU ECO LED Program Final Evaluation Report 22 Cobblestone Applied Research & Evaluation, Inc.

PVPLC’s Palos Verdes Blue Butterfly restoration project. This project will allow students

to have additional impact on the environment by clearing and replanting a parcel of land

at the Chandler Preserve, to assist with the endangered Blue Butterfly in the area.

Environment Summary

The long-term impact on the environment is yet to be determined and will only be

known through future tracking and research. However, even with the small number of projects

supported by the current funding, these service-learning courses have already provided the

opportunity to have a positive impact on southern California communities to work toward

greater conservation and restoration efforts.

Section 3: Community Building

The next area of interest for the evaluation team addresses Evaluation Question 4.

Specifically, the team investigated how the efforts of faculty, students, and community partners

combined to address community needs and create opportunities for the future. Three general

themes of community education, student community engagement, and developing community

partnerships resulted from analyzing the faculty reports, student surveys, and community

partner surveys.

Community Education

The faculty report regarding the campus garden project at Dominguez Hills stated the

garden “is an environmental showcase for the campus community and also for the community

at large.” Each plant in the garden was labeled, providing an informative experience for

everyone who visited or will visit the garden. Additionally, the faculty report stated the

students “cannot wait to begin the restoration involved in re-introducing the endangered PV

Blue Butterfly to [Palos Verdes Land Conservancy].”

Students participating at Channel Islands engaged with middle school students at a

nearby school. They taught the young students water testing protocols and the reasons behind

Evaluation Question 4: What was the effect of the ECO LED Program on promoting

community building?

CSU ECO LED Program Final Evaluation Report 23 Cobblestone Applied Research & Evaluation, Inc.

systematic investigation of water quality condition. Following their meetings with students, a

math and science teacher at the middle school wrote:

“… Thank you so so so so so much for an AMAZING experience with you and

your students at CSUCI this afternoon. They had beyond a good time, and I am so

impressed at the level in which they engaged themselves with the activities and

learning. I truly appreciate the opportunity and greatly enjoy being a part of this with

the students.”

Those in the Aquatic Ecology course at Fresno engaged in multiple types of service-

learning projects. One student worked with a middle school class and wildlife club. According to

the faculty report, through many encounters with them she “had clear mutual benefits… this

greatly reinforced her love of science teaching, especially in the area of wildlife and natural

resources.” All of the projects at Fresno, however, benefitted not only the students’ learning,

but the community partners as well. A good example of this was the project with a local citizen

group called Friends of Lost Lake Park. The Fresno students were able to develop inventories of

the flora and fauna in the park, thereby providing a service for the park that could not be met

by the community in general.

The faculty member who taught the Mine Reclamation course at Fresno noted about his

students:

“In addition, students informally transmitted their newfound awareness, skills, and

knowledge to the wider community. Some talked about how excited their co-workers were to

discuss project ideas. Others discussed career plans with partners and friends. In the process,

awareness and understanding has increased in the wider community.”

As demonstrated by the above comments and data, the service-learning courses not

only had an impact on the environment but on the community partners and local communities

as well. This aligns to the mission of service learning, which is utilized to not only enrich the

learning experience but ultimately strengthen local communities.

Student Community Engagement

Additionally, the ECO LED Program aimed to affect student community engagement.

The results of the student survey showed students’ attitudes of community engagement

CSU ECO LED Program Final Evaluation Report 24 Cobblestone Applied Research & Evaluation, Inc.

effectively remained the same from pretest to posttest (t(34) = -.567, ns). Further analysis

showed that growth in students’ community engagement was unlikely considering, as a group,

students started out with a high sense of community responsibility. The mean pretest score for

the composite variable measuring Community Engagement was 4.3 (SD = .52, n = 68). These

results also suggest that students with high community engagement may have chosen these

service-learning courses over more traditional courses to begin with. Figure 2 shows the

posttest results of each of the items that make up the Community Engagement composite

variable.

Figure 2. Student Posttest Mean Ratings of Community Engagement

While student attitudes about community engagement remained high over the course,

the students reported that the service learning allowed them to gain additional knowledge on

how to increase community involvement and awareness. Student survey results regarding

students’ agreement to these survey items (i.e., Course Influence on Community Engagement)

were a mean score of 4.0 (SD = .64, n = 52). Thus, results indicate that service learning was able

to link the students’ desires to help with the resources to do so. Figure 3 shows mean ratings of

the individual items of the Course Influence on Community Engagement composite variable.

4.7 4.5

4.2 4.4 4.5

4.2

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Students my agecan do things tomake the world

better.

I can make adifference in myneighborhood or

town.

I feel responsiblefor helping

others.

I often thinkabout the needs

of others.

Helping to solvecommunityproblems issomething

everyone shoulddo.

I intend tovolunteer

throughout mywhole life.

CSU ECO LED Program Final Evaluation Report 25 Cobblestone Applied Research & Evaluation, Inc.

Figure 3. Posttest Student Mean Rating of Course Influence on Community Engagement

Developing Community Partnerships

Though only four partners responded to the community partner survey, the results were

very complimentary toward both the students and faculty with whom they worked. Three

respondents worked with students in the Aquatic Ecology class offered at Fresno. The

remaining respondent worked with students in the Mine Reclamation course, also offered at

Fresno.

Figure 4. Community Partners Satisfaction, Frequency of Survey Responses

3.8 4.0 4.0

4.2

1.0

2.0

3.0

4.0

5.0

The communityparticipation aspect ofthis course showed mehow I can become more

involved in mycommunity.

I feel that the communitywork I did through thiscourse benefited the

community.

The community workinvolved in this coursehelped me to become

more aware of the needsof my community.

I have a responsibility toserve my community.

0 1 2 3 4

Quality of interaction with students

Level of trust with students

Overall communication with students

Quality of student work

Input into planning of student service-learning activities

Quality of interaction with faculty

Level of trust with faculty

Overall communication with faculty

Very Satisfied Satisfied Neutral Dissatisfied Very Dissatisfied Not Applicable

CSU ECO LED Program Final Evaluation Report 26 Cobblestone Applied Research & Evaluation, Inc.

The first part of the survey asked the community partners to rate their level of

satisfaction with various components of their experience with students and faculty. Figure 4

shows the frequencies of responses to these items. Overall, responses tended to be either Very

Satisfied or Satisfied with only three Neutral responses.

The survey also addressed the best aspects of the overall experience. Community

partners’ responses included, “The student I was able to work with was very open to taking in

as many experiences as she could while assisting me in the classroom as well as on field trips.

Her excitement around my students was infectious.” and, “The best aspect for us was

establishing a working relationship between our volunteer organization at Lost Lake Park and

the staff and students at Fresno State University. By partnering with the university, we are

better able to advance one of our goals: protecting sensitive habitat in the park. And

students are able to get practical field experience.”

Specific questions were also asked regarding the partners’ experiences with the

university. Again, only one partner provided feedback in these specific areas, but this partner

indicated that working with the students from Fresno provided the partner with access to

university technology and expertise and it also identified additional volunteers. Furthermore,

working with this group provided the community partner with new connections/networks with

other community groups and increased its geographical area served.

Every community partner indicated they were either Likely (n = 1) or Highly Likely (n = 3)

to serve as a community partner for another service-learning course in the future. Overall,

community partner responses indicated a high level of satisfaction with their experience with

students and faculty in service-learning courses.

Additionally, faculty indicated on their final reports that these service-learning courses

supported the goal of developing community partnerships. For example, students in the

Dominguez Hills class showed continued enthusiasm for planting native plants at home and

informing family, friends, and other community members about the benefits of a native plants

garden. One faculty member from Fresno provided the evaluation team with an extensive list of

options for community partners to potentially work with on future service-learning projects.

CSU ECO LED Program Final Evaluation Report 27 Cobblestone Applied Research & Evaluation, Inc.

Consequently, it is evident that these types of courses are developing local community

partnerships.

Community Building Summary

The ECO LED Program was reported to have contributed to developing relationships and

attitudes that support community involvement. Specifically, multiple opportunities were

provided to students to educate the community regarding environmental issues and proactive

activities. Also, students who were already willing to help the community were given the

information and skills needed to engage in community service on their own. Lastly, the

community partners reported their satisfaction in working with the faculty and students and all

indicated that they would be willing to partner with faculty and students for a future service-

learning course.

Successes and Challenges of the ECO LED Program

In this last section of the report, we answer the final evaluation question related to the

successes and challenges of the ECO LED Program. Through the successes, the positive impacts

on student learning and influence on students’ degrees and future careers is explored. With an

examination of challenges encountered during the courses, faculty can learn from these and

work to create optimal experiences for students and the communities they serve in future

service-learning courses.

Successes of the ECO LED Program

It was clear that faculty, community partners, and most students felt that the ECO LED

Program was a positive experience. Beyond the programs’ impacts on education, environment,

and community that have been discussed in detail, other possible benefits and successes were

mentioned by faculty and community partners.

Unfortunately, due to the duration of the ECO LED Program and scope of the evaluation,

we will be unable to assess the long-range impact of this program on students graduating from

Evaluation Question 5: What were the major successes and challenges of the ECO LED

Program?

CSU ECO LED Program Final Evaluation Report 28 Cobblestone Applied Research & Evaluation, Inc.

STEM disciplines. However, the evidence that was gathered suggests that the STEM students

intend to move toward graduation in their chosen degrees and pursue STEM careers. In fact,

faculty reports suggested that some students showed interest in pursuing careers that were

directly related to the service-learning projects in which they participated. This suggests that

service learning can be used in higher education to provide students with experiences that both

reinforce classroom learning and theory and provide students with opportunities to utilize what

they have learned in possible workplace environments.

The faculty also reported that their students were benefited by service learning in the

following ways:

Students appreciated the chance to work on a professional level.

Students were interested in the course content.

Students showed high levels of motivation.

Students showed excitement with course content and activities.

Students showed high levels of creativity.

Students collaborated well with each other and the community partners.

Students voluntarily took on leadership roles.

Students worked with each other despite being from different academic backgrounds.

Students had improved problem-solving skills.

These added benefits to students’ education and course experiences detail the

additional benefits of service learning in general and the ECO LED Program specifically.

Challenges of the ECO LED Program

The challenges that were encountered were logistical in nature, and while challenges

did occur, they could be mostly avoidable for future course offerings. Also, as previously stated,

we hope that other faculty will be able to learn from these challenges to aid in the creation of

new service-learning courses or adding the service-learning component to existing courses.

Project challenges are provided in Table 5 along with possible solutions. Some solutions were

identified by faculty themselves in written reports of their projects, while other possible

solutions are offered by the evaluation team. While the list of possible solutions is not

CSU ECO LED Program Final Evaluation Report 29 Cobblestone Applied Research & Evaluation, Inc.

exhaustive, they demonstrate that additional planning and repeated execution of these courses

will allow smoother execution of future projects.

Table 5. Challenges and Possible Solutions for the ECO LED Program

Challenges Possible Solutions

Travelling long distances to the project worksite

Find a closer project site to allow for multiple visits or laboratory exercises

Keeping students’ workload manageable More and smaller projects; field trips earlier in the course

Time of course; e.g., an evening course limits the amount of daylight for outdoor projects

Determine if time of the course is consistent with the proposed projects

Difficulty coordinating a project with a community partner

Coordinate with partners before course begins; students should begin the process of finding an SL partner early in the course

Obtaining permission for the project if multiple agencies are involved

Anticipate and request permissions prior to course

Coordinating schedule with middle school teachers, obtaining parental consent, and ensuring curriculum continuity

Hire an education coordinator; create a checklist of school/ district requirements ahead of time

The topic covered in the course was too extensive to be covered in a one semester course

Create two related courses; limit activities for the course

Two community partners responded with areas they would have changed. These

included, “We would recommend setting up a protocol between Fresno State University and

the 4 agencies governing the park so that there is easier access to and greater use of the park

by university students.” , and “The only drawback is that the period of the work is short and

not timed for maximum efficiency. But that is only a shortcoming of the schedule and the

timing of the semester.” Again, these challenges were logistical and may be avoided in the

future.

Conclusions

Faculty, students, and community partners were able to accomplish much during the

scope of the four service-learning courses that comprised the ECO LED Program in spring 2012.

The positive effects of these courses on students’ education, the environment, and community

building were evident given the analysis of student surveys, faculty reports, photographic and

video evidence, and community partners’ surveys. Additional information is needed to better

CSU ECO LED Program Final Evaluation Report 30 Cobblestone Applied Research & Evaluation, Inc.

understand the relationship of how service-learning courses such as these might have longer-

term impacts such as increasing interest in STEM disciplines, graduation in STEM degrees, and

selection of STEM careers. Given that the courses that resulted from the faculty sub-grants

were designed for junior, senior, and graduate students, there was little evidence to support

that STEM service-learning courses created from this program could influence students to

pursue STEM degrees if they were not already interested in these topics or careers. We suggest

that lower level courses be added in the ECO LED Program model to determine if students may

be influenced to pursue a STEM discipline instead of a different path. Also, the results

presented in this report and apparent success of the ECO LED Program warrant the future

expansion and evaluation of the program to investigate further the relationship of service

learning to graduation in a STEM degree and pursuit of a STEM career.

CSU ECO LED Program Final Evaluation Report 31 Cobblestone Applied Research & Evaluation, Inc.

References

Fabrigar, L. R., Wegener, D. T., MacCallum, R.C., & Strahan, E.J. (1999). Evaluating the use of exploratory

factor analysis in psychological research. Psychological Methods, 4, 272-299.

Gelmon, S.B., Holland, B.A., Driscoll, A., Spring, A., & Kerrigan, S. (2001). Assessing service-learning and

civic engagement: Principles and techniques. Boston, MA: Campus Compact.

Partnership for 21st Century Skills (2012). Retrieved from http://www.p21.org/overview/skills-

framework.

Altman, I. (1996). Higher education and psychology in the millennium. American Psychologist, 51(4),

371–378.

Batchelder, T. H., & Root, S. (1994). Effects of an undergraduate program to integrate academic learning

and service: Cognitive, prosocial cognitive, and identity outcomes. Journal of Adolescence, 17,

341–355.

Bringle, R. G., & Hatcher, J. A. (1995). A service-learning curriculum for faculty. Michigan Journal of

Community Service Learning, 2, 112–122.

Driscoll, A., Holland, B., Gelmon, S., & Kerrigan, S. (1996). An assessment model for service-learning:

Comprehensive case studies of impact on faculty, students, community, and institutions.

Michigan Journal of Community Service Learning, 3, 66–71.

Giles, D., & Eyler, J. (1994). The impact of a college community service laboratory on students' personal,

social and cognitive outcomes. Journal of Adolescence 17, 327-339.

Markus, G. B., Howard, J. P. F., & King, D. C. (1993). Integrating community service and classroom

instruction enhances learning: Results from an experiment. Educational Evaluation and Policy

Analysis, 15, 410–419.

Reeb, R. N., Katsuyama, R. M., Sammon, J. A., & Yoder, D. S. (1998). The Community Service Self-Efficacy

Scale: Evidence of reliability, construct validity, and pragmatic utility. Michigan Journal of

Community Service Learning, 5, 48–57.

Roth, C. (1992). Environmental literacy: Its roots, evolution, and directions in the 1990s. ERIC

Clearinghouse for Science, Mathematics, and Environmental Education, Columbus, Ohio.

Teksoz, G., Elvan, S., & Tekkaya-Oztekin, C. (2012). Modeling environmental literacy of university

students. Journal of Science Education and Technology, 21, 157-166.

CSU ECO LED Program Final Evaluation Report 32 Cobblestone Applied Research & Evaluation, Inc.

Appendix A: Student Survey Factor Analysis

Extraction Method: Principal Axis Factoring Rotation Method: Oblimin with Kaiser Normalization Four Factors Extracted Factor 1: Community Engagement (Cronbach’s alpha = .807)

1. Students my age can do things to make the world better. (.696) 2. I can make a difference in my neighborhood or town. (.550) 3. I feel responsible for helping others. (.775) 4. I often think about the needs of others. (.707) 5. Helping to solve community problems is something everyone should do. (.607) 6. I intend to volunteer throughout my whole life. (.466)

Factor 2: Interest in STEM Careers (Cronbach’s alpha = .929)

1. A career in the STEM fields would be enjoyable. (.938) 2. STEM fields are interesting to me. (.870) 3. I am confident that I can succeed in a STEM-based career. (.580) 4. I have always wanted a career in the STEM fields. (.856) 5. Reversed- I have no interest in pursuing a career in the STEM fields. (.471)

Factor 3: Course Influence on Community Engagement (Cronbach’s alpha = .776)

1. The community participation aspect of this course showed me how I can become more involved in my community. (.685)

2. I feel that the community work I did through this course benefited the community. (.670)

3. The community work involved in this course helped me to become more aware of the needs of my community. (.581)

4. I have a responsibility to serve my community. (.255) Factor 4: Course Influence on Degree and Career (Cronbach’s alpha = .854)

1. Performing work in the community helped me clarify or reaffirm which major I will pursue. (.795)

2. The community work in this course assisted me in defining which profession I want to enter. (.844)

3. The work I accomplished in this course has made me more marketable in my chosen profession when I graduate. (.663)

CSU ECO LED Program Final Evaluation Report 33 Cobblestone Applied Research & Evaluation, Inc.

Appendix B: Community Partner Survey Items

Please rate your level of satisfaction in working with [campus] in the [course name] course in the following areas. Scale: 1 = very dissatisfied to 5 = very satisfied with option of not applicable

Quality of interaction with students

Level of trust with students

Overall communication with students

Quality of student work

Input into planning of student service-learning activities

Quality of interaction with faculty

Level of trust with faculty

Overall communication with faculty How did working with faculty and students influence your capacity to fulfill the mission of your organization? If you did not experience any notable influence, please select "No influence". However, if working with faculty and students did influence your capacity, please select all that apply from the following list.

No influence

New insights about the organization's operations

Increase in number of clients served

Enhanced offerings of services

Increased leverage of financial/other resources

Increase in geographical area served

New connections/networks with other community groups

Increase in environmental impact

Change in organizational direction

Increase in number of services offered

Other (please specify)

We would like to know if you experienced any major challenges while working with students and faculty during the course. If you did not experience any major challenges, please select "No major challenges encountered". However, if you did experience any major challenges, please select all that apply from the following list.

No major challenges encountered

Demands upon staff time

Project time period insufficient

Students not well prepared

Number of students inappropriate for the size of my organization

Mismatch between course goals and my organization

Little contact/interaction with faculty

Students did not perform as expected

Other (please specify)

CSU ECO LED Program Final Evaluation Report 34 Cobblestone Applied Research & Evaluation, Inc.

What were some of the economic effects of your work with [campus] above and beyond what your organization normally could have done? Please select all that apply.

No economic effects

Increased value of services

Increased organizational resources

Completion of project(s)

Access to university technology and expertise

New products, services, materials generated

Increased funding opportunities

Identification of new staff

Identification of additional volunteers

Other (please specify)

Were there any significant negative impacts to your organization in working with [campus]?

No negative impacts

Excessive use of staff time

Excessive use of resources

Other (please specify) What was the best aspect of this experience for you?

What aspects of this experience would you change? How likely would you be to serve as a community partner for this or another service-learning course in the future? Scale: 1 = highly unlikely to 5 = highly likely with option of not sure/don’t know