Embed Size (px)
Citation preview
pp, 164-188 lournal of Experiential Education • 2005, Volume 28, No, 2
Student and CommunityPerceptions of the "Value Added"for Service-Learners
Lori Simons and Beverly Cleary
An explanatory mixed-methods design was used to evaluate a service-learning model on academic learning, personal and interpersonal devel-opment, and community-engagement for 59 service-learning students. Arepeated-method ANOVA demonstrates that students improve their aca-demic learning and participation in service but reduce their interests insocial institutions, local politics, and communication with communityrecipients from the beginning to the end of the semester. In addition, a 2(placement site) x 3 (placement activity) MANOVA indicates that com-munity recipients evaluate tutors' attitudes and skills more favorably thansocial-recreational leaders. Content analyses of student reflections explainthe mixed quantitative findings and show how the "value added" fromparticipating in service-learning leads to students' interpersonal and per-sonal development.
Keywords: Service-Learning, Academic Learning, Personal and InterpersonalDevelopment, Gommunity Engagement
Dr, Lori Simons is an assistant professor of Psychology in the Social ScienceDivision of Widener University in Chester, PA. Her recent research has focusedon the effects of service-learning for community recipients and K-5 teachers,multicultural service-learning and student development, and interdisciplinaryservice-learning and civic engagement. She can be reached regarding questionsfor this article via e-mail at: lorgold@aol,com
Beverly Cleary graduated from Widener University with a B.A. degree in Psychologyand is currently attending graduate school at Villanova University, in theCommunity Counseling program. Beverly works as a case manager at the ChesterTowers for the elderly and disabled in Chester, Pennsylvania. She was Dr Simons'research assistant for three years and also took part in the service-learning.
2005, Volume 28, No. 2 165
Duriug the past decade, there has been a substantial increase in thenumber of investigations on service-learning (Eyler, 2000; Harkavy,Puckett, & Romer, 2000; Moore, 2000), Past research has evaluated
the impact of service-learning from one of two research methodologies.Quantitative methods have measured changes in students' academic-learning (Eyler, 2000; Vogelgesang & Astin, 2000), personal and interper-sonal development (Eyler & Giles, 1999; Eyler, Giles, Stenson, & Gray,2003; Moely, McFarland, Miron, Mercer, & Illustre, 2002), and civicengagement (Eyler et al,, 2003; Gallini & Moely, 2003), before and afterservice. In contrast, qualitative methods have identified the processesassociated with students' development of tolerance and reduced stereo-typing (Rockquemore & Shaffer, 2000; Root, Callahan, & Sepanski, 2002)and social, community and civic responsibility (Battistoni, 2002; Eyler etal., 2003; Peters, 2004), Much research has been written about the benefitsfrom participating in service-learning; however, Eyler (2000), and Moore(2000) suggest that this research is fairly rudimentary and rigorous meth-ods are required if the field of service-learning is to advance. The purposeof the present study is to add to the service-learning research by examin-ing community recipients' perceptions of service-learners, and evaluatingservice-learning effects on students' academic learning, personal andinterpersonal development, and community engagement with an explana-tory mixed-methods design.
This article is organized as follows: First, an academic service-learn-ing model is defined, and previous research on service-learning is reviewed.Then, an explanatory methods design is described in two sections, one forthe quantitative analysis and one for the qualitative analysis. Finally, thetwo sets of findings are synthesized through a single discussion.
An Academic Service-Learning ModelThere are numerous definitions of service-learning (Eyler & Giles,
1999; Howard, 2003; Stanton, Giles, & Cruz, 1999), However, academicservice-learning combines academic study with community service (Eyler& Giles, 1999), and in this article, the term "academic service-learning"refers to a pedagogical method that intentionally integrates learning withservice, and within this framework, service and learning goals are of equalweight, and each enhances the other for all participants (Fin-co, 2003;Sigmon, 1996), This pedagogical method requires students to apply theo-retical knowledge to "real world" situations, and at the same time, they
166 lournal of Experiential Education
connect the "service experience" to the course content through activities,assignments, reflections, discussions, and examinations (Ramaley, 2000;Vogelgesang & Astin, 2000).
Service-Learning Research
Service-learning practitioners agree that service-learning providesstudents with an opportunity to learn beyond the bounds of the traditionalclassroom (Enos & Trope, 1996; Harkavy et al., 2000). However, investiga-tions on service-learning and academic learning outcomes are mixed. Insome studies, academic learning outcomes are measured by studentreports of academic ability, while in other studies learning outcomes areassessed through student performances on examinations, assignments andcourse grades. For example, Fredericksen (2000) compared 154 service-learners to 545 nonservice-Iearners enrolled in an introductory Americangovernment course and found that service-learners outperformed nonser-vice-Iearners on the second exam and course grades, but not on the firstexam. Similarly, Strage (2000) found that service-learning students earnedmore points on midterm and final exams but not on the first exam.Vogelgesang and Astin (2000) also found service-learning students hadbetter writing skills and higher grade point averages (GPAs) than nonser-vice-Iearners after conducting a national longitudinal study with 22,236undergraduate students enrolled at 177 institutions. In addition, after sys-tematically analyzing reflection papers for 60 service-learning students,Litke (2002) found that low and high achieving students self-reported hav-ing a better ability to apply and conceptualize course concepts because ofthe service experience. Ash, Glayton, and Atkinson (2005) assessed reflec-tion essays with a random sample of students eruroUed in academic, civicand personal articulated-learning courses and found that 74% of service-learners improve their application skills, but only 7% of them improvetheir analytical skills from the first to the final draft of the essay. Eyler andGiles (1999) found that more than half of all surveyed students reportedlearning more in service-learning courses.
Several studies suggest that service-learning has a positive effect onstudents' personal and interpersonal development (Eyler & Giles, 1999;Moely, McFarland, et al., 2002). In fact, service-learners often report gain-ing self-awareness and developing leadership skills (Ash et al,, 2005).However, research methods that assessed student development havevaried from self-report surveys to reflection papers and resulted in littleconsistency of outcomes. For example, Moely, McFarland, et al, conducteda pre-post test study on civic action, problem-solving, political awareness,leadership skills, social justice attitudes and diversity attitudes between217 service-learners and 324 nonservice-Iearners enrolled in the Gollege ofArts and Sciences. These researchers found that students did not differ in
2005, Volume 28, No, 2 167
scores at the beginning of the semester, but by the end of the semester serv-ice-learners were higher on civic action, social justice attitudes, leadershipskills and problem-solving skills than nonservice-Iearners. Similarly,Green (2001) found that service-learning students develop a greater level ofself-efficacy after conducting a qualitative analysis on 14 response papersthat describe students' experience of tutoring children in an urban andimpoverished neighborhood. In contrast, Vogelgesang and Astin (2000)found that students' self-perceived growth, interpersonal skills and leader-ship skills were not benefits from participating in service-learning. Boyle-Baise and Langford (2004) conducted a case study with eight studentsemolled in a social justice seminar and observed that service-learnerschange their social justice views from the beginning to the end of the serv-ice, although their views of poverty, race and activism were mixed. Boyle-Baise and Kilbane (2000) also conducted an ethnographic study with 25pre-service teaching students and found that teachers' links to understand-ing of families, and community strengths and problems, were weak.
Impact of Participation in Service-Learning Projects
Due to the increased interest in personal and interpersonal out-comes, a number of studies have examined the impact of participation inservice-learning projects on students' connections to community recipi-ents and their appreciation for other cultures. In spite of the growing num-ber of research studies in this area, there is still insufficient evidence aboutservice-learning's influence on diversity attitudes and multicultural com-petencies. For example, the process and the context of attitude-formationhas been investigated by Boyle-Baise and Kilbane (2000), who found thatstudents changed their attitudes toward themselves and the communityafter a semester of tutoring neighborhood children. Green (2001) alsofound that students modified their prejudicial attitudes after interactingwith culturally different community recipients and reflections onthoughts about race, class and economic injustices, Brody and Wright(2004), and Boyle-Baise (2002) suggest that service-learning provides stu-dents with an opportunity to develop relationships with communitymembers that they might not otherwise meet in their everyday lives, andthese interactions foster students' diversity attitudes and cultural aware-ness. On the other hand, Erikson and O'Gormor (cited in O'Grady, 2000)suggest that students' contact with community recipients that does notcontradict their cognitive biases will lead to reinforcing stereotypical atti-tudes and beliefs. Moely, Mercer, Ilustre, Miron, and McFarland (2002) didnot find differences in diversity attitudes between service-learning andnonservice-learning students. Root and colleagues (2002) found signifi-cant gains in only two out of four items inquiring about diversity for serv-ice-learning students.
168 Journal of Experiential Education
Many researchers suggest that service-learning is a prerequisite forcivic engagement (Boyle-Baise, 2002; Eyler & Giles, 1999; Wang & Jackson,2005). In an attempt to understand how service-learning fosters civicengagement, several researchers have evaluated students' communityservice. For instance, Payne (2000) conducted a pre-post study on explo-ration, affiliation, experimentation and assimilation preferences for com-munity involvement for 83 students enrolled in four sections of a service-learning course. In this study, students changed their exploration andassimilation preferences for community service by the end of the semester.Service-learners reduced their apprehension levels for (i,e,, explorationpreference), and increased their lifelong commitments to (i.e,, assimilationpreference), community service. Similarly, Gallini and Moely (2003)assessed community engagement, academic engagement and interperson-al engagement for 142 service-learning, and 71 nonservice-learning stu-dents at the end of the semester, and found that service-learners reportedgreater levels of engagement than nonservice-Iearners, In addition, Wangand Jackson (2005) assessed students' perceptions of their academicachievement, personal and interpersonal development, and civic involve-ment for 415 service-learning students and found that service-learnersrated their skills for civic involvement higher at post-course than at pre-course. Rockquemore and Schaffer (2000) also evaluated the processesthat occur while students are engaged in service-learning by analyzingjournals and surveys for 120 students enrolled in service-learning courses.These researchers found that 43% of all service-learners feU that they hada positive impact on local social problems. Reinke (2003) found that 72%of sin-veyed students reported that community involvement was impor-tant, and 63% of them feel that their community involvement improvedother people's lives, Eyler and Giles (1999) propose that participation inservice-learning leads to the values, knowledge, skills, efficacy, and com-mitment that underlie effective citizenship.
Assessment Methods of Service-Learning
It is important to recognize that there is a trade-off between externaland internal validity in assessment methods of service-learning (Wang,Kelly, & Hritsuk, 2003), For instance, pre-post studies neglect to measure"changes" that occurs while the students are engaged in service-learning,and these changes may provide insight into why some students benefitfrom service more than others. On the other hand, qualitative studies havebeen criticized for external validity issues of reliability and generalizabil-ity, and in some ways this criticism has been well-founded (Eyler, 2000;Moore, 2000; Wang et al., 2003), We have addressed these concerns aboutqualitative methods by using reflection questions to explain the missing
2005, Volume 28, No, 2 169
links in students' attitudes and behaviors before, during, and after service-learning. The current study adds to the service-learning research by eval-uating an academic service-learning model on students' academic-learn-ing, personal and interpersonal development, and community engage-ment using an explanatory mixed-methods design. Specifically, quantita-tive methods are used to test the following four hypotheses:
Hj Service-learners will improve their scores on writing assign-ments, examinations, and course grades from the beginning tothe end of the semester.
H2 Service-learners will improve their civic action, interpersonaland problem-solving skills, political awareness, leadershipskills, social justice attitudes, diversity attitudes, course values,and academic and field interests (i.e., GASQ scores) from thebeginning to the course end.
H3 Service-learners will decrease apprehension levels and increaselifestyle commitments to service (i,e,, GSIPI scores) from thebeginning to the end of the semester.
H4 School teachers will evaluate tutors more favorably than men-tors and social-recreational leaders compared to after-schoolteachers.
Qualitative methods are used to detect major themes of service-learningguided by two major research questions: "What" and "how" is the "valueadded" by participating in service-learning? The term "value added"refers to student learning that could not be achieved without participatingin service-learning (Eyler & Giles, 1999).
Method
Participants
A total of 59 undergraduate students enrolled in two sections of aneducational psychology course at a private teaching university in an east-ern metropolitan area during the spring of 2003 served as the sample. Asshown in Table 1, the majority of participants were Gaucasian (80%),female (85%), and either psychology or education majors (83%), and theaverage age was 20 years (SD = 8.03), Less than one-third of all partici-pants had volunteer experience (28%) prior to the course, and all of them(100%) self-selected to participate in service-learning. Service was carriedout at two different community sites, including a public elementaryschool and an after-school program.
170 lournal of Experiential Education
Table 1Demographic Characteristics of Participants
Characteristics n %
CenderMale 8 15Female 47 85
EthnicityGaucasian 47 80African-American 7 12Hispanic 5 8
MajorPsychology 15 25Education 31 52Arts and Sciences 8 13Nursing 4 7Business 1 2
Vear JD SchoolFreshman 36 61Sophomore 6 10lunior 7 12Senior 6 10Transfer 4 5
Previous VolunteerExperience 17 29
Service-Learning PlacementElementary School 43 73After-School Program 16 27
Service-Learning ActivityTutor 30 50Mentor 14 24Social-Recreational Leader 15 25
Service-Learning Gontinued 19 32Service-Learning in Future 40 67Note, n = 59,
Measures
Demographic questionnaire. A self-report survey was used to gatherdescriptive information on students' gender, race, age, year in school andvolunteer experience. Civic Attitudes and Skills Questionnaire (GASQ),
2005, Volume 28, No, 2 171
developed by Moely, Mercer, et al., (2002), was used to gather students'self-evaluations on civic attitudes and skills. The CASQ is an 84-item, self-report questionnaire that yields scores on six scales:
1. Civic Action—respondents evaluate their intentions to becomeinvolved in the future in some community service or action.
2. Interpersonal and Problem-Solving Skills—respondents evalu-ate their abilities to listen, work cooperatively, communicate,make friends, take the role of the other, think logically andanalytically, and solve problems,
3. Political Awareness—respondents evaluate themselves onitems concerning awareness of local and national events andpolitical issues,
4. Leadership skills—respondents evaluate their abilities to leadand effectiveness as a leader,
5. Social Justice Attitudes—respondents rate their agreement withitems expressing attitudes concerning the causes of povertyand misfortune and how social problems can be solved.
6. Diversity Attitudes—respondents describe their attitudestoward diversity and their interests in relating to culturally dif-ferent people.
Items are presented as statements and respondents express theirlevel of agreement with each statement by marking a scale ranging from 1(strongly disagree] to 5 (strongly agree). Internal consistencies for eachscale range from ,69 to ,88, and test-retest reliabilities for each scale rangefrom .56 to .81 (Moely, Mercer, et al,, 2002).
The CASQ also measrures students' views of their courses on fourcourse satisfaction scales including:
1. Course Value—respondents evaluate how important or usefulmaterial covered in the academic course had been,
2. Learning about Academic Field—respondents evaluate the con-tent of his/her academic coiu-se such as understanding and appli-cation of the course concepts, interest in the field, and an under-standing a professional's role in the field of study.
3. Learning about the Co/ninuuity—respondents evaluate how muchthey had learned about the community, different cultures, workingwith others effectively and seeing social problems in a new way.
4. Contribution to the Community—respondents indicate their per-ceptions of how useful their service activities had been in thecommunity.
Journal of Experiential Education
Items are presented as statements and respondents endorse theirlevel of agreement with each statement by circling a score on a scale rang-ing from 1 (strongly disagree) to 5 (strongly agree). Internal consistenciesfor each scale range from ,81 to .82 (Moely, McFarland, et al,, 2002).
Community Service Involvement Preference Inventory (CSIPI),developed by Payne (2000), was used to assess how students becomeinvolved in community service. The CSIPI is a 48-item paper and pencilinventory designed to assess four preferences:
1, Exploration Involvement Preference—this score reflects theapprehension common in new experiences: commitment is short-term and is usually at the convenience of the helper,
2, Affiliation Involvement Preference—this score reflects that behav-ior motivation for community service and commitments tend tobe infrequent and shorter in duration,
3, Experimentation Involvement Preference—this score reflects thedesire to make a difference in the lives of others and to learn moreabout the community.
4, Assimilation Involvement Preference—this score reflects cogni-tive processes with career and lifestyle decisions based on theservice experience as a way to understand what it means to be aresponsible citizen.
Items are presented as statements and respondents indicate theirlevel of agreement with each statement by circling a score on a five-pointLikert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Scoresfor each involvement preference range from 12 to 60, and the total scorefor each preference serves as an indicator for how the student prefers tobecome involved in community service! Internal consistencies forExploration (r = ,63), Affihation (r = .70), Experimentation (r = ,74), andAssimilation (r = ,70) preferences are modest.
Community service-learning evaluation. The researcher developeda 52-item questionnaire to gather information on students' performancesat the placement site. Specifically, items inquired about the students' atti-tudes, behaviors and skills. Items are presented as statements and teach-ers/supervisors ranked the students' performances by circling a score on afive-point rating scale ranging from 1 (unsatisfactory) to 5 (superior). Inter-item correlations range from .26 to ,93, and Cronbach's alpha for scaleitems (a - ,98) is high.
Learning outcomes. These outcomes were assessed using students'scores on examinations (n ^ 3), essays (n = 4), and midterm and finalcourse grades.
2005, Volume 28, No, 2 173
Reflection questions. Eight open-ended reflection questions weredeveloped by the researcher to inquire about students' service-learningexperiences. The reflection questions were as follows:
1. Explain how service-learning helped you with your career decisions,
2. Explain how service-learning helped you understand or apply thecourse content,
3, Identify the strengths and weaknesses associated with participat-ing in service-learning,
4, Explain how service-learning helped the placement site.
5, Identify the similarities between you and the children/teachers,
6, Identify the differences between you and the children/teachers,
7. Describe how your beliefs, attitudes, views and feelings changedthroughout the semester,
8. Describe what you gained from participating in service-learning.
Course Content
In-class time (50 minutes, 3 times per week) for service-learning stu-dents began with a lecture on service-learning. The next two classes con-sisted of a two-hour orientation on mentoring and tutoring by guest speakersrepresenting one of two placement sites. The guest speakers explained tothe students that they would work with groups of four children for 16hours at either an elementary school (grades K - 4) or an after-school pro-gram (grades K - 8), Then students were trained in one of following threeroles: (a) Mentors were trained to read to children using a district-approved curriculum that consisted of structured and non-structuredactivities; (b) Tutors were trained on a district-approved tutorial curricu-lum that consisted of semi-structured reading and math exercises; and (c)Social-Recreational Leaders were trained to expose the children to struc-tured activities using the physical education curriculum. The rest of thecourse was devoted to lecture, activities and discussion. Each section ofthe course had the same number of examinations (3), reflection questions(8), and written assignments (4),
Design and Procedure
An explanatory mixed-method design was used to collect bothquantitative and qualitative data before, during, and after service for 59students enrolled in an educational psychology course in the spring of2003, An explanatory mixed-methods design refers to a sequential phaseof quantitative and qualitative data collection, and an integrative analysiswhere quantitative findings provide a context for the qualitative results
174 lournal of Experiential Education
(Creswell, 1994, 2005). Several researchers have advocated for the use ofmultiple data sources (i.e., student and recipients) and the combination ofquantitative methods with qualitative techniques to address the concernsabout the purported benefits for service-learners (Eyler, 2000; Eyler &Giles, 1999; Moore, 2000). Quantitative data and results provide a generalpicture of the benefits for service-learners from students and communityrecipients, and the qualitative data and results refine participant out-comes. Students volunteered to participate in a study of service-learningand completed an informed consent form with a survey packet during thefirst week of the course. Then the students were re-administered the sur-vey packet after they began service, and then again after they completedservice (i.e., 16 hours). Students completed each survey at their own paceand placed it in a coded, confidential envelope and gave it directly to theresearcher. Each survey took about 45 minutes for the students to com-plete. Students also completed eight, open-ended, reflection questions(post-service) that generated two types of data: (a) A report of their emo-tional and cognitive reactions to the events they encountered, and (b) anevaluation of the "value added" by participating in service-learning. Inaddition, the researcher administered a survey to the teachers at the place-ment site. Teachers evaluated students' attitudes, behaviors and skills, andreturned the survey in a coded envelope to researcher. The teacherresponse rate was 42%, which is consistent with other survey responserates (Heppner, Kivlighan, & Wampold, 1999).
To analyze qualitative data, the constant comparative method(Creswell, 1994, 1998) was used to identify common themes that describe"what" and "how" students gain from participating in service-learning.Two coders read the open-ended responses and identified the units ofanalysis. The units of analysis were the isolated thoughts expressed bystudents, and open coding consisted of naming and categorizing the datathat describe the "added value" by participating in service-learning, andresulted in four major themes:
1. Service-learning fosters identification with community recipients.
2. Service-learning enhances academic-learning,
3. Service-learning enhances career decisions.
4. Service-learning shapes students' perceptions toward self and thecommunity (see Table 2).
After the open coding process was complete, the coders counted the num-ber of responses according to each category and divided the total number ofresponses by the number of participants in order to rank the major themes.
Once the four major themes were identified, an axial coding procedurewas conducted to explain how students learned while engaged in service-learning. The axial coding process consisted of the coders re-reading
2005, Volume 28, No, 2 175
Table 2Majar Themes of Academic Service-Learning, ParticipantReflections, Percentages of Participant Responses per Theme,and Axial and Selective Coding
Themes
1, Connection tothe community
2, AcademicLearning
Reflections
I was the only whitestudent but feltcomfortable andlearned that every-one can get alongregardless of race
This was my firstexperience in anAfrican AmericanCommunity
My leaming was sodifferent -1 learnedto appreciate what Ihad in terms ofresources
Socioeconomic/Cultural Differences
I felt as if I was in adifferent world
I learned to handlesituations different-ly—skills
I wanted to be there
The teachers gen-uinely cared for thestudents
Application ofterms, concepts andtheories
Application of real-life examples to thecoin-se
I didn't see a corre-lation between serv-ice leaming and thecourse
Service learninghelped me under-stand the courseconcepts
%
20
40
3
7
15
5
30
35
15
10
Axial Coding
Social
Personal & Social
Social
Personal
Personal
Social
Personal
Social
Personal
Personal
Selective Coding
Development of tol-erant attitudes& reduced stereo-typing
Orientations towardthe community
Self-identity
Reducedstereotyping
Tolerant attitudesStereotyping
Self-efficacy
Interpersonal
Transformedorientations
Understanding ofthe course
Application ability
Application ability
Application ability
176 lournal of Experiential Education
Table 2(continued)
Themes
3, CareerDevelopment
4, Transformationof students'perceptions ofthemselves andthe community
Reflections
Service learningconfirmed mychoice
Service learningchanged my mind—I am consideringbecoming a teacher
I do not want to bea teacher but I wantto work withchildren
Service learning didnot belp me withmy career choices
Service learninghelped me appreci-ate or understandwhat is involved inteaching
Service learningopened my eyes—I enjoyed helpingstudents
Empathy,,,I felt badthat the childrendid not have theprivileges that I did
I learned to under-stand the differ-ences betweenmyself and the chil-dren and toembrace these dif-ferences
It made me sad toleave the chil-dren,,,! loved thewhole experience
%
56
12
13
5
3
7
23
Axial Coding
Personal
Personal
Personal
Personal
Personal
Personal
Personal
PersonalSocial
Personal
Selective Coding
Career Decisions
Career Decisions
Career Decisions
Career Decisions
Self-efficacy
Self-identity
Orientations towardthe communityOrientation—self.Self-identity
Interpersonal andSelf-efficacy
2005, Volume 28, No, 2 177
Table 2(continued)
Themes
4, Transformationof students'perceptions ofthemselves andthe community(con't)
Reflections
My perception ofChester Schools as adangerous placechanged,,,there is alot of caring andconcern for thechildren
Minimal differencesbetween children inurban and suburbanschools ,,, they allwant to learn
My whole outlookchanged,,,I wasnervous and when Igot there I loved it
The first day Istepped foot in thedoor I knew I wasgoing to be there formore than 16 hours
I learned that teacb-ers have to genuine-ly care about theirstudents and tbiswas a view that def-initely changedthrough my serviceexperience
I was surprised byhow many parentsare involved in thescbool
This experienceconfirmed myperceptions
I realized how diffi-cult it is to be ateacher
I experienced nochange
I built relationshipswith the teachersand children—thecommunity
%
15
14
10
7
7
7
5
5
Axial Coding
Social &Personal
Personal &Social
Social
Personal &Social
Personal &Social
Personal
Personal
Personal
Selective Coding
Tolerant attitudesand reduced stereo-typing
Orientations towardthe community andself
Orientations towardself & community
Citizenship
Reducedstereo-typingOrientations—community
Orientations towardthe community.Reducedstereotyping
Orientation—communityReducedstereo-typing
Interpersonal
Lack of self-efficacy
Interpersonal
178 lournal of Experiential Education
students' reflections to each theme and categorizing their reflections aseither a personal or a social learning process. Data were coded as a "per-sonal process" when a student's reflection illustrated a deeper under-standing of oneself, and a "social process" was coded when a student'sreflection indicated attitudinal changes toward community recipients.After the axial coding was complete, the coders re-read the students'reflections and categorized them according to the framework of Eyler andGiles (1999). The reason for selective coding was to replicate previousresearch, which resulted in six major themes:
1, Career development.
2. Self-Efficacy.
3. Application Ability,
4. Reduced Stereotyping and Development of Tolerant Attitudes.
5, Interpersonal Skills.
6, Transformed Attitudes toward the Community,
Qualitative data served as a reliahility check for quantitative data by com-paring the dominant themes to students' surveys, and the discussion sec-tion addresses the discrepancies between qualitative and quantitativeresults.
Results
Academic Leaming
A repeated-measures ANOVA was used to assess differences inexams, essays and course grades. A Bonferroni pairwise comparisonshowed that service-learners had higher scores on exam two (x - 78,72)than on exam one (x = 70.21) and exam three (x = 73.42), F (l, 56) = 13,40,p < .05), Students improved their scores from the flrst to the second exam,but they did not improve their scores from the second to the third exam.Service-learners did not demonstrate improvements in writing assign-ments and course grades from the beginning to the end of the semester.
Personal and Interpersonal Development and Community Engagement
Using a repeated-method of self-report surveys, students' attitudes,skills and behaviors were observed at three points in time, A repeated-measvireANOVA was used to assess differences in the CASQ and CSIPI scores.Students made significant changes in course satisfaction, politicalawareness, social justice attitudes, problem-solving skills, and experimen-tation, exploration and affiliation preferences for community involvementas shovim in Table 3. A Bonferroni pairwise comparison showed expected
2005, Volume 28, No, 2 179
Tabte 3Significant With-In Suhject Effects for CASQ and CSIPI Scores farService-Learning Students
Source
CASQ Scores
Civic Action
Political Awareness
Diversity AttitudesSocial lustice
Problem-Solving
Leadership
Course Value
CSIPI Scores
Exploration
Experimentation
Assimilation
Affiliation
df
2
2
2
2
2
2
2
2
2
2
2
SS
19,05
349,183,62
353,4311840,77
40,05
253,85
459,06
294,45
112,05
1180,29
MS
9,52
174,59
1,81176,71
920,3820,03
126,92
229,53
147,22
56,03
590,14
F
,69
32,63***
,68
36,06***
40,31***4,27*
9,07***
20,83***
7,43***
3,05
38,25***
Note. ***p < ,001, **p < ,01, *p < ,05,
changes in stndents' course satisfaction and preferences for communityinvolvement, as well as unexpected changes in their political awareness,social justice attitudes and prohlem-solving skills as shown in Tahle 4,These findings partially support the second and third hypotheses.
Using a 2 (Type of Placement Site) x 3 (Placement Site Activity)MANOVA on community evaluation questions, teachers' perceptionswere observed for students' skills, attitudes and hehaviors. There were nosignificant interaction effects or main effects for elementary school andafter-school teachers. However, univariate analyses indicate that teachersrate skills, attitudes and behaviors differently for tutors, mentors andsocial-recreational leaders. A Tukey post-hoc comparison showed thatteachers rate tutors more favorably than leaders as shown in Table 5, andpartially support the fourth hypothesis.
The "Va7ue Added" hy Participating in Academic Service-Learning
Eyler and Giles (1999) suggest students' connections to the commu-nity foster civic involvement, and almost all (88%) of the coded datadescribe students' relationships with community recipients as a primarybenefit from participating in service-learning. One student commented:
180 lournal of Experiential Education
My views of inner-city children and their famihes changed after interacting
with their parents and realizing how involved they are with their children's
education and this motivated me to continue to work with the children after
my required hours were complete.
Another student commented, "My perception of Chester changed ,,. I builtrelationships with the teachers and the children and now I feel like I amapart of the community."
Eyler and Giles (1999) suggest that service-learning contributes tostudents' comprehension of course material, and 75% of the coded datareveal academic-learning as a secondary benefit from participating in serv-ice-learning. For example, one student commented, "I was able to applythe course material to real-life situations and this helped me understandthe content," Another student noted, "The material is difficult and thereis no way I could have comprehended it without real-life examples,"
Eyler and Giles (1999) suggest that service-learning allows studentsto identify with community recipients, discover that they have things incommon, and reduce personal biases, and more than half (72%) of thecoded data illustrate that students changed their stereotypes about otherswho are culturally different as a third service-learning outcome. One stu-dent wrote, "I assumed that poor people on welfare are just lazy and don'twant to work, don't care about their children, and expect everyone else totake care of their problems," and after this experience, "I realized howmuch parents are involved with their children and the school, and thattheir level of concern is no different from my own parents." He furtherwrote that, "I never considered working in an urban school until now." Afinal student's comments speak to changing her preconceived notions: "Iwas the only White person in the community placement but felt comfort-able and learned that everyone can get along regardless of their race,"
Eyler and Giles (1999) suggest that service-learning contributes togreater self-knowledge, personal self-efficacy and career development,and 56% of the coded data illustrates career development as a fourth andfinal benefit from participating in service-learning. For example, one stu-dent wrote, "Service-learning opened my eyes to the problems encoun-tered by teachers in an urban and impoverished school and I am more con-fident in my decision to become a teacher." Another student noted,"Service-learning confirmed my career choice .,,1 realized that I want towork with children but not as a teacher after this experience." A finalcomment illustrates how service-learning influences students' careerdevelopment: "After I met the children I realized how privileged I amdespite my own hardships; though I never considered teaching as a careerI would rethink this option as a result from this experience,"
In conclusion, there were four major themes that explained the
2005, Volume 28, No, 2 181
"value added" by participating in service-learning for educational psy-chology students. High frequencies were found for students' connectionsto community recipients, while moderate frequencies were found for stu-dents' academic-learning and transformed perceptions of the community.Students' career development was found as another benefit from partici-pating in service-learning, but it was cited less often.
DiscussionThere has been a tremendous amount of research on the impact of
service-learning, but despite these developments, few studies have evalu-ated service-learning with rigorous methods. This is one of the first stud-ies to use an explanatory mixed-methods design to demonstrate thatparticipation in service-learning affects academic-learning, personal andinterpersonal development, and community engagement. Incorporating arepeated-measures method, quantitative and qualitative data, and recipi-ent surveys allowed us to test for differences over time and assured us theconsistency of findings beyond that possible if only one method or datasource had been considered.
Impact of Service-Learning on Academic Learning
Research on the impact of service-learning and academic learning ismixed, in part, because of the various types of conceptualizations andmeasurements used. Service-learners earned better grades on the secondexam than on the first and third exam; however, the fact that they did notdemonstrate better writing skills or sbow improvements in course gradespartially supports the first hypothesis and is consistent with previousresearch (Strange, 2000; Vogelesgang & Austin, 2000). In addition, morethan half of students described a deeper understanding of theoretical con-cepts in their responses to reflection questions. Inconsistent findings foracademic learning between qualitative and quantitative findings may beattributed to the different assessment methods that were used to measureacademic learning, and academic learning may be better assessed withqualitative methods. This is an avenue that demands further research.
Personal and Interpersonal Development
Findings of changes in students' personal and interpersonal devel-opment partially supports the second hypothesis and are incongruentwith previous research (Moely, McFarland, et al., 2002; Payne, 2000;Reinke, 2003) that found service-learners increase their political aware-ness, social justice attitudes, and problem-solving skills from the begin-ning to the end of the semester. Service-learners' interests in learningabout local and national politics, understandings of the impact of socialinstitutions on the individual, and abilities to work with others decreased
182 Journal of Experiential Education
Table 4Means and Standard Deviations for CASQ and CSIPI Scores as aFunction of Time
Variable
CASQ Scores
Civic Action
Political Awareness
Diversity
Social Justice
Problem Solving
Leadership
Course Value
CSIPI Scores
Experimentation
Exploration
Assimilation
Affiliation
Time
M
26.43
17.37
14.11
28.31
40.80
14.51
34.86
44.95
34.40
40.60
36.74
1 (1)SD
3.25
3.46
2.79
3.66
4,78
2.92
4391
5.44
4.01
6.97
6.62
Time 2
M
26.47
17.47
13.98
28,37
32,23
13.17
36.90
45.75
37.61
41.88
39.18
(2)
SD
5.86
3.96
2.24
2.74
7.30
1.88
4.58
4.94
4.43
5.10
6.59
Time
M
25.68
14.12
13.74
25.02
39.78
13.88
38.00
48.56
38.95
39.66
31.90
3 (3)
SD
4.40
2.71
2.21
3.39
6.37
2.91
6,16
6.76
4.81
5.22
5.57
A
O _
3 <
3 =
3 <
3 <
3 =
1 <
3 >
1 <
3 =
3 <
Priori
2
1
2
2
2
2,
2,
2,
2
2
= 1
, 2
= 1
, 2
< 1
< 1
. 3
1
3
= 1
> 1
JVofe. The numbers in parentheses in column heads refer to the number for illustratingsignificant differences in the last column titled "A Priori."
throughout the semester. A possible explanation for inconsistent findingsin personal and interpersonal development is associated with the single-suhject design and the repeated-measures method that was used in thisstudy. Students' attitudes were measured before, during, and after serviceto address "good-subject effects" and "halo-effects" (Rosenthal & Rosnow,1991] linked with pre-post test studies, while we may have inadvertentlycontributed to "testing-effects" with this repeated-survey method(Heppner et al., 1999). Sample selection and experimenter control (i.e.,randomization, comparison group) biases associated with the single-sub-ject design may further contribute to the inconsistent results. In addition,service-learning provided students with opportunities to work with chil-dren in placements located in the poorest neighborhoods in the commu-nity. Unfortunately, this project exposed them to some of the worst aspectsof the local political process, including closing of the school and budget,and personnel reductions in the after-school program. Such exposure mayhave influenced students' awareness of local politics and injustices ofeducational institutions.
Service-Learning and Community Service
Service-learning influenced students' preferences for community
2005, Volume 28, No, 2 183
Table 5Means, Standard Deviations, and Univariate Analyses for Effects ofService Activity on 11 Dependent Variables
Tutor Mentor Leader ANOVA
Community M SD M SD M SD F (2,25)Recipient Questions
1, Understanding of schoolor agency'policies 4,78" .69 4,00 1,00 3,50*^ ,54 7.03**
2, ludgment in
performing tasks 4,78" ,42 4,20 ,44 4,00*' ,63 6,57**
3, Takes initiative 4,7l" ,46 4.20 ,83 3,66*' ,51 7,50**
4, Empathy 4,7l" ,46 4,40 ,54 3.66^ ,51 9.40***
5, Compassion 4,7l" ,46 4,40 ,54 3,83*' ,40 7,35**
6, Concern 4,7l" ,46 4,40 .54 3,83^ ,40 7,35**
7, Receptive to learning 4,50" ,85 4.40 ,54 3,SO*' ,54 3,93*
8, Actively seeks
new information 4,50" .85 3,80 ,44 3.16'' .40 7,73**9, Appears comfortable
interacting withstudents 4,69" ,48 4,60 ,54 4,00*^ ,63 3,57*
10, Respects students 4,76" ,43 4,60 ,54 4,16*^ ,40 3,60*
11, Is sensitive andresponsive tostudents 'needs 4,76" ,43 4,40 ,54 4,16^' ,40 3,91*
Note. ***p < ,001, **p < ,01, *p < ,05, Mean scores within rows with varying superscripts(̂ or t") are significantly different from each other. For all measures, higher means indicatehigher performance scores.
184 lournal of Experiential Education
service and participation in commnnity engagement. Service-learners'preferences for short-term community involvement and desires to make adifference in the lives of commnnity recipients increased throughout thesemester and partially supports the third hypothesis and the findings ofPayne (2000), who found that service-learning students make short-termcommitments and then transition to long-term commitments to communi-ty service. Service-learners also increased their behavioral motives forcommunity involvement from the beginning to the middle of the semester,bnt they also decreased their motives from the middle to the end of thesemester. Service-learning reqnired stndents to show initiative, creativityand flexibility in dealing with new and unexpected situations. Forinstance, many students wrote letters, worked with families, and partici-pated in school board hearings after the school was notified of its closing.At the same time, students in the after-school program performed activi-ties that were beyond their service role (i.e,, driving the children, takingthe children swimming) with little supervision because of budget and per-sonnel reductions. These service experiences gave students opportunitiesto see how the children are affected by inequities in resources, thusincreasing their motives for, and participation in, community service.
Teacher Evaluation and Thtors
Teachers evaluated students who worked as tutors more favorablythan those who worked as social-recreational leaders. This finding par-tially supports the fourth hypothesis. In comparison to leaders, tutors hada better understanding of the school or agency's policies, showed greaterinitiative when implementing activities, and demonstrated more empathy,compassion and concern for the children. Found consistent with service-learners' reflections, teachers also evaluated tutors as more respectful, sen-sitive and responsive to the children's needs than leaders. Tutoring pro-vided students with an opportunity to work with a group of culturallydiverse children in a public school system that consistently ranked low instudent achievement, thereby perhaps increasing their leadership skills,cultural competencies, and empathy for the children.
Relationship Retween Service-Learners and Service-Recipients
The formation of positive relationships between the service-learnersand the service-recipients (i,e., children and teachers) is consistent withprevious service-learning research (Boyle-Baise & Kilbane, 2000; Brody &Wright, 2004). The content analysis suggests that service-learners acquiredinsight into careers, gained self-knowledge, and developed relationshipswith community recipients. Discrepancies were observed betweenqualitative and quantitative data concerning students' interpersonal andpersonal development. For example, many students describe discoveringsimilarities between themselves and community recipients, and changing
2005, Volume 28, No. 2 185
their perceptions of the children with whom they worked. Consistent withprevious research, however, students did not change their diversity atti-tudes over the semester (Moely, McFarland, et al., 2002). It is important tonote that the survey measure of diversity attitudes assessed students'openness rather than changes in their preconceived notions ahout otherswho are culturally different. In addition, there were consistencies betweenthe qualitative and quantitative data. Service-learners continued to workwith the children beyond the required hours and advocated on behalf ofthe families at school board hearings, and this degree of communityengagement was consistent with student and teacher survey reports.Finally, qualitative and quantitative data suggest all participants involvedin this service-learning study benefited from their participation, and theuse of teacher and student reports, as well as both research methods,enhanced the reliability of the findings.
Directions for Future StudyThe current study adds to literature regarding service-learning by
using a rigorous research design which may be potentially useful toresearchers, although the results should be viewed in light of a few keylimitations. First, data were collected using self-report methods and stu-dents may not have answered all of the questions honestly. Second, datamay only be generalized to a private college in the eastern United Statesbecause of homogeneity effects and selection biases associated with thesmall number of participants who volunteered to participate in service-learning. Finally, the use of a single-subject design contributes to internalvalidity limitations and precludes one from making definitive conclusionsabout service-learning and student outcomes.
Although the combination of qualitative and quantitative measuresallowed for the disentanglement of pre-service, during-service and post-service effects, tJiere are several suggestions for future research. First,investigators need to improve qualitative measures of service-learning out-comes. Instead of collecting data with post-surveys, students should main-tain a daily journal to record their thoughts and feelings after each serviceexperience over the semester to measure specific areas under investiga-tion. Second, additional instruments that measure similar constructsshould be incorporated into the survey to enhance the reliability of thequantitative data. Third, a comparison group must be included to improvethe internal validity of service-learning. Fourth, larger sample sizes arenecessary to generalize results. Finally, the inclusion of quantitative andqualitative data throughout the semester will strengthen the rigor of thedesign and methods while providing an opportunity to compare quantita-tive data with qualitative data through different analysis techniques. This
186 Journal of Experiential Education
would make an interesting future study.
References
Ash, S. L., Clayton, P. H., & Atkinson, M. P. (2005). Integrating reflection andassessment to capture and improve student learning. Michigan Journal ofCommunity Service Learning, 11{2), 49-60.
Battistoni, R. M. (2002). What is good citizenship? Conceptual frameworksacross disciplines. Introduction to service-learning toolkit (2nd ed.,pp. 179-186). Providence, RI: Campus Compact.
Boyle-Baise, M. (2002). Multicultural service learning. New York: TeachersCollege Press.
Boyle-Baise, M., & Kilhane, J. (2000). What really happens? A look inside serv-ice-learning for multicultural teacher education. Michigan Journal ofCommunity Service Learning, 7, 54-64.
Boyle-Baise, M., & Langford, J. (2004). There are children here: Service-learningfor social justice. Equity and Excellence in Education, 37, 55-66.
Brody, S. M., & Wright, S. C. (2004). Expanding the self through service-learning.Michigan Journal of Community Service, 11(1), 14-24.
Creswell, J. W. (1994). Research design: Qualitative and quantitative approach-es. Thousand Oaks, CA: Sage.
Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing amongfive traditions. Thousand Oaks, CA. Sage.
Creswell, J. W. (2005). Educational research (2nd ed.). Upper Saddle River, NJ:Pearson Prentice Hall.
Enos, S., & Troppe, M. (1996). Curricular models for service learning.Metropolitan Universities, 7(1), 71-84.
Erikson, J. A., & O'Connor, S. E. (2000). Service-Learning: Does it promote orreduce prejudice? In C. R. O'Crady (Ed.), Integrating service learning andmulticultural education in colleges and universities (pp. 59-70). Mahwah,NJ: Lawrence Erlhaum Associates.
Eyler, J. S. (2000). What do we most need to know ahout the impact of service-learning on student learning? Michigan Journal of Community ServiceLearning, Special Issue, 11-17.
Eyler, J. S., & Giles, D. E. (1999). Where's the learning in service-learning? SanErancisco: Jossey-Bass.
Eyler, J. S., Giles, D. E., Stenson, C. M., & Gray, C. J. (2003). What we know ahoutthe effects of service-learning on college students, faculty, institutions andcommunities, 1993-2000. Introduction to Service-learning Toolkit (3rd ed.,pp. 15-22). Providence, RI: Campus Compact.
2005, Volume 28, No. 2 187
Fredericksen, P. J. (2000). Does service learning make a difference in student per-formance? Journal of Experiential Education, 23[2), 64-74.
Furco, A, (2003). Service-learning: A balanced approach to experiential educa-tion. Introduction to Service-learning Toolkit (pp. 11-14). Providence, RI:Campus Compact.
Gallini, S. M., & Moely, B. E. (2003). Service-learning and engagement, academicchallenge, and retention. Michigan Journal of Community Service Learning,10{l), 1-14.
Creen, A. E. (2001). "But you aren't white:" Racial perspectives and servicelearning. Michigan Journal of Community Service Learning, 8(1), 18-26.
Harkavy, I., Puckett, ]., & Romer, D. (2000). Action research: Bridging service andresearch. Michigan Journal of Community Service Learning, Special Issue,113-119.
Heppner, P P., Kivlighan, D. M., Jr., & Wampold, B, E. (1999). Research design incounseling (2nd ed.), Belmont, CA: Wadsworth.
Howard, J. P. E. (Ed.). (2003), Academic service learning: A counter-normativepedagogy. Introduction to service-learning toolkit (pp. 57-62). Providence,RI: Campus Compact.
Litke, R. A. (2002). Do all students "get it?" Comparing students' reflections tocourse performance. Michigan Journal of Community Service Learning, 8(2),27-34.
Moely, B. E., McFarland, M., Miron, D., Mercer, D., & IUustre, V. (2002). Changesin college students' attitudes and intentions for civic involvement as a func-tion of service-learning experiences. Michigan Journal of Community ServiceLearning, 9(1), 18-26
Moely, B. E., Mercer, S. H., Ilustre, D., Miron, D., & McFarland, M. (2002).Psychometric properties and correlates of the civic attitudes and skills ques-tionnaire (CASQ): A measure of students' attitudes related to service-learn-ing. Michigan Journal of Community Service Learning, 8(2), 15-26.
Moore, D, T. (2000). The relationship between experimental learning researchand service-learning research. Michigan Journal of Community ServiceLearning, Special Issue, 124-128.
Payne, C. A, (2000). Changes in involvement preferences as measured by thecommunity service involvement preference inventory. Michigan Journal ofCommunity Service Learning, 7, 41-53.
Peters, S. J, (2004). Educating the civic professional: Reconfigurations and resist-ances, Michigan Journal of Community Service Learning, 11(1), 47-58.
Ramaley, J. A. (2000). Strategic directions for service-learning research: A presi-dential perspective. Michigan Journal of Community Service Learning,91-97.
Reinke, S. J. (2003). Making a difference: Does service-learning promote civicengagement in MPA students? Journal of Public Affairs Education, 9(2),129-137.
Rockquemore, K. A., & Schaffer, R. H. (2000), Toward a theory of engagement: A
188 Journal of Experiential Education
cognitive mapping of service learning. Michigan Journal of CommunityService Learning, 7, 14-23.
Root, S., Callahan, J., & Sepanski, J. (2002). Building teaching dispositions andservice-learning practice: A multi-site study. Michigan Journal of CommunityService Learning, 8(2), 50-59.
Rosenthal, R., & Rosnow, R. L. (1991). Essentials of behavioral research: Methodsand data analysis (2nd ed.). New York: McGraw-Hill.
Sigmon, R. L. (1996). The prohlem of definitions in service-learning. In R. L.Sigmon et al. (Eds.), Journey to service-learning (pp. 9-12). Washington, DC:The Council of Independent Colleges.
Stanton, T. K., Giles, D. E., & Cruz, N. I. (1999). Service-learning. San-Francisco:Jossey-Bass.
Strage, A. A. (2000). Service-learning: Enhancing learning outcomes in a college-level lecture course. Michigan Journal of Community Service Learning, 7,5-13.
Vogelgesang, L. J., & Astin, A. W. (2000). Comparing the effects of communityservice and service learning. Michigan Journal of Community ServiceLearning, 7, 25-34.
Wang, O. T., Kelly, J. M., & Hritsuk, B. (2003). Does one size fit all?: Challenge ofsocial cognitive development. Journal of Community Service Learning, 9(2),5-14.
Wang, Y., & Jackson, G. (2005). Forms and dimensions of civic involvement.Michigan Journal of Community Service Learning, 11{2), 39-48.
