ePORTFOLIOS: USING TECHNOLOGY TO ENHANCEAND ASSESS STUDENT LEARNING

LEAH E . WICKERSHAM, P H . D .

SHARON M . CHAMBERS, P H . D .

Secondary and Higher EducationTexas A&M University - Commerce

Tracking and assessing student learning outcomes within onecourse can be accomplished with relative ease. However, the truechallenge lies within providing data for student learning out-comes for an entire program. The purpose of this descriptivestudy was to discover and explain the fundamental lessonslearned from the beginning stages of ePortfolio implementationin a cohort master's degree program for secondary teachers. Aquestionnaire was administered to all twenty-six students at theend of their first semester to assses three student leaming out-comes: self-knowledge, technological and organization skillsdevelopment, and knowledge and skills transfer. After the firstsemester of implementation, students still see the ePortfolio con-cept and process as an external component to their master'sdegree and a separate skill with the technology currently a bar-rier and fail to see how ePortfolios are connected to theirmaster's degree program. Results may reflect the current statusof traditional assessments that is both state and nationally driven,indicating a need to continue to make the shift in paradigm ofprogram design and assessment.

A paradigm shift in the past decade has To answer these calls for change, univer-changed the focus in education from a sities have begun to focus on studentteacher-centered instructional environment learning outcomes as a way to measureto a student-centered one (Brooks, 1997; what students have leamed and are able toTerheggen, Prabhu, & Lubinescu, 2000). do wben they complete their degree. Out-Asa result, universities are held responsi- comes assessments "include theble for and expected to provide evidence knowledge, skills, and atfitudes that deter-of the process and growth in student leam- mine what students know now that theying over time. Grades are no longer proof didn't know before their college experi-enough of leaming; multiple stakeholders ence" (p. 142). Leaming outcomes may bein education want documentation that evaluated through various measuresdemonstrates the entire process of leam- depending on the academic program, theing (Heaney, 1990; Terheggen, Prabhu, & philosophical beliefs of assessment prac-Lubinescu, 2000; Villano, 2005). Ruhland tices on the part of the faculty, collegeand Brewer (2001) call attention to the and/or department, and measures whetherincreased demands for accountability that students have reached the leaming expec-emphasize assessment of student leaming. tations. Due to the assessment emphasis

738

ePortfolios .../739

alternative processes to the traditional testare being developed as a means to meetaccreditation and accountability expecta-tions (Ruhland & Brewer, 2001).

Multiple researchers concur that thebest learning, which is retained, occurs inthe context of an active learning experi-ence. For example, learning is bestfacilitated in environments that provide forhands-on, experiential opportunities,accentuate student participation and inter-action with peers, and encouragestudent-teacher communication (Astin,1985, 1996; Tinto, 1987; Davis & Mur-rell, 1994; Kuh, 1996). Astin (1985,1996)provides a foundational framework thatstates students learn as a result of theirinvolvement level and demonstrated own-ership. Astin states that quality educationalprograms have a learning environment thatincludes students actively engaged, highexpectations, and continuous assessmentand feedback (Astin as cited in Skawins-ki & Thibodeau, 2002).

A regional university in North EastTexas, in partnership with a local inde-pendent school district, used Astin'sframework as a guide when planning a newand unique cohort master's degree pro-gram in Secondary Education. The goalwas to provide the best system for docu-menting student learning outcomes andways to assess the overall program quali-ty. It was important to be able to use anoutcomes assessment process that wouldactively engage students where they wouldbe responsible, reflective learners and pro-vide assessment information for feedbackto guide the student learning process andinform program goals, objectives and field-based learning experiences for quality

enhancement. Current investigationsemphasize the value of "student effort andinvolvement as decisive elements in pro-moting positive college outcomes" (Davis& Murrell, 1994, p. 2).

Since research shows that ePortfolioscan "enhance teaching, learning and assess-ment practices", this method of assessmentwas selected as a way to document andhighlight the process of student learningand to measure student learning outcomesduring their master's degree program ofstudy (Lorenzo & Ittelson, 2005, p. 3). TheEDUCAUSE Learning Initiative definitionof electronic portfolios was used in orderto have a collective way of thinking (Uni-versity of British as cited in Lorenzo &Ittelson, 2005). This definition describesePortfolios as "personalized. Web-basedcollections of work, responses to work,and reflections that are used to demon-strate key skills and accomplishments fora variety of contexts and time periods" (p.3).

There was a strong incentive to use elec-tronic portfolios as a means for learningoutcomes assessment because it providedadditional value and enrichment for learn-ers. "The electronic portfolio creates apersonal collection of thoughts and workthat enhances the use and knowledge oftechnology, improves instructional prac-tices and showcases the candidates forpotential employers, students and students'parents" (Hewett, 2005, p. 27). Throughthe use of electronic portfolios, the respon-sibility of learning is transferred to thestudents. It allows them to be involved andengaged in the learning process and there-fore keeps the focus on the learner-centeredenvironment. Hewett's (2005) research

740 / Education Vol. 126 No. 4

indicates that "electronic portfolios arebeing used by elementary, secondary andcollege educators to lead their classroomstoward learner-centered rather than teach-ing-centered learning environments.Hewett further states that "as a model forlearner-centered classrooms, e-portfoliosgive students ownership and responsibili-ty for their own learning" (p. 27).

Statement of ProbtemTracking and assessing student learning

outcomes within one course can be accom-plished with relative ease. However, thetrue challenge lies within providing data forstudent learning outcomes for an entireprogram.

PurposeThe purpose of this descriptive study

was to discover and explain the funda-mental lessons learned from the beginningstages of ePortfolio implementation in acohort master's degree program for sec-ondary teachers. Three student learningoutcomes, self-knowledge, technologicaland organizational skills development andknowledge and skills transfer, were iden-tified in order to document and assesslearning.

Self-knowledge is defined as anincrease in understanding of the relation-ship between theory taught in the graduateprogram as it pertains to the what, why andhow of individual careers. Technologicaland organizational skills development isdefined as an increase in technologicalskills, communication skills, and organi-zational skills. Knowledge and skillstransfer is defined as the ability for "Mas-ter Teachers" to transfer the knowledge

and skills developed in the program to theirclassrooms. The study sought to answerthe following research questions:1) Do ePortfolios provide evidence of the

process of learning and measure stu-dent learning outcomes?

2) Do ePortfolios provide evidence of theprogram's effectivenss as students mar-ticulate?

Methods & ProceduresStudents participating in the study were

members of a cohort in their first semes-ter of coursework in a secondary educationmaster's degree program. A questionnairewas administered to all twenty-six studentsat the end of their first semester. The ques-tionnaire consisted of thirteen Likert scaleresponse questions and two open-endedquestions.

At the beginning of each semester, stu-dents were given a structure to follow forthe design and development of their port-folio. Assignments were created whichrequired constant communication via elec-tronic means and the ability to expressthoughts clearly and effectively, electron-ic peer review and editing, and otheractivities to facilitate the utilziation of avariety of educational technoloiges. Thisstructure outlined for them the minimumrequired artifacts/documentation andreflections that needed to be included intheir ePortfolios. Organizational skills viathe structure of the portfolio itself beganthe process of students thinking systemi-cally.

Students were asked to identify quali-ties they believe a "Master Teacher" shouldpossess by outlining characteristics, skills.

ePortfolios .../741

abilities and performances in the class-room. As students completed a cohortcourse each semester, they conducted aself-evaluation in order to compare theirprogress toward high qualities of a "Mas-ter Teacher". Students were asked toinclude documentation of how they havechanged as a result of the knowledge andskills acquired within the course andexplain/describe how they have developedtoward their goal of becoming a "MasterTeacher".

Through their experience one goal wasfor them to understand the benefits to port-folio development and other methods ofassessment in the K-12 classroom. In addi-tion to students identifying how they havechanged as a result of the course and pro-gram, they were asked to provideevidence/documentation of how their stu-dents' learning has improved as a result ofthe knowledge and skills they have incor-porated into their classrooms.

The data was examined to determinegrowth and changes in the three studentlearning outcomes as a result of the elec-tronic portfolio development process. Theresults from the surveys and the experi-ences with the development andimplementation of ePortfolios may be ben-eficial to other universities as they considerways to document student learning out-comes and program evaluation.

Results & DiscussionResults from the ePortfolio learning

outcomes assessment survey after onesemester of portfolio implementationrevealed that overall students were evenlydivided in agreement, disagreement and

neutrality on the impact of the ePortfoliodevelopment process in assisting them inan increase in two of the student learningoutcomes: self-knowledge and knowledgeand skills transfer. However, the majorityof students did find that the ePortfoliodevelopment process led to an increase intheir overall technical skills and confidencein using technology. Results for each learn-ing outcome will be discussed in detail.

Self-KnowledgeIn the area of self-knowledge, students

were asked if the activities they wererequired to complete as part of the devel-opment of their ePortfolio resulted in anincrease of their knowledge in teaching.Students were also asked to respond towhether or not the reflection on their teach-ing was more apparent after utilizing theePortfolio and if this process made themmore aware of why they teach the way theydo, if they are more aware of what changesneed to be made in their teaching, and ifthey are more aware of how they are per-ceived by others.

In general, students were evenly divid-ed in agreement and neutrality, but almosthalf disagreed that the ePortfolio devel-opment process led to an increase in theirknowledge of teaching. However, the per-centages are not overwhelming in any areafor self-knowledge. These findings maybe a result of only one semester of imple-mentation and not be enough time forstudents to recognize how ePortfolios mayimpact their teaching. Table 1 provides thepercentage distributions for agree, neutraland disagree responses for each self-knowl-edge item.

742 / Education Voi. 126 No. 4

Table 1Self-KnowledgeItem

Increase Knowledge in TeachingReflectionsWhyWhatHow

Agree

33%46%42%46%46%

Neutral

21%25%25%21%21%

Disagree

46%30%33%33%33%

Technological & Organizational SkillsDevelopment

Upon introduction and implementationof ePortfolios within the cohort, it becameevident that many individuals were lack-ing the basic computer skills expected ofteachers today. This discovery was dis-couraging and may be an indication thatgraduates of teacher education programsare not receiving the technological train-ing and skills needed prior to going into theclassroom. This may also provide insightinto the disappointing results found in theself-knowledge and knowledge and skillstransfer learning outcomes. It is expectedthat once students become more familiarwith the concept of ePortfolios and com-fortable with the technology itself that thetechnology will no longer be seen as a bar-rier and students will be able to relate moreto the concepts of self knowledge andknowledge transfer.

Students were asked if working withtheir ePortfolio resulted in an increase intheir technological skills and abilities, ifthey were more confident with technolo-gy after working with their ePortfolio, ifdeveloping their ePortfolio helped them

become more organized, and if their com-munications skills were enhanced as aresult of working with the ePortfolio andits assignments. Results from the surveyrevealed that more than half surveyed,58%, believed that the ePortfolio devel-opment process (assignment upload,creation of the portfolio, emailing, pub-lishing and more) resulted in an increasein their technological skills as opposed to4% who disagreed. Thirty percentremained neutral in this area, but theseindividuals may already have the neces-sary technological skills required at thispoint in the ePortfolio developmentprocess. In fact, only seven individualsself-rated as novice users of technology asopposed to ten average users, sevenadvanced users and one expert user. Whenasked to rate their abilities after the ePort-folio development process, 43% originalnovice users scored themselves as nowaverage users of technology and 20%moved from average to advanced users.

Exactly half of those surveyed believedthat the ePortfolio development processled to an increase in their confidence inutilizing technology; however, results fororganizational and communication skillswere more evenly distributed. Organiza-

ePortfolios .../743

Table 2Technological and Organizational Skills Development

Item Agree

Technology Skills 58%Confidence 50%Organization 33%Communication 33%

Neutral

30%46%38%38%

Disagree

12%4%30%30%

tion and communication is still in its infan-cy and these results may indicate that overthe next two years there will be a positivechange in these areas as additional activi-ties and assignments are implementedrequiring more organization and commu-nication within the ePortfolio. Table 2provides the percentage distributions foragree, neutral and disagree responses foreach technological and organizational skillsitem.

Knowledge & Skills TransferStudents were asked if specific knowl-

edge and professional knowledge ofteaching transferred into their classroom asa result of the ePortfolio developmentprocess. They were also asked if theyobserved some changes in their students'learning as a result of their knowledgetransfer from the graduate program intotheir classroom, and if they integrated tech-nology into their classrooms due to anincrease in their confidence with technol-ogy. Finally, students were asked if theirthinking regarding assessment practices of

their students changed after exposure tothe ePortfolio assessment concept.

Results from specific and professionalknowledge transfer, similar to the self-knowledge outcome were evenlydistributed. Almost half, 46%, of therespondents disagreed that the ePortfoliodevelopment process led to a transfer ofspecific and professional knowledge intheir classrooms and careers. This may alsostem from the fact that they still see theePortfolio process as an external ratherthan integral part of their education.

Over half, 54%, of the students dis-agreed that this process and all they haveleamed from it has resulted in a change intheir students learning, indicating littletransfer of the ePortfolio concept hasoccurred in their classrooms. However,encouraging are the results from theirchange in thinking of assessment practices.Table 3 provides the percentage distribu-tions for agree, neutral and disagreeresponses for each knowledge and skillstransfer item.

744 / Education Vol. 126 No. 4

Table 3Knowledge and Skills Transfer

Item Agree Neutral Disagree

Specific KnowledgeProfessional KnowledgeChange in Student LearningChange in Assessment

33%25%38%38%

21%30%8%30%

46%46%54%33%

Advantages and Disadvantages ofePortfolios

Students were asked to address theadvantages and disadvantages they dis-covered with ePortfolios in twoopen-ended questions. Their responseswere surprising in that they contradictedwhat some of the survey results revealed.The disadvantage most cited by the stu-dents was the technological barrier andtheir lack of confidence to utilize the ePort-folio system environment. It is expectedthat over time students will become moreconfident with the technology and in factresults from the technological and orga-nizational skills learning outcome indicatea positive change in their technologicalabilities.

Students are beginning to realize theadvantages of the ePortfolio concept interms of assessment practices. A few indi-viduals are investigating the idea ofimplementing portfoho assessment in theirclassrooms which indicates knowledge andskills transfer. Students also liked the abil-ity to see other portfolios in order to learnabout the success and challenges faced bytheir peers in their classrooms and the abil-ity to share their thoughts with others.

Several discussed the added benefit of "lesspaper" and the fact that developing theirportfolio electronically "forced" them tobecome more organized with more spaceto store information and allowed for a pro-fessional looking documentation of theePortfolio.

ConclusionsAfter the first semester of implemen-

tation, students still see the ePortfolioconcept and process as an external com-ponent to their master's degree and aseparate skill with the technology currentlya barrier. Students fail to see how ePort-folios are connected to their master'sdegree program. More importantly, stu-dents do not view ePortfolio developmentas a method of alternative assessment indi-vidually, programmatically, and/or in theirclassrooms. It appears they are slow toaccept other methods of assessment withthe onslaught of standardized tests andaccountability measures for which thesestudents are held responsible. What waslearned from all results is the need to bemore effective in helping students makethat connection, and to integrate the ePort-folio concept firmly within the program.It is important to lay the foundation and

ePortfolios .../745

help students to bridge the gap betweenwhat they are learning in the cohort andwhat they do in their classrooms regardingePortfolio. In their reflection activities,they do make that transfer; however, theydo not see the connection of ePortfoliosassisting in their teaching practices.

Results may reflect the current statusof traditional assessments that is both stateand nationally driven, indicating a need tocontinue to make the shift in paradigm ofprogram design and assessment. The cohortis currently in its second semester of imple-mentation and steps have been taken tobetter connect the ePortfolio concept intothe program. Future courses include mod-els of teaching, curriculum design, actionresearch and leadership. As the cohort divesdeeper into the capabilities and function-alities of the ePortfolio system, studentswill design and develop units and lessonswhich are aligned with state and nationalstandards, create rubrics for grading, andpublish for peers and instructors to viewand evaluate. As a result of continuing todesign and develop their ePortfolio it isanticipated there will be an increase intechnological self-efficacy and progres-sion in thinking of assessment practices.The final ePortfolio will demonstrate stu-dents' learning process and progressthroughout the program and transferabili-ty of knowledge into their classrooms, inaddition to informing programmatic deci-sions.

ReferencesAstin, A. (1985). Achieving educational excel-

lence. San Francisco: Jossey-Bass.

Astin, A. (1996). Involvement in learning revisit-ed: Lessons we have learned. Journal ofCollege Student Development, 37, 123-34.

Brooks, M. (1997). Beyond teaching and learningparadigms: Trekking into the virtual universi-ty. Teaching Sociology (27), 1-14.

Davis, T. M. & Murrell, P. H. (1994). Turningteaching into learning. The role of Studentresponsibility in the collegiate experience.(Report No. HE027588). Office of Education-al Research and Improvement. (ERICDocument Reproduction Service No.ED372702 1994-08-00)

Heaney, B. (1990). The assessment of educationaloutcomes(Report No. EDO-JC-90-01). Officeof Educational Research and Improvement EDWashington, DC. (ERIC Document Reproduc-tion Service No. ED321834)

Hewett, S. (2005). Electronic portfolios: Improv-ing instructional practices. Tech Trends, 48(5),26-30.

Kuh, G. (1996). Guiding principles for creatingseamless learning environments for undergrad-uates. Journal of College StudentDevelopment, 37, 135-148.

Lorenzo, G. & Ittelson, J. (2005). An overview ofe-portfolios. Educause Learning Initiative, ELIPaper, 1-28. Retrieved September 9, 2005http://www.educause.edu/ir/library/pdf/ELI3001.pdf

Ruhland, S. & Brewer, J. (2001). Implementing anassessment plan to document student learningin a two-year technical college. Journal ofVocational Education Research, 25(2), 141-171.

Terheggen, S. L., Prabhu, R. P. & Lubinescu, E.S. (2000). Erom product to process: Enhanc-ing learning through the use of studentelectronic portfolios. U. S. Department of Edu-cation, Educational Resources informationcenter (ERIC). HE033685.

746 / Education Vol. 126 No. 4

Skawinski, S. & Thibodeau, S. (2002). A journeyinto portfolio assessment. The EducationalForam, 67, 81-88.

Tinto, V. (1987). Leaving college: Rethinking thecauses and cures of student attrition. Chica-go: University of Chicago Press.

Villano, M. (September, 2005). Hi-octane assess-ment. Campus Technology, 19(1), 45-48.