Embed Size (px)
Citation preview
1
Nov. 10, 2014
Nasser Paydar
Executive Vice Chancellor and Chief Academic Officer
RE: ePortfolio Initiative Annual Report
Dear Nasser:
As you know, the campus ePortfolio Initiative supports the efforts of our IUPUI faculty, staff,
and students to use electronic portfolios to improve learning, teaching, and assessment, and to
document student learning and achievement for various audiences, including prospective
employers. I am very pleased to provide you with the attached ePortfolio Initiative Annual
Report, which summarizes the progress of this work between January 1, 2013 and June 30, 2014.
(We decided to carry the report through June so that we could resume reporting on an academic
year basis for future reports.) Over this 18-month period, we have provided leadership and
support for campus ePortfolio projects that:
Promoted undergraduate student learning and success through the ePDP project led by
University College and many other innovative uses of ePortfolios in undergraduate
programs;
Supported student success and learning outcomes assessment and improvement in
graduate programs, including five health and life sciences graduate and professional
programs;
Contributed to and documented the effectiveness of student participation in RISE
experiences and other high-impact practices;
Advanced IUPUI’s reputation as an educational innovator through extensive publications
and presentations, participation in prominent inter/national ePortfolio initiatives, and
leadership roles in professional organizations and activities.
In addition to supporting these achievements, we co-led and staffed the intensive IU-wide review
of commercial and open-source ePortfolio platforms that culminated in the recent purchase of
TaskStream as IU’s new ePortfolio environment;
You will find more detailed information in the attached brief report and even more in the
appendices. As always, we present this report with a caveat: we try to capture as much of
IUPUI’s ePortfolio activity as we can, but we cannot compel ePortfolio adopters to report to us;
occasionally, we become aware of faculty members or programs that have been quietly using
ePortfolios without our knowledge. As we migrate from the Oncourse ePortfolio to TaskStream,
we should be able to track ePortfolio use more accurately.
2
I would be happy to respond to questions or to meet with you about the information contained in
this report. I hope that you find it helpful.
Thank you for your support of the ePortfolio Initiative—it is much appreciated by many.
Sincerely,
Susan Kahn
Cc: Trudy Banta
Kathy Johnson
Melissa Lavitt
Anastasia Morrone
Pratibha Varma-Nelson
ePortfolio Coordinating Committee
3
IUPUI ePortfolio Initiative Report January 2013 – June 2014
ACCOMPLISHMENTS
Supported Student Learning and Its Assessment
Susan Kahn, Susan Scott, and colleagues in the Center for Teaching and Learning and UITS
worked with approximately 43 ePortfolio projects in 16 schools (37 programs) and 6 other
campus-wide units. (See Appendix A for ePortfolio use statistics.) We helped faculty launch
ePortfolio pilots in International Affairs, Dental Hygiene, and the Fairbanks School of Public
Health. We advised several units in the process of expanding their ePortfolio initiatives to
new courses or programs, including the Herron School of Art and Design, the Department of
Biology, the Department of World Languages and Cultures, Organizational Leadership and
Supervision (Prior Learning Assessment), Philanthropic Studies, and University College. We
consulted on plans for ePortfolio pilots in the Paralegal Certificate Program in the
Department of Political Science, the career advising office in the McKinney School of Law,
and Intercultural Communications. (Several other units postponed plans for pilots, adoption,
or expansion of ePortfolio projects pending availability of a new platform.) IUPUI ePortfolio
projects continued to represent a wide range of possible uses and purposes, including
personal, academic, and professional development; integrative learning; assessment and
accreditation; academic and career preparation and planning; and academic and career
showcase. (See Appendix B for additional details on individual projects.)
We continued collaborating with University College on the electronic Personal Development
Plan initiative, focusing on applying the ePDP conceptual model to curriculum design and
pedagogy, and on designing a new faculty development program grounded in the model. We
co-led a successful pilot if this weeklong ePDP Summer Institute in June 2014 with 7 faculty
and staff members, who are continuing to meet regularly during the Fall 2014 semester.
Served IUPUI and IU Faculty and Students
With Lynn Ward of UITS, Kahn co-chaired the IU Joint Task Force on ePortfolio Platform
Review. A major effort begun in Fall 2012, the Task Force reviewed approximately 20
leading ePortfolio platforms; identified and prioritized university-wide ePortfolio
requirements; issued a Request for Information in Spring 2013; evaluated RFI responses;
hosted four vendor demonstrations based on detailed scenarios drawn from IUPUI ePortfolio
projects; and presented recommendations to UITS in Spring 2014. (See Appendices E and F
for the Task Force’s list of requirements and the use-case scenarios.) In addition to Kahn and
Scott, IUPUI Task Force members included representatives from University College, the
School of Nursing, the Center for Teaching and Learning, and the Center for Research and
Learning.
4
We presented well received new professional development programs focused on ePortfolio
pedagogy and assessment and on support for student users, and continued existing programs,
including the annual ePortfolio Symposium, hands-on software demonstrations, and a web
design workshop.
We supported participation in the July 2013 AAEEBL international ePortfolio conference for
12 IUPUI faculty and professional staff members, 2 students, and 2 CTL staff members. The
conference program featured 13 presentations by IUPUI faculty, staff, and students. In
addition, we supported participation and presentations by 9 IUPUI faculty and staff at the
first AAEEBL Midwest Regional Conference in Ann Arbor, MI in May 2014.
We provided financial and academic support to faculty and staff piloting the use of
ePortfolios in OIA Study Abroad programs during 2013 and to participants in University
College’s June 2014 ePDP Summer Institute.
Advanced IUPUI’s National and International Leadership in the ePortfolio Field
In 2013-14, 30 IUPUI faculty, staff, and students authored 5 print publications and 11 online
articles and made 27 presentations for international, national, and regional conferences on
ePortfolio topics. Faculty and academic staff from the Schools of Dentistry, Herron, Liberal
Arts, Medicine, Nursing, Philanthropy, Science, Social Work, and University College
contributed to strengthening IUPUI’s leadership in the ePortfolio field.
IUPUI contributed extensively during Year 3 of the FIPSE-funded Connect to Learning
Initiative, collaborating with 23 other institutions nationwide to advance ePortfolio theory
and practice. Our work included preparing and editing an IUPUI web site-within-a-web site
as part of a comprehensive new ePortfolio resource site, Catalyst for Learning
(http://c2l.mcnrc.org). (Scott served as one of several overall editors of the site.) We
successfully applied for a six-month funded extension of IUPUI’s Connect to Learning sub-
grant that provided additional time and support for fine-tuning and disseminating the IUPUI
section of the site; further developing our campus C2L project to expand and improve use of
the ePDP; and planning and conducting our pilot ePDP Summer Institute.
In 2013, we concluded our participation in Cohort VI of the Inter/National Coalition for
Electronic Portfolio Research. We are completing the qualitative data analysis for IUPUI’s
Cohort VI research project in Fall 2014, and will present our findings at the 2014 Assessment
Institute in Indianapolis and the 2015 AAC&U Annual Meeting. (See Appendix G for a
condensed summary of our preliminary findings on the use of reflection and assessment in
IUPUI ePortfolio projects.)
Kahn continued to serve as Chair of the Board of Directors of the Association for Authentic,
Experiential, and Evidence-Based Learning (AAEEBL), the inter/national professional
organization for the ePortfolio field, which she helped lead from start-up in 2009 to
recognized voice of a thriving community by 2014. In addition, Kahn and Scott coordinated
the ePortfolio track of the annual Assessment Institute at IUPUI, now one of the leading
venues for practitioner exchange of ePortfolio assessment expertise.
5
In November 2013, Kahn spent a week in Santiago, Chile consulting with DUOC UC, a 16-
campus institution affiliated with the Pontifical University of Chile. Over the course of the
week, she visited 10 DUOC UC sites, offering 4 workshops and 4 formal addresses,
facilitating 5 planning meetings, providing media interviews, and working with the DUOC
UC ePortfolio administrative team on launching its institution-wide ePortfolio initiative.
PLANS AND GOALS FOR 2014-15
Working with TaskStream, UITS, and CTL staff, support the implementation of TaskStream
as IUPUI’s new ePortfolio platform. Our plans include: offering hands-on workshops;
supporting campus pilots; and facilitating migration of existing projects from Oncourse to
TaskStream. Leverage the new platform’s advantages over the Oncourse ePortfolio to
increase ePortfolio use at IUPUI. (See Appendix H for screen shots of the TaskStream
environment and of Webfolios created with TaskStream.)
Working with University College and other interested units, expand adoption of the ePDP in
first-year seminars, degree programs, and RISE experiences and other high-impact practices.
Preliminary research by the Connect to Learning Project suggests that, when thoughtfully
incorporated and guided, ePortfolios can amplify the impact of HIPs. ePortfolios also offer
us a window into students’ experiences of these practices, support assessment of what
students are learning and achieving in these programs, and enable students to demonstrate the
outcomes of these experiences for prospective employers and graduate schools.
Support use of the ePortfolio and ePDP for learning and assessment of the PULs in the new
General Education Core.
Plan and implement an IUPUI ePortfolio Showcase, to be held in March 2015, to spotlight
outstanding IUPUI ePortfolio initiatives and individual students’ portfolios, and to advance
adoption of ePortfolios by demonstrating high-quality exemplars.
Expand campus workshop offerings on using ePortfolios to support student success, improve
teaching and learning, and provide data for authentic assessment of learning outcomes.
CHALLENGES
It is not yet clear to us what the migration to TaskStream will entail in terms of time and
resources. We are currently conducting training sessions with TaskStream staff, who, on their
part, still have much to learn about the variety of IUPUI and IU ePortfolio projects.
Support for student ePortfolio users has been a lingering issue. We will need to work closely
with units adopting TaskStream to determine whether and what local resources will be
needed (in addition to support available from TaskStream and UITS) and to develop an
appropriate support model(s).
6
APPENDIX A: ONCOURSE PORTFOLIOS BY THE NUMBERS*
Portfolio Sites (through December 2013):
Total portfolio sites in Oncourse: 779
Total non-unique users in all sites: 24,359
Total active portfolio sites (sites with evidence of activity in the last 2 years):
131
Total non-unique users in active sites: 10,097
Table at right shows breakdown of active sites by campus
It is difficult to determine whether a specific matrix instance is active because the
dates of most actions conducted by users are not recorded. As a result, the actual
total of active portfolio sites is higher than the number given (131).
Matrices:
Total saved matrix instances: 18,423
Total unique users with a saved matrix instance: 10,794
Table at right shows breakdown of matrix owners by campus
Numbers by campus were generated by the email domain, presentation, or
matrix owner. Some owners have a non-IU email address in their Oncourse
email field and were not included in the campus counts. Thus, the sum of the
campus totals is in all cases lower than the total number of users.
Portfolio Presentations (Webfolios):
Total presentations: 9,967
Total presentations created after December 31, 2011: 6,849
Total unique presentation owners/authors: 6,460
Total unique authors with presentations created after December 31, 2011:
4,565
Table at right shows breakdown of unique authors with post-December
31, 2011 presentations by campus
*Source: UITS
BL 17
CO 0
EA 1
IN 99
KO 1
NW 3
SB 3
SE 0
IU 1
Total 125
BL 2,449
CO 130
EA 333
IN 6,900
KO 125
NW 94
SB 96
SE 44
IU 95
Total 10,266
BL 962
CO 24
EA 49
IN 3,060
KO 46
NW 73
SB 85
SE 83
IU 33
Total 4,415
7
APPENDIX B: CURRENT EPORTFOLIO PROJECTS AND USES @ IUPUI
Pro
gra
m
ass
essm
ent
an
d
acc
red
itati
on
Ass
essm
ent
of
stu
den
t le
arn
ing
Inte
gra
tive
learn
ing
C
are
er
pre
para
tion
an
d
pla
nn
ing
Aca
dem
ic
pla
nn
ing &
ad
vis
ing
Sel
f
rep
rese
nta
tion
Pro
fess
ion
al
dev
elop
men
t
Sel
f –ass
essm
ent
(dev
elop
men
tal)
Work
flow
an
d
track
ing
American Studies (courses)
Art (Art History capstone & 300,
Preparing for Foundation Studies)
Biology (courses)
Center for Research and Learning
Center for Service and Learning*
(service scholars, civic-minded
graduate assessment)
Dentistry (Pediatrics*, Dental
Hygiene with ePDP)
Engineering and Technology (ET*,
ECT)
English (capstone, PRAC grant)
French (courses)
International Affairs (Study Abroad,
including German, SPEA, Biology,
Business)*
IUPU Columbus (Education,*whole
campus now using Chalk & Wire)
Law (career planning)
Library and Information Science
(MLS)*
Life-Health Sciences Internship
Program* (with ePDP)
Medicine (Pediatrics; planning
broader development with new
curriculum)
8
Pro
gra
m
ass
essm
ent
an
d
acc
red
itati
on
Ass
essm
ent
of
stu
den
t le
arn
ing
Inte
gra
tive
learn
ing
C
are
er
pre
para
tion
an
d
pla
nn
ing
Aca
dem
ic
pla
nn
ing &
ad
vis
ing
Sel
f
rep
rese
nta
tion
Pro
fess
ion
al
dev
elop
men
t
Sel
f –ass
essm
ent
(dev
elop
men
tal)
Work
flow
an
d
track
ing
Museum Studies (MS, using Epsilen)
Music Technology (BSMT)*
Nursing (CNS, DNP*, online BSN
completion programs with ePDP,
new BSN curriculum with ePDP,
MSN)
Organizational Leadership and
Supervision* (FYS ePDP, Prior
Learning Assessment)
Philanthropic Studies (capstone, BS)
Physical Therapy (DPT)
Political Science (Paralegal certif.)
Psychology (ePDP with advising,
Lifespan course, capstone)
Public Health (MPH capstone)
Social Work (BSW)
Spanish (capstone, addl. courses)
Technical Communications
Themed Learning Communities/First
Year Seminars in Business,
Education, Engineering &
Technology, Health Professions,
Mathematics, Psychology, SPEA,
Writing (ePDP)
University College (ePDP for FYS*
and peer mentors program)
Writing Program (DQP project;
piloting alternate platform)
*Received start-up financial support from ePortfolio initiative budget. Most others have been
supported via CTL consulting and ePortfolio faculty development workshops.
9
APPENDIX C: IUPUI PUBLICATIONS AND PRESENTATIONS, JANUARY 2013 – JUNE 2014
Publications
Buyarski, C. & Landis, C. (Spring 2014). “Using an ePortfolio to Assess the Outcomes of a
First-Year Seminar: Student Narrative and Authentic Assessment,” International Journal of
ePortfolio (http://www.theijep.com).
Kahn, S. (Winter 2014). “E-Portfolios: A Look at Where We’ve Been, Where We Are Now,
and Where We’re (Possibly) Going,” AAC&U Peer Review, Vol. 16, No. 1
(http://www.aacu.org/peerreview).
McGuire, L., Gentle-Genitty, C. & Galyean, E. (2013). “The ePortfolio: Product and Process
in Assessing Competencies for Social Work Education.” Special Edition on Technology and
Social Work Education, Journal of Baccalaureate Social Work.
Meek, J., Pesut, D., Riner, M., Allam, E., & Runshe, D. (2013). “A Pilot Study Evaluation of
Student Reflective Thinking in a Doctor of Nursing Practice Program.” Journal of Nursing
Education and Practice, Vol. 3, No 8, Sciedu Press, Toronto ON
(http://www.sciedu.ca/journal/index.php/jnep/index).
Technical Reports
Kahn, S., Scott, S., & Landis, C. “IUPUI Preliminary Report for I/NCEPR Cohort VI,”
Inter/National Coalition for Electronic Portfolio Research Cohort VI, Salt Lake City, UT,
March 2013 (http://ncepr.org/finalreports/cohort6/iupui_final_report.pdf).
Online publications, Catalyst for Learning web site (http://iupui.mcnrc.org)
Scott, S. “Who We Are,” September 2013.
Kahn, S. “Scaling Up ePortfolios at a Complex Urban Research University: The IUPUI
Story,” October 2013.
Scott, S. & Kahn, S. “IUPUI Professional Development: Learning Planned and Unexpected,”
November 2013.
Williams, C. “Peer Reflective Feedback in First Year Service Learning,” November 2013.
Scott, S. & Kahn, S. “There Are No Silver Bullets,” November 2013.
Kahn, S. “A Committee Bears Unexpected Fruit,” December 2013.
Scott, S. & Kahn, S. “Assessment Is Everyone’s Business,” December 2013.
Scott, S. “Our Student Voices,” December 2013.
Kahn, S., Scott, S., & Kinsman, P. “Working Together to Develop Metacognition and
Professional Identity,” December 2013.
Buyarski, C. “Reflection in the First Year: A Foundation for Identity and Meaning Making,”
December 2013.
Scott, S. & Buyarski, C. “What We’ve Learned,” December 2013.
Conference Presentations
Buyarski, C., & Kahn, S. “Giving Students a Compass: Seeking a Conceptual Model for a
Developmental ePortfolio,” Association of American Colleges and Universities Annual
Meeting, January 2013
10
Buyarski, C., & Landis, C. “Assessing First-Year Outcomes through Authentic Evidence:
Content Analysis of an ePortfolio,” IUPUI Assessment Institute, Indianapolis, October 2013
Buyarski, C., & Landis, C. “Content Analysis and Authentic Evidence: Using an E-Portfolio
to Assess the Outcomes of a First-Year Seminar,” AAC&U Annual Meeting, Washington,
DC, January 2014
Gilbert, B., Fierst, J., & Leitzell, J. “Life-Health Sciences Internships at IUPUI: Connecting
Classroom and Experiential Learning,” AAC&U Conference on General Education and
Assessment, Portland, OR, February 2014
Gilbert, B., Sherer, J., & Torline, E. “Using the Electronic Personal Development Plan
(ePDP) to Explore ‘Why?’ among Pre-Professional Undergraduate Interns,” E. C. Moore
Symposium, Indianapolis, April 2013
Kahn, S., Landis, C., & Scott, S. “IUPUI Research Final Presentation,” I/NCEPR Cohort VI
Conference, Westminster College, Salt Lake City, UT, March 2013
Kahn, S. “ Electronic Portfolios for Quality Assessment and Improvement,“ EAIR (European
Higher Education Society) Forum, Rotterdam, August 2013
Scott, S., & Kahn, S. “Straight Talk about Implementing ePortfolios,” AAC&U Annual
Meeting, Washington, DC, January 2014
AAEEBL National Conference, Boston, July 2013
Anton, M. “E-Portfolio Assessment and Students’ Reflections on Learning Outcomes in a
Second Language Program” (Spanish)
Johnson, K. & Kahn, S. “E-Portfolios in the Senior Year Experience” (English)
Kowolik, J. & Meadows, M. “Filling a Need: the Use of ePortfolio for Competency in
Pediatric Dentistry” (Pediatric Dentistry)
Turner, R. “Identity and Authority and Capstone ePortfolios in an Emerging Discipline”
(Philanthropic Studies)
Buyarski, C. & Kahn, S. “Coming of Age: The Need and Process for Developing a
Conceptual Model to Guide an ePortfolio Implementation” (University College)
McGuire, L. & Galyean, E. “Using the ePortfolio for Competency-Based Program
Assessment in Social Work” (Social Work)
Embree, J., Young, J., & Runshe, D. “Closing the Loop with ePortfolios and Program
Assessment Aimed at Improved Learning” (Nursing)
Runshe, D. “Supporting or Enabling: Developing a Sustainable Support Model for
ePortfolio Initiatives” (UITS)
Yard, M., Rehlander, N., & Runshe, D. “Expanding the Reach with ePortfolios: Inviting
High School Students into the College Environment” (Biology)
Buyarski, C. & Landis, C. “Developing Capacity for Assessment of Authentic Evidence
through an ePortfolio” (University College)
Gilbert, B. & Sherer, J. “Using the ePDP and Student Ambassadors to Encourage
Reflection among Pre-Professional Undergraduates in an Experiential Learning Program”
(Life-Health Sciences Internship Program)
Runshe, D. & Zhao, A. “Social Pedagogies: The Intersection of ePortfolios and
Academic Social Networking” (Course Networking)
Larrier, Y. & Runshe, D. “Developing a Professional School Counselor’s Disposition
with an ePortfolio” ( UITS and IUSB Education)
11
AAEEBL Midwest Regional Conference, Ann Arbor, MI, May 18-19, 2014
Kahn, S., & Sepulveda, D. “Fostering Engaged Learning with ePortfolios” [invited
keynote workshop]
Kahn, S., Anton, M., & Kinsman, P. “ePortfolios, Identity, and Integrative Learning in
the Senior Capstone at IUPUI” (English, Spanish, Art History)
Freeman, T., & Turner, R. “Capstone ePortfolios as Mechanisms for Integrative and
Engaged Learning” (Philanthropic Studies)
Williams, C. “The ePortfolio as a Place to Integrate Service Learning and Civic Identity”
(Psychology)
Kahn, S., & Scott, S. “Campus Strategies for ePortfolio Diffusion” (ePortfolio Initiative)
Powell, A., Alexander, M., & Ward, L. “IUPUI’s ePortfolio Support Model: A
Multifaceted Approach to Faculty Development and Technology Support” (CTL &
UITS)
Local Workshops and Presentations
ePortfolio Assessment, January 11, 2013 (Kahn, Kinsman)
Web Design Basics for ePortfolios, March 1, 2013 (Alexander, Ward)
ePortfolio Spring Symposium, April 19, 2013 (Kahn, Buyarski, Scott, Anton, Freeman,
Johnson, Kinsman)
Supporting Students Creating ePortfolios, October 11, 2013 (Scott, Murday, Gilbert, Fierst,
Leitzell)
ePortfolio Design, November 22, 2013 (Alexander, Ward)
ePortfolio Discussion Forums, January 17, March 18, and April 9, 2013 (Scott organizer)
ePortfolio in Study Abroad, IUPUI International Festival, February 2014 (Leslie, Scott,
Clark, Grossman, Raider, Powell)
Electronic PDP Faculty Institute, June 16 – 20, 2014 (Buyarski, Kahn, Powell, Scott)
Custom Workshops and Consultations:
World Languages and Cultures, March 22, 2013
Study Abroad, March 25, April 2, and April 15, 2013
Paralegal Studies, October 3, 2013
Life-Health Sciences Internship Program, October 4, 2013
IU East, October 15, 2013
Philanthropic Studies undergraduate faculty, December 13, 2013
Anthropology and Intercultural Communications, May 22, 2014
12
APPENDIX D: CONSULTATIONS, GRANTS, NATIONAL PROJECTS, AND
LEADERSHIP
External Consultation and Visits Hosted
Kahn, S., & Scott, S. Visitors to IUPUI from DUOC in Santiago, Chile, June 4, 2013
(meeting arranged with Anton, M., Gosney, J., and Grew, D.)
Kahn, S., & Scott, S. Consultation with accreditation administrators at Austin Peay State
College in Tennessee, September 13, 2013
Kahn, S. Extended on-site consultation, including four workshops, four formal addresses, and
visits to ten sites, at DUOC UC in Santiago,Valparaiso, and Viña del Mar, Chile, November
16 – 25, 2013
Buyarski, C., Scott, S., et al. Visitors to IUPUI from South Africa, February 20, 2014
Kahn, S., & Scott, S. Visitor from Stanford University & San Francisco State University,
March 26. 2014
Kahn, S., Buyarski, C., Scully, T., & Scott, S. Visitors from Ivy Tech East Central Indiana
region, June 27, 2014
Grants
“Connect to Learning,” project funded by FIPSE; original $20,000 grant through December
2013 plus a six-month $2,500 extension through September 2014
“Using an ePortfolio to Assess the Outcomes of a First-Year Seminar: Student Narrative and
Authentic Assessment,” $2,500 grant to Cathy Buyarski, University College, from IUPUI
Program Review and Assessment Committee, June 2012 – December 2013
National Projects
IUPUI Research Team, Inter/National Coalition for Electronic Portfolio Research Cohort VI:
Susan Kahn, Project Director, ePortfolio Initiative
Catherine Buyarski, University College
Karen R. Johnson, Department of English
Cynthia M. Landis, University College and ePortfolio Initiative
Kristin Norris, Center for Service and Learning
Susan B. Scott, ePortfolio Initiative
Katherine V. Wills, IUPU-Columbus
Previous members no longer with the university: Debra D. Runshe, Center for Teaching
and Learning/UITS; Katherine Steinberg, Center for Service and Learning; Kathryn
Wilson, Center for Research and Learning
IUPUI Project Leadership Team for Connect to Learning:
Catherine Buyarski, Co-Project Director, University College
Susan Kahn, Co-Project Director, ePortfolio Initiative
Brandi Gilbert, Life-Health Sciences Internship Program
Michelle Jeschke, Department of Psychology
Cynthia M. Landis, University College and ePortfolio Initiative
13
Susan B. Scott, ePortfolio Initiative
Khalilah Shabazz, Student African-American Sisterhood/Diversity Enrichment and
Achievement Program
Cynthia Williams, Department of Psychology
Previous member no longer with the university: Debra D. Runshe, Center for Teaching
and Learning/UITS
Leadership
Susan Kahn:
Chair, Board of Directors, Association for Authentic, Experiential, and Evidence-Based
Learning (AAEEBL)
Lead Coordinator, ePortfolio Track, Assessment Institute
Manuscript Review Board, International Journal of ePortfolio, published by Virginia
Tech and the University of Georgia in conjunction with AAEEBL
Book Review Editor, Assessment Update, published by Jossey-Bass (including electronic
portfolio coverage)
Editorial Board, RAPPORT (The International Journal for Recording Achievement,
Planning and Portfolios), published by the Centre for Recording Achievement (UK)
Catherine Buyarski, University College: Program Planning Committee, AAEEBL National
Conference 2014
Julie Meek, School of Nursing: Peer Reviewer for International Journal of ePortfolio
Amy Powell, Center for Teaching and Learning: Program Planning Committee, AAEEBL
Midwest Regional Conference 2014
Susan Scott:
Coordinator, ePortfolio track of the annual Assessment Institute at IUPUI
Reader/editor, Catalyst for Learning articles on practice (all C2L campuses), published
online by Connect to Learning project
14
APPENDIX E: IU EPORTFOLIO FUNCTIONAL REQUIREMENTS
1. Collection (storage, management, and retrieval of digital artifacts): The system must
provide robust and user-friendly capabilities for uploading, storing, locating, managing,
sharing, and viewing files (artifacts) in all common formats, including plain text, video,
audio, graphics, databases, URLs to external resources, etc. in a personal online digital
workspace/repository, including
1.1. Ability to control who has access to one's own intellectual property (artifacts) via
permission settings that can be easily understood and changed.
1.2. Ability for portfolio authors to upload digital audio and video artifacts (and for the
audience to play and/or view them without downloading them first).
1.2.1. Does your system transcode and optimize digital video and audio?
1.2.2. Does your system offer audio/video streaming or progressive download?
1.3. Unlimited personal storage quota and/or the ability to adjust quotas to accommodate
users and programs with special storage needs.
1.4. Ability to add metadata to individual artifacts.
1.5. Ability to group and organize portfolio artifacts via tagging, folders, collections, etc.
1.6. Can artifacts be moved, renamed, or duplicated?
1.7. Does your system offer a search feature for locating artifacts in the collection?
2. Reflection: The system must provide robust and user-friendly capabilities for creating,
editing, sharing, and discussing reflections on any component (an artifact, group of artifacts,
page, group of pages) of a portfolio or on the entire portfolio, including:
2.1. Ability for instructors and facilitators to scaffold the process of writing reflections
with prompts or custom forms
2.2 How does your platform distinguish reflections from other types of artifacts that the
user might create with your system?
2.2. Can individual reflections be shared and discussed with or commented on by other
users?
3. Self-presentation (custom free-form or template-based presentations): The system must
provide robust and user-friendly capabilities for creating, editing, managing, and sharing any
15
number of showcase portfolios and/or other types of web-based presentations, including:
3.1. Ability to incorporate artifacts and reflections from personal collection/repository into
presentations.
3.2. Ability to share presentations securely with specific individuals or groups within or
outside the university, or make the presentation public.
3.3. Ability to control the look and feel of a presentation by selecting from a collection of
professionally-designed visual themes or skins:
3.3.1. Does your system also allow users to create their own themes/skins by selecting
banner, colors, fonts, navigation layout, etc.?
3.4. Ability to request and receive feedback on an entire presentation or any part of it.
3.5. Ability to submit a presentation for formal evaluation (and/or evaluate a presentation)
3.6. Ability for the institution to archive and preserve student presentations that have been
formally evaluated.
3.7. Ability for the owner to delete a presentation.
3.8. Ability to create a presentation template (with a predefined structure and prompts) or
use such a template to create a presentation
3.9. Ability to access and update one's own presentations over time, across multiple
learning experiences and potentially multiple institutions.
3.10. Ability to save and view prior versions of presentation.
3.10.1. Does your platform support content versioning and rollback?
3.10.2. Is it possible to save a snapshot of a presentation at a specific point in time?
3.11. Ability to add metadata to individual pages or sections or to the entire portfolio
4. Outcomes/Standards/Competencies/Goals Tracking and Assessment: The system must
provide robust and user-friendly capabilities for creating, publishing, viewing, and aligning
items with learning outcomes/standards/competencies at the course, program, school,
campus, or institution level, including:
4.1. Ability for instructors, advisors, assessment coordinators, etc. to align course
assignments and other forms of student work with one or more outcome or goal.
16
4.2. Ability for students to select and align their own representative work with one or more
outcome or goal.
4.3. Ability to assess student mastery of outcomes/competencies by evaluating student
work with rubrics aligned with one or more outcome or goal.
4.4. Ability to easily track one's own progress or the progress of individual students and/or
groups of students for whom one is responsible in terms of meeting personal or
institutionally defined outcomes or goals.
4.5. Ability to map the curriculum of a course to department-, program-, school-, or
campus-level outcomes or goals (i.e., curriculum mapping)
4.6. Does your system allow students to set their own academic, co-curricular, career, and
personal goals?
5. Guided or Directed Portfolios for Learning and Assessment: The system must provide
robust and user-friendly capabilities for designing, facilitating, and/or participating in a series
of guided portfolio activities/assignments (artifact collection and selection, reflection,
feedback and evaluation) over time within a class or program, including:
5.1. Ability to align one or more parts of the guided portfolio to specific learning outcomes
5.2. Ability to easily create custom forms to guide the processes of reflection, feedback,
evaluation, or for ad hoc data collection
5.3. Ability to easily track one's own progress or the progress of individual students and/or
groups of students for whom one is responsible in terms of completing or evaluating
the activities in the guided portfolio
6. Feedback (Informal Review): The system must provide robust and user-friendly
capabilities for requesting, providing, and managing formative feedback on the entire
portfolio (guided or presentation) or any part of it (individual artifacts, pages, activities, etc.),
including:
6.1. Ability for the portfolio admin/manager to assign reviewers.
6.2. Ability for the portfolio owner to request feedback from assigned reviewers or other
users with whom they wish to share their work.
6.2.1. Can the portfolio owner control who can see feedback on their work?
6.3. Ability to provide rich text feedback.
17
6.4. Ability to provide feedback using a rubric.
6.5. Ability to include attachments with feedback.
6.5.1. Does your platform allow reviewers to annotate and comment on student artifacts
without downloading the original and uploading the annotated versions?
6.6. Workflow support and notifications to help users manage feedback activities (i.e.,
requests for feedback and availability of new feedback),
7. Evaluation (Formal Review): The system must provide robust and user-friendly
capabilities for assigning, providing, and managing the formal evaluation of an entire
portfolio (guided or presentation) or any part of it (individual artifacts, pages, activities, etc.),
including:
7.1. Ability to assign specific evaluators to assess specific groups of students and/or
specific parts of a portfolio.
7.2. Ability to easily create, share, and use rubrics to guide evaluation (including self-
evaluation) of entire portfolio or any part of it (an artifact, collection of artifacts,
reflection, etc.).
7.3. Workflow support and notifications to help users manage evaluation activities (i.e.,
dashboard/notifications of pending evaluation work or availability of new
evaluations)
7.4. Ability to view and track the rating status (unrated, in progress, complete) of items or
students to which an evaluator has been assigned.
7.5. Ability for external (non-IU) evaluators to participate in the evaluation process.
7.6. For guided and directed portfolios, ability for evaluators to view the guidance
(assignment instructions, reflection prompts, supporting materials etc.) that led to the
creation of a particular artifact or reflection.
7.7. Ability to lock (or make a snapshot of) student work that has been evaluated so that it
can no longer be changed by the student
7.8. Does your platform support blind and double-blind evaluation?
7.9. Does your platform have tools for ensuring inter-rater reliability?
18
8. Reporting: The system must provide robust and user-friendly capabilities for generating
predefined and custom reports on portfolio evaluation results and portfolio status, including:
8.1. Ability to integrate portfolio data seamlessly with data in the Student Information
Systems (SIS) via live links or nightly import.
8.2. Ability to aggregate data relative to outcomes or competencies at the institution,
campus, school, program or course level in order to evaluate student learning and
program effectiveness.
8.3. Ability to view summary data for any given population (average, median, mean,
standard deviation, counts)
8.4. Ability to drill down from summary to detailed view of assessment data
8.5. Ability to view portfolio or merged portfolio/SIS data in a tabular format.
8.6. Ability to save (as HTML), print and/or export to a delimited format any report
8.7. Ability to generate status reports of various kinds to assist with managing portfolio
process (e.g., how many students completed particular portfolio assignments or
submitted work toward a particular outcome; how many portfolio assignments need
evaluation; which evaluators are/are not keeping up with evaluation work).
8.8. Ability to extract representative samples of student work at course, program,
institutional levels, sorted by learning outcome, major or school, class level, grades
and other categories above.
8.9. Ability to extract examples that show individual students’ progress over time (e.g.,
by learning outcome, proficiency level, status, etc.)
8.10. Please provide a descriptive list of the predefined reports available through your
platform.
8.11. Is it possible to generate custom reports via the user interface?
9. Tracking and Workflow: The system must provide robust and user-friendly capabilities for
tracking one's own tasks and progress as well as for tracking the tasks and progress of the
persons (students, evaluators, etc.) for whom one is responsible.
9.1. Does your system provide dashboard views for each role?
9.2. Does your system provide email or other types of notifications to help users manage
their portfolio work?
19
10. Two-way and Multiuser Communication: The system must provide robust and user-
friendly capabilities to facilitate two-way and multiuser communication within and among
individuals and groups of users related to portfolio work.
10.1. Does your platform offer an internal email or messaging service?
10.2. Does your platform support threaded discussions?
10.3. Does your platform include the ability to add comments to portfolios that have been
shared?
10.3.1. Can portfolio owners control who can see comments on their work?
10.4. What other types of communications tools does your platform offer?
11. Collaboration: The system should provide robust and user-friendly capabilities for
collaborative authoring and editing of an entire portfolio (guided or presentation) or any part
of it (individual artifacts, pages, activities, etc.).
11.1. Does your platform allow the portfolio owner to give permission to others to create
or edit specific pages within a portfolio?
11.2. Does your system allow the portfolio owner to give permission to others to edit the
entire portfolio?
12. Social Networking and Web 2.0 Technologies: The system should provide robust and user-
friendly support for social media and Web 2.0 technologies in ways that support and enhance
learning, reflection, and social pedagogies.
12.1. Does your platform include built-in social networking capabilities? If so, please
describe.
12.2. Does your platform allow users to create and/or join common interest groups in
which portfolios are shared and discussed?
12.3. Does your platform allow users to create and maintain a blog or incorporate an
external blog into a portfolio?
12.4. Does your platform allow users to subscribe to portfolio feeds from other users of
the system?
12.5. Does your platform allow users to incorporate profile data from LinkedIn or other
social networking sites into their portfolios?
20
13. User Experience: The system must be accessible by persons with disabilities; extremely
easy to use, and offer a clean, modern, and attractive interface:
13.1. The system must be accessible to persons with disabilities (e.g., section 508
compliant, NFB Gold Certification, etc.).
13.2. Please provide screenshots and/or other evidence (user testimony, recorded feature
demos, awards or certifications) of the usability of your product.
13.3. Does your solution permit full rebranding of the logo and color schemes?
14. Text Editor: The system should provide a robust and user-friendly rich text editor for
creating and editing presentations, reflections, feedback, evaluative comments etc., including:
14.1. The text editor must allow users to easily link to and/or embed rich media files,
including images, audio clips, videos, presentations, etc.
14.2. The editor should provide fine control over page layout (for example, the ability to
wrap text around images or videos, the ability to organize content in columns, etc.).
14.3. The editor must be able to gracefully accept content copied and pasted from
Microsoft Word.
14.4. The editor should allow users to edit the source HTML.
15. Mobile Support: The system should offer all roles, but especially students, a robust mobile
experience including the abilities to view, provide feedback, and evaluate portfolios as well
as the ability to create and save all types of portfolio artifacts on one's mobile device.
15.1. Has your platform been optimized for access by mobile phones and tablets?
15.2. Do you offer mobile apps for your platform? if so, what mobile platforms are
supported and features are available? If not, is the development of mobile apps on
your roadmap and what is the estimated delivery date?
16. Documentation: The system must offer complete online documentation for users in all roles.
16.1. Describe system level documentation for administration, development, and
customization.
16.2. Describe documentation available to users within the application
16.2.1. Does your system offer contextual help?
21
16.2.2. Can the online documentation be customized by the institution?
17. Access, Roles, Groups, and Permissions: The system must provide a robust and flexible
model for roles, groups, and permissions that allows students, advisors, instructors, mentors,
evaluators, etc. to easily locate and access their own portfolios as well as those of the users
with whom they are collaborating or for whom they are responsible, including:
17.1. Ability to assign roles and permissions on per context basis (e.g., a single user can be
a student in one context, an evaluator in another, and an instructor or manager in a
third)
18. Ability for portfolio owner to control who can see, comment on, discuss, or collaborate on
entire portfolio or individual items.
19. Integration – General:
19.1. Identify all third-party integration tools required for your solution, i.e., messaging,
EAI. Do any known hardware/software incompatibilities exist?
19.2. Is your application XML compliant?
19.3. Does your platform offer native support for ad hoc SQL queries? Describe the
method and level.
19.4. Does your platform include a workflow component? If so, can it be integrated with
a homegrown workflow engine (via web services)? i.e. users would see only one
Action List for this application along with our other workflow applications?
19.5. Does your platform include the ability to exchange data with other enterprise
systems?
19.6. Are APIs available to customers who wish to develop custom integrations?
19.7. Estimate resources needed to integrate with Indiana University systems; can this
work be done in-house at IU or does it require 3rd party consultants?
20. LMS Integration: The system must provide robust and seamless integration with the LMS
(or an open API for building such an integration) to facilitate real time data sharing and
exchange (e.g., the ability for students to locate artifacts created in the LMS and easily
incorporate them into their portfolio, the ability for instructors to simultaneously assess and
grade portfolio work and push those grades to the LMS gradebook, etc.)
20.1. Does your product offer standard integrations with Sakai, Canvas, Blackboard,
and/or Desire2Learn. If so, please describe in detail the capabilities afforded by each
integration.
22
20.2. Can your platform function as an LTI tool provider? If so, please describe in detail
the capabilities afforded by the LTI integration.
20.3. How will users in the LMS be mapped to users in your system?
20.4. Does your LMS integration require the addition or modification of tables in the LMS
database?
20.5. Does your application require a synching mechanism for the data in the LMS and the
data in your system? How is this accomplished?
20.6. The system should allow users to push or pull artifacts from the LMS into the
portfolio or vice versa
20.7. The system should allow instructors to push grades or ratings earned in the portfolio
platform to the gradebook in the LMS
20.8. The system should allow users to navigate seamlessly to and from the LMS via
single sign-on.
21. SIS Integration: The system must provide robust and seamless integration with the Student
Information System (or an open API for building such an integration) to facilitate data
sharing and exchange for a variety of purposes including: provisioning users and groups (or
courses) in the portfolio system, generating portfolio reports filtered by academic and
demographic criteria stored in the SIS, monitoring indicators of academic risk in the portfolio
system and feed to the early warning system in SIS, etc.
21.1. Does your product offer standard integrations with PeopleSoft or Kuali Student? If
so, please describe in detail the capabilities afforded by these integrations?
21.2. Does your system accept automated batch or real-time feeds from the student
information system?
21.3. Can your system use data from the SIS or other enterprise systems to provision
users, groups, and/or courses?
21.4. Can your system use data from the SIS to generate reports for specific populations of
users (for example, all graduating seniors, all first year Hispanic females, all students
in the electronic engineering program, etc.?)
23
APPENDIX F: USE CASE SCENARIOS FOR EPORTFOLIO VENDORS
SCENARIO 1: Mechanical Engineering Assessment Portfolio
Context: Mechanical engineering program at a large research university
Project scope and duration: all students and select courses in program
Primary Purpose: assessment of student learning in relation to institutional and professional
(ABET) outcomes for accreditation and continuous program improvement
Personas:
Assessment coordinator: Alicia, a mid-career faculty member who hopes to move into
administration. Currently she represents her department on the college assessment
committee and she also chairs the curriculum and assessment committees in her
department.
Instructor: Girish, Tenure track engineering professor (primary)
Student: Courtney, 4th year mechanical engineering
External program evaluator: Byung, chair of the ABET program evaluation team that
will be conducting the review of Alicia's program
As the assessment coordinator for her department, Alicia is responsible for gathering and
analyzing evidence of student attainment of the program outcomes defined by ABET (the
accrediting body for programs in the applied sciences, computing, engineering, and technology
education) and her institution's undergraduate learning outcomes. Wishing to take advantage of
the new assessment capabilities in the campus portfolio system, Alicia and her colleagues on the
department's assessment committee have been meeting regularly over the last year to review
and revise their program assessment processes. The committee devoted most of their time to
mapping the curriculum to ABET and campus learning outcomes, mapping ABET outcomes to
institutional outcomes, and developing standard rubrics for evaluating student work. Another
important consideration was minimizing the additional work required of faculty and students
participating in the assessment process.
Alicia is now ready to implement the assessment process recommended by the assessment
committee. She begins by inputting the outcomes on which students will be evaluated, a mix of
ABET outcomes and institutional outcomes. Several of the ABET outcomes are almost identical
to the institutional outcomes, so instead of entering them twice, she links or maps the ABET
outcome to the institutional outcome. Next, Alicia creates the portfolio framework that will be
used to collect examples of student work over the course of their studies and to evaluate those
artifacts using rubrics. Alicia then inputs the rubrics that will be used to evaluate student work in
each section of the portfolio. Each rubric (or portfolio section) is aligned with one or more of the
learning outcomes for which the program is accountable.
Once the portfolio framework has been completed, Alicia designates the evaluators for each
section. For some sections of the portfolio, faculty will be evaluating the work of their own
students only. For others, designated evaluators with specialized expertise will be evaluating all
24
student work. After double-checking all of her work, Alicia publishes the portfolio, outcomes,
and rubrics so they can be seen and used by all of the faculty and students in program.
Alicia's department head and dean have already notified faculty about the new system and their
role in the process, so Girish is already aware of his responsibilities. First, in his syllabus, he
must include a section that identifies the ABET and institutional outcomes most heavily
emphasized in the course. He must also incorporate at least one assignment, paper, or project in
which students will be able to document their abilities related to the outcomes listed in the
syllabus. Before the semester began, Girish made the necessary modifications to his syllabus
and revised the final project so that it addresses all of the emphasized outcomes. He decides to
use the final project as the portfolio assignment.
At the start of the semester, Girish tells his students that they will be required to upload their
final project to a section of the Mechanical Engineering portfolio. The evaluation of the project
in the portfolio will also serve as the student’s grade on the project. It's near the middle of the
semester, and Courtney is ready to begin work on the final project. She opens the relevant
section of the portfolio so she can see the rubric that will be used to grade her project. She
studies the rubric carefully in order to better understand the grading criteria for the
project. Courtney is an avid tennis player, so she decides to build something that could be
useful to her and her tennis buddies -- a robot that picks up tennis balls from a practice
court. She works on the project intensively for eight weeks. Her final product is a working
prototype of the robot, a video showing the robot in action, and a paper describing how it was
constructed. Courtney uploads the video and paper to the relevant section in the portfolio. The
instructions in the section ask Courtney to write a one page reflection about the knowledge and
skills acquired over the course of her undergraduate program that she drew upon when designing
and constructing the robot. Courtney spends the evening writing the reflection and submits the
section for evaluation.
After the due date for the final project, Girish sits down to start grading them. It usually takes
him 45-90 minutes to grade each project, and he's concerned about the extra work associated
with assessing student work in the portfolio platform. He logs in to the portfolio and sees that
his students’ submissions are awaiting evaluation. Girish opens the first submission, which
happens to be Courtney's. Before reviewing Courtney’s work, he reviews the rubric for this
section of the portfolio to remind himself of the grading criteria for the project. He then watches
the video, carefully reads her paper and reflection, and examines the working prototype that
Courtney turned in. He returns to the rubric and rates her on the criteria aligned with four
ABET outcomes:
1. an ability to design a system, component, or process to meet desired needs within realistic
constraints
2. an ability to identify, formulate, and solve engineering problems
3. ability to apply knowledge of mathematics, science, and engineering
4. ability to communicate effectively
Girish gives Courtney the highest possible rating for all criteria in the rubric except the
communications criteria because her skills are just about average. After inserting some
25
suggestions for improving the writing and organization directly into Courtney’s paper, Girish
submits his evaluation. Upon submission, Courtney’s score on the rubric is transmitted to the
Gradebook for Girish’s course in the LMS and Courtney is notified that her portfolio work has
been evaluated. Courtney was a bit disappointed when she saw that she hadn’t received her
usual perfect score, but Girish's comments and ratings for the communications criteria in the
rubric helped her understand how her writing could have been improved.
Alicia has been running reports periodically to make sure students and faculty are participating in
the portfolio. Pre-defined reports provide summary and detailed information on which students
have submitted work to each section, which sections have been evaluated, and which evaluators
are keeping up with their assignments. Alicia can also generate reports with student
performance results for each outcome. The reports give summary statistics (mean, median,
mode, standard dev, as well as counts and percentages) for each criterion in the evaluation rubric
and detailed results for each student. Alicia's program is up for review by ABET in two years,
and she's already thinking about how much easier it will be to prepare the written reports and
assemble information for the program review team.
Fast forward two years: Alicia has prepared a self-study document for the ABET program
review team that describes her department's assessment and continuous improvement
processes. The self-study includes summary and detailed reports on student attainment
generated by the portfolio system. Some reports compare results for different populations of
students --for example, freshman vs sophomore; women vs men; all students vs
underrepresented populations, etc. She sends the self-study to Byung, chair of the ABET
program evaluation team, prior to the review team's campus visit. After reading the self-study,
Byung indicates that the review team would like to see more rated examples of student
work. Alicia helps the review team establish guest accounts and gives them read-only access to
her department's assessment portfolio. These accounts give Byung and his team access to the
online versions of the reports, allowing them to drill down to see student submissions (evidence)
on which ratings were based. Byung and his review team are delighted to see that Alicia's
program is taking such a systematic approach to program assessment. However, they noted that
some faculty raters were noticeably more or less rigorous than others. Before leaving, they
provide her with some suggestions for gauging and improving inter-rater reliability.
Scenario 2: Personal Development Plan
Context: Portfolio embedded in a first-year seminar designed to be revisited and revised
throughout the student’s college experience
Project Scope and Duration: Approximately 3,000 new users each fall with continued
engagement with the portfolio through graduation
Primary Purpose: To assist students in developing a foundational portfolio that allows for
documentation and reflection of the student’s background, educational goals and career
goals. As the student continues their enrollment, evidence and narrative of significant
educational moments, achievements, and evidence will be placed in the portfolio. At the point of
graduation, students will have a comprehensive portfolio that 1) facilitated the student’s sense of
26
purpose, meaning making, and integration of learning experiences, and 2) can be used to support
employment or graduate school applications.
Personas:
First-Year Seminar Instructor: Dr. Jones a faculty member in the first-year seminar
course. He is using the PDP for the first time.
Student: Victoria Martinez is a first-year student at the university. She is majoring in
biomedical engineering and is considering going to medical school.
Academic Advisors: Marit Smith is the student’s assigned advisor in University
Division. Once Victoria completes all pre-admission requirements, she will be assigned
an advisor (Terry Byers) in the biomedical engineering department.
Faculty Mentor: Dr. Warnicke is a professor in the biomedical engineering program
and coordinates internships and undergraduate research for all students in the department.
Personal Development Plan (PDP) Coordinator: Susan Anderson is responsible for
the implementation and evaluation of the Personal Development Plan (PDP) in the first-
year seminars. She also serves as consultant to programs utilizing the PDP in programs
and departments as they utilize the portfolio with students beyond the first semester.
Victoria Martinez is a new full-time freshman at State University. She has chosen to major in
biomedical engineering because it brings together her interest in building and creating things
with her passion for health. Her father recently had heart surgery and she was fascinated by the
technical equipment that was used to support both his surgery and recovery.
As part of her first-semester course load, Victoria was required to enroll in a first-year seminar
course designed to support her transition to college. Dr. Jones, her instructor for the course,
attended a faculty development workshop on how using an ePortfolio could help students clarify
their reasons for being in college and develop a plan for both degree completion and taking
advantage of the multitude of programs and services offered by State University. He was
particularly interested in using reflective writing coupled with visual artifacts to engage students
in this process as well as how use of the PDP might foster the development of critical
thinking. In developing his syllabus, he sees that there is an established structure to the PDP
with seven pre-set sections which cannot be altered so as to provide a consistent portfolio
structure as students move through the institution. Within each section, reflective prompts are
provided but they are very general and he wants to alter the prompts so as to provide more depth
to support the focus of his course. He spoke with the PDP Coordinator, Susan Anderson, and
found that out that while there were a few prompts that could not be deleted or edited, there were
some he could change to better align with his course goals. As this was his first time using the
PDP, he was also very pleased to find out that within the portfolio system there was a repository
of prompts that other faculty had written that he could cut and paste into his own course version.
27
As the semester progressed, Victoria really enjoyed putting together her PDP. She liked that the
prompts were easily viewable within the portfolio structure and she was easily able to customize
the look of her portfolio to reflect her personality. She added photos and clip art to make her
portfolio interesting and a reflection of who she is. Her favorite part of building the portfolio
was when Dr. Jones required that they share their PDP with two other students in the class. She
liked being able to look at other students’ portfolios and found that the feedback they provided
her within the portfolio structure helped her revise her own PDP. Because she was able to revise
her portfolio without losing her first version, she was amazed at how more clearly she was able
to state her goals than at the beginning of the term. Dr. Jones also appreciated the ability to have
multiple versions of student work so that he could see the progress in their critical thinking
which was part of his grading rubric for the PDP.
At the end of the term, Dr. Jones was a bit overwhelmed with the grading and reading of all the
portfolios his students had submitted. The workload was eased by the fact that he could use an
established rubric in the portfolio system (with adjustments made for the prompts he had
added). The student’s work and rubric appeared on the screen together and made grading fairly
straight forward. He was able to complete his assessment and grading on each individual section
and well as for the entire portfolio using different rubrics for each. Criteria in the rubrics were
mapped to institutional learning outcomes as well as outcomes specific to the biomedical
engineering program. When he completed his grading, the grades on the PDP were easily
uploaded into the course gradebook.
Near the end of the semester, Victoria met with her academic advisor, Marit Smith, to discuss
her progress and plan courses for the next term. Because a student’s assigned advisor is able to
view their advisee’s PDP, Marit was able to review Victoria’s academic and career goals before
she came in; this made for a much more meaningful conversation and Marit was able to suggest
elective courses as well as a minor in medical humanities that will make Victoria a stronger
candidate for medical school. Marit also highly encouraged Victoria to get involved in
undergraduate research. Victoria was thrilled to find out that the four-year course plan she had
completed as part of her PDP was directly linked to the campus academic planning/degree audit
system and that she could easily incorporate Marit’s suggestions.
The following year, as a sophomore, Victoria was feeling very confident about her ability to
succeed in college and be accepted to medical school. Her classes were challenging and very
interesting. She was particularly intrigued by one faculty member’s research on the use of a
specific technology to heal nerve damage. She spoke with the faculty member after class and
was excited to learn that he was hiring an undergraduate research assistant but in order to meet
the university’s guidelines to get this research experience to be transcripted as part of her degree,
she would need to meet with Dr. Warnicke who oversees all experiential learning activities in the
department. Dr. Warnicke explained to her that she would have to complete a research portfolio
in which she would document what she had learned through the research including evidence of
her work and results. In addition, she would have to keep a sort of on-going journal of her work
each week which would be uploaded in her portfolio as evidence to support a final reflection
paper on how what she learned in the research experience was linked to what she was learning in
her courses. At the end of the research experience, Dr. Warnicke would review the portfolio and
28
determine if she met the requirements to have this experience appear on her transcript. Marit
agreed and was hired into the position for the academic year.
As a junior, Victoria was meeting with her departmental advisor, Terry Byers, who indicated that
she should start thinking about developing a strong application for medical school and that some
medical schools would like to see evidence of her goals, knowledge, and skills through an
ePortfolio. Victoria explained that she had two portfolios–the PDP she had created in her first-
year seminar in which she had written about why she wanted to go the health field and another in
which she had documented what she learned through her research experience. Terry explained
that she could easily use the portfolio she created in her first-year seminar as the foundation and
move artifacts and narrative from her research portfolio into her PDP. Victoria used most of the
fall semester to update her PDP and add evidence of her learning and experiences. As her
assigned departmental advisor Terry was able to view her PDP and provide her with detailed
feedback within the text of the document so it was easy to make updates and, by the end of the
semester, she felt she had a great web-based presentation that told her story and provided support
for why she was a great candidate for medical school. She simply put a secure web address
linking to her PDP on her medical school applications and hoped the admissions committees
would be impressed by her commitment to the medical field and the work she had done.
In her senior year, Victoria got a bit nervous about being admitted to medical school and started
looking for jobs in the medical devices industry as a back-up plan. She hadn’t given much
thought to how to conduct a job search so she made an appointment with the campus career
center. The career counselor asked Victor to provide her access to her PDP and then talked with
Victoria about how employers also like to look at portfolios but they don’t necessarily want to
read the “story” of her interest in medical school. Victoria again was very pleased to find out
that she could create another version of her PDP (without losing the version for medical school)
and tweak it by altering text, deleting and adding different sections, and changing the look of it
to be more appealing to the medical device industry.
Behind the scenes, Susan Anderson was responsible for assessing the use and effectiveness of
the PDP across the campus. She did this through running reports and viewing individual student
portfolios. As the portfolio administrator, she had access to individual student portfolios
without requiring their permission (as long as she used them for assessment purposes only). In
addition, she could run reports on data such as usage rates, completion rates (% of the portfolio
completed by each student and averages), and rubric ratings by student and
section/course/project.
29
APPENDIX G: PRELIMINARY FINDINGS FROM RESEARCH ON REFLECTION AND ASSESSMENT IN IUPUI EPORTFOLIO PROJECTS
Finding #1: Only a few of those adopting ePortfolios began with reflection as a primary goal.
“I saw those connections between what they were trying to do with PBL, this integrative
learning, and what the portfolio promised.”
Finding #2: Whether or not adopters initially understood the importance of reflection in
ePortfolios, most recognized and prized that role within the first term of use.
“I’ve come to think that this is not only an incredibly impactful form of assignment for students
to do, but it’s also a way that instructors can be assured that the way that they’ve designed their
class . . . is working or not working.”
Finding #3: Instructors almost always expressed surprise at students’ limited ability to reflect,
and they subsequently devoted considerable effort to helping students learn how to think
reflectively.
“I went from hoping that students would draw these deep connections to hoping that they would
just simply answer all parts of the question!”
Finding #4: The purposes of reflection were related to wide-ranging course or programmatic
objectives but may be summarized in two primary categories: to help students make connections
and to build self-understanding and metacognition.
“If they’re showing evidence of being able to pull all those things together and relate them to
whatever particular area they’re investigating, that’s what I was really after.”
Finding #5: Instructors reported using a range of approaches to elicit reflection appropriate to
the context.
Explanation and advocacy
Demonstration and practice
Structure and pacing of assignments
Social pedagogies
Formative and summative assessment
Finding #6: Assessment practices vary widely according to both students’ abilities and
instructors’ own understandings of reflection.
“How can you grade reflection? . . . It’s like grading somebody on their opinion of something.”
Finding #7: Many respondents perceived greater success with self-evaluation than with depth or
quality of reflection about course or program content.
“For usually a few students each year, it’s that light bulb kind of moment: ‘Oh that’s why I’m
drawn to this kind of work!’”
30
Finding #8: Respondents often described success in terms of the professional reward of seeing
evidence of their students’ learning and of seeing that the program or course had positively
affected their students.
“Really, really gratifying . . . The students seemed to use their individual ePortfolios as a
transformative, reflective learning experience.”
Finding #9: Respondents also noted direct benefits for themselves and their projects from
improved understanding of their own curricula as they “closed the loop” on their assessment and
reflected ever more deeply on their own teaching practice.
“We made a major curricular change . . . and a lot of that was due to the way we’re doing the
capstone portfolios.”
31
APPENDIX H: TASKSTREAM WEBFOLIO AND DIRECTED RESPONSE FOLIO (ASSESSMENT EPORTFOLIO) SCREEN SHOTS TaskStream WebFolio
32
Directed Response Folio: Program Coordinator View of Template Builder
33
Directed Response Folio: Student View of Template
Directed Response Folio Report: Instructor/Coordinator View of Performance by Rubric Dimensions
34
Directed Response Folio Report: Instructor/Coordinator View of Performance by Rubric Totals
Directed Response Folio: Student View of Results Rubric
