Upload
piers-stanley
View
221
Download
0
Tags:
Embed Size (px)
Citation preview
Building bridges to constituents: Building bridges to constituents: Communicating through performance Communicating through performance
indicators and electronic portfoliosindicators and electronic portfoliosTrudy Banta, Victor Borden, & Susan Trudy Banta, Victor Borden, & Susan
KahnKahn
Sharon Hamilton Policy and governance issues for electronic student portfolios
OverviewOverview
Planning for improvement PIs @ IUPUI
The tip of the assessment iceberg
iPort: The institutional portfolio Placing PIs in context
ePort: The electronic student portfolio Demonstrating student learning
Completing the cycle Linking accountability to improvement
Planning for ImprovementPlanning for Improvement
Planning & Budgeting 1. Mission, Vision, Goals developed 2. Unit goals aligned 3. Annual reports on web 4. Programs based on assessable goals,
with performance indicators 5. Biennial planning/budgeting hearings
conducted
Implementation
(Everyone on campus implements goals)
Evaluation
1. Academic and administrative program reviews
2. Evaluation of process effectiveness 3. Assessment of learning outcomes
in major in general education
4. Course evaluations 5. Student assessment 6. Constituent surveys 7. Management information and analysis 8. Program cost analysis 9. Web-based evaluation tools 10. Annual campus performance report 11. NCA accreditation
Assessable
Outcomes
Instrumentation
Data Collection
Analysis
Application
of FindingsCulture
OfEvidence
Improvement 1. Reporting to internal constituents
2. Demonstrating accountability to external stakeholders
3. Applying findings in campus improvement initiatives
4. Proposing improvement initiatives
5. Improving assessment methods Web-based
data
Electronic portfolios
Planning and BudgetingPlanning and Budgeting
1. Mission, Vision, Goals developed2. Unit goals aligned3. Annual reports on web4. Programs based on assessable
goals, with performance indicators5. Biennial planning/budgeting hearings
conducted
ImplementationImplementation
Everyone on campus implements goals
EvaluationEvaluation
1. Academic and administrative program reviews2. Evaluation of process effectiveness3. Assessment of learning outcomes
in major in general education
4. Course evaluations5. Student assessment6. Constituent surveys7. Management information and analysis8. Program cost analysis9. Web-based evaluation tools10.Annual campus performance report11.NCA accreditation
ImprovementImprovement
1. Reporting to internal constituents2. Demonstrating accountability to
external stakeholders3. Applying findings in campus
improvement initiatives4. Proposing improvement initiatives 5. Improving assessment methods
• Web-based data• Electronic portfolios
Planning for ImprovementPlanning for Improvement
Planning & Budgeting 1. Mission, Vision, Goals developed 2. Unit goals aligned 3. Annual reports on web 4. Programs based on assessable goals,
with performance indicators 5. Biennial planning/budgeting hearings
conducted
Implementation
(Everyone on campus implements goals)
Evaluation
1. Academic and administrative program reviews
2. Evaluation of process effectiveness 3. Assessment of learning outcomes
in major in general education
4. Course evaluations 5. Student assessment 6. Constituent surveys 7. Management information and analysis 8. Program cost analysis 9. Web-based evaluation tools 10. Annual campus performance report 11. NCA accreditation
Assessable
Outcomes
Instrumentation
Data Collection
Analysis
Application
of FindingsCulture
OfEvidence
Improvement 1. Reporting to internal constituents
2. Demonstrating accountability to external stakeholders
3. Applying findings in campus improvement initiatives
4. Proposing improvement initiatives
5. Improving assessment methods Web-based
data
Electronic portfolios
PIs @ IUPUIPIs @ IUPUI
Performance Indicators as the tip of the assessment iceberg
PIs @ IUPUI: BackgroundPIs @ IUPUI: Background
Status of PIs in the early 1990s Kells (OECD)
Failure of PIs for national accountability in Europe and elsewhere. Movement toward quality assurance
Borden & Banta (NDIR 82) Greater possibilities for PIs if they emerge from
comprehensive planning efforts Gaither, Nedweck, & Neal (ASHE ERIC)
Increasing demand for accountability and continuing inability of HEIs to communicate about performance to external audiences
The Rise of CSFs, KSFs, KPIs Rockhart (1979) Critical Success Factors (CSFs) NACUBO/ Peat Marwick – 67 Indicators Dolence (1989) KPIs for enrollment management
Balanced Score Card (BSC)Balanced Score Card (BSC)
Kaplan & Norton propose business model Financial performance Customer service and satisfaction Process effectiveness and efficiency Organizational learning
Ruben (1999) applies BSC to HEIs Teaching/Learning - Programs/Courses, Student Outcomes Service/Outreach - University, profession, alumns, state,
prospective students, families employers Scholarship/Research - Productivity/Impact Workplace satisfaction - Faculty/staff Financial - Revenues/expenditures
Problems with Current Problems with Current SystemsSystems
Despite Ruben’s challenge to focus on mission critical activities, most focus on administrative functions
There is a disconnect between performance indicators and core performance objectives
What is the organizational link between the accountability focus of campus executives and the performance of mission critical functions?
Making the ConnectionMaking the Connection
Doing assessment right (Banta, 2002) Participative planning
Assessment must be explicitly tied to institutional functions and be associated with goals that faculty and administrators value
Implementation Assessment is most powerful at the unit level where
changes will most immediately affect the faculty and students
Improving and sustaining while the results of assessment may be used to respond
to various external accrediting and stakeholder constituents, it will first and foremost provide the means by which the organization can continuously assess and improve itself
PIs @ IUPUI: Performance PIs @ IUPUI: Performance ObjectivesObjectives
Excellence in Teaching and Learning Attract and support a better prepared, more diverse student
population. Support and enhance effective teaching Enhance undergraduate student learning and success Provide effective professional and graduate programs and support
for graduate students and post-doctoral fellows. Excellence in Research, Scholarship, and Creative Activity
Conduct world-class research and creative activities relevant to Indianapolis, the state, and beyond.
Provide support for scholarly activity and external funding. Enhance infrastructure for research and creative activities.
Excellence in Civic Engagement, Locally, Nationally, and Globally Enhance capacity for civic engagement. Enhance civic activities, partnerships, and patient and client
services. Intensify commitment and accountability to Indianapolis, Central
Indiana, and the state.
Co
llab
ora
tio
n –
Div
ers
ity
– B
es
t P
rac
tic
es
PI Development ProcessPI Development Process
Vested interested groups (handout) Brainstorming session
What evidence would convince a (neighbor, prospective student, legislator) that you we were doing well?
What evidence do you (or could you) use in your unit to assess performance?
3-4 performance indicators per objective
Three level rubric (green, yellow, red) Adapted Nominal Group Technique for voting
Status and ResultsStatus and Results
In Handout Summary of performance objectives and
performance indicators, with measures shown for one set of indicators
Published in… Campus annual performance report Chancellors State of Diversity address Institutional portfolio
iPortiPort
The Electronic Institutional Portfolio
““Institutional Portfolio”Institutional Portfolio”
A focused selection of authentic institutional work, combined with narrative analysis, that demonstrates specific institutional achievements and shows learning and improvement over time—i.e., “institutional effectiveness”
IUPUI’s iPortIUPUI’s iPort
Begun in 1998 as part of national grant-funded project
Organized around main components of institutional mission and major strategic priorities
Intended for purposes of both internal improvement and accountability to multiple stakeholders
Used as platform for accreditation self-study in 2002
Is on the Web—available to all
What do we gain from work onWhat do we gain from work onan electronic institutional portfolio?an electronic institutional portfolio?
Occasions for focused, campus-wide discussions of areas needing improvement
A means of demonstrating institution’s commitment to accountability and effectiveness
An easy, accessible medium for communicating with stakeholders about accomplishments and effectiveness
Next StepsNext Steps
Develop new iteration/update as annual performance report
Update the design and technological infrastructure
Foreground links to other relevant campus Web sites
Incorporate more direct evidence of student learning through interface with student electronic portfolio
ePortfolioePortfolio
The Electronic Student Portfolio
Authn/Authz Security WorkflowComm.Tools
Storage
“Unbundling”
In Touch
Syllabus
Testing
Gradebooks
Schedule
EreservesDigital content
Full Text articles
Federated Searching
Learning ProfileLearning Matrix
Advising
Resume/Vita BuilderResearch Manager
Content Manager
Knowledge Mapper
Authn/Authz Security WorkflowCommTools.
StorageOther
ServicesOther
Services
ePortfolio Goals
Support students' learning of and engagement with the PULs over their entire undergraduate experience at IUPUI beginning in the freshman learning community and culminating in the capstone experience
Assist both faculty and students to reach a clearer, more coherent understanding of how various aspects of the curriculum support students' increasing mastery of the PULs
Contribute to assessment of student learning of the PULs at the levels of the individual student, the course, program, and institution
ePortfolio and the PULs
The ePortfolio is organized around IUPUI's Principles of Undergraduate Learning
1. Core communication and quantitative skills
2. Critical thinking
3. Integration and application of knowledge
4. Intellectual depth, breadth, and adaptiveness
5. Understanding society and culture
6. Values and ethics
ePortfolio Levels of Competence
Introductory: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 26 credit hours
Intermediate: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 56 credit hours
Advanced: What all baccalaureate students at IUPUI should know and be able to do in relation to PULs in their major or profession or academic program
Learning MatrixLearning Matrix
Planning for ImprovementPlanning for Improvement
Planning & Budgeting 1. Mission, Vision, Goals developed 2. Unit goals aligned 3. Annual reports on web 4. Programs based on assessable goals,
with performance indicators 5. Biennial planning/budgeting hearings
conducted
Implementation
(Everyone on campus implements goals)
Evaluation
1. Academic and administrative program reviews
2. Evaluation of process effectiveness 3. Assessment of learning outcomes
in major in general education
4. Course evaluations 5. Student assessment 6. Constituent surveys 7. Management information and analysis 8. Program cost analysis 9. Web-based evaluation tools 10. Annual campus performance report 11. NCA accreditation
Assessable
Outcomes
Instrumentation
Data Collection
Analysis
Application
of FindingsCulture
OfEvidence
Improvement 1. Reporting to internal constituents
2. Demonstrating accountability to external stakeholders
3. Applying findings in campus improvement initiatives
4. Proposing improvement initiatives
5. Improving assessment methods Web-based
data
Electronic portfolios