DIFFERENT APPROACHES TO QA AND THEIR IMPACT ON EFFICIENCY, EFFECTIVENESS AND SUSTAINABILITY

DIFFERENT APPROACHES TO DIFFERENT APPROACHES TO QA AND THEIR IMPACT ON QA AND THEIR IMPACT ON

EFFICIENCY, EFFECTIVENESS EFFICIENCY, EFFECTIVENESS AND SUSTAINABILITYAND SUSTAINABILITY

Ralph A. WolffRalph A. WolffPresident and Executive DirectorPresident and Executive Director

Accrediting Commission for Senior Colleges and UniversitiesAccrediting Commission for Senior Colleges and Universities Western Association of Schools & CollegesWestern Association of Schools & Colleges

Wolff - INQAAHE March 2009Wolff - INQAAHE March 2009 22

"Tell me and I forget. "Tell me and I forget.

Teach me and I remember.Teach me and I remember.

Involve me and I learn."Involve me and I learn."

Ben FranklinBen Franklin


Forces Leading to New Forces Leading to New ApproachesApproaches

New types of institutionsNew types of institutionsMergers, acquisitions, joint programs, dual Mergers, acquisitions, joint programs, dual degrees, transnational provisiondegrees, transnational provisionNew modalities of deliveryNew modalities of deliveryHeightened expectations of QA agenciesHeightened expectations of QA agenciesIncreased emphasis on student learningIncreased emphasis on student learningGlobal competitionGlobal competitionDissatisfaction of premier institutionsDissatisfaction of premier institutionsEconomic downturnEconomic downturn


A Personal Learning JourneyA Personal Learning Journey

Receipt of grants from two foundationsReceipt of grants from two foundations

Work with principles of learning Work with principles of learning organizationsorganizations

Study of British QAA Audit processStudy of British QAA Audit process

Participation in Hong Kong TQLPRParticipation in Hong Kong TQLPR

Involved with 4 Handbook revisionsInvolved with 4 Handbook revisions

Member of Dubai UQAIBMember of Dubai UQAIB


Opening the Door to Change: The Opening the Door to Change: The Questions We AskedQuestions We Asked

What do you most want from your What do you most want from your accreditation experience? Did you get it?accreditation experience? Did you get it?

What was the half-life of your last report? What was the half-life of your last report? Did it lead to real change/improvement/Did it lead to real change/improvement/

Who was the report for? WASC or the Who was the report for? WASC or the institution?institution?

Were key stakeholders deeply engaged in Were key stakeholders deeply engaged in self review and preparation of the report?self review and preparation of the report?


The Answers We ReceivedThe Answers We Received

Reviews need to be value adding – they aren’tReviews need to be value adding – they aren’tThe process costs too much The process costs too much Focus on continuous improvement Focus on continuous improvement Have high standards that lead and guideHave high standards that lead and guideRely more on existing data/evidenceRely more on existing data/evidenceProvide external accountability to avoid Provide external accountability to avoid governmental interferencegovernmental interferenceFocus on key issues Focus on key issues Address “accreditation fatigue”Address “accreditation fatigue”


Mental Models of QualityMental Models of Quality

Can you identify the top institution in your Can you identify the top institution in your country in your jurisdiction?country in your jurisdiction?


Evolution of QA GoalsEvolution of QA Goals

Meeting institution’s stated goalsMeeting institution’s stated goals

Meeting external standardsMeeting external standards

Assuring adequate quality systemsAssuring adequate quality systems

Assuring effective learning resultsAssuring effective learning results

Benchmarking- comparing resultsBenchmarking- comparing results

RankingRanking


Developing and Applying Developing and Applying Standards – Striking A BalanceStandards – Striking A Balance

Compliance -- -- -- -- -- ImprovementCompliance -- -- -- -- -- Improvement

Minimum -- -- -- -- -- -- AspirationalMinimum -- -- -- -- -- -- Aspirational

Prescriptive -- -- -- -- -- AdaptivePrescriptive -- -- -- -- -- Adaptive

Quantitative -- -- -- -- -- QualitativeQuantitative -- -- -- -- -- Qualitative

Inputs -- Processes – - OutcomesInputs -- Processes – - Outcomes

Comprehensive -- -- -- - FocusedComprehensive -- -- -- - Focused


Reframing the Review ProcessReframing the Review Process

Single Review -- -- -- -- Multiple ReviewsSingle Review -- -- -- -- Multiple Reviews

Episodic -- -- -- -- -- -- - ContinuousEpisodic -- -- -- -- -- -- - Continuous

Formative -- -- -- -- -- -- SummativeFormative -- -- -- -- -- -- Summative

Standards Focused -- --Theme FocusedStandards Focused -- --Theme Focused

Interview Based -- -- -- Evidence BasedInterview Based -- -- -- Evidence Based

Atomistic -- -- -- -- -- -- - SystemicAtomistic -- -- -- -- -- -- - Systemic

Judgmental -- -- -- -- -- - DevelopmentalJudgmental -- -- -- -- -- - Developmental


Applying Quality PrinciplesApplying Quality Principles

Leadership commitment Leadership commitment

Responsibility for all studentsResponsibility for all students

Range of variability Range of variability

Sampling/Audit trailsSampling/Audit trails

Process mappingProcess mapping

Continuous improvementContinuous improvement


US Accreditation ReformUS Accreditation Reform

The shift from an episodic ‘add-on’ to a The shift from an episodic ‘add-on’ to a value-adding processvalue-adding process The shift from inputs to alignment of The shift from inputs to alignment of processes to learning outcomesprocesses to learning outcomes A shift from a single comprehensive visit A shift from a single comprehensive visit to a variety of new visit processes to a variety of new visit processes A redefinition of accreditation from a A redefinition of accreditation from a conservator of traditional values to an conservator of traditional values to an agent for changeagent for change


Southern Association (SACS)Southern Association (SACS)

Significant reduction in the number of Significant reduction in the number of standards standards

Off-site compliance reviewOff-site compliance review

Institutional Quality Enhancement Plan Institutional Quality Enhancement Plan (QEP) review (6 months)(QEP) review (6 months)


Academic Quality Improvement Academic Quality Improvement Program (AQIP)Program (AQIP)

Only for institutions in good standingOnly for institutions in good standing

Quality systems centeredQuality systems centeredSelf assessmentSelf assessmentStrategy forumStrategy forumAction projectsAction projectsAnnual external reviewsAnnual external reviewsSystems portfolio/review (4 years)Systems portfolio/review (4 years)Institutional review (7 years)Institutional review (7 years)


WASCWASC

Learning-centered 3 stage review processLearning-centered 3 stage review processProposal (peer reviewed)Proposal (peer reviewed)Capacity and Preparatory Review (20-24 months))Capacity and Preparatory Review (20-24 months))Educational Effectiveness Review (18-24 months) Educational Effectiveness Review (18-24 months)

Emphasis on student and organizational Emphasis on student and organizational learninglearningPage limits; reliance on institutional Page limits; reliance on institutional portfoliosportfoliosEvidence centeredEvidence centered


Institutional ImpactInstitutional Impact

New approaches are having a significant New approaches are having a significant impactimpactShifting to targeted areas of improvement Shifting to targeted areas of improvement that will make a real differencethat will make a real differenceRevealing huge “capacity” issues relating Revealing huge “capacity” issues relating to faculty training and institutional datato faculty training and institutional dataCited by many as “transformational”Cited by many as “transformational”Involves more work than ever before and Involves more work than ever before and greater valuegreater value


Where We Are Still ChallengedWhere We Are Still Challenged

Changing institutional culture to focus on student Changing institutional culture to focus on student learninglearningCompeting with the rankings/reputational Competing with the rankings/reputational paradigmparadigmShifting from mission-centered to public Shifting from mission-centered to public accountabilityaccountabilityDeciding what to make publicDeciding what to make publicDefining what level of learning is “good enough”Defining what level of learning is “good enough”Addressing under preparedness of studentsAddressing under preparedness of studentsImproving rates of student completionImproving rates of student completion


Lessons Learned/ImplicationsLessons Learned/Implications

QA agencies need to define and periodically QA agencies need to define and periodically challenge their own models of quality challenge their own models of quality Revisions to visit processes are as important Revisions to visit processes are as important and impactful as revising standardsand impactful as revising standardsQA agencies have tremendous intellectual QA agencies have tremendous intellectual capital that makes us all leaders capital that makes us all leaders Focus initially is on capacity; needs to move to Focus initially is on capacity; needs to move to effectivnesseffectivnessWe all need to address what we will do after the We all need to address what we will do after the second round of reviews under existing modelssecond round of reviews under existing models


Stages of QAA DevelopmentStages of QAA Development

Formational – defining scope, creating Formational – defining scope, creating standards, generating consentstandards, generating consent

Operational – implementing, creating Operational – implementing, creating support materials, building capacitysupport materials, building capacity

Developmental – training institutions to Developmental – training institutions to meet new expectationsmeet new expectations

Transformational – redefining purposes to Transformational – redefining purposes to serve new needsserve new needs


ConclusionConclusion

We have a long way to go and a lot to learnWe have a long way to go and a lot to learnWe are competing with budget cuts in many We are competing with budget cuts in many places that shift priorities and focus away from places that shift priorities and focus away from q.a. systems and assessment of learningq.a. systems and assessment of learningInstitutions continue to build recognition through Institutions continue to build recognition through increased research and selectivity increased research and selectivity Innovation, experimentation and adaptation Innovation, experimentation and adaptation worksworksOur QA processes holds great promise to lead Our QA processes holds great promise to lead the world in defining, evaluating and improving the world in defining, evaluating and improving qualityquality


Personal ReflectionsPersonal Reflections

New approaches involve riskNew approaches involve risk

Risk-taking requires courageRisk-taking requires courage

Meaningful change requires persistence Meaningful change requires persistence and timeand time

Consent-building is ongoingConsent-building is ongoing

Founded on passion for student and Founded on passion for student and organizational learningorganizational learning

Documents

DIFFERENT APPROACHES TO QA AND THEIR IMPACT ON EFFICIENCY, EFFECTIVENESS AND SUSTAINABILITY