This article was downloaded by: [RMIT University]On: 27 August 2014, At: 15:22Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Quality in Higher EducationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/cqhe20

Quality Monitoring in HigherEducation: the impact on studentlearningMargaret Horsburgh aa Auckland Institute of Technology, Director (AcademicDevelopment) , Private Bag 92006, Auckland 1020, NewZealandPublished online: 02 Nov 2010.

To cite this article: Margaret Horsburgh (1999) Quality Monitoring in Higher Education:the impact on student learning, Quality in Higher Education, 5:1, 9-25, DOI:10.1080/1353832990050102

To link to this article: http://dx.doi.org/10.1080/1353832990050102

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information(the “Content”) contained in the publications on our platform. However, Taylor& Francis, our agents, and our licensors make no representations or warrantieswhatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions andviews of the authors, and are not the views of or endorsed by Taylor & Francis. Theaccuracy of the Content should not be relied upon and should be independentlyverified with primary sources of information. Taylor and Francis shall not be liablefor any losses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly or indirectly inconnection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expresslyforbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Quality in Higher Education Vol. 5, No. 1, 1999

Quality Monitoring inHigher Education: the impact onstudent learningMARGARET HORSBURGHDirector (Academic Development), Auckland Institute of Technology, Private Bag 92006,Auckland 1020, New Zealand

ABSTRACT This paper reports on a case study set in the context of one New Zealand educationalinstitution, the Auckland Institute of Technology, which considered the impact of quality monitor-ing on the improvement and enhancement of student learning within two undergraduate degreeprogrammes. Quality in higher education is multifaccted and complex, and although there aredifferent perceptions of quality monitoring in higher education, quality, whatever its focus, hasbecome the vehicle through which accountability is addressed. It is argued that the focus for qualityshould, in a rapidly changing world, be on the attributes of graduates, where transformation of thelearner is central. Quality monitoring should be concerned with improvement and enhancement ofstudent learning. In the study, quality monitoring processes had quite a narrow impact and werenot concerned with the complexity of a whole teaching programme, or issues such as leadership orthe culture in which students learn. The influence of the social, economic, political and personalcontext in which the two programmes are situated was also considerable. Overall analysis confirmedthat, for quality monitoring to have an impact on student learning, the emphasis must be oncurriculum, learning, teaching and assessment.

Introduction

Although there are different perceptions of quality monitoring (the term quality monitor-ing is used to refer to the broad set of quality-related activities or evaluations that occureither external or internal to an organisation) in higher education, quality whatever itsfocus has become the vehicle through which accountability is addressed. Accountability isassociated with efficiency and effectiveness, with concepts of quality based on dimensionsof higher standards, zero defects, value for money or fitness for purpose (Harvey & Knight,1996). It is argued that none of these concepts directly encompass the core activities ofhigher education, those associated with teaching and learning. The focus for quality shouldin a rapidly changing world, be on the attributes of graduates, where transformation of thelearner is central. Quality monitoring should be concerned with improvement andenhancement of student learning. This paper reports on a study concerned with thequestion: to what extent does quality monitoring impact on the student experience oflearning?

A case study set in the context of one New Zealand (N.Z.) educational institution—aN.Z. polytechnic, the Auckland Institute of Technology (AIT)—considered the impact of

1353-8322/99/010009-17 © 1999 Taylor & Francis Ltd

9

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

10 M. Horsburgh

quality monitoring on the improvement and enhancement of student learning within twoundergraduate degree programmes (Horsburgh, 1998). The author interacted with theparticipants (staff, students, employers) on two degree programmes over a two and a halfyear period.

Quality as Transformation

A review of the literature included consideration of quality as transformation. Understand-ing quality as transformation involves regarding education as a transformative process inwhich the student is an active participant rather than a passive receiver. In short, theeducational process may transform by enhancing and empowering the student. Enhance-ment is reflected in the addition of knowledge and skills. Empowerment is the develop-ment of students' critical ability, that is their ability to 'think and act in a way thattranscends taken-for-granted preconceptions, prejudices and frames of reference' (Harvey& Knight, 1996, p. 4). At its core, transformation refers to the evolution of the way studentsapproach the acquisition of knowledge and skills and relates them to the wider context.A recent study (Harvey et ah, 1997a) showed that in a changing world, employers needgraduates who are not only bright and have the higher level academic abilities of analysis,synthesis and critique, but must also be adaptable, flexible, self-motivated and self-assuredand able to interact effectively in teams and have good interpersonal skills. Graduates needto have a range of personal attributes, including knowledge, intellect, willingness to learn,flexibility and adaptability, self-regulatory skills, self-motivation and self-assurance. Theyalso require interactive attributes such as interpersonal skills, team work and communi-cation skills. Nightingale (1997), in reviewing the role of higher education in Australia,explains that graduates need to be flexible and adaptable and, in addition to being creative,resourceful, lateral and logical thinkers, must also be able to synthesise information andconstruct meaning from the flood of data available in a technological world. They will needto be lifelong learners and they will need transformative skills. The Review of HigherEducation Financing and Policy (West, 1998) in Australia formalises these ideas, and recog-nises that quality in education must be seen in terms of what students can do at the endof their education experience. This is what employers and society want. Employers wantpeople who can go forward, who see change as an opportunity not a threat. Theywant transformative agents.

Transformation provides an overarching concept of quality in education, with otherconcepts of quality, such as 'fitness for purpose' or 'consistency' possible means towardsachieving transformation but not ends in themselves (Harvey & Knight, 1996). If highereducation is not about transforming the life experiences of students, then it is unlikely tobe fulfilling its potential (Barnett, 1992). This is reinforced by other authors, 'the goal oftertiary education... should be to change students' interpretation of their world'(Biggs, 1989, p. 10). Quality is seen as the extent to which education transforms theconceptual ability and self-awareness of the student (Ramsden, 1993) and providesthe skills and abilities for actively contributing to a rapidly changing world.

If achieving quality in higher education requires a focus on developing transformativegraduates, then the total learning environment is important. 'Quality' needs to be under-stood as a transformative process, it cannot be separated from learning, teaching, assess-ment, institutional practices and structures and the institutional, departmental and facultyculture and climate. At issue is that, to date, the predominance of accountability as a focusfor quality, means that quality monitoring is frequently concerned with inputs, outputsand systems, rather than processes and learning outcomes, and may have little to do with

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

Quality Monitoring in Higher Education 11

Curriculum

Learning Processes

Assessment

Teaching and Teachers

Learning Resources and Support

Institutional Systems andStructures

Organisational/Facuhy/School

Culture

Socio-Economic and PoliticalContext

• Influence

Transformation

Resulting

in graduates

with vocationally relevantknowledge, skills and abilities

•who are life-long learners andunderstand how they learn

• who can act independently

•who self-assess

•who can communicate effectivelyand have appropriateinterpersonal skills

•who are effective team players

•who create new knowledge

•who can analyse, critique andsynthesise

• who act as well as think, andreflect on action and thinking

• who see new ways to solve oldproblems and think laterally

•who are creative and innovative

• who can adapt to differentenvironments

•who are able to use up to datetechnological developments

FIG. 1. Elements contributing to transformation.

learning and teaching. The troublesome area of outcomes and quality is avoided (Caveci al., 1990). A compliance culture amongst academic staff is a possible outcome, ratherthan a culture where improvement and innovation in teaching, which will lead totransformation of students, is valued.

To provide the basis for focusing the study, a synthesis of the literature identified theelements needed for transformation. These are presented as a framework for transform-ation (Figure 1). Taking this a step further, the factors that make up the various elementsof the framework are outlined in Table 1. The concepts and framework depicted providedthe theoretical basis for the study and helped to structure the observations, interviews anddocument reviews or information gathering for the study.

Methodology

Two educational programmes were chosen for in-depth study over a two-and-a-half-yearperiod. The intent was to enhance and maximise the author's understanding regarding theimpact quality monitoring may have on student learning, particularly in relation totransformation. It was further the intention to understand what other factors may beimpacting on student learning.

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

12 M. Horsburgh

TABLE 1. Factors which contribute to transformation

CurriculumCurricula must:

—enhance with knowledge and skills;—include opportunity for the development of a

wide range of attributes;—define programme aims and learning outcomes;—empower with flexibility and choice and allow

for student input;—encourage personal development.

Learning processesCurriculum implementation must:

—focus on learning and the developmentof metacognition;

—allow for the achievement of a wide rangeof attributes;

—encourage deep learning;—encompass student centred learning;—see students as active participants;—empower through the development of

critical ability.

AssessmentAssessment must:

—support learning;—be outcomes based;—be credible and reliable;—provide for useful and swift feedback;—incorporate multiple methods;—allow for integrated and holistic assessment.

Organisational/faculty/school cultureThe culture within the organisation and thedifferent levels of an organisation must:

—be self-critical;—value teaching and learning;—seek to continuously improve/change

with a changing environment;—be one where student and staff input is

integral;—foster collegiality which is learning-oriented

and outward-looking.

Teaching and teachersTeaching must:

—be innovative with a variety of approaches;—be student-centred;—emphasise learning;—involve appraisal which focuses on learning

and teaching improvementTeachers must:

—engage in shared reflective practice;—aiscuss pedagogical issues;—engage in professional development which

allows for transformative learning.

Learning resources and supportThe learning environment needs to be onewhere:

—learning and teaching are valued;—research enhances teaching;—library, computing and other learning

resources are appropriate and adequate;—learning support is available.

Institutional systems and structuresAn institute's systems should encompass:

—recognition and reward for transformativeteaching or good learning facilitation;

—structures and regulations that allow for bothcurricula and learning integration as wellas flexibility;

—student centred learning;—management action in response to

student feedback, and monitoring reportswhere appropriate.

Socio-economic and political contextThe broader context of education needs tobe one where:

—resources support 'quality7 learning;—political discourse values learning;—national systems understand learning.

The New Zealand Education Amendment Act, 1990, enabled degrees to be awarded outsidethe New Zealand university system, and undergraduate degrees commenced at AIT in1991. All undergraduate degrees are subject to the same internal and external qualitymonitoring processes and, in 1996 when the study commenced, AIT was offering 15undergraduate degrees spanning a range of implementation dates commencing from 1991.In recognition of this, and to ensure that different stages of quality monitoring might be

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

Quality Monitoring in Higher Education 13

reflected, the selection of the two cases included consideration of the timeframe in whichthe programmes had been in place. One degree (Programme A) had commenced in 1992and the second (Programme B) in 1995. Stakeholders for the two degrees also varied andincluded for one, professional body interest, as graduates are eligible to apply for regis-tration as an accountant. The literature review had indicated that professional bodies as aparticular stakeholder group in education may have impact on educational programmes,which is quite specific to their role and different from the influence and impact of externalquality agencies. It was intended to capture this within Programme A. No particular bodyor employer group was associated with Programme B.

Data sources included:

1. Information, both published and unpublished literature related to the development ofexternal quality monitoring in New Zealand, and with reference where appropriate tointernational developments.

2. Existing documentary evidence relating to the development of internal quality monitor-ing policies, practices and procedures within the Auckland Institute of Technology. Thisincluded evaluative reports (organisation, faculty, department, division, programme),committee minutes and Academic Board decisions.

3. Detailed documentation relating to the two programmes. This included curriculumdocuments, student handbooks and student assessment plans.

4. Interviews where appropriate with key informants involved in the development andreview of external (national) and internal (institutional) quality-monitoring systems.

5. Quantitative data from existing surveys such as student evaluations of teaching, andstaff, student and graduate satisfaction surveys.

6. Qualitative data directly related to the student experience and student learning, derivedfrom focus groups and in-depth interviews with students, programme staff, and otherkey respondents such as employers.

7. Current and ongoing research into 'quality' processes and practices internationally andwithin New Zealand.

Analysis was shaped by a critical approach where interpretation only makessense against a background of social rules, practices and beliefs. Social, political andorganisational structures matter, and may have enabling and constraining effects.

What Does Have an Impact on Student Learning

Overall, the analysis of quality monitoring at AIT, the interviews with employers and theinteraction of the researcher over time with staff and students on the two programmesprovided contrasting examples of curricula and student groups. Both programmes operatewithin the environment of a rapidly changing educational institution, interacting withinthe influence of many social, political and economic factors. The programmes themselvesare dynamic and constantly changing.

Despite the contrasts, the same factors were found to be impacting on each programmeand either directly or indirectly on the student experiences of learning. However, theywere impacting in different ways and there was considerable interaction between thefactors. The factors that emerge as impacting include some elements of quality monitoring(both internal and external). Overall, the following eight factors were identified as havingsignificant impact:

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

14 M. Horsburgh

• the curriculum intent and philosophy, along with the programme-development teamwho determine what this will be;

• leadership;• external approval panel members, external programme monitors and the influence of a

professional body;• the teaching staff and how they teach and assess students;• the teaching staff and the environment in which they teach;• programme-specific internal quality monitoring, both formal and informal;• resource issues;• student expectations.

The Curriculum and the Programme Development Team

The philosophy of the curriculum, what its overall intent or purpose is, the structure of thecurriculum and the strategies developed to implement the curriculum, all had an impacton the student learning experience. The team who developed and implemented thecurriculum also had an impact. Donald (1997) believes that, to a large extent, the disciplinedetermines the learning objectives for programmes and modules, together with the kind ofteaching and learning strategies and assessment methods that will be used. Differences incurricula, depending on the discipline base from which they originate, are well describedin the literature (Becher, 1989; Donald, 1997) and can account for some of the differencesseen in this study. Academic disciplines represent ways of thinking and provide the basicbuilding blocks and a framework around which teaching is organised. For the programmesin this case study, having or lacking a disciplinary base at the time of initial curriculumdevelopment strongly impacted on the curriculum outcome and the way in which theprogramme was initially implemented. To achieve the broad outcomes intended from acurriculum, which might help students to achieve transformative skills, Donald (1997)argues that it is necessary to bridge disciplinary boundaries: there must be an emphasis onvaluing generic learning outcomes, rather than just the knowledge and skills associatedwith a discipline. An alternative is to focus on the development of research skills in adisciplinary context (Harvey & Knight, 1996).

Programme A is a complex multi-disciplinary programme offering majors in sevendisciplines. All students irrespective of their major complete a commmon core, an initialthree semesters of integrated studies, before choosing to major in one or two areas. InProgramme A the curriculum philosophy required discipline boundaries to be bridged andthere to be communication across business disciplines. The outcome is a curriculum wherethe philosophy, intent and student experience is one of integration and achievement ofbroad attributes. This is particularly evident in the first three modules (or first threesemesters) where modules involve integration and where the teaching team is multi-disciplinary. The curriculum-development team comprised lecturers from a mix of disci-plines but also with a strong base in education. Staff knew about educational researchrelating to generic attributes, understood that employers value generic attributes and werecommitted to developing a curriculum that would help students achieve these attributes.The strength of discipline-related feeling about how a programme should be structuredand taught was recognised and experienced. The difficulties in putting together anintegrated curriculum, in having it approved and in implementing it provided evidence ofthis. A whole change to learning, teaching and assessment approaches was seen, when thestudents move from the integrated modules to professional studies, and provides support

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

Quality Monitoring in Higher Education 15

for Donald's (1997) view that disciplinary barriers must be broken down if broad learningoutcomes of the kind needed by graduates in the next century are to be achieved.

A change in management structure within Programme A's Faculty coincided with theintroduction of this programme. It reflected the determination of the programme develop-ment team, certain of the value of a curriculum intent on achieving generic graduateattributes, to take whatever steps were needed to break down barriers and achieve anintegrated curriculum. A matrix management structure meant the end of the relativeisolation of discipline specialists.

Programme B (a language programme) is a major area of study in a degree with acommon framework of core modules and subject specialisations. For Programme B therhetoric was for a curriculum intended to achieve broad learning outcomes. However,the development and implementation processes made this difficult. The curriculum wasdeveloped by administrators with little communication across the disciplines that wouldbe involved in implementation. Core modules were to provide students with generic skillsand the discipline base of the major area of study would reinforce these. There wasminimal involvement in the curriculum development from the lecturers of the majordiscipline and no reference as to how, where or why the generic learning outcomes mightbe appropriate for integration into their discipline. Economic issues prevailed. Another'discipline' was to 'teach' the core modules, and opportunities to communicate or discussbridging the disciplinary boundaries presumably were not deemed necessary. Donald(1997) is clear that the differences between disciplines are authentic, in terms of theterminology they use, the way things are documented and in fact whether it is evenpossible to agree conceptually. At the time of implementation of Programme B, theteaching staff involved in the core modules were not using learning outcomes, theirassessment methods were different from what was intended in the 'new' degree and theyfelt no commitment to the degree other than in economic terms, it brought more teachingto their school. The outcome has been a curriculum where both staff and students havestruggled to clarify the overall intent or purpose, and where the achievement of the statedbroad learning outcomes in the original curriculum documentation are not yet recognisedby staff and students as being achieved. Economic and political factors have interfered and,even when two discipline areas were able to agree on an educational rationale, economicissues prevailed and compromises were made. There are still issues to resolve, one of theseis the purpose of a practicum module, with the original emphasis or requirement for aproject in a workplace having shifted to a self-directed learning project. A recent UK report(Harvey et ah, 1998) on work experience in curricula makes it clear that with the increasingrequirement for graduates to be work ready, actual learning in a workplace is important.Not only can learning in a workplace play an important role in helping students achievework readiness, it also has a role in the development of students as lifelong learners.Students see that learning goes on in the workplace as well as in the institution. Lifelonglearning is a requirement for transformative graduates and certainly a trait valued andexpected by employers. The opportunity for workplace learning in Programme B is nowoptional and it is yet to be clarified just what the practicum should achieve.

Leadership

Leadership impacted significantly, positively through the development and implemen-tation of Programme A, whereas a lack of leadership impacted negatively in the earlystages of Programme B. Only when programme leadership related to experience in the

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

16 M. Horsburgh

discipline, and the need for it was recognised, did the Programme B teaching team beginto clarify overall direction and address how this might be achieved.

Both programmes are relatively new, change was required to introduce both. Leadershipwas critical to achieve the intended changes and introduce an innovative and integrateddegree such as Programme A, with all the associated risks (for example, competitordegrees all require only 3 years of study and this programme requires 4 years or equivalentstudy), and the shake-up of staff and discipline traditions that was needed. Similarly,leadership to achieve change from a language diploma to a degree was needed forProgramme B. Leadership is defined in numerous ways but a common understanding ofleadership is as 'a process of social influence whereby a leader (or group of leaders) steersmembers towards a goal' (Bryman, 1992, p. 2). Change and leadership have a symbioticrelationship. Change creates the need for leadership and leaders are, or are perceived to be,initiators or drivers of change (Middlehurst, 1997). Change creates instability, uncertaintyand a need for adaptation in individual roles and attitudes, as well as in organisationalstructures and cultures. There becomes both a psychological and a practical need forleadership (Middlehurst, 1997). Neither practical nor psychological leadership for changewas provided with the development of Programme B. It is only more recently when thevalue for and the need for programme leadership is actually in place, that the curriculumis beginning to develop positively. In contrast, the overall leadership for Programme A hasbeen significant. The programme leaders have all recognised that psychological issuesmust be addressed alongside the practical. Staff are supported in a variety of ways:professional development opportunities to understand the philosophy of the degree andthe strategies needed to achieve the overall learning outcomes are provided, practical toolssuch as module handbooks are in place and a core staffing policy is now being addressed.

The impact of leadership on the programmes was further clarified using Middlehurst's(1997) framework. The first dimension of leadership is an analytical and creative dimen-sion. Here leaders have a capacity to generate ideas and new perspectives, they can engageindividuals intellectually with notions of change and create a shared understanding.Leadership and innovation go together. Analytical and creative leadership was the key tothe initial and ongoing development and implementation of Programme A. Educationalleadership is strong within this faculty overall and a leader for the programme wasappointed wilh a capacity to engage colleagues intellectually and help them generatesolutions to achieve a vision. Within Programme B, this dimension was missing, or notrecognised as being important in terms of implementing a programme quite different fromthat which was already in place.

Leadership also has a structural dimension where leaders recognise the need to restruc-ture functions and activities to achieve change. The leadership within Programme A'sFaculty recognised that structural change was needed to achieve the curriculum intended,whereas Programme B's Faculty did not establish management processes for the newdegree, it was not clear at the time of programme approval, nor until some time later, whowas in "charge'.

The final leadership dimension in Middlehurst's (1997) framework is a motivationaldimension. Change is more likely to be made and sustained if account is taken of espousedvalues and these are harnessed positively to achieve the desired change. Middlehurst(1997) recognised that appeals to academics to change their practices, solely on grounds ofeconomy and efficiency, are unlikely to inspire commitment beyond what can be achievedthrough compliance measures. This is exactly what happened with Programme B. Theteaching staff did not know why they should change and, in the first couple of years of theprogramme, they did not. The outcome was a curriculum where discipline teaching

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

Quality Monitoring in Higher Education 17

continued to take precedence over the stated intention to include generic learningoutcomes; core modules were a nuisance and contributed little.

External approval and monitoring

All degrees in New Zealand require external approval before funding is granted. Polytech-nic degrees are also subject to external monitoring, with monitors appointed by theexternal quality assurance agency, the NZQA. The impact of external approval andmonitoring on both degrees was significant. The differences in the way approval panelmembers, from different disciplines, approached course approval was the first obviousimpact. Questions that were asked and the different emphases given during the approvalpanels were quite contrasting. Donald (1997) notes that it is unusual for courseapproval to be more than very general in terms of course learning outcomes and content.'Curriculum is protected by a close association with academic freedom' (Donald, 1997,p. 37). The exceptions are professional bodies who tend to look at content and other inputswith a great deal of interest. For the professional component of Programme A, a separateprofessional accreditation visit was needed before approval by the accountancy pro-fessional body was given. This professional body's influence determined the content forthe professional modules and, although the professional body has little to do with studentlearning, their impact is hugely significant on the knowledge and skills incorporated intothe professional studies modules. Staff liaison with the professional body is also consider-able and the majority of lecturers in this discipline undertake regular ongoing educationthrough the professional association.

The original external approval panel for Programme A did not recommend approval forthe degree. Their report indirectly resulted in a much more carefully thought out curricu-lum, with clearly stated strategies as to how integration across disciplines would work inpractice. The panel questioned teaching strategies, content and curriculum structure. Forthe second panel, the programme developers knew they would have to defend and explainvery carefully a curriculum that was quite unlike the traditional curriculum to which theacademics, on the approval panel, were accustomed.

In contrast, the discipline area was not represented by an academic on Programme B'sapproval panel. Rather, the discipline was represented through a business interest, uncer-tain of educational issues. There were few issues raised about the discipline, other thanconcerns over how the core modules and the practicum would be integrated into thediscipline area. It was also noted that no-one seemed to be in charge of the programme,overall, at the time of the approval visit. The external approval process added nothing tothe quality of the programme. Concerns were noted but, unlike Programme A, thecurriculum was not sent back for further work.

Overall, the impact of external programme approval was dependent on the discipline(s)represented on a panel, and the importance different disciplines gave to different curricu-lum issues. Both external panels raised questions about learning resources and staff onboth programmes found this helpful in terms of lobbying for extra resources. However,there is not any evidence that comments or recommendations regarding resources, madeby external panel members, have actually been significant factors in improving learningresources, such as library and computing resources.

Of greater significance has been the impact of the external monitors appointed tomonitor the implementation of the programme. Their stated role is very broad andincludes visits to the programme and interviews with staff and students. Programme A'steam recognised that it could be valuable to have external academics assisting with

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

18 M. Horsburgh

development and, immediately after the programme was approved, set about establishinga process that would provide useful advice to them. Although Programme B's team didnot take a deliberate approach to establishing how an external academic might be useful,the impact has been similar, and even more significant where a very small team of staffbenefit from the collegial opportunity that external monitoring provides for discussingcurriculum, learning and teaching issues. The external monitors see their role in a way akinto that of using references in a written paper; there is an opportunity to put ideas upagainst external views and compare them. External monitors act as peer reviewers whocan comment on the whole programme, or on particular aspects, in order to judgecomparability of standards or ideas.

For both programmes, the monitors have acted as mentors, or 'critical friends' and beenavailable informally in addition to contact during formal visits. The collegial relationshipshave provided a focus for discussing a range of issues associated with the curriculum,learning and teaching. Alongside this, they bring an element of impartiality and staff feelthey can react differently from the way they are able to with their own colleagues. Theproblems of 'hollowed collegiality' described by Massy et al. (1994) do not exist. Despitecoming from universities that might be deemed competitors, the willingness of monitorsto be open and to share ideas was impressive. The monitors for both programmes wereequally as positive as staff about the value of external advice.

Teachers, Learning, Teaching and Assessment

The most direct impact on the student experience of learning came from the teachers: howthey teach and help students to learn and the sort of assessment practices they adopt.

Interviews with students and lecturers made it clear that the biggest impact on studentlearning comes from the teachers. The students in this study were very clear about what,to them, makes a good teacher. A sound knowledge base and an ability to relateknowledge to the workplace or the discipline were seen as important. Students in thisstudy endorsed the characteristics of good teaching described by Ramsden (1991), that is,clarity of explanation, the level at which the learning material is pitched, the enthusiasmand interest of teachers, feedback on learning, relevant assessment methods and appropri-ate workloads. However, the students also identified and valued teachers who adopted adeep approach rather than a surface approach, that is teachers who asked critical andchallenging questions and stimulated discussion. Clear teacher organisation and goalswere important. For quality monitoring to be of any value to students, observation ofteaching with follow-up action to improve teaching or remove poor teachers, would haveto be a feature. Students of both programmes experienced a variety of teachingmethods and this was seen positively. Understanding how to learn helped them becomeself-directed in their learning and this was valued.

Assessment was recognised as the major motivating factor in learning. Assessments thatencouraged superficial learning were not valued by students. An over-emphasis on writtenand theoretical assessment in Programme B was not, for example, as valuable as oralexaminations in stimulating learning. Programme A students found a critical thinkinginterview and open-book examinations challenging and 'real'. Feedback to students andthe processes engaged in to give feedback has been identified as one of the most importantaspects where assessment may contribute to transformative learning (Harvey & Knight,1996). Again, students on both programmes volunteered that the feedback they received,both the type and the way it was delivered, was very important to their learning.Providing feedback only on summative assessments was quite inadequate, particularly if

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

Quality Monitoring in Higher Education 19

this was no more than a tick at the bottom of a page. Lander et ah (1995) consider that anyemphasis on summative assessment should be reversed in favour of formative assessment,feedback requires reflection by both the student and the teacher. In both the programmesin this study summative assessment was emphasised over formative and was impactingnegatively on, or was a limiting factor in the development of, transformative skills.

Teachers and the Environment in which they Practice

Several other factors that impact on the teachers also impacted indirectly on studentlearning. These included, in particular, the professional-development opportunities andexperiences that the teachers had and, in addition, the appraisal systems and the criteriathat operate for lecturer promotion.

Professional development to support the introduction of Programme A was veryimportant. The programme leaders recognised that if a new programme, which requiredteachers to teach and think about learning and teaching in a different way, was to beintroduced, staff would need considerable professional development. Formally, this hasbeen provided through allocation of a teaching position to support staff development.Informally, however, the professional development has been even more considerable andoccurs in numerous module team meetings, formal academic committee meetings, as wellas discussions in staff rooms and lecturer offices. Perhaps the most significant developmenthas been the collegial discussions on learning and teaching, the opportunities for staff toincrease the amount of thinking about practice. For several authors (Dill et ah, 1996;Harvey, 1995; Massy et ah, 1994), collegialism which is learning-oriented, focused onstudent learning, self-critical and concerned to continually improve, is essential if qualityis to be seen in terms of student learning outcomes. Professional dialogue, which ischaracterised by the transfer of ideas between colleagues, is one way in which teachersthemselves are part of transformative learning (Nightingale & O'Neill, 1994). The outcomeof a focus on learning and teaching amongst staff has been for Programme A, 'the biggestchange that has taken place'. This is borne out by students who had transferred into thedegree from earlier diplomas. They perceived the programme to be more academicallyrigorous than the diplomas, more problem- and case study-based, more capability-focusedrather than technically task-based and taught in such a way as to encourage criticalthinking and independence in students.

In contrast, lecturers in Programme B had no initial professional development to supportthe introduction of their new degree and little opportunity to find time or a forum in whichto discuss the curriculum, and learning and teaching issues. The impact on studentlearning was seen here in the lack of professional development opportunities.

Authors such as Mezirow (1990) are very clear that if students are to achieve transforma-tive learning, teachers must role-model critical reflection. Again, this was difficult for somelecturers in Programme B. Opportunities for transformative professional developmentexperiences were not available, and without them it is of considerable doubt that theselecturers will be able to facilitate transformative learning (Fish, 1991). In contrast, Pro-gramme A staff-development sessions included workshops on critical thinking and theProgramme Leader believes that the reflective ability of lecturers has also improved withthe requirement to write a reflective report each time a module is taught.

Promotional opportunities are very important to teaching staff and until recently theonly significant way that excellence in teaching has been recognised at AIT. Learningfacilitation is a criteria for promotion, and lecturers must provide evidence of facilitationof independent and self-directed learning. There is some congruence in the skills required

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

20 M. Horsburgh

to support student transformative learning and lecturer promotion criteria. Although thesame promotion criteria apply across the Institute, faculties implement processes differ-ently. Programme A's Faculty recognised the need to adapt lecturer appraisal forms inaccord with the philosophy of the programme. Appraisal of lecturers is a formativeexercise, although evidence of appraisal is a requirement for promotion. Students on bothprogrammes recognise that they have opportunities to appraise their teachers but none ofthe students interviewed felt that the appraisals had much value in terms of any improve-ments in teaching that they might see. Feedback to them was never given, and they werereally honest in their appraisals only when a neutral person collected the feedback.Students perceived staff to have power and the appraisal aspects of professional develop-ment were found to be impacting quite negatively on student empowerment. Students sawno evidence that they could make a difference.

Programme-specific Internal Quality Monitoring

Several aspects of formal internal quality monitoring impacted on the programmes andon student learning. The first of these was internal moderation of student assessments.All assessments are reviewed by several lecturers before they are submitted to studentsand marking is reviewed before material is returned to students. This has helped teachersto understand marking and grade criteria and provide some level of consistency forstudents.

The internal quality monitoring requirement for a programme annual report from eachprogramme has an impact in terms of documenting areas for improvement. The intentionis that the report is a reflective team report, which identifies areas for overall improvementand an annual action plan for achieving improvements. The process of preparing thereport requires teaching staff to reflect on their teaching and all aspects associated withtheir programme; lecturers participate in a learning process. The reports themselvesprovide evidence of continuous improvement within the programmes, each year there isa report on how improvement actions have been achieved and new areas for improvementare identified.

For Programme A, the key unit for internal monitoring is the module or programmecomponent. Monitoring is accomplished through a written module evaluation completedby the module co-ordinator at the completion of each module occurrence. While moduleevaluations require reflection on every aspect of the module (learning outcomes, content,teaching strategies, assessment, student outcomes, resources and so on), by the moduleteaching team, a set format for reporting is provided. Reference must be made to studentevaluations of the module. The impact of these module reviews or evaluations is assignificant as the overall programme annual report. A focus on continuous improvementis evident. However, there was some indication that the prescribed requirements andformat for module reviews may not, in fact, be leading to sustained continual improve-ment and may just become a compliance requirement. Not all staff saw module reviews asan opportunity to reflect on the module and how to improve it but rather as a task to becompleted.

Informal monitoring, such as occurs with professional dialogue and exchange ofideas, was clearly of considerable importance and, although the impact was notobvious, informal monitoring cannot be overlooked. Ultimately, informal monitoring,might turn out to be the most valuable in terms of improvement and enhancement ofstudent learning.

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

Quality Monitoring in Higher Education 21

Resources

The whole learning environment is important to students. This is borne out by the studentsatisfaction survey approach taken by the University of Central England in Birmingham(Harvey et ah, 1997b) and adopted by AIT. The research instrument is based on theexpressed views of students gathered during focus-group discussions. All areas that mayimpact on a student's experience are identified for inclusion in the survey (Meiklejohnet ah, 1997).

In the case study, students identified several elements as impacting directly on theirlearning, including library and related resources such as language tapes, and access tocomputer laboratories. Students considered resources to be inadequate. In both pro-grammes these areas were also identified by approval panels as weaknesses. Resourceissues have been slow to resolve. Resourceful students have found other sources to supporttheir learning, such as computers in their workplace and language videotapes belonging tofriends. There is also direct evidence of an impact on student learning where insufficientlanguage material in the library made the continuation of a language research moduleimpractical. An opportunity for developing transformative skills was lost.

Student Expectations

Student expectations were impacting on the programmes in several ways. The first impactwas seen in respect of structural arrangements. Timetabling was in accord with expressedstudent need, particularly part-time student requirements, regulations were changed toaccommodate demand for 'double majors' and enrolment procedures altered to assiststudents with course planning. However, a greater impact was seen in the recognition, bystudents, of their 'rights' in terms of being 'consumers' and having an understanding oflegislation. Students are consumer-oriented and quite sophisticated about the economicdimensions of their study, in terms of future employment prospects and immediatefinancing demands. Employment prospects are important. In response to the recognitionthat students understand they have rights to appeal and complain, documentation is veryclear. Regulations and policy and processes for handling assessment appeals are carefullydrafted and checked by lawyers before being given to all students. Special assessmentopportunities are also provided, such as 're-sits', special examinations and aegrotat passes.Teaching staff in both programmes are aware that students expect 'value for money'.To date this area of impact has been positive, programmes and services recognise the needto be student-focused.

Conclusion

The case study provided the opportunity to understand the complexity of the factors andthe interaction between them as they impact on undergraduate degree programmes andthe student experience of learning (Figure 2). While it may be expected that the interactionsbetween factors would be even more complex, for example that quality monitoring (eitherexternal or internal) would impact directly on factors such as actual teaching performanceor on learning resources, this was not evident. Quality monitoring processes had quite anarrow impact, and were not concerned with the complexity of a whole teaching pro-gramme, or issues such as leadership or the culture in which students learn. The influenceof the social, economic, political and personal context in which the two programmes aresituated was also considerable.

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

External QualityMonitoring

Accountability

ProfessionalDevelopment •

PromotionalOpportunities andTeaching Rewards

Professional BodyExpectations Leadership

Innovation

StudentExperience of

Learning

Employer Expectations

tudent Expectations

Learning Resources

FIG. 2. Factors impacting on the student experience of learning.

IS

I

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

Quality Monitoring in Higher Education 23

The curriculum, its overall intent, purpose and structure impacted significantly on thestudent experience. The influences on the curriculum were many and included economicand social factors. The context in which the curricula were developed, in particular thedisciplinary context, influenced curriculum design. External quality monitoring, whereexternal academics contributed to the approval of a curriculum and acted as mentors or'critical friends' during implementation of a new programme, had an impact. Collegialdiscussions on learning and teaching were enhanced. Where academic staff questioned therequirements of quality monitoring and deliberately determined to ensure that benefitswould flow from quality activities, enhancement of teaching was an outcome. Where anexternal professional body had an interest in the graduates, their impact on the curriculumwas significant.

Of significance for both curriculum development and implementation was the leader-ship provided by internal academic staff. Curriculum innovation and the implementationof curriculum, where graduates are intended to have transformative attributes, was theoutcome of 'creative and motivational' leadership at a programme level.

The most direct impact on student learning came from the teachers, their teachingpractices, how they help students learn and the sort of assessment practices they adopt.The professional development opportunities and experiences the teachers have, teacherappraisal systems and promotion criteria all impact on the teachers. Internal qualitymonitoring requirements for programme annual reports and component parts impactindirectly as teachers reflect on their practice. Professional development is enhanced andactivities result that continually improve learning and teaching.

The impact of resources to support learning was significant, and deficits are notedthrough internal and external quality processes. However, there was no evidence of anyimpact of quality monitoring leading to resolution of resource issues. Finally, studentexpectations that programmes will provide 'value for money' in terms of future employ-ment prospects and immediate financing demands had a positive impact. Programmes andservices are student-focused.

Overall, the greatest impact on student learning was the curriculum, factors thatinfluence the curriculum, and the teachers. The most direct impact on student learning wasfrom teacher practices, how they help students learn and the assessment practices theyemployed.

There was minimal impact of quality monitoring activities on actual learning andteaching practices. AIT has comprehensive quality monitoring systems in place and thedevelopment of these has been influenced by external requirements. There are also someinnovations, such as programme annual reports. Whilst innovations such as this providea focus for improvement and enhancement of programmes, it may be that the balancebetween activities that staff see as requiring compliance and those that foster enhancementof learning and teaching is still to be achieved.

The factors impacting on the student experience of learning confirmed what manyhigher education commentators report, that for quality monitoring to have an impact onthe effectiveness of higher education, the emphasis must be on curriculum, learning andteaching. Quality monitoring must focus on more than systems, inputs and outputs, ifeffectiveness is to be enhanced.

Massy (1997) suggests that there is a quality monitoring continuum with activitiessupporting accountability at one end and activities supporting continual improvement atthe other. The accountability end tends to look backwards on what has happened and thecontinual improvement end looks forward. This study suggests that the mid-point of thecontinuum is internal quality monitoring, and if we are to recognise the impact that

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

24 M. Horsburgh

curriculum, learning and teaching processes have on the outcomes for graduates, theemphasis needs to be on internal quality monitoring. Trow's (1996) suggestion of distin-guishing between internal and external accountability is helpful here. Internal accountabil-ity is about an institution carrying out its own inquiry and analysis into its own operations.It is aimed at continual improvement through investigation, with critical self-analysis andeffective action outcomes. Critical self-review institutionalises the principle of reflection,and where staff engage in reflection they automatically focus on development andimprovement.

There is little intrinsic value in monitoring for monitoring's sake, the real value must bein information, which leads to action being taken continually to improve. Internal account-ability does this. For Kells (1992) the focus on local responsibility is also the key factor inestablishing an improvement-oriented environment. The notion of internal accountabilityis appropriate at all levels of an institution, and the challenge is then to ensure thatrequirements for internal accountability are not imposed in such a way as to stifleinnovation or bottom-up improvement. Teaching units or programme teams, as well asteams involved in providing particular aspects of student services, such as the library, canbe just as involved in regulating their own activities and assessing their own performanceas is the institution as a whole. If the emphasis shifts from external accountability tointernal accountability, external agencies and external quality monitoring could act as trueauditors of a continuous quality improvement process, a process where the focus is onlearning and teaching, and quite distinct from the current accountability focus with itsemphasis on systems (Harvey, 1995).

It is suggested that the following principles are appropriate to guide internal account-ability. These principles emerge both from the literature and from this study. Internalquality monitoring needs to incorporate principles that seek to put in place an agenda forcontinuous quality improvement. Bureaucratic accountability requirements do not fit intothis model. However, leadership at all levels is important. Continuous quality improve-ment is forward' looking, it recognises that enhancement and development emerges fromgood practice and acknowledges the professionalism of teaching staff and the importanceof intrinsic motivation. The following general principles are recognised as central tocontinuous quality improvement:

• the parameters for a quality improvement and enhancement process must be estab-lished;

• effective action and appropriate change must flow from monitoring;• quality processes must be non-burdensome;• a partnership between the centre and the teachers must be established, with the centre

arbiter of key values and principles and the ways of doing things decided by the peoplewho must actually do them;

• delegation of responsibility for quality must be to those people able to effect change atthe teaching-learning interface;

• teaching units and other teams involved in providing student services must defineimprovement and put in place processes needed to foster and implement improvement.

References

BARNETT, R., 1992, Improving Higher Education: total quality care (Buckingham, Society for Research intoHigher Education and Open University Press).

BECHER, T., 1989, Academic Tribes and Territories, intellectual enquiry and the culture of discipline (Buckingham,Society for Research into Higher Education and Open University Press).

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4

Quality Monitoring in Higher Education 25

BIGGS, J., 1989, 'Approaches to the enhancement of tertiary teaching', Higher Education Research andDevelopment, 8(1), pp. 7-26.

BRYMAN, A., 1992, Charisma and Leadership in Organisations (London, Sage).CAVE, M., KOGAN, M. & SMITH R., (Eds), 1990, Output and Performance Measurement in Government: the state

of the art (London, Jessica Kingsley).DILL, D.D., MASSY, W.F., WILLIAMS, P.R. & COOK, C.M., 1996, 'Accreditation and academic quality assurance?

Can we get there from here?' Change, 28(5), pp. 17-24.DONALD, J., 1997, Improving the Environment for Learning: academic leaders talk about what works (San Francisco,

Jossey-Bass).FISH, D., 1991, 'But can you prove it? Quality assurance and the reflective practitioner', Assessment and

Evaluation in Higher Education, 16(2), pp. 22-36.HARVEY, L., 1995, The New Collegialism: improvement in accountability', Tertiary Education and

Management, 2(2), pp. 153-160.HARVEY, L. & KNIGHT, P., 1996, Transforming Higher Education (Buckingham, Society for Research into

Higher Education and Open University Press).HARVEY, L., PLIMMER, L., MOON, S. & GEALL, V., 1997a, Student Satisfaction Manual (Buckingham, Society for

Research into Higher Education and Open University Press).HARVEY, L., MOON, S. & GEALL, V., 1997b, Graduates Work: organisational change and students' attributes

(Birmingham, Centre for Research into Quality, UCE).HARVEY, L., GEALL, V. & MOON, S., 1998, Work Experience: expanding opportunities for undergraduates

(Birmingham, Centre for Research into Quality, UCE).HORSBURGH, M., 1998, Quality Monitoring in Higher Education: a case study of the impact on student learning,

unpublished doctoral thesis, Charles Sturt University.KELLS, H., 1992, Self-Regulation in Higher Education (London, Jessica Kingsley).LANDER, D., WALTA, A., MCCORRISTON, M. & BIRCHALL, G., 1995, 'A practical way of structuring teaching for

learning', Higher Education Research and Development, 14(1), pp . 47-60.MASSY, W.F., 1997, Teaching and Learning Quality-Process Review: the Hong Kong programme', Quality

in Higher Education, 3(3), pp. 249-262.MASSY, W.F., WILGHR, A.K. & COLBECK, C., 1994, 'Departmental cultures and teaching quality: overcoming

hollowed collegialiry', Change, 26(4), pp. 11-20.MEIKLEJOHN, R., HORSBURGH, M. & WRIGHT C , 1997, The 1997 Report on the Student Experience at AIT

(Auckland, Auckland Institute of Technology).MEZIROW, J., 1990, Fostering Critical Reflection in Adulthood: a guide to transformative and emancipatory learning

(San Francisco, Jossey-Bass).MIDDLEHURST, R., 1997, 'Reinventing higher education: the leadership challenge', Quality in Higher Education,

3(2), pp. 183-198.NIGHTINGALE, P., 1997, 'Submission to the review of higher education funding and policy', HERDSA News,

19(1), pp. 1-4.NIGHTINGALE, P. & O'NEIL, M., 1994, Achieving Quality in Higher Education (London, Kogan Page).RAMSDEN, P., 1991, 'A performance indicator of teaching quality in higher education: the Course Experience

Questionnaire', Studies in Higher Education, 16(2), pp. 129-150.RAMSDEN, P., 1993, Theories of learning and teaching and the practice of excellence in higher education',

Higher Education Research and Development, 12(1), pp. 87-98.TROW, M., 1996, Trust, markets and accountability in higher education: a comparative perspective', Higher

Education Policy, 9(4), pp. 309-324.WEST, R., 1998, Review of Higher Education Financing and Policy (Commonwealth of Australia, Department of

Employment, Education, Training and Youth Affairs).

Dow

nloa

ded

by [

RM

IT U

nive

rsity

] at

15:

22 2

7 A

ugus

t 201

4