EMQT project (2009-2011)

Erasmus Mobility Quality Tools

Lupo Donà dalle Rose University of Padova

This project has been funded with support from the European Commission.

Agreement Number: 2009 – 3668/001-001

CONFERENCE “STRENGTHENING THE IMPACT OF CONFERENCE “STRENGTHENING THE IMPACT OF LEARNING MOBILITY”LEARNING MOBILITY”

CMEPIUS, 8TH DECEMBER 2011, LJUBLJANA CMEPIUS, 8TH DECEMBER 2011, LJUBLJANA

When? When? ORIGIN & STARTING POINTSORIGIN & STARTING POINTS

The Bologna Process impact and its progression towards the European Higher Education Area

“mobility is at the heart of the Bologna Process”

Difficulties in recognition, Nancy 2008; preliminary discussion took place between Deusto and Padova…

… the idea subsequently became a concrete project, with the main support provided and role played by the “Education Training and

Mobility” Task Force of the Coimbra Group.

1. European Quality Charter on Mobility2. Erasmus University Charter

3. Erasmus Student Charter4. Green Paper on Mobility

Important documents are:

Project coordination: University of Padova

14 Universities : Padova, Bologna, Deusto, Aarhus, Åbo, Charitè Berlin, Granada, Graz, K. U. Leuven, Leipzig, Iaşi, Jena, Paris-Sud, Thessaloniki

3 Associations: ESN-Erasmus Student Network, Brussels EuroPACE ivzw (BE)Coimbra Group Office, Brussels

1 National Agency: Agencia Nacional de Evaluación de la Calidad y Acreditación ANECA, Madrid

1 Private counselling partner: CHE Consult (DE)

2 Associated Partner Universities: Trinity College Dublin, Turku University

Who? Who? The EMQT Consortium…. The EMQT Consortium….

What?The EMQT project is an “Erasmus structural network”

Why? AIMS and PURPOSES:

• To define a QTB – Quality Tools’ Box and develop guidelines for different “Quality patterns or profiles”, where a given institution can position itself, and for mechanisms of accountability and of external validation.

• To identify organisational models, good practices, benchmarking procedures and related indicators, which should characterise the quality of the students’ exchange mobility.

How? Physical (5+1x2) and virtual meetings (8)*

– through online and telephone conferences -

of the Managing Committee and the Plenary Assembly

* Skype and Flashmeetings

and Where?

5+1 General Meetings workshops

Padova (kickoff meeting)BrusselsLeuvenBologna (1st Open Seminar)Brussels (Final Validation Conference)

Task Forces (6) and respective chairsTask Forces (6) and respective chairs

General organisational models within HEIs

CHE Consult

Language preparation Universidad de Granada

Information and orientation Aristotle University of Thessaloniki

Students’ performances and recognition

Alma Mater Studiorum Università di Bologna

Reception of host students Karl-Franzens-Universität Graz

e-Coaching or ICT mobility tools Katholieke Universiteit Leuven

The Methodological framework• By acting through 6 Task Forces (TFs) each one having its own deliveries &

each TF following the GAIN sequence

G : identification of Goals

A : Actions

IN : INdicators

• By producing Questionnaires, both at internal and at external level of HEIs

• By drafting Mapping Reports

• By listing key Indicators

How…

And what about the academics’ point of view?• EMQT statement on the meaning of academic quality

during a study period abroad• 85 interviews with academics in partner institutions

• By identifying Good Practices

What are the main mobility goals?

I1. Opening doors to other kinds of mobilityI2. Boosting reputation and increasing visibility of the HEI through ERASMUS and cooperationI3. Enriching the institution teaching offer and services by international mobilityI4. Achieving institutional awareness of intercultural diversity

S1. Allowing every student an ERASMUS mobility according to his/her academic needsS2. Achieving transversal competencies and awareness of intercultural diversity (links also to the society)S3. Ensuring the most successful stay with emphasis on academic achievement

So1. Building awareness of European citizenshipSo2. Fostering interaction between HEIs and non-HEI organisations as well as the civil society

SOCIETYSOCIETY

STUDENTSSTUDENTS

INSTITUTIONINSTITUTION

order found in the questionnaire lay-out

GOALSMERGED SAMPLE INTERNAL EXTERNAL

7

S3. Ensuring the most successful stay with emphasis on academic achievement

1° 1° 1°

5

S1. Allowing every student an ERASMUS mobility according to his/her academic needs

2° 3° 2°

2

I2. Boosting reputation and increasing visibility of the HEI through ERASMUS

3° 4° 3°

1

I1. Opening doors to other kinds of mobility and cooperation

4° 2° 6°

4

I4. Achieving institutional awareness of intercultural diversity

5° 7° 4°

3

I3. Enriching the institution^s teaching offer and services by international mobility

6° 6° 5°

6

S2. Achieving transversal competencies and awareness of intercultural diversity (links also to the

society)7° 5° 7°

9

So2. Fostering interaction between HEIs and non-HEI organisations as well as the civil society

8° 9° 8°

8So1. Building awareness of European citizenship 9° 8° 9°

Priority in the goals according to EMQT survey

EMQT Tools’ Box: Main components EMQT Tools’ Box: Main components

1. EMQT Questionnaire and Glossary

2. List of key/main Indicators

3. Suggestions for Good Practices4. Report on Academic Quality in

exchange mobility

What are the EMQT outputs?What are the EMQT outputs?

EMQT reference documentsEMQT reference documents1. Quality in mobility: how to measure and

assess it, a paper by Maria Sticchi Damiani

2. EMQT Mapping Report

3. Paper on the Recognition Process

What are the EMQT outputs?What are the EMQT outputs?

Facts and figures: the mobility rate

Erasmus area: 7.7 students out of 1,000 (SMS, EC data for 2008-09)

• Internal sample 14.9 (max 31.8 ÷ min 4.8)

13 universities

• External sample 21.4 (max 77.7 ÷ min 3.4)

65 HEIs

• merged sample 18.8 (max 77.7 ÷ min 3.4)

The EMQT questionnaire

1. early version – tested internally≈ 200 questions, 340 fields

2. revised version (95% overlap, very similar) – tested externally

3. Toolbox version ≈ 200 questions, 340 fields

PAGE 1 … PAGE 3EMQT Questionnaire

For those terms which seem unclear, we advise consulting the EMQT glossary. Initials and acronyms

are also included in the glossary under the term “Initials”. The numbers you fill in should refer to the last academic year for which these are available, keeping in mind to

fix that academic year for all the numbers. So, if for certain questions the last available data refer to the academic year 2008/2009 and for others these refer to 2009/2010, please keep 2008/2009 as the academic

year of reference for all the questions, unless otherwise explicitly specified.

Part I - General Information I.I. CONTACT PERSON / ADDRESS

Contact Person / Address Please indicate the details of a contact person in charge of this questionnaire and the address of the structure/service of your Higher Education Institution (HEI) to which the contact person belongs. Name of HEI: original name & (official) translation into English__________________________________ ERASMUS ID code__________________________________________________________________________ Country_______________________________________________________________________________ Contact Person Name of contact person_________________________________________________________________ Department/Unit_________________________________________________________________ ____ E-mail-address_________________________________________________________________________

I.II. CENTRAL-LEVEL DATA

I.II.I Total number of staff in your HEI (in FTE - full time equivalents)

a) Academic staff (teachers and researchers) _________ b) Administrative and technical staff _________

I.II.II Number of offered Degree Courses 1st cycle a.

2nd cycle b.

One-tier degrees1 c.

3rd cycle d.

Pre-Bologna organisation of studies e.

I.II.III Total number of enrolled students 1st cycle a.

2nd cycle b.

One-tier degrees c.

3rd cycle d.

Pre-Bologna organisation of studies e.

I.II.IV Mobility

1 See glossary

Section 1: Organisational Models

1.1 Institutional backbone for ERASMUS mobility

1.1.1. Is mobility made explicit in the mission statement of your institution? yes no

1.1.2. Does your institution have a strategy on ERASMUS agreed upon at institutional level? yes no 1.1.2.1. If yes, is the ERASMUS strategy encompassing the following cycles? a) 1st cycle (BA) yes no b) 2nd cycle (MA) yes no c) 3rd cycle (doctoral) yes no

1.1.3. Does your institution have an office for the ERASMUS programme? a) As an independent unit yes no b) As part of an IRO yes no c) Either independent or part of an IRO but in strong synergy with other services (registrar office, career guidance service, computing centre, etc.) yes no d) No, but there is multi-task administrative staff taking care of ERASMUS yes no

1.1.4. What is the number of staff at your University for the administration of the ERASMUS programme? _________

1.1.5. What is the total number of personnel; academics, administrative and technical staff involved at the ERASMUS programme management? a) Academic staff (i.e. ERASMUS Coordinators/Advisors1) ________ b) Administrative and technical staff (ERASMUS Offices) ________

1.1.6. Does your institution provide incentives for staff to get involved? a) Academic staff yes yes, in most cases yes, in few cases no b) Administrative and technical staff yes yes, in most cases yes, in few cases no 1.1.6.1. If yes, could you briefly describe them? _______________________________________________

1.1.7. Management system on quality for the ERASMUS programme; Does your institution use a quality management strategy for the ERASMUS programme? yes no 1.1.7.1. If yes, could you briefly describe it?___________________________________________________

1.1.8. Does your institution monitor the mobility of students according to numerical indicators? yes yes, in most cases yes, in few cases no

1.1.9. Does your institution use the results of the monitoring process for strategic decisions? yes yes, in most cases yes, in few cases no

1.1.10. Does your institution run regular satisfaction surveys related to ERASMUS with: a) Students? yes no b) Academics? yes no c) Others: ___________________________________________________________________

1.1.11. Does your institution draw on results from monitoring and surveys for quality improvement activities? yes yes, in most cases yes, in few cases no

1.1.12. Does your institution set corridors on (some) key numerical indicators for improvement in respective value? yes no 1.1.12.1. If yes, on which indicators? ______________________________________________________

1.1.13. Does your institution adjust corridors according to actual indicator development? yes no

1 Ibidem

list of EMQT main indicators - 1• Indicators as a hint to the quality of istitutional actions

aimed at a certain mobility goal (a priori group reflection)

• indicators were then translated into questions, to be asked for in the questionnaire

most answered / seldom answered

• Internal reflection/debate based on agreed methodology: after several stages, from a questionnaire with 340 questions, the TFs selected 4 general indicators plus 31 indicators

list of EMQT main indicators - 2

General Part• 1. number of outgoing study mobility• 2. study mobility rate• 3. number of incoming study mobility• 4. average duration of study mobility

list of EMQT main indicators - 2

General Part• 1. number of outgoing study mobility• 2. study mobility rate• 3. number of incoming study mobility• 4. average duration of study mobility

LIST OF INDICATORS – TFaorganisational models

1. Existence of a strategy on ERASMUS agreed upon at institutional level2. ERASMUS Office – Function

independent unit; part of an IRO; in strong synergy with other services (registrar office, career guidance service); multi-task administrative staff taking care of ERASMUS)

3. Implementation of a quality management system for ERASMUS4. Provision of incentives for staff to get involved5. Monitoring the mobility of students according to numerical indicators…..7. total number of academic, administrative and technical staff (FTE)

involved in counselling and in managing ERASMUS student mobility…

Suggestions for Good Practices

1. The point of view of a QA National Agency: extraction of good practices from questionnaire returns (care of ANECA)

2. cases of good practices identified by the 6 TFs

3. Document and example of implementation strategies

• Task Force b: Language preparation and related issuesFor this dimension, both the provision of language courses by the institutionand the actions the institution takes to standardize the different languagelevels are analysed.

Good Practices:• GP2.1. The institution offers pre-departure language units for outgoing• students and incoming students.• GP2.2. The institution offers semester/year-long language course units for• outgoing students and incoming students.• GP2.3. The institution offers language course units for specific purposes (i.e.:• Law, Medicine, Engineering, etc.) for outgoing and incoming students.• GP2.4. The institution offers language course units at different levels.• GP2.5. The institution follows strictly the Common European Framework of• Reference, CERF.

Interviews with academics

• 20 QUESTIONS

Institutional aspects 6 questions

Exchange coordinator aspects 7 questions

Recognition issues 4 questions

Problems and future 2 questions

Suggestions 1 question

THE INTERVIEWS

- 14 H.E.I.’S PARTICIPATING IN THE EMQT PROJECT

- AVERAGELY 6 ACADEMICS INTERVIEWED

OVERALL: 83 INTERVIEWED ACADEMICS (AND 2 ADMINISTRATIVE STAFF)

METHODOLOGY OF ANALYSIS (1)

EXTRACTING RECURRING ELEMENTS FROM THE ANSWERS AND COUNTING THE NUMBER OF ANSWERS IN WHICH THESE ELEMENTS WERE MENTIONED.

WE USED MATRIXES FOR THIS (EXCEL FILES) IN WHICH WE COULD “TICK” THE RELEVANT COLUMN ANY TIME THESE ELEMENTS TURNED UP IN AN ANSWER.

METHODOLOGY OF ANALYSIS (2)

EXTRACTING INTERESTING AND/OR SURPRISING ANSWERS WHICH RECURRED IN DIFFERENT INTERVIEWS

SOME INTERESTING OUTCOMES (2/3)

Academics from 4 different universities mentioned the differences in courses as interesting aspects that could make them decide to set up agreements (different types of learning or assessment, or different approaches to contents), instead of the much more expected argument of compatibility.

Interesting as well that in 5 out of the 6 quotes, the academic was from the subject area of Science, who, notably, tend to be more rigid on the recognition issue.

EMQT Mapping ReportEMQT Mapping ReportGeneral Index

PageGeneral Part.........................................................................................2

Section 1: Organisational Structures (Tfa) 13Section 2: Language issues (TFb) 42Section 3: Information and Orientation (TFc) 49Section 4: Students’ performance and recognition (TFd) 59Section 5: Reception of incoming students (TFe) 80Section 6: e-coaching or ICT tools (TFf) 90

Appendix FACTS AND FIGURES: an accompanying paper to the EMQT survey

106

The different sections offer an accurate description of the Erasmus landscape, its physiology, established practices, etc.

Distribution of the external sample institutions over geographical areas and mobility size

DISTRIBUTION OF RETURNS FROM EXTERNAL TESTING

MOBILITY SIZE

GROUP 1 GROUP 2 GROUP 3

GE

OG

RA

PH

ICA

L A

RE

A

A – NORTHERN EUROPE

11 3 1

B – EASTERN EUROPE

6 7 10

C – SOUTHERN EUROPE

2 9 4

D – CENTRAL EUROPE

3 4 5

GROUP 1: from 1 to 100 outgoings ; GROUP 2: from 101 to 300; GROUP 3: above 300

•identifying and extracting important aspects and elements of mobility as witnessed by the EMQT returns,•giving a smart/realistic interpretation to the several – sometimes conflicting – data collected, on the basis of personal experience based competences, •extracting ranges of variability for meaningful indicators,

the whole being offered in an coherent and usable manner.

Important ingredients of the landscape are the values of several “derived” indicators (e.g. the mobility; the reciprocity ratio; etc.).

building the landscape of the European mobility consisted in

examples of derived indicators - 1derived indicator Average

value Minimum

value Maximum

value Number of

usable returns Overall mobility rate

18.8

3.4

77.7

40

First cycle mobility rate 17.0 2.0 78.4 30 Second cycle mobility rate 20.7 2.2 50.0 25

One-tier degree courses mobility rate 20.1 1.4 23.9 (115.0)

14

Third cycle mobility rate 15.3 0.3 33.3 (138.9)

15

Number of outgoing students per degree-course (overall)

5.5

0.7

26.8

39

Number of outgoing students per degree-course (first cycle)

5.9 0.6 21.0 27

Number of outgoing students per degree-course (second cycle)

1.6 0.1 5.9 23

Number of outgoing students per degree-course (one-tier degree-courses)

15.3 2.0 65.0 12

Number of outgoing students per degree-course (third cycle)

0.5 0.1 3.3 10

examples of derived indicators - 2

derived indicator Average value

Minimum value

Maximum value

Number of usable returns

Ratio “academics” over “administrative & technical staff”

2.05 0.48 7.8 46

Reciprocity ratio 1.01 0.11 2.9 48

average number of outgoing students per Erasmus Office staff member

63.5

3

185

38

average number of outgoing students per counselor/manager

11.5

1

52

38

AVERAGE VALUE OF EU MONTHLY GRANT per STUDENT (in euro)

246

74

460

28

number of incoming students per buddy/peer tutor

4.5

0.5

26 20

total number of outgoing students over available places (as from BAs)

0.44 0.11 0.98 29

percentage of degree-courses offered in English language

10.5%

0.2%

60%

23

The concept of profile

• for a group of respondents

• for a single institution

external versus internal testing

question number

QUESTIONinternal sample

external sample

NOTES

Section 11.1.4.1 average number of outgoing students per Erasmus Office staff member 57 831.1.5 a&b2 average number of outgoing students per counselor/manager 13 5,4

1.1.15.1 3 are students included in Erasmus QA? 67% 44%

1.2.4.0support schemes for students with special needs - outgoing

92% 45%

1.3.1.3 ratio STA over SMS 0,073 0,172 Erasmus area average = 0,170ratio STT over SMS 0,027 0,051 Erasmus area average = 0,039

Section 2

2.1.1 .1 pre-departure / pre-arrival language course units, outgoing 75% 32%

pre-departure / pre-arrival language course units, incoming 75% 61%

2.1.2 .3 semester/year-long language course units, outgoing 50% 61% semester/year-long language course units, incoming 75% 87%

2.1.3 .5language course units for specific purposes (subject related…), incoming 67% 39%

1. long term preparation is quite stronger in ET;2. the students of the ET seems to be more equipped with language skills

Section 33.1.2.1e Social network tools - addressed to incoming 33% 77% ET more open to new technologies

3.1.8.1ecomunication of satisfaction monitoring results with partners

42% 69%

3.2.1.1e buddy system in place: a) for outgoing 8% 50%

buddy system in place: b) for incoming 67% 91%3.2.4.1e motivate local students to get engaged as buddies 67% 80%

Section 44.1.2.1e BAs used both ways 60% 65%

4.1.2.1eBAs used only for outgoing

12% 7%IT is more open to asymmetric flows

4.1.2.1e BAs used only for incoming 16% 5%

4.1.2.1e sleeping BAs 13% 16%

4.1.3 - derived indicator

average ratio "SMS number/available places" 37% 47%

4.3.1.1e pre-departure ToR sent to host institution 50% 73%offer special preparatory activities (discipline specific) tooutgoings

17% 27%

offer special preparatory activities (discipline specific) toincomings

8% 36%

Section 5

5.2.7.10 course units taught in English 58% 88%

5.2.8.11degree programmes entirely taught in English

33% 79%

derived indicator percentage of degree-courses offered in English language 0,5% 12,5%

Section 66.3.2.13 monitoring feedback from mobile students online 17% 46% ET more open to new technologies

4.3.2.1emutual trust vs modern QA?

…. Thus any institution can find out how to position itself in the landscape (institutional profile)…

the EMQT open questions

QUESTIONNAIRE SECTION

number of question

columns in EXTERNAL

testing

number of open

questions

number of numerical questions

number of closed

questions

general part50 8 33 9

Sect. 1 79 11 19 49Sect. 2 27 2 6 19Sect. 3 41 7 2 32Sect. 4 45 8 16 21Sect. 5 60 5 4 51Sect. 6 38 3 8 27

total 340 44 88 208100% 12,9% 25,9% 61,2%

An example of open question returns

Open question 4.3.9.1. – If your institution does not send out students to go abroad to do exclusively research – why not? (12 valid returns)

1) it is forbidden by the Erasmus rules (e.g. as stated by National Agency), 3 returns as a whole!

2) research can be carried out only together with other activities (only 1 return!)

3) research project is considered as a study activity 4) thesis work in first cycle is not considered as research

(some testing institutions have only or prioritize the first cycle)

5) almost no mobility in the 2nd cycle, where research can be carried out

6) not required by students (in a medical faculty) 7) other European or National programs are best suited for

research activities abroad (3 returns)

EMQT Tools’ Box: Main components EMQT Tools’ Box: Main components

1. EMQT Questionnaire and Glossary

2. List of key/main Indicators

3. Suggestions for Good Practices4. Report on Academic Quality in

exchange mobility

What are the EMQT outputs?What are the EMQT outputs?

EMQT reference documentsEMQT reference documents1. Quality in mobility: how to measure and

assess it, a paper by Maria Sticchi Damiani

2. EMQT Mapping Report

3. Paper on the Recognition Process

What are the EMQT outputs?What are the EMQT outputs?

For any further information please visit the EMQT website

www.emqt.org

or contact: sara.pittarello@unipd.it

Thank you very much for your kind attention!