TEACHERS MEAN WELL BUT ...

How Students Perceive Assessment in Translation Training

Tamara Mikolič Južnič

University of Ljubljana

INTRODUCTION

Translation Quality Assessment Forms of assessment

Formative Summative

Holistic or analytic (criterion-referenced)? Assessment grids (e.g. González Davies 2004) Subtracting points from a given number Often focussed on negative aspects and end

product Any positive feedback?

How to present TQA to students How students perceive teachers‘ feedback on

TQA

DEPARTMENT OF TRANSLATION

BA in Interlingual communication MA in translation (and MA in interpreting) over 400 students L1: Slovene; L2: English; L3:

German/French/Italian http://www.prevajalstvo.net/department Teaching staff:

30 full-time professors, assistants and lecturers + 17 external part-time trainers;

23 have courses in translation Co-operation and tuning

METHOD AND POPULATION

Online questionnaire for students in 2012 and 2014; parallel questionnaire for trainers (in 2012) www.surveymonkey.com

Given answers + comment space for all answers Structure of student responses in 2012 (100

responses)

BA year 1 BA year 2 BA year 3 candidate for BA

graduation

MA year 1 MA year 2 candidate for MA

graduation

former student

0

2

4

6

8

10

12

14

16

18

English-GermanEnglish-FrenchEnglish-Italian

METHOD AND POPULATION

Structure of student responses in 2014 (72

responses):

BA yea

r 1

BA yea

r 2

BA yea

r 3

cand

idat

e fo

r BA g

radu

ation

MA y

ear 1

MA y

ear 2

cand

idat

e fo

r MA g

radu

ation

form

er st

uden

t0

1

2

3

4

5

6

7

8

9

English-GermanEnglish-FrenchEnglish-Italian

METHOD AND POPULATION

Structure of traner responses (2012; 16 responses)

BA Year 1 BA Year 2 BA Year 3 MA Year 1 MA Year 20

1

2

3

4

English-Slovene

Slovene-English

German-Slovene

Slovene-German

Italian-Slovene

Slovene-Italian

French-Slovene

Slovene-French

RESPONSESCOMMENTS IN ASSESSING TRANSLATION

oral written oral and written0

10

20

30

40

50

60

70

80

students in 2012students in 2014trainers

FREQUENCY OF ASSESSMENT

always 7 out of 10

5 out of 10

3 out of 10

less than 3 out of

10

never0

10

20

30

40

50

60

70

students in 2012students in 2014trainers

LENGTH OF THE COMMENTS

0

20

40

60

80

students in 2012students in 2014trainers

CONTENT OF COMMENTS

comments are limited to grammatical and/or lexical errors

comments include notes on stylistic errors

comments include notes on errors regarding the text type

comments include notes on the function/purpose of the text

comments are focused exclusively on the negative aspects of the product

comments explicitely confirm good solutions to translation problems (e.g. with a tick, a plus or other postivie character)

comments explicitely stress the postive aspects with regard to the lexical or stylistic solutions

0 20 40 60 80 100 120

trainers

students in 2014

students in 2012

DETAILS ON THE FORM OF COMMENTS

underlined words or parts of sentences without further comments

underlined words or parts of sentences with 'correct' answers added

underlined words or parts of sentences with comments on the type of error

exclamation marks, question marks or other similar characters (without further

comments)

brief comments on the translation/summary beside the

text

extended comments on the translation/summary beside the text

biref oral comment in class

extended oral comment in class

brief oral comment at office hours

extended oral comment at office hours

0 10 20 30 40 50 60 70 80

trainers L1-->L2trainers L2-->L1students in 2014 L1-->L2students in 2014 L2-->L1students in 2012 L1-->L2students in 2012 L2-->L1

SUMMATIVE ASSESSMENT METHODS

only with a numerical grade (without any particular explanation, points etc.)

with a numerical grade based on a system with points

with a numerical grade based on a verbal explanation

0 10 20 30 40 50 60 70 80 90 100

trainersstudents in 2014students in 2012

COMPARISON BETWEEN BA AND MA LEVEL

no, the system remains unchanged

I notice minor changes I notice major changes0

10

20

30

40

50

60

students in 2012students in 2014trainers

Answers of those students/trainers which are at MA level

TRAINERS‘ ASSESSMENT SYSTEMS

it is useful to the students because based on the system I can clarify my grade

I use an assessment grid, in order to be able to grade translations in the most objective way possible

I use the absoulte method: a translation gets the grade that it deserves, regardless of the other translations in the group

I use the relative method: the best translation gets the highest grade etc.

I have trouble assessing and grading translations: I can't always find the right way to explain my corrections/grades to the students

my system works well forclarifying my grading to the students but it's (too) time-consuming

I don't really spend a lot of time explaining my grading

0 10 20 30 40 50 60 70 80

STUDENTS‘ RECEPTION OF ASSESSMENT SYSTEMS

the teacher explains the grade objectively and I understand why it is as it is

the teacher does not explain the grade, the criteria seem subjective and I don't know why the grade is as it is

when I compare myself to my colleagues I don't know why their grades are higher/lower

0 10 20 30 40 50 60 70 80

students in 2014students in 2012

CONCLUSIONS

Informative feedback vs. Trainers‘ workloads

What trainers and students agree on:Comments in written form are the least

frequentThe most frequent comment is underlining

and brief oral comments in classThe way summative assessment is

achievedThe assessmet system does not change

(much) between BA and MA levels

CONCLUSIONS

What they do not agree about:Frequency with which translations are

assessed Length of the comments (extensive

enough?)Contents of the comments

positive feedback (not enough?) lack of understanding of the criteria:

align the expectations of the students by stating the criteria in a clear, unambiguous way

CONCLUSIONS

Generally, students‘ responses in 2012 and 2014 are quite similar

Greatest differences:Comments tend to be written more often

in 2014Explicit positive feedback is more frequent

in 2014Comments are more brief in writing and

more extensive orally at office hours in 2014

More students are unsure of the trainers‘ criteria in 2014

CONCLUSIONS

To train highly competent, self-confident translatorsPositive feedback is essentialKnowing when you succeed is essential

Future research:Follow assessment reception through the

yearsFollow trainers‘ perceptions as well Include other potentially interesting areas

and questions

THANK YOU FOR YOUR ATTENTION