View
217
Download
2
Category
Preview:
Citation preview
Practices, policies and tensions in classroom‐based assessment
Dina Tsagari Pavlos Pavlou University of Cyprus
The 6th Annual EALTA Conference, 4 ‐
7 June 2009
Department of Teacher Education,
University of Turku
Finland
Classroom Assessment:
‘process of obtaining information which is used to make educational decision about students, and to judge instructional effectiveness and curricular
adequacy’
American Federation of Teachers, National Council on Measurement in Education
, American
Association of Colleges for Teacher Education & National Education Association
http://www.unl.edu/buros/bimm/html/article3.html
Teachers should be skilled in:
•
choosing
and developing
assessment methods appropriate for instructional decisions.
•
administering, scoring
and interpreting
the results of both externally‐produced and teacher‐produced assessment methods.
•
using assessment results when making decisions about individual students, planning teaching, developing curriculum, and
institutional improvement.
•
developing, using
and evaluating
valid student grading procedures which use student assessments.
•
communicating assessment results to students, educational decision makers and other concerned stakeholders.
‘Standards for Teacher Competence in Educational Assessment of Students’
1990,
American Federation of Teachers, National Council on Measurement in Education
,
American Association of Colleges for Teacher Education & National Education Association
http://www.unl.edu/buros/bimm/html/article3.html
Literature review
•
Studies have yielded contradictory results depicting teachers
•
heavy users of tests (Gullickson, 1984)
•
showing preference for informal assessment methods (Brindley, 1989; Mavrommattis, 1997; Stiggins and
Conklin, 1992).
Literature review
•
Studies depict teachers as ‘assessment illiterate’, e.g.Teachers•
had superficial knowledge of test use, esp. in interpreting
standardized tests (Goslin, 1967; Gullickson, 1984) •
reported using a variety of assessment methods but their most
frequently used formats corresponded to those used by formal external examinations (Rogers, 1991; Wilson, 1998; 2000).
•
did very little reflection on what was being assessed and were unaware of the assessment work of their colleagues (Black and Wiliam, 1998; Harlen & Deakin‐Crick, 2002; 2003).
•
had little knowledge of assessment frameworks and adapted whatever assessment activities were in their disposal to suit
their own teaching contexts (Breen et al., 1997; Arkoudis and O’Loughlin, 2004; Davison 2004).
Literature review
•
Teachers’
assessment
practices
vary
according teachers'
experience,
their
views
of
the
role
of
assessment
in
the
curriculum,
collegial
expectations and external reporting demands (McCallum et al, 1995; Breen et al, 1997).
Literature review
•
Studies
also
highlight
the
tensions
between administrative and educational purposes for the use of assessment
instruments
and
state‐mandated
assessment
policies
on
teacher
assessment
(Rogers, 1991;
Breen
et
al.,
1997;
Wilson,
1998;
2000;
Arkoudis
and O’Loughlin, 2004; Davison 2004).
Conclusions of the literature review
•
The
role
of
the
EFL
teacher
in
classroom‐based assessment has received little attention in the research literature, e.g.
•
What
are
the
purposes
of
classroom‐based assessment?
•
What
assessment
methods
do
teachers
use
for classroom‐based purposes?
•
How
do
they
choose
and
develop
their
assessment methods?
•
How do they use them in the classroom?•
Are
teachers
sufficiently
trained
in
classroom
assessment?
• These issues remain largely unproblematized and unresearched
Aims of the study:
To investigate the
a) purposes
b) methods
c) procedures
d) training needs
of EFL teachers’
classroom assessment in state
schools Cyprus
•
Survey questionnairesa.
Biographical Information
b. Purposes of LTA
c. Methods of LTA
d. Procedures of LTA
e. Training in LTA
Types of data
A. Biographical information
Types of schoolsTeachers(n=191)
Primary Schools 13%Junior High Schools 33%Senior High Schools 47%
Technical & Vocational Schools 7%
A. Biographical information
Teaching experience(years)
Teachers (n=191)
0‐1 4%
2‐5 12%
6‐10 14%
11+ 70%
B. Purposes of LTA
2. Why do you assess your students? Teachers (n=191)
Student‐centred purposes
b. To obtain information about my students’
progress 88%
d. To motivate my students to learn 66%e. To make my students work harder 54%
f. To prepare my students for external tests 50%
Instruction‐based purposes
c. To plan my teaching 69%Administration‐based purposes
g. To determine students’
final grades 61%
a. To place students at appropriate levels 44%h. To provide information to central administration
(e.g. school, Ministry)24%
B. Purposes of LTA
3. What kind of tests do you use? Teachers(n=191)
Tests(n=262)
b. Progress tests 88% 72.2%
c. Achievement tests 75% 11%
a. Mini‐quizzes 72% 16.3%
d. Diagnostic tests 54% ‐
e. Placement tests 26% 0.3%
C. Methods of LTA
•
Teacher‐oriented assessment methods (methods designed and administered by the teachers)
C. Methods of LTA ‐
Reading
6.1 What types of tasks do you use in your tests when you test reading?
Teachers(n=191)
Test tasks(n=278)
True‐false items 84% 26%
Matching items 74% 2.5%
Short answer questions 68% 48.2%
Multiple‐choice items 64% 21.2%
Sentence completion items 60% 0.35%
Fill in forms 49% 0.35%
Jumbled sentences 63% ‐
Cloze 50% ‐
Information transfer 33% ‐
Visual representation 9% ‐
•
Integrating Reading + Vocabulary + Writing = 41% tests
•
English paper of the School leaving exam (university admission)
C. Methods of LTA –
Reading/Writing
Follow‐up writing tasks Test tasks(n=42)
Summary writing 26%
Extended follow‐up writing 74%
C. Methods of LTA –
Reading/Vocabulary
Vocabulary task types Test tasks (n=160)
Find words in text 38%
Match words from text with definitions 29.3%
Find opposites of words in text 10%
Explain the meaning of underlined words in text 5.6%
Fill in discrete sentences with words from text 8.7%
Other 8.4%
C. Methods of LTA ‐
Reading
6.8 Do you use other methods to test your students’
language skills?
Teachers (n=191)
Reading
Ask oral questions based on text 76%
Reading aloud 44%
C. Methods of LTA ‐
Writing
6.2 What types of tasks do you use in your tests when you test writing?
Teachers(n=191)
Test tasks(n=62)
True‐false items 80% ‐
Matching items 77% ‐
Joining sentences 57% ‐
Multiple‐choice items (to identify grammar errors) 53% ‐
Editing a piece of writing 53% ‐
Writing short paragraph 83% 27.5%
Writing composition/essay 81% 72.5%
•
Task environment (audience, purpose, word limit…)•
Task types (limited)
•
Marking criteria (rare)
C. Methods of LTA – Listening
6.3 What types of tasks do you use in your tests when you test listening?
Teachers(n=191)
Test tasks(n=6)
Short answer questions 72% ‐
True/False 65% ‐
Multiple‐choice 61% 2
Order jumbled pictures 57% ‐
Retelling a story 52% ‐
Taking notes 48% ‐
Following directions 42% 2
Filling in forms 41% ‐
Dictation 37% ‐
Summary writing 35% 2
C. Methods of LTA – Speaking
6.4 What types of tasks do you use in your tests when you test speaking?
Teachers(n=191)
Test tasks(n=0)
Oral descriptions (event/object/photo) 43% ‐
Oral directions 37% ‐
Sentence response cues 2% ‐
C. Methods of LTA ‐
Speaking
6.8 Do you use other methods to test your students’
language skills?
Teachers (n=191)
Speaking
Interviews 62%
Oral discussion with each student 59%
Oral presentations 53%
Oral reading 39%
Role plays 35%
Group discussions 35%
Public speaking 29%
Info gap activities 28%
C. Methods of LTA – Grammar
6.5 What types of tasks do you use in your tests when you test grammar?
Teachers(n=191)
Test tasks(n=504)
True/False 81% 8%
Sentence Completion 76% 42%
Sentence Transformations 72% 13.5%
Multiple‐choice 66% 5%
Cloze 64% 4%
Identify grammar errors 63% 0.2%
Re‐writing sentences 41% 2.6%
Editing a piece of writing 38% 3.6%
Put words of a sentence in order 33% 3.2%
Other ‐ 18%
C. Methods of LTA ‐
Vocabulary
Do you assess your students’
vocabulary skills?
Yes
98%
No
1%
No answer
1%
C. Methods of LTA –
Vocabulary
6.6 What types of tasks do you use in your tests when you test vocabulary?
Teachers(n=191)
Test tasks(n=504)
True/False 82% 5.8%
Sentence Completion 76% 34%
Crosswords 67% 0.4%
Cloze 65% 14.5%
Multiple‐choice 64% 6%
Matching 57% 5.2%
Derivatives 57% 21.3%
Synonyms/Antonyms 52% 2.4%
Labeling 41% 0.4%
Translation from L1 to L2 17% ‐
Other ‐ 10%
C. Methods of LTA –
Language Functions
Do you assess language functions in your tests?
Yes
83%
No
15%
No answer
2%
C. Methods of LTA –
Language Functions
6.7 What types of tasks do you use in your tests when you test language functions?
Teachers(n=191)
Test tasks(n=0)
Multiple choice 80% ‐
Responding to situations 46% ‐
Matching 43% ‐
Filling in gapped dialogues 33% ‐
Discourse chains 26% ‐
Content of teacher‐made tests
Language areasTeachers(n=191)
Test Tasks(n=1513)
Vocabulary 98% 41%
Grammar 97% 33.3%
Writing 97% 7%
Reading 92% 18.3%
Listening 92% 0.4%
Speaking 84% ‐
Functions 83% ‐
C. Methods of LTA
•
Teacher‐oriented assessment methods (methods designed and administered by the teachers)
•
Student‐oriented assessment methods (methods that directly involve students’
participation in the
assessment process)
C. Methods of LTA
6.8 Do you use other methods to test your students’
language
skills?
Teachers (n=191)
Diaries/Journals
Peer‐
assessmentSelf‐
assessmentPortfolios Projects
Reading 20% 28% 33% 23% ‐
Writing 22% 28% 31% 23% 49%
Listening 20% 27% 27% 20% ‐
Speaking 20% 25% 25% 20% ‐
Grammar 21% 26% 27% 20% ‐
Vocabulary 21% 26% 27% 19% ‐
Functions 4% 7% 9% 2% ‐
Other interesting characteristics of teacher‐made tests
• Parallel versions (59%)
• Odd scoring (tasks & test papers)
•Errors in aggregating final scores
•
(Parts of the) tests comprised photocopied samples of external test papers & test booklets
• Handwritten (15%)
D. Procedures of LTA – Sources of test items
7. Which of the following represents your primary source for test questions/items?
Teachers (n=191)
a. Items developed by myself 85%
c. Items from published textbook materials 70%
b. Items prepared together with other teachers 49%
d. Items found on the Internet 41%
e. Items from external exam papers 33%
f. Items from test booklets (textbooks) 22%
External Test papers
Language areas Teachers (n=191)
Vocabulary 26%
Writing 23%
Listening 23%
Reading 22%
Grammar 20%
Speaking 14%
Functions ‐
D. Procedures of LTA ‐
Feedback
8. When you give feedback to your students after a test, how do you do so?Teachers (n=191)
a. Verbal feedback 82%
c. Written comments 72%
e. Total test score 62%
d. Conference with student 39%
b. Checklist 29%
f. A letter grade 28%
D. Procedures of LTA ‐
Feedback
9. When you give final evaluation feedback to your students, how do you do so?
Teachers (n=191)
c. Total test score 61%
b. Written comments 61%
d. A letter grade 43%
a. Checklist 22%
Other 2%
D. Procedures of LTA ‐
Stakeholders
10. Who will see the test results? Teachers (n=191)
a. Students 100%
b. Parents 74%
c. Head teachers 33%
d. Inspectors 26%
E. Training in LTA
a. During your pre‐service or in‐service teacher training, have you learned something about testing and assessment
(theory and practice)?
Teachers (n=191)
Yes 72%
E. Training in LTA
b. Types of trainingTeachers (n=191)
Completed a full course on testing and assessment ‐
Completed a course in which testing and assessment were topics54%
Completed a workshop on testing and assessment 39%
E. Training in LTA
c. Who organised your training in LTA? Teachers (n=191)
Pedagogical Institute 31%
University Department 28%
School Advisors/Inspectors 13%
External Examination Boards 6%
Teacher’s Associations (e.g. CyTEA…) 4%
Publishers 4%
E. Teachers’
training needs in LTA
•
classroom‐focused LTA
•
purposes of LTA
•
content and concepts of LTA
•
the use of the CEFR as the basis of test design
E.1 Training of teachers in classroom‐focused LTA
1.1 Please specify if you need training in the following domainsYes No
g) Using the European Language Portfolio,
an adaptation of it or some other portfolio type60% 16%
a) Preparing classroom tests 49% 26%
d) Using self/peer‐assessment 48% 21%
c) Giving feedback to students based on information from tests / assessment
43% 25%
f) Using informal, continuous, non‐test type of assessment43% 23%
b) Using ready‐made tests from textbook packages 34% 32%
e) Using diary/journal assessment 27% 11%
E.2 Training of teachers in purposes of LTA
2.1 Please specify if you need training in the following domainsYes No
c) To place students onto courses, programmes, etc 49% 24%
a) To give grades 44% 26%
b) To find out what needs to be taught / learnt 41% 29%
d) To award final reports (at school) 41% 24%
E.3 Training of teachers in content and concepts of LTA
3.1 Please specify if you need training in the following domains:Yes No
e) Testing aspects of culture 52% 22%
f) Establishing reliability of tests / assessment 52% 20%
g) Establishing validity of tests / assessment 52% 19%
h) Using statistics to study the quality of LTA 49% 18%
d) Testing integrated language skills 44% 24%
a) Testing receptive skills (reading / listening) 44% 26%
b) Testing productive skills (speaking / writing) 42% 27%
c) Testing grammar & vocabulary 41% 26%
f) Testing language functions 28% 7%
E.4 Training of teachers in other LTA activities
4.2 Please specify if you need training in the following domain:
Yes No
Using the Common European Framework as the basis for your tests or assessment practices 81% 7%
Overall ‐
Teacher training in LTA
Areas of training in LTA
Teachers (n=191)
Positive Negative
d) use of the CEFR 81% 7%
c) content and concepts of LTA 45% 21%
b) purposes of LTA 44% 26%
a) classroom‐focused LTA 43% 22%
Purposes of classroom‐based assessment
• are diverse and give rise to different teacher positioning in assessment
• tensions arise: need for summative and formative assessment data
• create imbalance in the range of assessment opportunities that teachers provide for their learners
Practices and procedures of classroom‐based assessment
Teachers • rely on the paper and pencil format to more informal,
student-oriented approaches to assessment• do not seem to vary their testing methods very much• are not always efficient in scoring or setting their test
papers • do not easily share their assessments among them• report using a variety of feedback mechanisms (?)• favour external examination, textbook and test booklet
formats• test a limited range of language skills
Practices and procedures of classroom‐based assessment
Classroom-based• is operationalised as the testing of part of students’
linguistic knowledge by presenting students with a series of summative mini-achievement tests
• places emphasis on language ‘product’ rather than ‘process’
• is limited to assessment of learning rather than assessment as learning
Training in classroom‐based assessment
Teachers• are not sufficiently competent in classroom assessment• teacher education programmes do not provide adequate
training in classroom assessment
Recommendations•
Clear assessment guidelines and specifications – Ministry
Education/Pedagogical Institute
•
Pre‐/in‐service LTA training courses: regular/relevant•
Teacher assessment teams
•
Collaborative assessment projects
•
Assessment training booklets
•
Self‐access materials
•
Technology (internet, online tutorials)
•
Teacher publications and presentations in LTA
•
Formal LTA courses in University programmes
Further research
•
Follow‐up questionnaires
•
Classroom observations/recordings (discourse analysis)
•
Assessment documents
•
Follow‐up interviews with teachers, school advisors/inspectors, parents and students
•
CBA research projects in other contexts
To give feedback or for further details please contact:
Dina Tsagari
Pavlos
Pavloutsagaric@otenet.gr
ppavlou@ucy.ac.cy
Thank you!
Recommended