Upload
arella
View
83
Download
1
Embed Size (px)
DESCRIPTION
Week 5 Lecture 4 . Principles of language assessment: validity, reliability, practicality ,authenticity ,and beneficial backwash Brown, p. 19-41. Lecture’s objectives. Understand the principles of language assessment. Use language assessment principles to evaluate existing tests. - PowerPoint PPT Presentation
Citation preview
PRINCIPLES OF LANGUAGE ASSESSMENT: VALIDITY, RELIABILITY, PRACTICALITY ,AUTHENTICITY ,AND BENEFICIAL BACKWASH BROWN, P. 19-41
Week 5Lecture 4
Lecture’s objectives
Understand the principles of language assessment.
Use language assessment principles to evaluate existing tests.
Use language assessment principles to design a good test (next week)
How do you know if the test is effective?
To answer this question, you need to identify five criteria for “testing a test” Practicality Reliability, validity, authenticity, and washback
To understand these principles let’s read the story on page 20.
1-Practicality
When do you say that this test is practical? When
It is not expensive The time is appropriate It is easy to administer The scoring and evaluation procedures are
clear and efficient.
Applying the practicality principle Read page 31
2- Reliability “a reliable test is consistent and
dependable: this means that if you give the test to the same students but in two different occasions, the test should yield=give the same results. There are five factors that might
contribute to the unreliability of a test : Students related reliability Rater reliability Test administration reliability Test reliability
Students-related reliability is related to factors that might contribute to the students themselves such as illness, fatigue, anxiety or any other physical and psychological factors which might make the observed score unreflective to the actual students performance.
Rater reliability is related to human subjectivity, biased and error.
inter-rater reliability is when two or more scorers yield inconsistent scores of the same test. Due to lack of attention, inexperience, or even biases.
Intra-rater reliability is common in the classroom teachers because of unclear criteria or biases toward particular bad or good students. (how to solve this problem? Read P,21)
Test administration reliability is related to the condition in which that test is administered.
Test reliability is related to the test itself.
Is it too long?, the time limit, poorly written test
is there more than one answer to the question?,
is the wording of the test clear?
Applying the reliability principle Example : Read page 31-32
3- Validity(Relevance to the curriculum)Test validity is the most important
principle among the other four.Test validity refers to “ the extent to which
inferences made from he assessment rests are appropriate , meaningful , and useful in terms of purpose of the assessment”p,22
A test of reading ability must actually measure reading ability.
Does the test content/ items match the course content or unit being taught?
Evidences of validity a- content validity is achieved when the
students where asked to perform the behavior that being measured. For example, when you are assessing a student's ability to speak a second language but you are asking him/her to answer a paper and pence multiple choice questions that require grammatical judgment, then you are not achieving the content validity. Does the test follow the logic of the lesson or unit? Are the objectives of the lessons clear and present in the test?
Read page 32-33
b- criterion validity refers to fact that the specified classroom objectives are measured.
c-face validity refers “ is to the extent students view the assessment as fair, relevant , and useful for improving learning” p,26
Students will judge a test to be face valid if Directions are clear Organized in a logical way No surprises Time appropriate Appropriate in difficulty
Applying Face Validity Principle
Read page 34-35
4- Authenticity If you are claiming that your test is
authentic then you are saying that “ this task is likely to be enacted in the real world” * (do you remember one of the TOEFL problems?)
A reading passage is selected from real world sources that the students are likely encountered or will encounter.”
“Are the items contextualized rather than isolated? “ the sequencing of items that show no
relationship to one another lacks authenticity” p,28
Applying the Authenticity principle Read page 35-36
What do you notice in the two examples provided on page 35-36
5- Washback
Washback is “ the effect of testing on teaching and learning” p, 28
Teaching to the test For students to diagnose their strength
and weakness Provide feedback to the students in case
of formal and informal assessments.
How can you enhance backwash: Praise students on their correct responses. Comment on the test performance. Give constructive comments/criticism on their
weakness. Give strategies on how can they improve their
performance. Use the questions on the test that show area of
weakness to diagnose students’ weakness and where they need to focus their efforts on.
Give a chance for your students to “feed back on you feedback” to be able to focus their efforts in the coming weeks.
Samples of English Tests to Evaluate using the five principle of language testing http://
www.nysedregents.org/Grade3/EnglishLanguageArts/samplelisteningselection.pdf
http://www.henry.k12.ga.us/wle/home/crct/first%20grade/English%20test2.pdf
http://www.henry.k12.ga.us/wle/home/crct/first%20grade/Grammar%20and%20Mechanics%20Test.pdf
Class activity
Read Exercise 6 page 38-30, in small group evaluate the scenarios provided.
Week6Quiz
Constructing Classroom Tests p, 48-65
Next week
Brown, H. Douglas, 2004. Language Assessment: Principles and classroom practices. Pearson Education, Inc.
References