Embed Size (px)
Citation preview
The EXAVER Proficiency Test:
Customers’ Experiences for
Test Improvement
Mexico City
March 22nd, 2019
Janeth Tello Carrillo
Universidad Veracruzana
Contents1. Location2. Current Test Administrations3. Research Question4. Key concepts5. Methodology6. Results7. Conclusion
History
Location
▪ eight examination centers administering tests twice a year
▪ 2,500 candidates every year
Current Test Administrations
Research QuestionHow do our customers’ experiences in theSeptember – October 2018 test administrationprovide us with information to improve ourtests?
Test Impact“any of the effects that a test may haveon individuals, policies or practices,within the classroom, the school, theeducational system or society as a whole”
Wall (1997), in Cheng L., Watanabe Y., & Curtis, A. 2004
Customer Satisfaction“…an outcome of the comparisoncustomers make between theirprepurchase expectations of the serviceperformance and postpurchaseperceptions of that performance”
Cardozo, 1965, Churchill & Suprenant 1982; Szymanski & Henard, 2001; Yi, 1990 in Walker, J. 2015
Quality ControlAccording to the American Society for Quality(ASQ) it consists of the observations,techniques and activities used to fulfillrequirements for quality
MethodologyData from a twenty closed-ended questionsand one open question questionnaire wereanalyzed following both quantitative andqualitative paradigms
Results
Level of Education
B.A. – 45.8%High School – 39.4%M.A. students – 11.9%
Gender
Female – 42.3%Male - 57.5%
Opinion before taking any of the EXAVER tests
Positive – 61.2%Neutral – 36.5%Negative – 2.3%
Gender
Female – 42.3%Male - 57.5%
Yes – 78.5%No – 18.7%Not sure
After taking the test would you
recommend EXAVER?
EXAVER A2, B1 and B2
adminproblems
Academicprocedures
Qualitymaterials
Administrative procedures related to
academic decisions
«The topic for the Speaking test should be the same
for every one»
«More time should be given to take the written test»
«I had to wait for more than an hour to take the
speaking test»
Administrative problems
«there should be a download option to later print the
proof of registration»
«We should be reminded that the test administration
is coming, so we don’t forget.»
«There aren’t enough seats for the test in the capital
city»
Quality materials
«The acoustics of the room made it difficult to
understand the dialogues in the listening part»
«the air conditioning was on and I was freezing»
«there was something wrong with the CD player. The
volume was too low »
ConclusionsTest improvements in three main areas:Administration of the test
a) Adding google maps to the registration proofs,
test enrollment procedures which includes options
to save, print and edit candidates’ profiles on
EXAVER web page, two reminders a week prior
to the test administration.
Cont’Quality materials
a) Some of the examination rooms were changed
for the February test admin
b) The CD’s are recorded with more sophisticated
devices to avoid poor sound quality
c) Air conditioning is on before the test begins. After
that, (or at the listening section) it’s off
Cont’Academic test procedures
a) results of the test are published two weeks earlier
than the ones last term
b) interview times in the Speaking test lists
c) B2 test offered in more university regions
d) training was offered last December to hire more
examiners
* Test taking individually, long speaking test, more time for the
written test
ReferencesCheng L., Watanabe Y., & Curtis, A. (2004) Washback in Language Testing.
Research Contexts and Methods. Lawrence Erlbaun Associates,
Publishers. USA
Denzin, N. & Lincoln, Y. (2011) El campo de la investigación qualitativa. Gedisa
Editorial. Barcelona
Fulcher, G. Davidson, F. (2007) Language Testing and Assessment. Routledge.
USA
McNamara,T. (2000) LanguageTesting. Oxford University Press. Oxford
Walker, J. (2013) Exploring Service Logic in ESL. TESL Canada Journal. Volume 30,
No 2, Spring 2013
Walker, J. (2015) Using Critical Incidents to Understand ESL Student
Satisfaction.TESL Canada Journal.Volumr 32, Issue 2, 2015
The EXAVER Proficiency Test:
Customers’ Experiences for
Test Improvement
Mexico City
March 22nd, 2019
Janeth Tello Carrillo
Universidad Veracruzana
