Annotated Bibliography and Literature Survey:Annotated Bibliography and Literature Survey:Assessment of Language ProficiencyAssessment of Language Proficiency

in the Health and Social Services Sectorin the Health and Social Services Sector

Research Report

Presented by Juliane Bertrand

November 24, 2005

Terms of Reference

Survey the following:

1) ESL proficiency tests for adults working in a professional environment, with emphasis on health and social services workers

2) Materials generated by each such test

3) Studies on ESL proficiency assessment

Main ESL Proficiency Tests

• Administered directly by the design agency

- Test of English as a Foreign Language (TOEFL)- Michigan English Language Assessment Battery (MELAB) - Test of English for International Communication (TOEIC) - ETS Test of Professional English (ETS TOPE)

• Administered by the client institution

- Institutional Testing Program (TOEFL ITP) - Speaking Proficiency English Assessment Kit (SPEAK) - Secondary Level English Proficiency Test (SLEP) - Michigan Test of English Language Proficiency (MTELP) - Comprehensive English Language Test (CELT)

Description of ESL Proficiency Tests

Type of questions Skills assessed Environment Test Duration

Multiple choice

Short answer

Compo-sition and/or

interview

Oral compre-hension

Written compre-hension

Written produc-

tion

Oral produc-

tion

School Work-place

Area of health

and social

services

TOEFL paper

3.20 hrs X X (written)

X X X (TWE)

X *

TOEIC 2.30 hrs X X X X X

MELAB 3.30 hrs X X (both)

X X X X X X

TOEFL ITP

1.55 hrs X X X X X

SLEP 1.20 hrs X X X X

SPEAK 20 min X (oral)

X X

CELT 2.15 hrs X X X

MTELP 1.30 hrs X X X

Conditions forLanguage Proficiency Testing

Test Type of room Staff training Cost

TOEFL Regional centres N/A Test taker: US$140

TOEIC

At the institution Standard room or computer lab

TOEIC staff, or TOEIC-trained institutional staff Institution: $45 + taxes per test taker

TOEFL ITP

At the institution Standard room

Instruction manual supplied by ETS Marking by ETS representatives

Institution: $42.50 + taxes per test taker

MELAB At Ann Harbour N/A Test taker: US$120

SLEP At the institution

Standard room Starter kit:: US$235

10 question booklets: US$85 100 answer sheets: US$85

CELT At the institution Standard room N/A

Examiner's kit: $45 Cassette: $45

10 question booklets: $120.50 100 answer sheets: $55

Other costs per section: $24

MTELP At the institution

Standard room Starter kit: US$165

20 question booklets: US$75 100 answer sheets: US$10

Strengths / Weaknesses Test Strong Points Weak Points

TOEFL paper

- Standardized test - International reputation

- Does not assess oral production - Intended for the school environment - Too general (not geared to health and social services)

TOEIC - Standardized test - International reputation - Designed to assess workers

- Does not assess oral production - Too general (not geared to health and social services)

MELAB - Standardized test - Assesses the four language skills

- Not recognized in the health field - Administered in Michigan (USA)

TOEFL ITP/ SLEP/ CELT/ MTELP

- Can be administered locally - Does not assess oral or written production - Intended for the school environment - Too general (not geared to health and social services)

SPEAK - Can be administered locally - Does not assess oral production - Intended for the school environment - Too general (not geared to health and social services)

ETS TOPE - Assesses the four language skills - Referenced to typical workplace tasks

- Not tailored specifically to the health and social services sector - Administered in China

Canadian Tests

• Canadian Test of English for Scholars and Trainees (CanTEST)

Positives

- Assesses the four language skills- Uses authentic material

Negatives

- Administered only in Toronto, Ottawa, Saskatoon, Winnipeg and Halifax - Not geared to the needs of health and social services workers

Canadian Tests

• Canadian English Language Benchmark Assessment for Nurses (CELBAN)

Positives

- Assesses the four language skills- Referenced to nursing tasks

Negatives

- Administered only in Vancouver, Toronto, Edmonton and Winnipeg

Canadian Tests

• CanTEST

• Cost: about $200, at test-taker’s expense+ travel to a city where the test is administered

• Duration: 3 hrs

• CELBAN

• Cost: $300 plus taxes, at test-taker’s expense + travel to a city where the test is administered

• Duration: 3 hrs 30 min

Other Assessment Instruments

• Canadian Language Benchmarks (CLB)

- Set standards for the four language skills- Centred on the test-taker’s social and professional activities- Lay the ground for designing tests for specific job sectors (e.g. CELBAN)

• Portfolio

- Tracks the learner’s progress- Avoids psychological factors that could affect test performance- Requires ongoing assessment

Materials Generated by these Tests

• Works on theory and practice

- More than 70 research reports published by ETS (e.g. TOEFL, TOEIC, SLEP, TSE) since 1977

- Numerous CCLB documents describing the theoretical bases of Canadian language standards, the characteristics of those standards, and resulting ideas for activities

- Technical manuals on administering language proficiency tests; charts to help interpret test results

Materials Generated by these Tests

• Validity studies

- Frequently re-assessed for standardized tests

- Difficult to produce for locally administered tests given widely varying test management procedures

- CELBAN = a first study suggests a positive link between success with CELBAN and withTOEFL (CCLB, 2004)

Materials Generated by these Tests

• Testimonials from institutions using these tests

- Overall satisfaction with TOEFL and TOEIC

- TOEFL unsatisfactory for nurses

- CELBAN = a study on nurse and administrator satisfaction should be undertaken in 2006

Studies on ESL Proficiency Assessment

• Background

- Prior to the communicative approach, language proficiency was the sole focus of assessment.- Integrated test introduced- Leading works: Lado, Robert (1961). Language Testing, London, Longmans, xxiv + 390 p. Valette, R. M. (1967). Modern Language Testing: A Handbook, New York, Harcourt, Brace & World. Cooper, R. L. (1968). “An elaborated language testing model,” in Problems in Foreign Language Testing: Proceedings of a Conference. Language Learning, 18, p. 57-73. Spolsky, B. (1978). Approaches to Language Testing: Advances in Language Testing, Series 2, Arlington, VA, Center for Applied Linguistics.

Studies on ESL Proficiency Assessment

• Present trends

- With the communicative approach, assessment covers all language skills.- Introduction of alternatives to the standardized test and of tests referenced to quasi-authentic tasks.- Leading works: Canale, M. and M. Swain (1980). “Theoretical bases of communicative approaches to second language teaching and testing,” Applied Linguistics, 1, p. 1-47. Bachman, L. (1990). Fundamental Considerations in Language Testing, Oxford,

Oxford University Press. Celce-Murcia, Z. Dörnyei, S. Thurrell (1995). Communicative competence: A pedagogically motivated model with content specifications. Issues of Applied

Linguistics, 6 (2), p. 5-35. Milanovic, Michael and Nick Saville (Eds.) (1996). Performance Testing, Cognition and Assessment. Selected papers from the 15th Language Testing Research Colloquium, Cambridge and Arnhem, Cambridge, Cambridge University Press, coll. “ Studies in Language Testing 3,” 314 p.

Conclusion

• None of the tests surveyed can be used to assess proficiency in English as a second language for all job categories in the health and social services sector.

• CELBAN, designed with reference to CLB, seems to be a model to follow. It is consistent with current language acquisition theories (e.g. emphasis on language objectives rather than specific grammatical constructs and on the use of authentic tasks).