Assessing English Language Proficiency A Training Moduleell.tamucc.edu/files/module_10_script.pdf · Assessing English Language Proficiency A Training Module ... For instance a non-native

1

Assessing English Language ProficiencyA Training Module

PUBLISHED AS PART OF THE LEP STUDENT SUCCESS INITIATIVE GRANT

Department of P-16 InitiativesTexas Education Agency

In collaboration withThe Institute for Second Language Achievement

(ISLA)Texas A&M University – Corpus Christi

andJoAnn Canales, Ph.D.

Professor, Texas A&M University – Corpus Christi

Slide 1: The information contained in this PowerPoint is largely drawn from Texas state documents so that educators who work with Limited English Proficient students will understand the requirements and be able to appropriately assess their English language proficiency. It also includes a section on what constitutes language and language proficiency to help educators understand the complexity of language and the importance of a comprehensive assessment to determine the appropriate placement and instruction.

Once you review and “own” the material, feel free to utilize the custom animation feature to enhance points you wish to emphasize..

2

Workshop FocusI. Review Assessment Standard and Competency for

Becoming an ESL Certified Teacher in TexasA. English as a Second Language (ESL) Standards (Standard

VI)B. TExES – Texas Examinations of Educator Standards

(Competency 007)

II. Define Language and Language ProficiencyIII. Examine Requirements

A. State Requirements1. LPAC – Language Proficiency Assessment Committee2. HLS - Home Language Survey 3. OLPT - Oral Language Proficiency Test

4. NRT - Norm Referenced Test

B. Federal (NCLB) Requirements - TELPAS – Texas English Language Proficiency Assessment System

1. TOP – Texas Observation Protocols2. RPTE – Reading Proficiency Tests in English

Slide 2: Note to facilitator: Review workshop objectives with the audience. Emphasize that they will have time to engage in application and discussion, particularly during the section numbered III B.

3

BECOMING AN ESL CERTIFIED TEACHER IN TEXAS

Assessment is addressed by Standard VI of the English as a

Second Language (ESL) Standards

http://www.sbec.state.tx.us/SBECOnline/standtest/standards/allesl.pdf

Slide 3: Say: “Standard VI of the ESL Standards addresses assessment. Let’s review what teachers are expected to know and be able to do.”

[Note to facilitator: Review the standard with the audience]ESL StandardThe ESL teacher understands formal and informal assessment procedures and instruments

(language proficiency and academic achievement) used in ESL programs and uses assessment results to plan and adapt instruction.

What teachers know and understand:6.1k basic concepts, issues, and practices related to test design, development, and interpretation;6.2k types of assessment used in the ELS classroom (e.g., recognition, production portfolio,

observation, student self-assessment) and their characteristics, uses, and limitations;6.3k standardized tests commonly used in ESL programs in Texas;6.4k state-mandated LEP identification procedures and placement and exit criteria, including the role

of LPAC; and6.5k relationships among state-mandated standards, instruction, and assessment in the ESL

classroom.What the Beginning ESL teacher can do:6.1s select, adapt, or develop appropriate assessments for different purposes in the ESL program

(e.g., diagnosis, evaluation, proficiency);6.2s interpret results of standardized tests commonly used in ESL programs in Texas;6.3s implement LPAC recommendations for LEP identification, placement, land exit; and6.4s use ongoing assessments to plan and adjust instruction that addresses individual student

needs and enable ESL students to achieve learning goals.

4

BECOMING AN ESL CERTIFIED TEACHER IN TEXAS

Assessment is also Competency 007 on the TExES for ESL Certification

http://www.texes.nesinc.com/prepmanuals/PDFs/TExES_fld154_prepmanual.pdf

Slide 4: Say: “Competency 007 on TExES for ESL Certification focuses on Assessment. Listen for the similarity to Standard VI we just reviewed.”[Note to facilitator: Review the competency below with the audience]TExES CompetencyThe ESL teacher understands formal and informal assessment procedures and instruments used in ESL programs and uses assessment results to plan and adapt instruction.The Beginning ESL teacher:Knows basic concepts, issues, and practices related to test design, development, and interpretation and uses this knowledge to select, adapt, and develop assessments for different purposes in the ESL program (E.g., diagnosis, program evaluation, proficiency).Applies knowledge of formal and informal assessments used in the ESL classroom and knows their characteristics, uses, and limitations.Knows standardized tests commonly used in ESL programs in Texas and knows how to interpret their results.Knows state-mandated LEP policies, including the role of the LPAC, and procedures for implementing LPAC recommendations for LEP identification, placement, and exit.Understands relationships among state-mandated standards, instruction, and assessment in the ESL classroom.Knows how to use ongoing assessment to plan and adjust instruction that addresses individual student needs and enables ESL students to achieve learning goals.

5

How English Proficient Are you?

To FAT32You can use the FAT32 conversion tool to easily convert your hard

disk to the FAT32 file system. However, before you convert filesystems, you should read the following information carefully:

If you didn’t make a Windows 98 Startup Disk, before you convert to FAT32, you should create one by following the steps in the ”Using the Startup Disk” section earlier in this chapter.

You shouldn’t convert any drives on which you also want to run an operating system that doesn’t support FAT32. Also, if you’re running Windows 98 and another operating system in a dual-boot environment, converting your primary disk drive to FAT32 may cause the other operating system to be unusable. This is true even if the other operating system is installed on a different drive.

If anti-virus software is running, it may detect the request to update the partition table and book record and prompt you to allow the updates. If this occurs, instruct the anti-virus software to allow the updates.

Once you convert to FAT32, you can’t compress stored information or convert back to FAT16 unless you use a third-party partition management utility designed for that purpose.

Slide 5: [Note to facilitator: The purpose of this exercise is to create a mindset that English language proficiency is relative – it is very contextually driven. Before clicking on the text,]

Ask, “How many of you consider yourselves proficient in English?”

Then say:“Before we get started talking about language proficiency, let’s see how we stack up. I am going to ask someone to read this aloud and then ask someone to explain the content and the process. Please follow along, listen carefully, and read silently.”

Points to be made:While you may deduce it’s something having to do with computers, what specifically is it trying to tell us can be done?What is FAT32?How do we go about using this tool?What is dual-boot environment mean?What is a partition table?What does it mean to “compress stored information”?What is a third-party partition management utility?

Any language is very contextually driven. While it is also interactive dependent, it is possible to be able to read and not comprehend what was read. Which is why it is important to assess discreetly, as well as holistically.

Let’s spend a few minutes defining language as a way to better understand how to determine English language proficiency.

6

Language and Language Proficiency Defined

Linguistic Structures– Graphophonemics– Lexicon– Morphology– Semantics

Communication Skills

WritingReadingWritten

SpeakingListeningOral

ExpressiveReceptive

Slide 6:Language is a multi-dimensional concept that requires the interaction of linguistic structures, communication skills, and sociolinguistic variables such as the domains of language, registers, dialect, and knowledge of the language. (Canales, 1988)Linguistic structures refers to the four subsystems that include grapho/phonemics (letters/sounds); lexicon (vocabulary); morphology (grammar and word order); and semantics (meaning). These subsystems exist in all languages. Levels of proficiency may differ between any or a combination of these subsystems. For example, an individual's pronunciation of English may be difficult to understand and word order may not be grammatically correct, however the verbal delivery of a complex concept may be achieved. Thus, the individual's language proficiency may be "high" with respect to vocabulary and meaning but "low" regarding the use of sounds and grammar/syntax. Additionally, these subsystems are also interactive, not mutually exclusive operations in everyday communication.Communication skills are either receptive (listening/reading) or expressive (speaking/writing) and are accomplished in pairs related to the mode of communication , e.g., oral (listening/speaking) and written (reading/writing). In either case, the pairing of the skills does not suggest that proficiency in one skill, e.g., listening, necessarily equals proficiency in its paired skill, e.g., speaking or reading. For instance a non-native speaker of English whose language learning efforts have not focused on oral production (speaking), or on academic reading materials, might easily be able to understand a set of oral directions, but would have a difficult time giving these same directions, or comprehending the directions if they were written.The process of learning a second language differs from learning one's first language in that the order of first language acquisition is speech to print, i.e., listening/speaking to reading/writing while for some individuals the order of second language acquisition might possibly be listening, reading, writing, speaking. Thus, measures of language proficiency ought to allow for students to demonstrate their levels of proficiency while employing different communication skills simultaneously (integratively).

7

Language and Language Proficiency Defined (Continued)

Sociolinguistic Variables– Social domains– Registers– Dialect– Knowledge of the language

Slide 7: Sociolinguistic variables refer to those environmental variables which impact on language, namely, the social domains, the registers, the dialects, and the knowledge of the language.Social domains of language include four categories: home, peers, school, and community. This aspect of language suggests, and rightly so, that the language spoken within the context of the four domains may differ, not only regarding the pronunciation of the words, but the vocabulary used and the level of complexity required of the utterances (phrases/sentences). While there is a commonality among each of these domains to a certain extent, each of the domains have their unique demands requiring users of language to adapt according to the personal and social needs of the domain participants. The content of discourse (connected speech) and the vocabulary utilized must vary from setting to setting if communication is to be the primary outcome of the communicative exchange.Registers of a language refer to a range of expressive opportunities that depend on such factors as situation, participants, relationships, roles, topics, and locale. For example, the formality of a teacher's conversation will differ considerably when talking with students, versus colleagues, versus the school principal.Dialect refers to the form of speech employed by a given community. This may consist of shortened or altered forms of words found in the native language such as `mon' for `man' commonly used by natives of the Caribbean or Bahamian islands. Some easily distinguishable dialects include Southern, Bostonian, and New Jersey dialects.Knowledge of the language refers to the extent of the exposure an individual has in a given language. The average monolingual child from a home environment where there are frequent opportunities for verbal exchange, including family discussions and reading of children's literature, enters kindergarten at age five with at least 23,741 hours (365.25 days/year x 5 years x 13 hours of wakefulness/year) of exposure to the oral (listening and speaking) skills of the language spoken at home, with peers, or in the community. Thus, it is possible for children to know the sounds of a language without the letters that represent them and to use them in meaningful combinations to form words and sentences in a variety of settings before the first day of school. [Note: It is this standard against which linguistically different children or children limited in their own native language must compete in the school environment.)None of the above characteristics of language such as dialect, listening, and sound system are mutually exclusive, that is, they do not occur in isolation. Rather these characteristics function integratively to communicate meaning. Thus, it follows that assessment of language proficiency should also be integrative (assess the use of language in context) if it is to be a useful measure of proficiency.

8

Stages of Language Learning

Child learns difficult phonemes/complex grammar. “If I were you. I would have gone.”

Child is able to create his/her own language. “Mommy, I love you 1,000 trees.

Child can generate original language. “When I get big, I’m gonna be an astronaut..”

Child makes errors by overgeneralizing. “I goed to the store today.”

Language has features of adult language. “I want to go outside with you.”

Speech is abbreviated. Child uses 2 –word utterances. “Baby, go?”

Child vocalizes. Babbles “ma-ma-ma”.

Grade 2-6 Level

Communication Development Stage

Grade 1 LevelCreative Stage

Kindergarten LevelAutomatic Stage

60 MonthsStructural Awareness Stage

48 MonthsExpansion & Delimiting Stage

24 MonthsUnitary Stage

12 MonthsInfant Stage

Slide 8: When children learn a language from birth, they have the luxury of developmental growth over time. Generally speaking, children begin verbalizing at the age of 12 months and gradually build on sounds to add words, then phrases and sentences in more complex structures as they mature. They also have the luxury of developing their oral language skills before they develop their written skills.

Unfortunately, our students do not have the same luxury. That is why we need to assess their proficiency in an integrative way, as well as facilitate the development of their skills integratively, i.e., engaging them in activities that use listening, speaking, reading, and writing in as many settings as possible.

9

More Than Meets the Eye!

• Appearances• Dialect• Foods

and • Customs•Religious beliefs•Values•“Grandmother’s stories”•Interpersonaldynamics

Slide 9: It is also important to understand that there is more to the students than meets the eye. They may not only have different physical characteristics, speak a different language, and eat different foods, they also bring with them different •customs (celebrating “Christmas” at different times of the year, e.g., January 6 – El Dia de Los Reyes);•religious beliefs (believing in a different deity called by a different name, e.g., Buddha);•Values (cooperative vs competitive); •Grandmother’s stories (beliefs handed down through the generations in the form of “sayings” or behaviors, e.g., if someone drops a knife it means a male will be visiting, or not exercising after eating for at least 1 – 2 hours); and •Interpersonal dynamics (personal distance between individuals or avoidance of eye contact).

These characteristics are also important to determine, in order to appropriately work with the students initially, but also to prepare them to function in our society.

10

Texas Education Agency Requirements

HLS - Home Language Survey

OLPT - Oral Language Proficiency

Test

NRT - Norm Referenced Test

LPAC – Language ProficiencyAssessment Committee

Slide 10: Since 1975, as a result of the LAU remedies following the Supreme Court ruling in the Lau v. Nichols case required schools to offer instruction in a language comprehensible to all students, the Texas Education Agency has required:

Home Language Survey (HLS) – a brief questionnaire intended to determine the need for further assessment.

Oral Language Proficiency Test (OLPT) – intended to assess listening and speaking proficiency skills in the student

Norm Referenced Test (NRT) – a standardized achievement test (as opposed to a criterion referenced test) intended to assess reading and writing

LPAC – Language Proficiency Assessment Committee – a group of individuals responsible for determining the best placement for children who are linguistically different

Following are some additional points about each of these assessment components.

11

Home Language Survey

Administered, only once, at time of enrollment

Sample survey questions:– What language is spoken in your home

most of the time?"– What language does your child (do you)

speak most of the time?"

Slide 11: Note to facilitator: Read the slide and then read below.

Additional questions may be asked as well, e.g.,Is a language other than English used in the home?What was the student’s first language?Does your child speak a language other than English?

A positive response to any of the questions warrants administering an OLPT

PROS and CONS of Home Language Surveys

PROS1. Brief questionnaire - Can be completed in less than 5 minutes2. Can serve as the initial screening3. Can be administered in any language4. Does not require specialized training

CONS1. Questions can be misleading, e.g., Grandfather may speak a language other than English but the

child may speak both.2. Parents are required to have literacy skills.3. Administrators may allow bias to influence parents.

4. Because this is essentially a self-report, parents tend to negate the use of a language other than English in the home.

12

Oral Language Proficiency Test

Examples: IDEA, LAS

Example of Linguistic Structures Measured by the LAS

XXXXSpeaking

XXListening

PhonologySemanticsSyntaxVocabularyCommuni-

cation Skills

Slide 12: With respect to OLPTs, the state has a list of recommended tests. The two most commonly used are the IDEA and the LAS.

For prekindergarten through Grade 1, scoring below the grade appropriate score is sufficient to designate the child as LEP.

For students in Grades 2-12, the same applies as well as their performance on the NRT.

OLPTs are administered individually, and usually by an adult unknown to the LEP student. Such a situation can be intimidating to the student causing limited oral language production, particularly in English.

Two key points about the most widely used OLPTs. 1. The IDEA was first a language development kit developed by a Speech Pathologist and a classroom

teacher. The OLPT emerged from the kit, thus linking the exam performance to the activities in the kit.2. The LAS assesses English proficiency in the areas indicated by the “Xs” in the cells. The secondary version

has a pictorial story sequence designed to elicit talk from the student. The title of the story is “The Lady in Red”. In some cultures, a “lady in red” does not represent something that is discussed publicly, and especially to a stranger in a one-on-one situation.

Additionally, in the late 1980s, the TEA convened panels of experts and practitioners, to review these instruments for their appropriateness. Every committee convened had the same finding . . . These measures would not meet American Psychological Association standards. They are too brief and too limited in their assessment items to accurately and comprehensively determine an individual’s language proficiency. However, most of the OLPTs can be administered in 20 minutes or less, which is what makes them appealing to school district personnel. It’s important to consider, however, that these OLPTs simply assess Basic Language used socially, and a bare minimum of skills as evidenced by the chart indicating the linguistic skills measured.

13

Norm Referenced Test

Examples:

– CTBS – Comprehensive Test of Basic Skills– SAT – Stanford Achievement Test– CAT – California Achievement Test

Bilingual or ESL Program Placement Criteria

– Score below the 40th percentile or– Unable to take the test due to limited English proficiency

Issues with SATs

– Assess reading skills, not writing– Challenging for students with

Poor reading skillsLearning disabilitiesLimited attention span

Slide 13: Examples of Norm Referenced Tests include the CTBS, SAT, and theCAT (or you can simply mention the one used in your district}.

For students in Grades 2-12 to qualify for bilingual or ESL programs, they must score below the 40th percentile on the NRT or evidence such a limited ability in English at his or her grade level on the reading and language arts sections of a TEA-approved norm-referenced assessment instrument or other test approved by TEA at the time of administration, so as to warrant it invalid.

It’s important to remember that while NRTs were created to compare students’performance, they are primarily a reading test, and there are many reasons why a student would not score well on such a test including having poor reading skills, learning disabilities, and/or a limited attention span.

Thus, while the intent to assess all four communication skills with OLPTs (listening and speaking) and NRTs (reading and writing) may evidence sound reasoning, the basic assumptions are ill-conceived.

14

Texas Education AgencyRequirements

LPAC – Language ProficiencyAssessment Committee– Role– Composition– Some Decision Making Points– Documentation

http://www.tea.state.tx.us/student.assessment/resources/guides/lpac/index.html

Slide 14: LPAC – Language Proficiency Assessment Committee is a fourth requirement of TEA.Role of the LPAC

• Adhere to the provisions stipulated in the LPAC manual• Function as a committee to make assessment decisions on an individual student basis within four

weeks of the enrollment of ELLs.• Monitor students’ progress and needs• Determine appropriate instructional interventions• Document assessment decisions, instructional interventions for exempted students, and the

reason for each exemption in the student’s permanent record file. (see Assessment Decision Record on TEA LPAC Manual website).

Composition of the LPAC implementing a bilingual program (including but not limited to):A campus administratorA professional bilingual educatorA professional transitional language educator (a bilingual teacher or ESL teacher)A parent of an ELL student (not employed by the school district or charter school)Composition of the LPAC implementing an ESL program (including but not limited to):One or more professional personnel, including an ESL teacherA parent of an ELL student (not employed by the school district or charter school)Some Decision Making Points:Administration of the TAKS in Spanish vs. English – limit of three years of administration in SpanishStudents in ESL classes are administered the English TAKSMay exempt students from TAKS testing only if the student is having academic or linguistic difficulties that stem

from school experiences outside the U.S., e.g., not yet received a rating of advanced high on RPTE. (see exemption flow charts on pages 21, 23, 29, and 31 of the LPAC Manual).

15

Federal (NCLB) Requirements

TELPAS – Texas English Language Proficiency Assessment System

http://www.tea.state.tx.us/student.assessment/telpas/telpas_guide.pdf

– Meets NCLB federal requirements– Assesses ELLs in Listening, Speaking, Reading, Writing– Parent brochure available

http://www.tea.state.tx.us/student.assessment/resources/guides/parent_csr/telpas.html

– Uses 4 proficiency ratings – Beginning, Intermediate, Advanced, and Advanced High

– Has two parts: TOP (Texas Observation Protocols) – Grades K-12RPTE (Reading proficiency Tests in English) – Grades 3-12

Slide 15: In addition to the state requirements set forth by TEA, new requirements have been added as a result of the No Child Left Behind (NCLB) Act. These requirements call for a more comprehensive assessment of students’ English Language Proficiency, in each of the communication skill areas and in a variety of settings. This comprehensive, and very complementary assessment is known as:

. TELPAS – Texas English Language Proficiency Assessment SystemDeveloped to meet the federal testing requirements of the No Child Left Behind Act of 2001 (NCLB)Requires that ELLs be assessed annually in Listening, Speaking, Reading, and Writing beginning in Kindergarten until the Language Proficiency Assessment Committee (LPAC) determines that they are proficient in English.Brochures available for parents to:

provide basic information about TELPAS (available at TEA website);explain a sample test report sent to child’s school;provide summaries of proficiency ratings used to gauge progress in learning English.Uses four proficiency ratings – Beginning, Intermediate, Advanced, and Advanced HighBeginning—Students who receive this rating are in the early stages of learning English. These students have a small vocabulary of very common words and little ability to use English in academic settings. These students often communicate using English they have memorized. Intermediate—Students who receive this rating are able to use common, basic English in routine academic activities but need considerable English-language support to make learning understandable. Socially, these students are able to communicate simply about familiar topics and are generally able to understand conversations but may not comprehend all the details. Advanced—Students who receive this rating are able to understand and use academic English in classroom activities when given some English-language support. In social situations, these students can understand most of what they hear but have some difficulty with unfamiliar grammar and vocabulary. Advanced High—Students who receive this rating are able to use academic English in classroom activities with little English-language support from others, even when learning about unfamiliar material. Students at this level have a large enough vocabulary in English to communicate clearly and fluently in most situations.TELPAS has two parts:Reading Proficiency Tests in English (RPTE) – Grades 3 – 12 only - one day testTexas Observation Protocols (TOP) – Grades K – 12 - classroom observations to measure students’ progress in listening, speaking, and writing.

16

TOP – Texas Observation Protocols (READING - Grades K – 2)

Reading (K-2) – Formative assessment, using grade level rubrics found in the TOP Rater Manual, based on the following types of activities:

Paired readingSing-alongs and read-a-longs, including chants and poems;Shared reading with big books, etc.Guided reading with leveled readers/textsReading subject-area texts and related materialsIndependent readingLiterature circlesCooperative group workReading response journalsSustained silent reading

Slide 16: The Texas Observation Protocols (which are actually performance rubrics) are designed to be used as formative assessment tools. This means that the teacher can assess the students as they engage in their everyday class activities, without stopping instruction to administer an assessment instrument.

There are two different instruments, [click to next slide]

17

http://www.tea.state.tx.us/student.assessment/admin/rpte/Descriptors_06.pdf

TOP Proficiency Level Descriptors Grades K-1 Reading

Slide 17: This rubric is for use with children in K-1. It follows the same four levels of proficiency mentioned earlier and takes a holistic approach to assessing reading. This means that the teacher is asked to make a judgment of students’ reading proficiency based on children’s ability to recognize print, decode, fluency, and comprehension holistically, as opposed to rating each of these aspects discreetly.

18

TOP Proficiency Level Descriptors Grade 2 Reading

Slide 18: The rubric for use with children in Grade 2 is similar in nature extending the context to include academic, as well as social settings. [Note to facilitator: If time permits ask for a volunteer to read each of the columns.]

Children in third grade and beyond are administered the RPTE – Reading Proficiency Test in English. [Click to next slide].

19

RPTE – Reading Proficiency Tests in English

Grades 3 – 12Designed especially for ELLsVariety of reading selections and questionsSpan range of English reading abilityDifferent RPTE test for grade groups (3, 4-5, 6-8, 9-12)

http://www.tea.state.tx.us/student.assessment/resources/release/rpte/index.html

http://www.tea.state.tx.us/student.assessment/teachers.html

Slide 19 Different RPTE test for grade groups (3, 4-5, 6-8, 9-12)Beginning level–lots of pictures/common wordsAdvanced level-requires near native understanding of English

20

RPTE – Reading Proficiency Tests in EnglishEXEMPTIONS

ELL students in special education may be exempted by ARD Committee

2nd semester non-English speaking immigrants may be exempted by LPAC

ELLs with parental denials may not be exempted

Slide 20: No notes.

21

TOP – Texas Observation Protocols (WRITING -- Grades K – 12)

Writing (K-12) – Formative assessment, using grade level rubrics found in the TOP Rater Manual, of 3-5 writing samples, including academic and non-academic work, taken at time of rating. Examples of writing taken from TEA website may include:

Journal writing for personal reflectionsShared writing and language experience dictationOrganization of thoughts and ideas through prewriting strategiesWriting assignments in various subject areasPublishing and presentingLabeling pictures, objects, and items from projectsCooperative group workLearning logs for content-area concept attainmentFirst draftsRevising and editing skill application

http://www.tea.state.tx.us/student.assessment/telpas/telpas_prompts.pdf

Slide 21: Writing Protocols consist of similar holistic formative rubrics. There are also two versions: [click to next slide]

22


TOP Proficiency Level Descriptors Grades K-1 Writing

Slide 22: The first protocol is used with students in grades K – 1 and the second protocol is used with students in Grades 2 - 12. [Note to facilitator: Ask participants to “put their heads together” and review the protocol that best reflects the grade level taught. Then ask the following questions:

1. How similar are these descriptors to what you have used in the past.2. As you reflect on your own students, how accurately do these descriptors

describe your students’ capabilities?

23

TOP Proficiency Level Descriptors Grades 2-12 Writing

Slide 23: Points to address:

Teachers must be very familiar with analyzing students’ products to use these products well, as they too are holistic in nature and require the teacher to make an overall judgment of quality based on a variety of characteristics.

Say to audience: “As a way to familiarize ourselves with TEA’s expectations, we are going to assess three separate writing samples. We will be using the protocol for Grades 2 – 12. Please refer to this protocol as one of you reads the example on the screen. Then, we will determine and discuss the appropriate response.

24

Grade 4 Writing Example

Slide 24: GRADE 4 WRITING SAMPLE

This paper presents features of the beginning level of second language proficiency in writing. While some knowledge of English is evident, the writing is characterized by high frequency words (toys, beds, furniture, TV dog, etc.) and phrases presented in a list-like fashion. Despite the highly familiar, concrete topic of describing a room, the ability to express original ideas using short, simple sentences is not evident. While some cohesive devices (for example, and, also) are present, they are used in a manner that indicates a memorized or artificial pattern. Primary language features are also frequent (furniture small, pillows of color red, the bed of my kitty and the of my dog). When the attempt is made to express an idea that goes beyond listing words and phrases, the attempt is unsuccessful (And in the dark if see whats glow, and if see scary), and the ideas cannot be understood even by individuals accustomed to working with English language learners. [Example and Analysis taken from p. 35 of the 2005 TOP Rater Manual]

25

Grade 8 Writing Example

Slide 25: GRADE 8 WRITING SAMPLE

This paper shows features of the advanced level of second language proficiency in writing. This paper about a girl’s experiences on a soccer team in Mexico shows an ability to narrate in some detail about a past event dealing with a familiar subject. The paper includes a variety of sentence structures used to link ideas. There is evidence of sustained control over basic grammar features, tenses, and sentence patterns and partial control of more complex constructions. While the student successfully uses the past tense to tell the story, she substitutes present tense for past tense when reporting dialogue (the teacher told me that I can be the leader, I told him that I want to be the leader, etc.), which is a more complex grammatical construction. This problem and the occasional presence of primary language features (in a soccer team, organize with all the girls, win the first play, I felt very proud of me) do not hinder understanding. [Example and Analysis taken from p. 55 of the 2005 TOP Rater Manual.]

26

GRADE 4 Writing Example (High Level)

Slide 26: This paper shows features of the advanced high level of secondlanguage proficiency in writing. The student uses appropriate content-based vocabulary (materials, battery, bulb, wires) to clearly explain a scientific process. The paper shows evidence of the ability to link ideas smoothly and control complex structures (If it doesn’t work that means that this battery must be dead or that the bulb is broken. Try a new battery and if it still doesn’t work try a new bulb). While there is some unnaturalness of expression (get one end of a wire touching the bulb), it does not detract from the overall effectiveness of the communication. The command of English demonstrated in this sample suggests an ability to address grade-appropriate writing tasks with minimal need for support in second language development. [Example and Analysis taken from p. 59 of the 2005 TOP Rater Manual.]

27

Other ideas for assessing writing:

Develop essay questions from text material covered.

Be directive, e.g., compare, critique, define, etc.

Allow students adequate time for a full response.

Slide 27: Besides the ideas presented in the Protocol Manual, here is an additional suggestion that integrates the content areas, as well.

28

Content Area Examples –Science

Science Example 1: (6th – 8th grade)Scientists tell us that there are hundreds of asteroids

that could collide with the earth anytime. One such asteroid did in fact collide with the earth millions of years ago during the time of the dinosaurs. Describe where you think this asteroid might have hit the earth and what kinds of physical changes took place afterwards.

Imagine that you have just heard on the news that an asteroid is going to hit the earth in a few days. Describe ways that scientists will try and keep the asteroid from hitting the earth. How will people react and what will you do?

Science Example 2: (9th-12th grade)You are given two test tubes, one labeled Protein Q, the

other labeled Protein Z. How could you tell if these tubes really contained different proteins? Outline the experimental procedure you would follow.

Slide 28: These writing prompts were created for use in a middle school and a high school in the Denver Public Schools System using the students’ content area textbooks.

[Note to facilitator: Read the sample writing prompt or have someone in the audience read them and discuss their potential for eliciting students’knowledge of the content area as well as conventional writing skills.]

29

Content Area Examples – Social StudiesSocial Studies Example 1: (6th – 8th grade)

Step back into time to the 1600’s. You are a woman with lots of intelligence, lots of ideas and lots to offer the world. Write an entry in your diary explaining your frustration because you are about to get married to a man you don’t even know. Your marriage has been arranged by your parents and you have no choice but to marry this man. He is quite a bit older than you. You are much more interested in using your intelligence and education for something. Write in your diary about what you want to do, and why your society probably won’t let you.

Social Studies Example 2: (9th – 12th grade)

An important function of the United Nations is to help settle disputes between nations. Describe how one dispute was handled successfully, point out how the settlement illustrates a general strength of the United Nations. Describe also how one dispute was handled unsuccessfully, pointing out how this illustrates a general weakness of the United Nations.

Slide 29:These writing prompts were also created for use in the same middle school and high school in the Denver Public Schools System using the students’content area textbooks.


30

Describe the characteristics of the party system in the U.S. illustrated in the cartoon below.

Social Studies Example 3(9th – 12th grade)

Slide 30: For students with limited reading proficiency, but who have goodcomprehension skills or schooling in their own country, a pictorial representation of concept makes an excellent writing prompt.

31

Content Area Examples –Language Arts

Language Arts Example 1: (6th – 8th grade)

The main character in this story is an eighth grade girl who overhears her teachers arguing about her. She is a straight A student and seems to do well in class. Describe what you think the teachers are discussing and what it might have to do with the green and gold “Scholarship Jacket”.

Language Arts Example 2: (9th – 12 grade)

Use a picture or a cartoon and ask students to create a story depicting what they see.

Slide 31: These are additional examples of writing prompts created for use in the middle school and high school in the Denver Public Schools System using the students’ content area textbooks.


32

USING PICTURES AS A STIMULUS

Slide 32: For students with limited reading proficiency, pictures are an excellent stimulus for written, as well as oral, production.

33

Content Area Examples – MathGrouping: Assign students to groups of 2

Materials:– 2 sheets of paper– 1 pencil– 2 patterns per group

Directions:Each of you has a pattern that you are going to ask your

partner to draw. BE SURE that your partner does not/cannot see the pattern. Sit back to back and take turns describing your particular pattern to your partner. Be sure to call each shape by its name, e.g., Draw a triangle in the upper right hand corner. Be specific and remember to use the terms we have learned in math class. Each of you has 10 minutes to describe your pattern to your partner.

Scoring: Use the TOP rubrics for listening and speaking

Slide 33: Content area subject matter can be used to assess all skill areas –listening, speaking, reading, and writing. Although in this particular instance, the receiver of the information is being asked to draw, it provides an excellent example of an activity that can be used to assess multiple modalities.


34

TOP – Texas Observation Protocols (LISTENING -- Grades K – 12)

Listening (K-12) – Assess formatively, using a rubric, during informal and formal academic tasks. A rubric is provided on p. 23 of the TOP Rater Manual found on the TEA website. Examples of activities provided on the TEA website include:

Reacting to oral presentationsResponding to text read aloudFollowing directionsCooperative group workInformal, social discourse with peersLarge-group and small group interactions in academic settingsOne-on-One InterviewsIndividual student conferences

Slide 34: In contrast to the protocols for reading and writing, there is only one listening protocol for students in Grades k – 12.

35


TOP Proficiency Level Descriptors Grade K-12 Listening

Slide 35: Put your heads together and review the descriptors for each of the ratings. Think about your own students and try to mentally apply this protocol to their performance.

[Note to Facilitator: Discuss with the audience the challenges in administering this instrument, e.g., structuring activities that demonstrate students’ listening proficiency; monitoring talk to determine what constitutes use of high frequency words, “with support” vs. “minimal support”; documenting the varying contexts (social and academic).

36

TOP – Texas Observation Protocols (SPEAKING -- Grades K – 12)

Speaking (K-12) -- Assess formatively, using a rubric, during informal and formal academic tasks. A rubric is provided on p. 24 of the TOP Rater Manual found on the TEA website. Examples of activities provided on the TEA website include:

Cooperative group workOral presentationsInformal, social discourse with peersLarge-group and small group interactions in academic settingsOne-on-One InterviewsClassroom discussionsArticulation of problem-solving strategies Individual Student Conferences

Slide 36: As with the Listening Protocol, there is only one Speaking Protocol to be used as students engage in both informal and formal academic tasks.

Following is an example of an informal task.Getting acquainted activity:

1 piece of chart paper4 studentsIn respective corner, write descriptors about yourself. – 4 brothers, like hotdogs, prefer action movies, etc.As a group, take 20 minutes to share what characteristics you have in common, don’t have in common.Use the TOP rubric for speaking to rate students.

Speaking Activity 2: Planning a dream party

In the year, 20__, you will be graduating from high school. You have been granted an unlimited amount of money to plan your dream Graduation Party. As a group, decide on the following:Where will your party be?How will your guests get to your party?When will your party be held?Who will you invite?What will you serve?What kind of entertainment will you have?

You have 35 minutes to plan your party.

SET PARAMETERS to keep students from going off on inappropriate tangents!!

Use the TOP Speaking Rubric to rate students

37

TOP PROFICIENCY Level Descriptions Grades K-12 Speaking

Slide 37: Please put your heads together and review the descriptors for each of the ratings. Imagine your students. How well does this protocol work for you?

[Note to facilitator: Engage the audience in a discussion of the importance of structuring classrooms where the teacher gets “off center stage” and students have multiple opportunities to engage in talk.Discuss challenges with participants, e.g., using a holistic instrument to accurately determine students’ proficiency in discreet areas such as pronunciation, vocabulary, morphology, and semantics (the linguistic structures discussed earlier in the workshop.]

38

FORM for Documenting Listening and Speaking Activities

Slide 38: Managing the collection of the data can be quite cumbersome. This is an example of a form that can be easily used as students engage in listening and speaking activities, while the teacher “clipboard cruises” documenting students’proficiency based on the rubrics shared earlier. Performance on six different activities can be captured on this one form. The overall proficiency can be determined by averaging the individual’s performance across the activities. Every effort should be made to include three social events and three academic events so as not to skew the average. Or, another option is to generate a social AND an academic oral language proficiency score which would require modification of the form by inserting an additional Overall Proficiency column and labeling them accordingly – Overall Social Proficiency and Overall Academic Proficiency.

[Note to facilitator: Review the top portion and the importance of documenting this information.

39

FORM for Documenting Reading and Writing Activities

Slide 39: Managing the collection of the data can be quite cumbersome to document reading and writing, as well. This is an example of a form that can be easily used as students engage in reading and writing activities, while the teacher “clipboard cruises” documenting students’ proficiency based on the rubrics shared earlier. Performance on six different activities can be captured on this one form.

Be sure to note pertinent information at the top includingObserver’s NameGrade/SubjectCampusDistrict

40

FORM for Capturing Proficiency in Communications Skills

Slide 40: This form captures students’ proficiency performance for all four communication skills during three separate activities which is a state requirement.

Again, it’s important to note pertinent information at the top includingObserver’s NameGrade/SubjectCampusDistrict

41

More To Assessing Language Proficiency Than Meets the Eye!

• HLS• OLPT• NRT

versus• Reading Rubrics• Writing Rubrics• Speaking Rubrics• Listening Rubrics• Reading Proficiency Test

Slide 41: Using the glacier metaphor again, assessing English Language Proficiency is far more complex than has been historically practiced, as it should be, given the complexity of language. We need to be more comprehensive and use complementary practices as those being required under the NCLB Act.

42

One-shot Snapshots May not Capture True Capabilities

Slide 42: Borrowing from “Far Side”, if we resort to one-shot snapshots, we may not be capturing the students’ TRUE capabilities.

43

Recommending a Student for Exit?Ask yourself . . .

1. Does the oral language proficiency test used by your school district measure, not only the kind of language needed in your class, but also that which will be needed at the next higher grade level?

2. Has the student sufficiently mastered the basic language skills that will prepare him/her to deal successfully with the shifting emphasis of language skills at the next level of schooling?

3. Are you familiar with the textbooks that the student will be expected to use during the next school year? Are you sure s/he can handle both the language and content demands of these books with a minimum of help?

4. Have you challenged the student in terms of vocabulary development, a variety of thinking and problem solving skills, and on a wide range of topics?

Slide 43: The multiple data will not only help you answer these critical questions more aptly, but actually serve as an advocate for the student and his/her educational success and well-being.

[Note to Facilitator: As a member of the audience to read each of the questions while the rest follow silently along as they think about their own LEP population.]

44

Recommending a Student for Exit? Ask yourself . . . (Continued)

5. How is the student’s reading rate in English? How is his/her comprehension, not only of materials in the reading text, but in the content-area materials as well?

6. What are the student’s scores in language arts and reading on the most recently-administered achievement test? Are his/her scores at least as high as the average student in the school (i.e., the student’s scores compare favorably with the school or district expectations?

7. Has the student mastered the district standards in math, science, and social studies? Is the progress that s/he has made toward achieving the content standards what you would expect compared to other students in the same grade?

8. How high is the student’s anxiety level in your class? Is school stressful for the student or is s/he self confident and able to handle frustration or failure?

Adapted from B. Mace-Matluck, Southwest Educational Development Laboratory (SEDL)

Slide 44: No Notes

45

Getting Started

1. Identify a cross section of grade representatives and select a “chief worrier”

2. Access all of the available resources from the TEA and ISLA websites

3. Determine the assessment activities to be used campus wide

4. Organize the assessments in a notebook to be disseminated to every classroom teacher

5. Schedule “reliability training” to ensure that everyone is utilizing the rubrics in similar fashion – use student work samples including voice recordings for this training

6. Develop a district-wide consistent record keeping system.

Slide 45: Using a “David Letterman” approach, it’s important to identify the top 6 steps necessary for adequately and appropriately developing your English language assessment skills. [Proceed by reading each of the items on the slide.]

46

Reasons Why the English Language is Hard to Learn:

1. The bandage was wound around the wound. 2. The farm was used to produce produce. 3. The dump was so full that it had to refuse more

refuse. 4. We must polish the Polish furniture. 5. He could lead if he would get the lead out. 6. The soldier decided to desert his dessert in the

desert. 7. Since there is no time like the present, he

thought it was time to present the present. 8. A bass was painted on the head of the bass

drum. 9. When shot at, the dove dove into the bushes. 10. I did not object to the object.

Slide 46: [Note to the Facilitator: To help create empathy among your participants for the challenges faced by their students, share the following circulated on e-mail (Author Unknown).]

47

Reasons Why the English Language is Hard to Learn (Continued)

11. The insurance was invalid for the invalid. 12. There was a row among the oarsmen about

how to row. 13. They were too close to the door to close it. 14. The buck does funny things when the does are

present. 15. A seamstress and a sewer fell down into a

sewer line. 16. To help with planting, the farmer taught his

sow to sow. 17. The wind was too strong to wind the sail. 18. After a number of injections, my jaw got

number. 19. Upon seeing the tear in the painting, I shed a

tear. 20. I had to subject the subject to a series of tests. 21. How can I intimate this to my most intimate

friend?

Slide 47: No Notes

48


Let's face it - English is a crazy language. There is no egg in eggplant nor ham in hamburger; neither apple nor pine in pineapple. English muffins weren't invented in England or French fries in France. Sweetmeats are candies while sweetbreads, which aren't sweet, are meat. We take English for granted. But if we explore its paradoxes, we find that quicksand can work slowly, boxing rings are square and a guinea pig is neither from Guinea nor is it a pig. And why is it that writers write but fingers don't fing, grocers don't groce and hammers don't ham?

If the plural of tooth is teeth, why isn't the plural of booth beeth? One goose, 2 geese. So one moose, 2 meese? One index, 2 indices?

Doesn't it seem crazy that you can make amends but not one amend, that you comb through annals of history but not a single annal?

If you have a bunch of odds and ends and get rid of all but one of them, what do you call it? If teachers taught, why didn't preachers praught?

Slide 48: No Notes

49


If a vegetarian eats vegetables, what does a humanitarian eat?

In what language do people recite at a play and play at a recital? Ship by truck and send cargo by ship? Have noses that run and feet that smell? How can a slim chance and a fat chance be the same, while a wise man and a wise guy are opposites?

Have you noticed that we talk about certain things only when they are absent? Have you ever seen a horseful carriage or a strapful gown? Met a sung hero or experienced requited love? Have you ever run into someone who was combobulated, gruntled, ruly or peccable? And where are all those people who are spring chickens or who would actually hurt a fly?

You have to marvel at the unique lunacy of a language in which your house can burn up as it burns down, in which you fill in a form by filling it out, and in which an alarm goes off by going on.

English was invented by people, not computers, and it reflects the creativity of the human race (which, of course, isn't a race at all). That is why, when the stars are out, they are visible, but when the lights are out, they are invisible. And why, when I wind up my watch, I start it, but when I wind up this essay, I end it?

Slide 49: No Notes

50

Sample TExES Item:

A fifth-grade student arrived from his home country, El Salvador, last year with no prior formal education. He is now in his second year in a Texas school and is receiving bilingual and ESL services. He is still at the beginning stages of Spanish literacy development, English language development, and academic development.

What would be the state policy with regard to the assessment of academic skills in this student’s case?

a. The language proficiency assessment committee (LPAC) may recommend that the assessment of English language skills be waived; however, an assessment of academic skills must be administered in either English or Spanish.

b. Since he is enrolled in the bilingual program, assessment of academic skills must be administered in either English or Spanish.

c. The LPAC may determine that neither English nor Spanish proficiency tests would be an appropriate measure for school accountability.

d. Since he is now in his second year of enrollment in a U.s. school, the school must administer an assessment of academic skills in English.

Slide 50: And, now, to get a sense of how you will be assessed on the TExES, try your hand at this sample item.

“C” is the correct response.

51

Sample TExES Item:

A middle school ESL teacher is working with a group of ESL students whose English-language abilities vary. Which of the following would be the most appropriate strategy for evaluating the progress of students who are at different proficiency levels in English.

a. Using multiple measures, such as observations, test scores, and samples of daily work

b. Selecting language achievement tests that have been normed on a similar student population.

c. Establishing a grading curve and distributing students’test results along the curve

d. Assessing students only in those areas of English in which they have achieved competence.

Slide 51: A is the correct response.

52

RESOURCES

Canales, J. (Fall, 1988). “Assessment of language proficiency: Informing policy and practice.” Position paper prepared for Southwest Educational Development Laboratory (SEDL) to assist state education agencies in defining language assessment policy and practices.

Canales, J. (1993) Innovative assessment in traditional settings. The power of two languages: Literacy & biliteracy for Spanish speaking students. New York: MacMillan-McGraw Hill Publishing Company, pp. 132-142.

Canales, J. (1994) Linking language assessment to classroom practices. TABE Compendium. San Antonio, Texas: Texas Association for Bilingual Education, Fall, pp. 59-73.

Peregoy, S.F. & Boyle, O.B. (1997). Reading, writing, and learning in ESL: A resource book for K-8 teachers (3rd ed.). White Plains, New York: Longman Publishing Group.

Websites:

http://www.sbec.state.tx.us

http://www.tea.state.tx.us

Slide 52: No Notes

Documents

Assessing English Language Proficiency A Training Moduleell.tamucc.edu/files/module_10_script.pdf · Assessing English Language Proficiency A Training Module ... For instance a non-native