DLIFLC 7-9 FEB 01 Diagnostic Assessment Thomas S. Parry Directorate of Continuing Education Defense Language Institute BILC Professional Seminar 2005 Sofia,

DLIFLC 7-9 FEB 01

Diagnostic AssessmentThomas S. Parry

Directorate of Continuing Education

Defense Language Institute

BILC Professional Seminar 2005

Sofia, Bulgaria

DLIFLC 7-9 FEB 01

Language Learning

• Language is among the most complex of human behaviors to acquire and maintain

• Learning is among the least understood of human endeavors

• Language + Learning = no small task!

DLIFLC 7-9 FEB 01

Assumptions

• Acquiring working proficiency in a foreign language requires significant investment of resources

• Maintaining that proficiency is an ongoing challenge

• Language learners need specific, periodic feedback and meaningful recommendations on skill development/improvement in order to be successful

DLIFLC 7-9 FEB 01

Diagnostic AssessmentA Language Learner’s Roadmap

• Language learners obtain meaningful feedback and recommendations on specific skill development based on ILR Scale

• Language Learners will, with time and practice, advance to the next higher level of language proficiency in each skill area if recommendations are followed

DLIFLC 7-9 FEB 01

ILR Proficiency Level Summary

5

LEVEL FUNCTION/TASKS CONTEXT/TOPICS ACCURACY

4

3

2

1

0

All expected of an educated NS

All subjects Accepted as an educated NS

Tailor language, counsel, motivate, persuade, negotiate

Wide range of professional needs

Extensive, precise, and appropriate

Support opinions, hypothesize, explain, deal with unfamiliar

topics

Practical, abstract, special interests

Narrate, describe, give directions

Concrete, real-world, factual

Intelligible even if not used to dealing

with non-NS

Errors never interfere with

communication & rarely disturb

Q & A, create with the language Everyday survivalIntelligible with

effort or practice

Memorized Random Unintelligible

DLIFLC 7-9 FEB 01

Diagnostic AssessmentDesign Considerations

• Formative, not summative• Face-to-face, not paper & pencil • Brings together teaching, testing and

curriculum development• Validity:

– Instrument, text selection– Questions in English– Quality authentic materials

DLIFLC 7-9 FEB 01

Diagnostic AssessmentDesign Considerations

• Reliability:– Selection of personnel– Training– Standards, governing principles

• Practicality:– Duration– Means of delivery

DLIFLC 7-9 FEB 01

Diagnostic AssessmentGeneral Description

• A 1.5 to 2-hour, individually-administered face-to-face interview with two trained specialists

• Four skills: speaking, listening, reading and writing– Speaking is patterned after the OPI– Listening and reading based on carefully

selected and graded authentic materials

DLIFLC 7-9 FEB 01

Content of Interview

• Speaking, reading, listening and written essay portions include: – A variety of topics– Specific target country-related issues

Content is regularly updated based

on current world events

DLIFLC 7-9 FEB 01

Results of the Interview

• Diagnostic profile – Highlights patterns of linguistic

strength and weakness

• Learning plan– Outlines learning strategies,

resources and recommendations

DLIFLC 7-9 FEB 01

The Diagnostic AssessmentProcess

Text Selection

Grading of Texts

Diagnostic Assessment

Specialist Training

Learner Styles and Strategies

Assessment

The Four-Skill Interview

Speaking

Listening

Reading

WritingThe Learner Profile

The Individual Learning Plan

DLIFLC 7-9 FEB 01


Text Selection

Grading of Texts


Specialist Training


Assessment


Speaking

Listening

Reading



DLIFLC 7-9 FEB 01

Text Selection• Content of authentic materials used for

reading and listening skills includes– Target country-related issues– Variety of topic areas– Culture-specific current events– Societal issues

• Differences between texts selected for reading and those selected for listening

• Source considerations for authentic materials

DLIFLC 7-9 FEB 01

Grading of Texts

• Authentic materials are graded (assigned an ILR level) as follows:– Determine text mode (author intent)– Separate core from periphery– Establish what the text is and what

a linguist can do with the text – Take into account language-specific

factors (i.e. structure, accent, applications)

DLIFLC 7-9 FEB 01


Text Selection

Grading of Texts


Specialist Training


Assessment


Speaking

Listening

Reading



DLIFLC 7-9 FEB 01

DA Specialist Training

• Prerequisite knowledge, skills and abilities for DA specialists include:– Native speaker of the target language– High-level (level 3) English

proficiency in all skill modalities– Through grounding in ILR skill-level

descriptions for all skill modalities– Effective interpersonal skills– Background in general computer

applications

DLIFLC 7-9 FEB 01

DA Specialist Training• Specific DA specialist training includes:

– Applying text typology theory in selecting/grading authentic materials

– Eliciting oral dialogue and language samples– Structuring the Diagnostic Assessment

interview– Creating collection of meaningful learning

activities to incorporate into learning plans– Creating diagnostic profiles– Writing comprehensive learning plans– Utilizing specific software applications for

selecting source materials and recording speech

DLIFLC 7-9 FEB 01


Text Selection

Grading of Texts


Specialist Training


Assessment


Speaking

Listening

Reading



DLIFLC 7-9 FEB 01

Learning Styles and Strategies Assessment

• The Student Orientation Scale (SOS) is used to assess individual student learning style preferences and establish a psychological profile in order to:– Appropriately structure the DA interview– Recommend activities that align with

dominant student learning preferences– Recommend strategies that will aid in

developing recessive learner preferences

DLIFLC 7-9 FEB 01


Text Selection

Grading of Texts


Specialist Training


Assessment


Speaking

Listening

Reading



DLIFLC 7-9 FEB 01

Diagnostic InterviewPhases

• Warm-up/introductions• Speaking (descriptive preludes)• Reading (at least 7 texts at each

level)• Listening (at least 7 passages at

each level)• Written Language Sample • Wind Down/conclusion

DLIFLC 7-9 FEB 01


Text Selection

Grading of Texts


Specialist Training


Assessment


Speaking

Listening

Reading



DLIFLC 7-9 FEB 01

The Learner Profile• The Learner Profile highlights patterns

of learner strength and weakness through three primary means:– General content task list for all four skills– In-depth analysis of all language factors

for each skill• Discourse Competence• Structural Accuracy• Lexical Precision • Socio-cultural Competence

– Analysis of language structure

DLIFLC 7-9 FEB 01


Text Selection

Grading of Texts


Specialist Training


Assessment


Speaking

Listening

Reading



DLIFLC 7-9 FEB 01


• The Individual Learning Plan consists of:– Thorough analysis of learner strengths and

weaknesses in the language– Descriptions of areas needing additional work– General recommendations on how to

overcome the weaknesses– Specific recommendations (tasks) on how to

advance to the next higher level in each skill– Recommendations/considerations of optimal

venue for language study– Recommendations of resources for further

study

DLIFLC 7-9 FEB 01

Diagnostic Assessment• Applications

– Intermediate and advanced language instruction

– Second or third-semester of basic language instruction

– Immersion programs– Distance learning (VTT and online learning)

• Future Directions– Computer-mediated delivery– Inform the foreign language learning/teaching

process including curriculum development, pedagogy

DLIFLC 7-9 FEB 01

Questions?

Discussion?

Documents

DLIFLC 7-9 FEB 01 Diagnostic Assessment Thomas S. Parry Directorate of Continuing Education Defense Language Institute BILC Professional Seminar 2005 Sofia,