1

CooperativeLanguage Training Programme Assessments

“Mapping the Road: Success in Language Training”

Keith L. WertAssociate BILC Secretary for Programme Assessment

Director, Partner Language Training Center EuropeGeorge C. Marshall European Center for Security Studieswertk@marshallcenter.org

2

CooperativeLanguage Training Programme Assessments

• History:– 2000:

• Requested by a NATO Assistant Secretary General to Chair, Nato Training Group who wrote BILC secretariat

– 2001 • Conducted first assessment

– 2001-2010• Before Accession:

– Slovakia , Latvia, Slovenia, Estonia,• After Accession:

– Czech Rep ., Bulgaria , Romania• PfP and IPAP countries

– Macedonia, Georgia , Serbia

– Countries that have asked for follow up visits: Czech Republic, Bulgaria, Slovakia, Georgia, Macedonia and Serbia.

3

Language Training Programme Assessments

• Team members from: – Slovenia, Sweden, Germany,

Canada, UK, and US.

• Level of Interest:– Latvia: Deputy State Secretary – Slovakia: Director of Military

Education– Macedonia: Chief of Staff– Bulgaria: Deputy Chief of Staff– Georgia: 1st Deputy Minister

of Defense– Romania: Head of MoD

Human Resources– Serbia: The J-7 and MoD

Personnel Sector

Serbia: May 2009

4

Items of Interest

Development of a language policy Integrating language policy into military personnel policies Development of a language training structure that meets the objectives of the language

policy

Establishing effective and efficient use of language training resources - Appropriate emphasis on and balance between intensive and non intensive programs - Ensuring resources are allocated in a transparent and ‘objective’ manner - Effective and efficient language testing programs to NATO standards - Transparent procedures for faculty professional development - Harmonizing bilateral support for language training - Development of a modern military lexicon, based upon agreed NATO nomenclature - Development of Syllabi at STANAG 6001 Level 3

Pre-Visit Process

• Provide the potential scope of what you are going to be looking over to the appropriate coordinating office so that everyone knows what the objectives are

• Not a check list. You never really know what aspects of the assessment will require the most attention

• We provide “Outlines” with points for potential discussion and observation. – Policies– Schools– Testing

Pre-Visit Process (Outlines)

What is the focus?

• We are generally not:– Components• Book Orders• Teacher training• Mgt courses• Technology

– Labs, IMI etc• Doing some legwork for the bilateral donors:

– Book orders– Course recommendations

• We are:– Processes• Language and Testing Policies • Integration with military personnel policies• Language Training Management• Resource allocation• Faculty Development plans

Visit Process

• Extensive orientation briefings by MoD/General Staff– Policy and organization explained

• Visits to schools• Classroom observations• Discussions with management• Discussions with faculty and students• Meetings with bilateral language assistance responsibles, e.g.

British Council, Offices of Defense Cooperation. (“Donor countries”)

Analytical Process: Overarching Objective

• To review how the language policy fits with military personnel policies.

• To see if the language training structure meets the policy objectives.

• To see if the structure can produce the required numbers of graduates at the required proficiency levels in a somewhat predictable manner.

Areas of Interest

• Language Policy (Personnel policies)• Language resources allocation• Language laboratories and self-access centers• Testing policies and processes• Syllabi standardization or how long does a student take to

reach Level 1,2,3? (and now4!!!!!)• Teachers contra management• Management contra teachers• Military language instruction• Professional Development (transparency thereof)• Intensive vs. non intensive language programs

Language Policy

• A language policy must be an inseparable component of personnel policy. If it is not, that is the first sign that there is a problem.

• Does the language policy exist in a vacuum?

– Were the schools involved in making the policy?– Is the policy realistic? – Do the policy makers ever check to see if the policy is

successful?– Are personnel assigned to language training for a reason?

Language and Personnel Policy

MOD/General/Joint Staff

Language Schoolhouse Management

Are they talking to each other?

Language Policy

• Simple reality test:

– Visit several classrooms in different locations and ask a range of teachers and students the same question:

“Why are you in this classroom?”

Language Resources Allocation

• Why are some locations well resourced and others not?

– Is a language lab at a military unit as important as one at a school house with a full time intensive language program?

– Who is in charge of language training resources?– Is the allocation process “transparent”?

Testing Policies and Processes

• Is there a STANAG 6001 Testing Frenzy?– Overused STANAG tests?– Simpler ways to determine Level 1?– Is testing focus on particular assignments? (Personnel

policy loop)

– Don’t test more than necessary

Language Labs and Self-Access Centers

• They have a life of their own: I know this because they keep reproducing themselves everywhere we go– Great in concept, difficult in execution– Tend to draw resources away from more fundamental

needs: like books

Syllabi Standardization

• There should be evidence of language training predictability, especially at the lower levels

– If no one knows how long it takes the normal language student to reach Levels 1 and 2, there is a problem

– If it is known how long it takes and authorities shorten course lengths anyway, there is a problem

– Course lengths determined arbitrarily– Stability and predictability at lower levels is important

“ If you can’t measure it, you can’t manage it.”Dr. Ray Clifford,

SwedenOctober, 2001

Syllabi Standardization

• Certificates from different courses do not tell you anything– The students passed the “test” and received credit• No comparability across classrooms• No comparability across institutions in the same country

Teachers contra management

• Highly educated and professional teachers don’t always grasp the larger scale training management issues and confuse higher level management with the wrong input:– “These books are boring, that is why the students are not

succeeding!”– “The test is bad, not a truly professional test.” (I don’t like

this test format.)

Management contra teachers

• Senior management has to pay attention to the real results teachers and school houses produce– If it is working, leave it alone and let the language

professionals improve it incrementally– Stability is important– Having the power to do something is not synonymous with

knowing what to do

Military language

• Primary warning signs are:– Teachers who do not feel it is in their profession to teach it– Courses that introduce highly technical language at low

levels of language ability– Leaders who think that just the language and terms of the

specialty need to be learned– Intensive basic courses that rely on military language as the

primary component

Teacher Professional Development

• There are limited numbers of opportunities for teachers and these opportunities must be handled in a rational and transparent manner

• Professional institutions everywhere have application procedures and published standards for awarding teachers with research grants and development opportunities

• There should be a well-thought out process controlled by the country, not the bilateral (donor) providers

Intensive vs. Non intensivePrograms

• “Military language training” has unique attributes:– Large numbers of students/High proficiencies to attain

• Non-intensive basic courses rarely succeed

• Putting language training resources at local military bases looks good on PowerPoint – The students are frequently pulled from class

• The issue is diffusion of resources: what percentage is spread thinly to poor effect

Line Management

Students not grouped /regrouped by abilityClasses too largeListening materials not used enoughTeachers not changing classesStudent attrition policiesWorking conditions in classrooms poorEvaluations of teacher performance

Potential Positive Outcomes

• Country takes fundamental look at language system

• Country uses “external experts” to help make hard internal decisions

Questions?