Upload
conrad-garrett
View
228
Download
0
Tags:
Embed Size (px)
Citation preview
Bureau for International Language Coordination
Julie J. DubeauBILC Secretary
Istanbul, TurkeyMay 24, 2010
General information
LNA report
BAT Report
Outline of Presentation
What does BILC do?
Annual Conference in Spring June 2009 hosted by Italy - “Bridging the Gap:
Language Requirements vs. Language Reality”
May 2010 hosted by Turkey - “Mapping The Road: Success in Language Training”
Professional Seminar in Fall October 2009 hosted by Denmark –
"The 21st Century Classroom: Keeping up with the Times!“
October 2010 hosted by Bulgaria – “Aligning Training and Testing in Support of Interoperability”
Ratified by the nations and promulgated by NSA in February 2009
English and French versions can be downloaded from the BILC website www.bilc.forces.gc.ca
Steering Committee will be discussing a proposal for admin change during conference
Responsible for STANAG 6001
What else does BILC do?
Language Training Assessments Assistance to National Testing Programmes Language Testing Seminars:
LTS & ALTS
Special Projects:Benchmark Advisory Test (BAT)Language Needs Analysis (LNA)
General Information ACT is tasked by IMS to investigate the
feasibility of introducing a computer based learning tool for use by nations to familiarize with NATO terminology…
BILC is investigating requirement through JSSG
“ACT should develop ADL solutions that will complement nations’ efforts in language training, understanding of different local government structures and understanding familial, clan and tribal cultures”.
ACT has requested BILC’s assistance
Please contact the BILC Secretariat if you are interested in providing an ADL product
Complementary ADL SolutionsIMS Report Dated November 26 2009, Bratislava Follow-up Tasking on
Counter Insurgency (COIN)
November 2008 Chairmen’s Meeting of the NTG in Brussels
The aim of this study was to show whether language requirements appeared to be set at appropriate levels to enable military personnel to perform their duties adequately in the NATO OPS context, in this case ISAF.
LANGUAGE NEEDS ANALYSIS OFNATO CRISIS ESTABLISHMENT (CE) POSTS
To broadly identify whether the mandatory SLP requested for the post was:
At Level ~ A: meaning that from the tasks described in the JD, the profile appeared to be adequate;
High ~ H: meaning that the SLP requested for the post appeared to be higher than the functions defined in the job description; or
Low ~ L: meaning that the SLP requested for the post appeared to be lower than the functions defined in the job description.
LANGUAGE NEEDS ANALYSIS OFNATO CRISIS ESTABLISHMENT (CE) POSTS
Summary of Results
Ratings Total = 609 CE Post JDs
A - At Level 355 (58.2%)
H - High 203 (33.3%)
L - Low 41 (6.7%)
H/L - Mixed 10 (1.6%)
Job Descriptions requiring SLPs of Level 1: 30 JDs required SLPs of 1111 & all were High.
Job Descriptions requiring mixed SLPs of level 1 & 2:
4 required SLPs of 2211 & all were at level. 2 required SLPs of 2221 & both were at level.
Job Descriptions requiring SLPs of level 2: 61 JDs required SLPs of 2222. 15/61 (26.2%) were at level. 43/61 (70.4%) were High.
Results: Levels 1 & 2
Job Descriptions requiring mixed SLPs of Levels 2 & 3: 156 (25.6%) JDs required mixed SLPs of Levels 2 and 3. Of these 156, 66 (42.3%) were considered as having
SLPs that were at level
Further Breakdown: 1 required SLP 3221 (High/Low depending on skill) 9 required SLP 3222 (7/9 High) 1 required SLP 3223 (High) 4 required SLP 3232 (3High/1Low) 118 required SLP 3322 (47 at level (39.8%), 56 High
(47.4%) & 3 Low (2.5%)) 23 required SLP 3332 (17 at level, 6 Low note that these
6 also req Dari)
Results: Levels 2 & 3
Results: Level 3
Job Descriptions requiring SLPs of Level 3:
301 (49.4%) required SLP 3333 240/301 (79.9%) were considered at level 44/301 (14.6%) were considered High 13 were considered as Low SLPs for functions, (all
req expert language knowledge with 3s in Dari and Pashto)
Job Descriptions requiring mixed SLPs of level 3 & 4:
43 profiles of 4343, from which 22/43 were at level (51.1%), and 21(48.8%) were not scored at level (some H some L)
1 profile of 4344 (Low), 1 profile of 4443 (High)
Results: Levels 3 & 4
Results: Level 4 Job Descriptions requiring SLPs of level 4:
8 profiles of 4444 (6/8 were considered at level, with comments referring to the functions as requiring expert language ability, diplomatic or sophisticated language use, 2 were H)
Out of the 53 JDs requiring some skills at level 4, 10 (18.8%) also req languages other than Eng such as French, Dari, Pashto, etc.
Out of these 10, 6 (60%) were at level.
Recommendations NATO CE Post JD SLPs should be reviewed SLPs should be based on an analysis of the
tasks performed by incumbents in relation to STANAG 6001 Ed 3.
It is strongly recommended that the next analysis of requirements be done with the assistance of BILC language experts.
CE posts master sheet should include the SLPs requested for the posts.
IMS-Tasking on Language Proficiency and Education on NATO Standards & Terminology. 12 February 2010
Analysis of CE/PE posts Setting realistic requirements Review of linguistic requirement for Officers for operationally
deployed duties‘ has been initiated
Benchmarking
What’s next for interoperability?
Benchmark Advisory Test (BAT)- PURPOSE
To provide an external measure against which nations can compare their national STANAG test results
To promote relative parity of scale interpretation and application across national testing programs
To standardize what is tested and how it is tested
Benchmark Advisory Test (BAT)
Allocation to 11 Nations (200 ‘free’ tests) Tests administered by LTI, the ACTFL
Testing Office via the Internet and Telephone
January 2010 – End of Benchmarking Process
Positive Feedback from Nations
Benchmark Advisory Test (BAT) Results
Listening Speaking Reading Writing
Black (11) 64% (18) 72% (10) 60% (11) 55%
White (18) 61% (18) 56% (18) 94% (18) 39%
Red (18) 89% (18) 83% (18) 83% (18) 50%
Blue (20) 85% (19) 47% (20) 55% (20) 60%
Teal (8) 0% -- -- (8) 75% (8) 38%
Maroon (16) 69% (15) 47% (14) 64% (16) 44%
Pink (18) 67% (18) 50% (18) 78% (18) 67%
Purple (12) 8% -- -- (13) 54% (13) 62%
Orange (13) 77% (13) 46% (13) 54% (13) 62%
Yellow (17) 24% (18) 0% (18) 33% (18) 0%
GreenNation did not release national test scores, but indicated that, on average, they were higher than those attained on the BAT.
Test purpose Testing method Use of plus ratings Alignment of author purpose, text type, and
reader/listener task Inadequate tester/rater norming (productive skills) Inconsistencies in the interpretation of STANAG
6001 Cut-off score setting, etc.
FACTORS CONTRIBUTING TO NON-ALIGNMENT
BAT – Way Ahead
Project successful Demand for administrations will dictate future
development needs and modes
BILC SC to formulate recommendations
Nations who require BILC support (post-BAT) can request it
QUESTIONS?
Enjoy the Conference!