English Language Proficiency Test - Aerosolutions · The English Language Proficiency Test, or ELP Test, has been designed by Aerosolutions to validate the English proficiency level

English Language Proficiency Test

ELP TEST Operations Manual

Released by : Peter Goethals

Date of issue : 13/02/2012

Checked by : Steven Wille

Version : V3.0

This Manual is to be used only for the purpose of individual, private study and may not be reproduced in any form or medium, copied, stored in a retrieval system, lent, hired, rented, transmitted, or adapted in whole or in part without the prior written consent of Aerosolutions. Copyright in all materials bound within these covers or attached hereto, excluding that material which is used with the permission of third parties, belongs exclusively to Aerosolutions. Whilst every effort has been made to ensure the accuracy of the information contained within this book, Aerosolutions does not give any warrants as to its accuracy or otherwise. This Operations Manual has been written and published as a reference work to assist Airlines and Training Centers in using the ELP Test and the ELP Test Data Management System (EDMS). Aerosolutions excludes all liability for any loss or damage incurred or suffered as a result of any reliance on all or part of this Manual except for any liability which may not legally be excluded. For further information on products and services from Aerosolutions, visit our website at www.aerosolutions.be This Operations Manual may be amended from time to time. Such amendment service will be provided by Aerosolutions. Any comments about the content or requests for clarification should be directed to the Head of Training. Version 3.0 - February 2012

© Aerosolutions - Confidential in Use Page 3

List of Effective Pages

Page Version Page Version

Cover Version 3 33 Version 3

Verso Version 3 34 Version 3

3 Version 3 35 Version 3





















24 Version 3

25 Version 3

26 Version 3

27 Version 3

28 Version 3

29 Version 3

30 Version 3

31 Version 3

32 Version 3


Amendment Record

Version number Revision Date Chapters affected

Version 1 (original)

Version 2 January 2010 ALL

Version 3 February 2012 ALL


Table of Contents

ELP Test Delivery ................................................................................. 7 Introduction ................................................................................................ 7 ELP Test Platform ...................................................................................... 8

System architecture .................................................................................. 8 Flash Player ............................................................................................ 11 Adobe Flash software and Flash Media Server ...................................... 12

ELP Test Data Management System ....................................................... 13 EDMS Administration concepts .............................................................. 13 EDMS System Administrator .................................................................. 13 EDMS Testing Center Administrator ....................................................... 14 Rater ....................................................................................................... 14 Test Taker ............................................................................................... 15 Accessing the ELP Data Management System ...................................... 15 User Management .................................................................................. 18 Rating the ELP Test ................................................................................ 22 View Test results .................................................................................... 25 Change settings ...................................................................................... 27 Logging off or leaving your computer ...................................................... 28 Launching the ELP Test ......................................................................... 29

Test-delivery procedures ......................................................................... 32 Introduction ............................................................................................. 32 EDMS System requirements ................................................................... 32 Configuring video and audio devices ...................................................... 33 Test Administration Supervisor and Proctors .......................................... 35 Physical Supervision ............................................................................... 36 Hardware & software .............................................................................. 36 ELP Practice Test ................................................................................... 36 Testing space ......................................................................................... 36 Personal Belongings in the Testing Room .............................................. 37 Candidate Verification ............................................................................. 38 Rules of Test Conduct ............................................................................ 38 Unethical behaviour ................................................................................ 39 Answering Questions Regarding Test Items ........................................... 39 Candidate Test Appeals Process ............................................................ 39

ELP Test procedure – Check list ............................................................. 40 Test Security ............................................................................................. 41

Test materials ......................................................................................... 41 Test system ............................................................................................ 41

Ongoing support for external Raters ...................................................... 43 Ethical issues ........................................................................................... 45

International Language Testing Association - Code of Ethics ................. 45

Glossary .................................................................................................. 48

Appendixes ............................................................................................ 48

Version 3.0 ELP Test Operations Manual


Intentionally left blank



ELP Test Delivery

Introduction The English Language Proficiency Test, or ELP Test, has been designed by Aerosolutions to validate the English proficiency level of Pilots and Air Traffic Controllers in accordance with the International Civil Aviation Organization language proficiency requirements. The ELP Test assesses speaking and listening proficiency in both ICAO radio phraseology and plain English and evaluates language proficiency in the six proficiency criteria (for the six ICAO levels). The 60-minute test is computer-delivered to allow for easy and uniform administration and for maximum authenticity. There are two test sections: The listening section (Part 1) is computer adaptive and measures the examinee's* ability to understand English in both routine and non-routine situations. Candidates are required to recognize and understand communications in both plain English and phraseology. This section consists of traditional multiple-choice questions with four answer choices. The speaking section (Part 2) assesses how effectively candidates can communicate in English and measures the examinee's ability to respond to different aviation related situations and tasks. Candidates are required to listen and respond to questions and/or to give required information. All responses are recorded for later rating. Both parts of the test are web-delivered through an Internet-based delivery platform and can be administered at multiple test sites (Testing Centers) anywhere in the world. To achieve a license endorsement it is necessary to obtain a pass in both Part 1 and Part 2. The lowest level achieved in either Part 1 or Part 2 is the final assessment. * Examinee (test taker, candidate)



ELP Test Platform

System architecture The ELP online language testing system uses the following hardware:

Client computers Several network servers (Application Server, Flash Media Server …) A Network which connects the Servers and the Client computers.

In general terms, all Users (Test Takers, Raters, Testing Center Administrators, System Administrators and Aviation Authorities) interact with the online system through a web browser such as Microsoft Internet Explorer, Mozilla Firefox, Apple Safari … (see Figure 1) At the core of the entire online system is the Application Server. It accepts user data, processes them and generates feedback to be sent to the user through the Web Server. The Web Server is the front-end of the system and coordinates data communication between the back-end servers (Application Server & Flash Media Server) and the user browser (Figure 2).

SystemAdministrator

Test Takers Testing CenterAdministrator

AIRLINE

Figure 1 – Flash Clients



A set of scripts and programs installed on the Application Server allows the Test Takers, Testing Center Administrator, Raters, System Administrator and Aviation Authorities to interact with the online system. One of these software programs is the ELP Test Data Management System (EDMS), which was specifically designed and developed by Aerosolutions to efficiently and effectively control, distribute, manage English Language Proficiency Tests (ELP Tests). The EDMS also facilitates ELP Test Rating. The ELP Test Data Management System software application is based on the same technology as our full feature web-based ICARUS Learning Management System (LMS), which is used by many aviation training organizations worldwide to organize and manage distance learning courses and online assessments. The LMS provides complete solutions for managing and tracking all types of training - from e-Learning courses to traditional classroom training and self-paced study programs. The EDMS includes extensive testing and assessment tools, reporting, collaboration and communication tools, management and quality control capabilities - all in one integrated application that is highly scalable to precisely meet the needs of the Testing Centers, Raters and System Administrators. The system also allows Aviation Authorities to monitor testing activities.

ClientInterface

ApplicationServer

Flash MediaServer

Database File System

HTTP

EDMS, LINUX LINUX

Flash Player, browser

RTMP

MySQLPHP

TestTaker

Rater

Training Center

SystemAdministrator

Figure 2 – Client/Server Architecture



The main features of the ELP Test Data Management System are:

Centralized and automated administration Reduced workload Flexible tracking and reporting capabilities Monitoring test taker activities through a web-based graphical interface High security High reliability and repeatability Transparency

The Media Server and the Flash client application communicate over a persistent connection using Real-Time Message Protocol (RTMP). This is a TCP/IP protocol designed for high-performance transmission of audio, video, and data messages. The Web Server communicates with the Flash client to Flash Player over HTTP. The Flash client then uses RTMP to establish a persistent connection to Flash Media Server, allowing for an uninterrupted data stream to flow between client and server. The Flash Media Server cannot initiate a connection to a Flash movie; the connection must be initiated from the Flash Player running on the client. The Flash movie streams audio and video to the Flash Media Server which stores the recorded streams. Multiple Users (Flash clients) can connect to the same application running on the Flash Media Server. Once the server accepts a client connection, the connection can be used to exchange audio, video, and ActionScript1 data until either the client or server disconnects. The bandwidth usage of the ELP online language testing system is large because the system is used for streaming audio and video files to and from Flash clients. A Flash Media Server application can be deployed (installed) over a cluster of nodes and handle progressively larger loads. At the same time, the application will appear as a single virtual application to the client. In clustered systems, any node is ready to accept a connection from any client. As a result, all nodes have the same set of applications installed, resulting in a homogenous cluster. The Flash Media Server can handle at least 500 concurrent users (without adding more hardware Servers).

1 Actionscript is a dialect of ECMAScript (meaning it is a superset of the syntax and semantics of the language more widely known as JavaScript).



Flash Player The Adobe Flash Player is a widely distributed multimedia and application player created and distributed by Adobe Systems. Flash Player runs SWF files that can be created by the Adobe Flash authoring tool or by a number of other tools. Flash Player supports an embedded scripting language called ActionScript (AS). Most computers nowadays have this player installed2. If not, this component is freely distributed and requires very little disk space. To make sure that the Flash Player is working correctly on your computer, an online testing site is provided by Adobe (http://get.adobe.com/flashplayer/ ). Updates to Flash Player ensure that Flash Player works properly and may contain important changes to security. Adobe recommends that all clients update to the latest version of Flash Player whenever a new version is available, especially when a security update is mentioned. The Adobe Flash Player is used to run AS data and to capture real-time audio and video from the Client’s microphone and camera and stream it to Flash Media Server. To publish camera and microphone content, the Flash movie should be played within Flash Player 11 or higher. Although the Flash Player may be running within the Standalone Player or within a web browser, a web browser should be used to access the ELP Data Management System. All Users (Test Takers, Raters, Testing Center Administrators and Aviation Authorities) must have the Adobe Flash Player correctly installed. Most computers ship with this player already installed. Flash Player 8 and higher support a broad range of platforms and browsers, including browsers such as Firefox, Safari and Chrome.

2 Installation of Adobe Flash Player may require administrative access to your PC, which is normally provided by your IT department. It is recommended that you close all other open browser windows before continuing with the installation.



Adobe Flash software and Flash Media Server Aerosolutions uses Adobe's Flash software to author interactive content and animations and Flash Media Server for creating rich media applications and for streaming audio and video to all Client computers (Test Takers, Raters, Testing Centers and System Administrators). Flash and Flash Media Server, a graphical front-end and an object-oriented back-end, have enabled Aerosolutions to create a highly sophisticated English Language Proficiency Test. It’s easy for end users because the Flash Player is small and quick, and it’s great for the Test System Developers and Test Item writers because of the many design and development options.



ELP Test Data Management System

EDMS Administration concepts The ELP Data Management System supports restricting access based on pre-defined roles. EDMS System Administrators can assign different levels of access to the system based on pre-defined roles. There are several roles in the English Language Proficiency online testing system, such as: EDMS System Administrator Testing Center Administrator (TC Administrator) Rater Test Taker Test Supervisor Aviation Authority Representative

Each of the different roles has specific access privileges (see Figure 3).

EDMS System Administrator An EDMS System Administrator has full access privileges to all Flash Clients on the EDMS system. An EDMS System Administrator is the highest level of access available on the system and is reserved for Aerosolutions employees. EDMS System Administrators are able to: Manage user registrations and profiles Keep user profiles up-to-date Define roles Assign raters Manage content Monitor workload and progress Manage Testing/Training Centers (e.g. set number of ratings per test) Create customized reports on individual and group performance Manage the ‘Rated Speech Samples’ library (reliability scripts) Oversee Configuration Management Monitor activity

All features are manageable using automated, user-friendly interfaces. EDMS System Administrators also have complete access to the ELP Test database.



All Authorized EDMS System Administrators receive specific training3 in the appropriate use of the EDMS system. This training takes approximately 2 days to complete.

EDMS Testing Center Administrator A Testing Center Administrator (TC Administrator) has limited access privileges to the ELP Data Management System. Each Testing Center (e.g. Airline, Air Navigation Service Provider, Training Provider …) should assign at least one TC Administrator. Users with this level of access are able to: Manage user registrations and profiles Keep user profiles up-to-date Monitor progress Access the ‘Rated Speech Samples’ library (reliability scripts) Download documents from the Document Library Create customized reports on individual and group performance Print ELP Test Certificates Verify Rater Certificates

All Authorized EDMS TC Administrators must receive specific training in the appropriate use of the EDMS system. This training4 takes approximately 1 day to complete.

Rater Raters have very limited access to the online testing system because their responsibilities lie mainly with Rating ELP Tests. Users with this level of access are able to: Mark ELP Tests within the time frame specified by the Testing / Training

Centers Report cheating Report damaged tests Access the ‘Rated Speech Samples’ library (reliability scripts) Access the Document Library Maintain training record up-to-date (e.g. rater training certificates, refresher

training certificates etc.) All Authorized EDMS Raters receive specific training in the appropriate use of the EDMS system. This training takes approximately 1 day to complete and can be part of the English Language Rater Training Course.

3 Only for Aerosolutions personnel 4 Course reference: AS ELP/EDMS Short Course



Test Taker A Test Taker is the person who actually takes the Test. Although the EDMS manages the whole testing process, Test Takers have a very limited access to the online testing system. Login codes can only be used once and are made obsolete after finishing the test.

Accessing the ELP Data Management System Logging in is the process of securely authenticating yourself to the ELP Data Management System by using your User Name and password. By logging in, you not only prove you are authorized to access the EDMS, but you gain access to customized resources. To access the ELP Data Management System, all Users (Test Takers, Raters, Testing Center Administrators, System Administrators and Aviation Authority Representatives) should go to the ‘Online Testing Center’ webpage and perform these steps:

1. Open a browser window5 (Internet Explorer, Netscape Navigator, Apple Safari, Google Chrome or other).

2. In the address bar at the top of the screen, type the URL (the Web

address) for accessing the EDMS. The system administrator should give you this URL, which begins with http://...

Your IT Manager may have set up one or more shortcuts to help you access the EDMS:

If there is an EDMS shortcut or alias on your desktop, double-click

it to open the EDMS. If the EDMS is your home page, it will appear automatically when

you open your Web browser. If the EDMS is set as a Favorite, you can select it from the

Favorites menu (Internet Explorer), without having to type the URL.

3. A web page similar to the one shown (Figure 4) should appear on your screen.

4. Type your User Name and Password.

5. Click the login button.

5 Make sure that pop-up windows are allowed.



Figure 3 - EDMS different levels of access based on pre-defined roles

EDMS Management

System Administrator

TC Administrator

Rater

Test Taker

Test Taker Management

Configuration Management

Rater Management

Create/update user accounts

Assign Tests

Review Test results

Training Center Management

Sampling Rate

Number of Raters

Test Taker Management

Document library

Rate ELP Tests

Document library



EDMS Users that have been assigned more than one role6 (e.g. System Administrator and TC Administrator) will be redirected to the ‘MY ACCOUNT’ web page (see Figure 5). The ‘MY ACCOUNT’ webpage can be accessed at any time by clicking the "My Account" link at the top of any page. Users that have been assigned one role only will be redirected to the ‘HOME’ page of their account.

Figure 4 - Testing Center Home Page Depending on the role that has been assigned to you by the System Administrator, you will be presented with different options on the left hand MENU BAR.

6 Multi-role account



After logging in Test Takers will see the ELP Test Introduction web page similar to the one shown in Figure 14.

User Management The Aerosolutions EDMS System Administrator is responsible for editing and managing all Rater, TC Administrator, Test Supervisor and Aviation Authority Representative accounts. Only the System Administrator can create or delete these accounts. To create a new Rater, TC Administrator, Test Supervisor or Aviation Authority Representative account, the following personal information is required:

Name Email address Date of Birth Telephone number Country Training Center name (Airline, Air Navigation Provider, Pilot school,

Training Center, Aviation Authority etc. ) TC Administrators and Raters can use the ‘Change Settings’ link on the left hand PERSONAL INFORMATION BAR to update this information at any time. The SAVE CHANGES button should be pressed to update the changes.



Figure 5 – EDMS ‘My Account’ web page The TC Administrator and/or the Aerosolutions EDMS System Administrator are responsible for managing all Test Taker accounts. Before a Candidate can take an online ELP Test, the TC Administrator will need to setup an account for the Candidate. To access the ELP Data Management System, the TC Administrator should go to the ‘Online Testing Center’ webpage (see Figure 4) and enter his/her User Name and Password. If the TC Administrator has been assigned only one role, he/she will be redirected to the ‘Testing Center Module’ home page. To access the ‘ADD USER’ webpage, click on the ‘Add User’ link on the left-hand MENU BAR. A web page similar to the one shown (Figure 6) should appear.



Figure 6 - Add User (TC Administrator Module) If the TC Administrator has been assigned more than one role, he/she will be redirected to a web page similar to the one shown in Figure 5. To access the ‘ADD USER’ page in this case, click on the Testing Center Module icon, and select the ‘Add User’ link on the left-hand MENU BAR. When the ‘Add User’ link is selected several fields are presented that allow the TC Administrator to enter information about each new Test Taker. It is mandatory to enter at least the following information for each new candidate: Name (as it appears on the ID document) ID Card Number Country Birth Date

When the required information has been entered into the appropriate fields, the TC Administrator must select the SAVE button which automatically creates a Test Taker account for the new candidate. After the template has been populated with necessary candidate information, the TC Administrator can select one or both of the following options:



1. Assign Test 2. Print Test Book(let) 3. Schedule an ELP Practice Test

To assign an ELP Test to the candidate, choose the test (version) you want to assign from the selection box. Select the ‘Print Test Book’ option if the Test Book needs to be printed immediately. Before taking the actual ELP Test, a testing candidate should have the option of taking an ELP Practice Test. The ELP Practice Test is a fixed sampling of questions similar to the actual ELP Test. To assign an ELP Practice Test to the candidate, select the ‘Practice Test’ option. Press SAVE to save and create the new User. A PDF file will open within a new browser window and display all the Test Book pages once you've clicked on the SAVE button. Otherwise, click the ‘Print Test Book’ link which will appear on the screen. Use the ‘Edit User’ link to edit the information of particular candidate. On the ‘EDIT USER’ page, enter selection criteria and press FIND. A list of names will appear retrieved by your selection. To edit the details of a specific User, highlight the desired name and click on it. Make necessary changes and press SAVE to update the changes.



Rating the ELP Test Only Raters are allowed to rate ELP Tests. To rate an ELP Test, Raters should go to the ‘Online Testing Center’ webpage (see Figure 4) and enter his/her User Name and password. A list of ELP Tests will appear (see Figure 7) that have to be rated. The ‘RATE TEST’ webpage can be accessed at any time by clicking the "Rate Test" link on the left-hand MENU BAR. The following information is displayed: Test ID User ID (Test taker identity is not revealed) Test version Airline or Testing Center name Test date and time Duration of the ELP Test Number of days left to rate the ELP Test

Figure 7 – Rate Test web page



All Test Takers remain anonymous, with User IDs referenced by the system, once generated upon registration. To edit the details of a specific ELP Test, highlight the desired Test and click on it. A web page similar to the one shown in Figure 8 will appear. Scores (ratings) of the speaking component of the ELP Test are entered into the EDMS on this page.

Figure 8 – ELP Test details for Raters The following general information is displayed: Test version Test date and time

Provided that the Adobe Flash Player has been installed correctly, Raters can access all sound files and rater scripts wherever and whenever they want.



Raters should use the ‘Fragment’ icons (see Figure 9) to look and listen to each of the task items. For the following tasks, a rater script is also available: Task 2 – Listening to a conversation Task 4 – Read backs7 Task 5 – Dealing with emergencies Task 6 – Describing a situation Task 7 – Short answer questions Click on the ‘Rater Script’ icons (Figure 9) shown next to the ‘Fragment’ link to open a script.

Figure 9 – Fragment and Rater script icons The EDMS Player (Figure 10) should be used to play audio streams from the Internet, allowing Raters to listen to the sound files. Use the player's playback controls to play, pause, or skip to the beginning or end. You can also drag the playhead to move through a sound fragment. Use the audio section sliders to adjust the volume.

Figure 10 – EDMS Player After listening to each sound fragment, Raters should enter the scores for each of the tasks into the system8. Click the RATE TEST button to rate the selected ELP Test. Following the requirements in the Appendix to Annex 1 (ICAO Manual), the lowest score a candidate receives on any of the six ICAO criteria is the score which is reported as the ICAO level. The EDMS automatically identifies differences in scores and flags these differences. If required, ELP Tests are distributed files to another rater.

7 When applicable 8 Individual task item scores are used to calculate split-half reliability.



View Test results TC Administrators can access the ‘VIEW TEST’ webpage at any time by clicking the "View Tests" link on the left-hand MENU BAR. Select Test Status, Training Center, Test Version and User Name (as appropriate) in the selection panel above and click on the FIND button. A list of ELP Tests will appear (see Figure 11) that meet the selection criteria, including the following information: Test ID User Name Airline or Testing Center name Test version ‘Test finished’ date and time ‘Rating completed’ date and time

Figure 11 – View Tests web page To edit the details of a specific ELP Test, highlight the desired ELP Test and click on it. A web page similar to the one shown in Figure 12 will appear. Test scores of the ELP Test obtained from different Raters can be reviewed on this page.



Figure 12 – ELP Test details for TC Administrators Provided that the Adobe Flash Player has been installed correctly, TC Administrators can access all sound files and rater scripts wherever and whenever they want. Use the ‘Fragment’ icons and the ‘Rater Script’ icons (see Figure 9) to look and listen to each of the task items. TC Administrators can also listen to the sound files. Use the player's playback controls (Figure 10) to play, pause, or skip to the beginning or end. You can also drag the playhead to move through a sound fragment. Use the audio section sliders to adjust the volume. If an ELP Test has been rated by several Raters, the TC Administrator is able to view each rating. Click the REQUEST ADDITIONAL RATING button if the ELP Test has to be rated by another rater.



Change settings TC Administrators, Raters, Test Supervisors and Aviation Authority Representatives can manage personal data and privacy settings from the CHANGE SETTINGS page (Figure 13). Use the ‘Change Settings’ link on the left hand PERSONAL INFORMATION bar to update this information at any time. Press SAVE CHANGES to update the changes. You should NEVER give anyone your password, and if you suspect that someone else knows it, you should change it as soon as possible. To change your password, click the ‘Change Password’ link on the left hand PERSONAL INFORMATION bar. If you have forgotten your password, go to the ‘Online Testing Center’ webpage and click the “Forgot password?” link.

Figure 13 – Change settings page



Logging off or leaving your computer During the day Raters, TC Administrators, System Administrators, Test Supervisors and Aviation Authority Representatives may occasionally need to leave their computer for a few minutes. When this happens, in order to protect the security of the ELP Testing System, the workstation should always be locked. To lock the screen, hold down the Control, Alt, and Delete keys simultaneously (as you do when you log on) and choose the Lock Computer button (or press the k key on your keyboard). This will keep anybody else from accessing your computer while you’re away. When you get back, just press Control-Alt-Delete, log in as you normally would, and your programs and documents will be exactly as you left them. Make sure that the EDMS workstations are configured to automatically lock when the screensaver comes on. At the end of the day, or during the day if you will be gone from your computer for an extended period of time, it is a good idea to log off. To log off, click the “Log Off” link at the top of the screen.



Launching the ELP Test Before a Candidate can take an online ELP Test, Test Taker should go to the ‘Online Testing Center’ webpage (see Figure 4): www.aerosolutions.be/Testing/ and enter his/her User Name and password. After login, ELP Test Takers will see the ELP Test Introduction web page similar to the one shown below (see Figure 14).

Figure 14 – ELP Test introduction page The Flash Player Settings panel opens automatically (Figure 15) when the Flash Application attempts to obtain data from a microphone or camera. If the Test Taker allows access, Flash will have access to the audio and video devices. However, if the User denies access to the audio and video devices, the ELP Test will not start.



Although the Flash Player allows test takers to make decisions about what happens with regard to the computer they are using, including the Player’s access to audio and video devices, test takers must grant permission via the Flash Player Settings panel, as shown in Figure 15.

Figure 15 - Asking permission to access microphone and camera The Settings panel can be manually opened by choosing Settings from the Flash Player context menu (available by right-clicking atop the Flash Player window on Windows or Ctrl-clicking on Macintosh). When opened manually, the panel offers more options, as shown in Figure 16. This panel also opens automatically as long as the User is denying access to the audio and video devices.

Figure 16 - Full-fledged Flash Player Settings panel



The following information is displayed on the ELP Test introduction page: “Welcome to the English Language Proficiency Test (ELP Test) for Pilots. The ELP Test is used to assess speaking and listening proficiency in both phraseology for routine communication and plain English for non-routine and emergency situations. The Test consists of a 10-minute listening comprehension test (Part I) and a 35-minute speaking test (Part II). For Part I, you will have to answer multiple-choice questions. This part of the test is computer-adaptive and measures your ability to understand English. The speaking section (Part II) assesses how effectively you can communicate in English and measures your ability to respond to different aviation situations and tasks. This part of the test is recorded. The media window located near the top-right corner of the screen displays your image. For identification purposes, we require you to remain visible for the whole duration of the test. A high-quality microphone and speakers are required, although a microphone headset performs best. You will now have the opportunity to test the Recording Volume. Speak clearly into the microphone. While you are speaking, the input meter should be green. Red means you have to speak louder. You should be able to hear your own voice through the earphones or speakers after the recording has stopped. Once you are satisfied that you have set the Recording Volume to the right level, check the statement below and click on the button in the bottom-right corner of the screen to start the ELP Test.” Test takers are required to run a test of the audio hardware before taking the ELP Test. They should also verify if their image is displayed in the media window located near the top-right corner. Once all of these requirements have been satisfied, test takers should click on the button in the bottom-right corner of the screen to start the ELP Test. If the Test Taker needs to leave the room for any reason, the ELP Test cannot be paused. When the candidate completes the test, the Test Administration Supervisor or the TC Administrator should: Verify if the test has been completed Close the test window Collect the ELP Test Book



Test-delivery procedures

Introduction Before any candidates arrive at the testing location, it is important that the testing room be ready for testing. Be sure that all hardware is in order and that the web-based testing system is functioning properly. TC Administrators should monitor ELP Test Takers during the entire session to ensure all rules are followed and that no cheating takes place. While a person is taking an ELP Test, they should be prohibited from performing any other tasks on the testing workstation. This maintains the security and protected environment of the Test.

EDMS System requirements ELP Tests are delivered on computers via a web browser. Users (Flash Clients) can access the ELP Test (and the EDMS application) from any computer connected to the Internet. In addition to having a broadband Internet Connection, all users need a computer that meets the minimum standards listed below. Keep in mind that the minimum requirements may be slow and cause frustration during an actual ELP Test or when using the ELP Data Management System. If the Client computer is located behind a (corporate) firewall, Flash Clients may experience problems to access the system. Using pop-up killer software may prevent users from being able to view parts of the EDMS pages or ELP Tests as well as parts of the Aerosolutions website. To disable the firewall and/or pop-up killer software, ask your network administrator for assistance.

Screen Resolution 1024 Width x 768 Height or higher

Sound card Sound card, speakers or headphones and a microphone required for audio

Internet Connection

High-speed / Broadband (DSL, Cable, T1 or LAN)

Video USB or Firewire webcam

Other equipment Mouse / pointing device



Browsers*

Microsoft Internet Explorer V8.0 (or higher) Mozilla Firefox V8.0 (or higher) Apple Safari V4.0 (or higher) Google Chrome V15.0 (or higher)

Processor Pentium II class or better

Plug-ins Flash Player V9.0 (or higher)

Table 1 – Minimum system requirements *Other browsers might NOT compatible with the testing system

Configuring video and audio devices In order to take the ELP Test, Test Takers will need a headset with a microphone and a camera attached to the computer (Figure 17). The headset should have two earpieces and an attached microphone. We do not recommend using a standalone microphone as the sound will vary too much as examinees might move back and forth in their chair.

Figure 17 - Audio and video devices required for the ELP Test



There are two types of headsets that can be used. They look almost identical but vary in the way they are connected to the computer:

1. Miniplug 2. USB.

The first has dual miniplugs, one marked with a speaker icon and the other marked with a microphone icon. Be careful to plug the correct plug into the correct port. If the plugs are switched around, you will not be able to hear or speak. The miniplug system is the easiest to get running and we recommend it if you system has the miniplug ports or you do not have an available USB port. The second type of headset is connected via USB. This may be the preferred connection method if you have USB connectors on the front of your machine. USB system headsets, however, have to be installed through software. Most likely though, your system will automatically recognize the headset after about 20 seconds, the very first time you plug it in, and then the headset will install itself. Some headsets come with an additional microphone on/off switch and earpiece volume control knob. If that is the case, make sure the headset is switched to on and the ear piece volume is turned up. You may have to tell your system to route all audio through the headset. Anti-spyware and anti-virus control may try to block the sound test from running, so you may be prompted to allow it. To install a camera or microphone, follow the instructions that accompany the audio and video device. For a list of cameras that are known to be compatible with Flash Media Server, see the documentation on camera compatibility on the Adobe website. Although many cameras contain a built-in microphone, we recommend the installation of a separate high-quality microphone, or for best results, a high quality microphone-headset combination. After you’ve installed the audio and video devices, you can specify which camera or microphone Flash should use by default. Microphones and speakers often cause unwanted echo, resulting in feedback. Feedback is normally a symptom of the microphone being placed so that it picks up audio from speakers. The best way to avoid speaker audio feeding back into the microphone is to use a closed headset instead of speakers. When using speakers, make sure that the microphone is unidirectional. If the microphone is omni-directional, it will pick up speaker output no matter how it is placed relative to the speakers. The Test Administration Supervisor or the TC Administrator is required to run a preliminary test of the audio and video hardware EVERY time the computer settings are changed.



Test Administration Supervisor9 The Training Center shall provide facilities for the administration of the test that do not disadvantage any test taker. A Test Administration Supervisor should be carefully prepared and trained so that each administration of the test can be uniform, assuring that all test takers receive the same instructions and access to any permitted aids. If something occurs that calls into question the uniformity of the administration of the test, the problem should be identified and any remedial action to be taken to offset the negative impact on the effected test takers should be promptly communicated. At each testing site, there must be a Test Administration Supervisor who is responsible for oversight of the entire test administration process. Generally this will be a representative of the Testing Center where the ELP Tests are being administered, managed and supported. It is the responsibility of the Test Administration Supervisor to ensure that test security standards are maintained and that the tests are administered in accordance with these instructions and standards. The Test Administration Supervisor can select responsible individuals (Proctors) to assist with the administration of the examination. Potential examinees or relatives of examinees are not permitted to serve as Proctors or Test Administration Supervisors. The Proctors assist the Test Administration Supervisor in keeping order in the testing room, monitoring examinees to deter cheating, and distributing and collecting test books. The duties and responsibilities of the Test Administration Supervisor are: Printing, distributing and collecting Test booklets Scheduling and supervision of all Proctors Communication between Training/Testing Center and Aerosolutions Maintenance of test security Supervise test sessions Arrange for testing space and ensure that the testing room(s) meets the

specifications for testing space (see next page) and for the secure handling of test materials

Verify, fully document, and report any irregular incidents Promptly inform Aerosolutions of any breach in security of test materials Collect, inventory and store all test books and forms Verify the identity of testing candidates Management of the EDMS Testing Center account10

9 These responsibilities can also be assumed by the TC Administrator. 10 Only if the Test Administration Supervisor is also the TC Administrator.



Physical Supervision Physical supervision is an important part of ELP Test administration. The Test Administration Supervisor and/or Proctors must be present on site to start tests and to monitor test sessions. Test Administration Supervisors may observe in the testing room, through a testing window, or use a video surveillance system11.

Hardware & software Hardware and software suitable for the Tests must be available and operating correctly. All hardware must be checked and in good working order. Software must be correctly licensed, checked, in good working order, and set appropriately for each ELP Test.

ELP Practice Test Before taking the actual ELP Test, a testing candidate should have the option of taking an ELP Practice Test. The ELP Practice Test is a fixed sampling of questions similar to the actual ELP Test. It is highly recommended that all Test Takers take the ELP Practice Test so that they know what to expect when their actual ELP test begins and to become comfortable with the testing format and questions. The ELP Practice Test is available online. To access the ELP Practice Test, Test Takers should go to the ‘Online Testing Center’ webpage:

1. Open a browser window (Internet Explorer, Netscape Navigator, Safari, or other). Make sure that pop-up windows are allowed.

2. In the address bar at the top of the screen, type the URL (the Web address) for EDMS. The system administrator should give you this URL, which begins with http://...

3. A web page similar to the one shown in Figure 4 should appear on the screen.

4. Type User Name and password 5. Click the login button 6. The ELP Practice Test will start automatically …

Testing space ELP Test workstations should be in separate rooms in order to provide privacy for the candidate and to prevent candidates disturbing each other when tests 11 Including the ELP Test webcam surveillance system.



are delivered. Test workstations must have adequate desk space that allows the testing candidate to work comfortably when using the computer monitor and keyboard. The testing room must have adequate lighting in order for the testing candidate to view the keyboard and make notes during the test in their ELP Test Book (recommended 400 – 800 lux). Temperature and ventilation should be appropriate. The testing room must be exclusive to testing and the testing room and immediate surrounding areas must be quiet when testing is in progress. These two requirements help to ensure that a candidate is not distracted when tests are in progress. Testing rooms should also be barrier-free design such that the Test Administration Supervisor or the Proctors have an unobstructed view of the test taker. Make sure that construction or repairs are not scheduled near the testing room, and/or fire drills are not scheduled on the examination day.

Personal Belongings in the Testing Room Do not permit Test Takers to bring personal belongings into the seating area of the testing room. All materials, except pencils and erasers must be deposited in the area designated for personal belongings. The following items are not permitted in the testing room: Personal Digital Assistants, laptops, calculators Watches with computer or memory capability Paging devices Cellular telephones, tape recorders … Recording/filming devices Radios Reference materials (books, notes, papers) Briefcases, coats, hats, crash helmets … Beverages or food of any type No smoking is permitted during the test sessions or in the test rooms

The following items should be supplied by the Testing Center: Pencils, erasers Test taker’s personal ELP Test booklet

Candidates are not to bring or write on scrap pieces of paper. Test takers are allowed to make notes in their test booklets, either in the margins or on the blank pages at the back of each test booklet.



Candidate Verification When an ELP Test candidate comes to the Testing center to take an ELP Test, the Test Administration Supervisor should complete a Test Sign-off Sheet with information such as test date, time in, name of candidate, ID type, and ID number. The candidate should sign his or her name on the signature line of the sign-off sheet. A Test sign-off sheet is attached as Appendix 2. Candidates must present valid and acceptable identification (ID) with a signature and photograph. Acceptable forms of ID include: A valid pilot’s license A valid passport A valid ID

Expired documents are not acceptable. The photograph on the ID document must be recent and recognizable. Original documents must be presented: copies are not acceptable. ID requirements should be strictly enforced. Candidates who do not have acceptable identification should not be allowed to take an ELP Test.

Rules of Test Conduct The Rules of Test Conduct should be made clear by the Test Administration Supervisor or a Proctor to any candidate taking an ELP Test. Test Takers are required to indicate that they understand and agree to these rules before the test is started. The Rules of Conduct for ELP Test Takers are as follows: Testing session might be monitored by a Test Administration Supervisor or

Proctor. Communication with the Test Administration Supervisor or a Proctor is

acceptable only in an emergency. No supplemental materials may be used during the test. This includes

books, scrap paper, etc. Personal belongings will be kept in a safe and secure area until the test is completed.

Test Takers are not allowed to leave the testing room before the test has ended except in case of an emergency.

If any of these rules are broken, the ELP Test must be terminated. Once these steps have been taken, the candidate can be escorted to the testing room and the test start time may be marked on the ELP Test sign-off sheet.



Unethical behaviour The giving of false or misleading information during the registration process, and any form of cheating during the test itself, are viewed as serious breaches of ethical behavior and will attract penalties. Any candidate found to be cheating will automatically have his/her ELP Test scores cancelled. Practices that constitute breaches of test rules include: attempting to gain access to test questions prior to the test attempting to take the test on behalf of another person allowing another person to attempt to take the test on their behalf attempting to remove a test book or part thereof, an answer sheet, or any

notes, from the testing room failure to follow Test Administration or Proctor’s instructions at all times creating a disturbance using unauthorized aids (including notes, note paper, video camera,

mobile phone, tape recorder, etc.) Any irregularities must be reported to the Aviation Authorities and the System Administrator.

Answering Questions Regarding Test Items If an test taker thinks a test item is incorrect, instruct him/her to answer the item to the best of his/her ability with the information provided. Do not answer any questions regarding examination content. Assure the examinee that the test item will be identified in a report to Aerosolutions. Prepare a report identifying the item(s) in question and send this report to Aerosolutions.

Candidate Test Appeals Process If a test candidate has a problem with his or her test results, a test item, or issues concerning the testing environment, a test appeals form could be filed and submitted to the Testing Center. Any appeals process must be in accordance with the local Aviation Regulations. A senior Rater from Aerosolutions must be responsible for finalizing ELP Test scores in the event that test results (not content) are appealed.



ELP Test procedure – Check list A typical sequence of events in the ELP testing process is as follows:

1. Set up the testing room(s) 2. Check lighting, temperature, noise level … 3. Place a sign12 saying "QUIET--TESTING IN PROGRESS" on the door

and in adjoining areas, to limit outside noise 4. Check Test Taker identity 5. Fill in Test Sign-off Sheet 6. Ask Test Taker to sign the Test Sign-off sheet 7. Create an account for the Test Taker using the ‘Add User’ link in the

EDMS 8. Assign ELP Test to Test Taker using the ‘Assign Test’ option in the

EDMS 9. Print ELP Test booklet and give it to the Test Taker 10. Explain rules of conducts (e.g. emergency situations) 11. Direct Test Takers to testing room (allowing only approved items into

the room) and keep personal belongings in a safe and secure area until the test is completed

12. Monitor Test Taker during test 13. Collect ELP Test booklet at the end of the testing session 14. Complete Test Sign-off Sheet 15. Store ELP Test booklet in a safe place (Test booklets should be

retained at the Testing Center or at any other location deemed appropriate by the Testing Center)

16. Raters analyze test taker’s performance 17. Testing Center publishes test taker’s scores 18. Candidates receive ELP Test Certificate13

12 Whenever required 13 Only when level 4 score or higher has been obtained.



Test Security

Test materials All test materials are highly confidential and must be kept secure at all times. It is the responsibility of the Test Administration Supervisor and the Proctors to see that no persons copy questions at any time. Test Administration Supervisors and Proctors are required to safeguard all aspects of test security, to ensure that test takers are not disturbed, and to account for all test materials distributed. They should be alert and observant of any attempts to cheat during the ELP Test. If the Test Administration Supervisor and/or Proctors suspect that an examinee is cheating in any way, including giving or receiving assistance during the examination, communicating with others, retaining or copying examination questions, or using prohibited aids, then: document incidents and descriptions of the incidents. warn the test taker, advising him that test irregularities will be reported to

the Aviation Authorities and the System Administrator. If the examinee tries to copy or retain test examination material, confiscate

the material and expel the test taker from the testing room.

If an examinee is disruptive, engages in clear or flagrant cheating, or attempts to copy questions or retain or record test materials, and is expelled from the testing room, the candidate should be advised that his or her examination will not be scored. Documentation of the expulsion and the reasons for it should be maintained for the Aviation Authorities. All security rules are to be enforced without exception.

Test system Flash has very strong, built-in digital media protection capabilities when streaming content with Flash Media Server. Some typical protection features built into all content delivered with Flash Player are: No client cache Unique transfer protocol limits stream ripping No exposed media on the server Authentication at the SWF level Authentication at the stream level Control over information that is exposed



The Flash Media Server supports several ways of controlling whether an end user may connect to an application instance. Identity information, such as name and password, is transmitted over RTMP. The server verifies the name and password, and allows the client to continue as appropriate. The Flash Media Server has functions for server-side scripts that may disconnect users. End user data can be stored in shared objects, a mechanism for persistent storage on either the client or server computer. Applications only have access to the shared objects associated with that application, preventing another Flash program from reading that data. Applications cannot access the resources or files of other applications. This is enforced by saving resources for one application in a unique directory that can't be accessed by other applications.



Training and ongoing support for (external) Raters Initial Rater Training, Refresher Rater Training and on-going quality control checks assure that the scoring process is working as intended. Aerosolutions offers the following methods of training, certification and on-going support to external raters: Initial Rater Training Refresher Rater Training Blind sample ratings for certification and re-certification Standardization using online accessible rated speech samples14 Reliability monitoring

Aerosolutions15 provides Initial Rater Training Courses and Rater Refresher Courses on a regular basis. Aerosolutions internal policy regarding validity of training certificates is as follows:

- Rater Training Certificate: 3 years - Rater Refresher Training Certificate16: 3 years

For certification purposes, candidates are required to demonstrate the ability to rate language proficiency according to an agreed standard17 by rating five Rater Assessment Samples. The five tests for certification are displayed in the Rate Test webpage. The criteria for rater certification are as follows: The overall ICAO level must agree with the senior rating team Raters are allowed a discrepancy of no more than one level in the

individual language profiles providing this does not affect the overall ICAO level

Raters can extend the validity period of their certificate by successfully rating five additional Rater Assessment Samples. To maintain the agreed standard, the ‘Rated speech samples’ section of the EDMS includes sample tests that include scores with detailed rater rationales and commentaries on the candidate’s performance. The samples are a permanent feature of the EDMS and there is no limit to the number of times raters or trainees can listen to them.

14 Rated Speech Samples are accessible through the ELP Data Management System 15 In cooperation with Emery-Roberts 16 Only valid if a candidate had obtained a valid Rater Training Certificate in the past 17 Standards are defined in Aerosolutions’ Manual for Raters



To monitor test reliability and rater performance, a number of tests are double-rated. The tests are selected at random and a second rater conducts a ‘blind’ rating (i.e. without knowing the first rater’s scores). The system is anonymous; neither the first nor the second rater is aware that the sample in question is being rated for a second time. The EDMS automatically identifies differences in scores and flags these differences to Aerosolutions. If ratings vary from the standard18, the rater will be asked to re-standardize with a new set of rater speech samples. If the rater’s ratings continue to vary from the standard, he will not be able to continue work as an Aerosolutions rater. The sampling rate is set as follows:

‐ 1 % for all experienced raters ‐ 5 % for all inexperienced raters

In addition, Aerosolutions provides ongoing support to raters of accredited test centers, including rater reliability checks. All external raters must satisfy the following requirements19: A Rater Training Course certificate issued by Aerosolutions or any other

Language Training Center approved by Aerosolutions Hold a valid rater certificate Perform a minimum of 50 ELP Test assessments per year ICAO Level 6 score

18 Standards are defined in Aerosolutions’ Manual for Raters 19 See also Appendix 3: Recommended qualifications for raters to meet ICAO LPRs



Ethical issues

ILTA - Code of Ethics Every individual makes decisions about how they conduct themselves professionally. However, there are very few people who would knowingly work with an individual whose behavior is unethical. It is important to follow the ethical guidelines and seek consultation for situations that call for actions that are questionable. Language testers (Raters) are independent moral agents and sometimes they may have a personal moral stance which conflicts with participation in certain procedures. They are morally entitled to refuse to participate in procedures which would violate personal moral belief. Language testers accepting employment positions where they foresee they may be called on to be involved in situations at variance with their beliefs have a responsibility to acquaint their employer or prospective employer with this fact. Employers and colleagues have a responsibility to ensure that such language testers are not discriminated against in their workplace. Principle 1 Language testers shall have respect for the humanity and dignity of each of their test takers. They shall provide them with the best possible professional consideration and shall respect all persons’ needs, values and cultures in the provision of their language testing service. Language testers shall not discriminate against nor exploit their test takers on grounds of age, gender, race, ethnicity, sexual orientation, language background, creed, political affiliations or religion, nor knowingly impose their own values (for example social, spiritual, political and ideological), to the extent that they are aware of them. Sexual relations between language testers and their test takers are always unethical. Principle 2 Language testers shall hold all information obtained in their professional capacity about their test takers in confidence and they shall use professional judgement in sharing such information. In the face of the widespread use of computerized test records and data banks, the increased demand for accountability from various sources and the personal nature of the information obtained from test takers, language testers are obliged to respect test takers’ right to confidentiality and to safeguard all information associated with the tester-test taker relationship.



Principle 3 Language testers should adhere to all relevant ethical principles embodied in national and international guidelines when undertaking any trial, experiment, treatment or other research activity. Principle 4 Language testers shall not allow the misuse of their professional knowledge or skills, in so far as they are able. Principle 5 Language testers shall continue to develop their professional knowledge, sharing this knowledge with colleagues and other language professionals. Continued learning and advancing one’s knowledge are fundamental to the professional role; failure to do so constitutes a disservice to test takers. Language testers shall make use of the various methods of continuing education that are available to them. These may involve participation in continuing language testing programmes and professional conferences, and the regular reading of relevant professional publications. Language testers shall take the opportunity to interact with colleagues and other relevant language professionals as an important means of developing their professional knowledge. Principle 6 Language testers shall share the responsibility of upholding the integrity of the language testing profession. Language testers shall promote and enhance the integrity of their profession by fostering a sense of trust and mutual responsibility among colleagues. In the event of differences of opinion, viewpoints should be expressed with candour and respect rather than by mutual denigration. Language testers who become aware of unprofessional conduct by a colleague shall take appropriate action; this may include a report to the relevant authorities. Principle 7 Language testers in their societal roles shall strive to improve the quality of language testing, assessment and teaching services, promote the just allocation of those services and contribute to the education of society regarding language learning and language proficiency.



Principle 8 Language testers shall be mindful of their obligations to the society within which they work, while recognising that those obligations may on occasion conflict with their responsibilities to their test takers and to other stakeholders. Principle 9 Language testers shall regularly consider the potential effects, both short and long term on all stakeholders of their projects, reserving the right to withhold their professional services on the grounds of conscience. As professionals, language testers have the responsibility to evaluate the ethical consequences of the projects submitted to them. While they cannot consider all possible eventualities, they should engage in a thorough evaluation of the likely consequences and, where those consequences are in their view professionally unacceptable, withdraw their services. In such cases, they should as a matter of course consult with fellow language testers to determine how far their view is shared, always reserving the right, where their colleagues take a different view, to make an individual stand on the grounds of conscience.


Glossary ACCENT A distinctive pronunciation of a language which is usually associated with a geographical region (for first-language speakers) or with the phonological influence of another mother tongue (for second or foreign language speakers). All speakers of all languages have an accent. ADMINISTRATION The actions involved in the delivery of a test to a group of test takers under specified conditions. CUE The spoken input from an audio-recording or a live interlocutor which requires the test taker to provide a spoken response. Prompt is synonymous with cue. DESCRIPTOR A brief description accompanying a band on a rating scale, which summarizes the degree of proficiency or type of performance expected of a test taker to achieve that particular level. The band may contain several descriptors. DIALECT A distinctive variety of a language, usually associated with social or geographical distinctions, which is characterised by differences in accent, vocabulary and grammar with regard to other varieties of the same language. FACE VALIDITY The extent to which a test appears to test takers, or those choosing it on their behalf, to be an acceptable measure of the ability they wish to measure. FORMULAIC SPEECH A restricted or coded use of language comprising fixed standard phrases or lexical and syntactical routines, developed either by consensus for highly repetitive communications (e.g. everyday exchanges of greetings) or formally prescribed for special or professional purposes. (ICAO standardized phraseology is an example of formally prescribed formulaic speech.) INTERLOCUTOR A suitably qualified and trained person with whom a test taker interacts during a test in order to complete a speaking task. INVIGILATOR A person ensuring that the specified test conditions are maintained while a test is being conducted. Proctor is synonymous with invigilator. LANGUAGE PROFICIENCY SKILLS The knowledge and abilities which impact on the capacity of a given individual to communicate spontaneously, accurately, intelligibly, meaningfully and appropriately in a given language.


LINGUISTIC RATER A rater whose assessment will focus on the linguistic features of a test taker’s performance in a test (compare with “operational rater”). NEEDS ANALYSIS A detailed study by means of questionnaires submitted to the target population and descriptions of samples of language as it is actually used. The results of such studies provide data for establishing language test objectives. OPERATIONAL RATER A rater with working knowledge of professional standards and procedures of radiotelephony communications whose assessment will focus on the appropriateness of a test taker’s performance with regard to the holistic descriptors (compare with “linguistic rater”). PLAIN LANGUAGE The spontaneous, creative and non-coded use of the resources of a given natural language, as constrained by the functions and topics that are required to conduct radiotelephony communications and to meet safety-critical requirements for intelligibility, directness, appropriateness, nonambiguity and concision. RATE To assign a level of language proficiency to a test taker’s performance. RATER A suitably qualified and trained person who assigns a rating to a test taker’s performance in a test based on a judgment usually involving the matching of features of the performance to descriptors on a rating scale. RATING SCALE A scale consisting of several ranked categories (i.e. the six language proficiency skills of the ICAO Rating Scale) used for making judgments of performance. They are typically accompanied by band descriptors which make their interpretation clear. REGISTER A style of speech (involving distinctive vocabulary, syntax, speech rate, etc.) that is adopted by the speaker to be appropriate for a given situation or activity. RELIABILITY The consistency or stability of the measures from a test. RUBRIC The instructions given to test takers to guide their responses to a particular test task. TEST CONSTRUCT A hypothesized mental ability which cannot be directly observed or measured in language testing, for example listening ability. Language tests attempt to measure the different constructs which underlie language ability.


TEST ITEM Each testing point in a test which elicits a response from the test taker and which serves to gather evidence. TEST MAINTENANCE The activities of a testing organization intended to preserve the reliability, validity and security of the test over time. These activities include monitoring test results and rater reliability, designing and trialling new test items, issuing new versions of the test, reviewing instructions for test administrators, etc. TEST OBJECTIVE The language behaviors for which a test requires test takers to demonstrate their ability. TEST PACKAGE All documentation and other materials required for the administration of a test. This may also include provision of the logistical and human resources of a testing service. TEST TAKER The person who is tested. Candidate is synonymous with test taker. TEST TASK The combination of a single rubric and the associated test items. TEST USER The person or institution making use of a test and to whom test results are made available in order to inform choices or actions. TESTING SYSTEM A combination of all provisions for administrating a given test, including the -test materials, but also the organisation of test maintenance, test delivery and rating. TRIAL, TRIALLING A stage in the development of test tasks where tasks are administered to a representative sample of the target population to ascertain whether the test functions as expected. VALIDATE To undertake actions during test development and test maintenance that demonstrate the validity of a test. VALIDITY The extent to which test results enable inferences to be made about language proficiency which are appropriate, meaningful and useful given the purpose of the test. VARIETY The effects on spoken language of dialect or register.

Appendix 1 – ICAO Rating Scale


ICAO RATING SCALE

LEVEL PRONUNCIATION STRUCTURE VOCABULARY FLUENCY COMPREHENSION INTERACTIONS

Expert

Level 6

Pronunciation, stress, rhythm, and intonation, though possibly influenced by the first language or regional variation, almost never interfere with ease of understanding.

Both basic and complex grammatical structures and sentence patterns are consistently well controlled.

Vocabulary range and accuracy are sufficient to communicate effectively on a wide variety of familiar and unfamiliar topics. Vocabulary is idiomatic, nuanced, and sensitive to register.

Able to speak at length with a natural, effortless flow. Varies speech flow for stylistic effect, e.g. to emphasize a point. Uses appropriate discourse markers and connectors spontaneously.

Comprehension is consistently accurate in nearly all contexts and includes comprehension of linguistic and cultural subtleties.

Interacts with ease in nearly all situations. Is sensitive to verbal and non-verbal cues and responds to them appropriately.

Extended

Level 5

Pronunciation, stress, rhythm, and intonation, though influenced by the first language or regional variation, rarely interfere with ease of understanding.

Basic grammatical structures and sentence patterns are consistently well controlled. Complex structures are attempted but with errors which sometimes interfere with meaning.

Vocabulary range and accuracy are sufficient to communicate effectively on common, concrete, and work-related topics. Paraphrases consistently and successfully. Vocabulary is sometimes idiomatic.

Able to speak at length with relative ease on familiar topics but may not vary speech flow as a stylistic device. Can make use of appropriate discourse markers or connectors.

Comprehension is accurate on common, concrete, and work-related topics and mostly accurate when the speaker is confronted with a linguistic or situational complication or an unexpected turn of events. Is able to comprehend a range of speech varieties or registers.

Responses are immediate, appropriate, and informative. Manages the speaker/listener relationship effectively.

Operational Level 4

Pronunciation, stress, rhythm, and intonation are influenced by the first language or regional variation but only sometimes interfere with ease of understanding.

Basic grammatical structures and sentence patterns are used creatively and are usually well controlled. Errors may occur, particularly in unusual or unexpected circumstances, but rarely interfere with meaning.

Vocabulary range and accuracy are usually sufficient to communicate effectively on common, concrete, and work-related topics. Can often paraphrase successfully when lacking vocabulary in unusual or unexpected circumstances.

Produces stretches of language at an appropriate tempo. There may be occasional loss of fluency on transition from rehearsed or formulaic speech to spontaneous interaction, but this does not prevent effective communication. Can make limited use of discourse markers or connectors. Fillers are not distracting.

Comprehension is mostly accurate on common, concrete, and work- related topics when the accent or variety used is sufficiently intelligible for an international community of users. When the speaker is confronted with a linguistic or situational complication or an unexpected turn of events, comprehension may be slower or require clarification strategies.

Responses are usually immediate, appropriate, and informative. Initiates and maintains exchanges even when dealing with an unexpected turn of events. Deals adequately with apparent misunderstandings by checking, confirming, or clarifying.

Appendix 1 – ICAO Rating Scale


ICAO RATING SCALE

LEVEL PRONUNCIATION STRUCTURE VOCABULARY FLUENCY COMPREHENSION INTERACTIONS

Pre-operation Level 3

Pronunciation, stress, rhythm, and intonation are influenced by the first language or regional variation and frequently interfere with ease of understanding.

Basic grammatical structures and sentence patterns associated with predictable situations are not always well controlled. Errors frequently interfere with meaning.

Vocabulary range and accuracy are often sufficient to communicate on common, concrete, or work-related topics, but range is limited and the word choice often inappropriate. Is often unable to paraphrase successfully when lacking vocabulary.

Produces stretches of language, but phrasing and pausing are often inappropriate. Hesitations or slowness in language processing may prevent effective communication. Fillers are sometimes distracting.

Comprehension is often accurate on common, concrete, and work-related topics when the accent or variety used is sufficiently intelligible for an international community of users. May fail to understand a linguistic or situational complication or an unexpected turn of events.

Responses are sometimes immediate, appropriate, and informative. Can initiate and maintain exchanges with reasonable ease on familiar topics and in predictable situations. Generally inadequate when dealing with an unexpected turn of events.

Elementary Level 2

Pronunciation, stress, rhythm, and intonation are heavily influenced by the first language or regional variation and usually interfere with ease of understanding.

Shows only limited control of a few simple memorized grammatical structures and sentence patterns.

Limited vocabulary range consisting only of isolated words and memorized phrases.

Can produce very short, isolated, memorized utterances with frequent pausing and a distracting use of fillers to search for expressions and to articulate less familiar words.

Comprehension is limited to isolated, memorized phrases when they are carefully and slowly articulated.

Response time is slow and often inappropriate. Interaction is limited to simple routine exchanges.

Pre-elementary Level 1

Performs at a level below the Elementary level.






Appendix 2 – Test Sign-off Sheet


TEST SIGN-OFF SHEET

Date Time in Name ID type and number Date of birth Workstation Test Start Test Stop

Appendix 3 – Recommended qualifications for Raters of tests


RECOMMENDED QUALIFICATIONS FOR RATERS TO MEET ICAO LPRs

Raters of tests:

1. Shall have experience in aviation adequate to evaluate performance up to level 6 in compliance with the holistic descriptors and the ICAO Language Proficiency Rating Scale. 1.1. English language experts:

Their level of the English language proficiency (speaking, listening, reading and writing) shall be proved by a certificate/diploma such as:

University degree in Applied Linguistics in English, or The International English Language Testing System (IELTS)

examination – Academic module (the average score of 7.0 or above, including minimum of 7.5 on speaking and listening), or

The Cambridge Certificate of Proficiency in English (CPE) examination (grades A, B or C), or

Common European Framework (CEF) level C2; etc.

The above or similar certificate/diploma shall be accompanied by proof that language proficiency has been actively maintained. Note: The above mentioned qualifications are only applicable to raters whose first language is not English

1.2 Operational experts (pilots and air traffic controllers) should demonstrate proficiency in English at ICAO level 6 (expert).

2. Shall have experience in teaching Aviation English or operational experience as an air traffic controller or pilot.

3. Shall have successfully completed a training course in oral proficiency examination and testing systems, particularly in assessment procedures based on the ICAO Language Proficiency Rating Scale.

4. Shall pass a refresher course every 5 years to maintain standards in rating and assessment techniques.

Appendix 4 – Confidentiality Agreement


CONFIDENTIALITY AGREEMENT (Test Administration Supervisor and Proctors) Please email the signed confidentiality agreements to: Aerosolutions BVBA, [email protected] Note that ELP Tests may not be issued until the Confidentiality Agreement form has been returned. Please copy this page, and ensure that all Proctors and the TC Administrator complete an agreement. Please retain the original copy of the confidentiality agreements. _________________________________________________________________________ In signing this Confidentiality Agreement I undertake: to ensure that all staff assisting with the administration of the ELP Test are

fully briefed in the procedures set out in this Manual and that all procedures are followed;

to maintain the security of the ELP Test and all test items at all times and to allow no copies or transcripts of the tests to be made.

to ensure that the identity of all candidates is verified by requiring the candidate to produce photographic ID;

to allow NO persons to access any examination materials excepting the candidate(s) during the administration;

to store every test booklet after the test administration; I understand the need for the maintenance of the strictest test security and confidentiality. I promise that I will not, directly or indirectly, in any way compromise the security of any examination materials. Excepting the test taker in the examination room, I will not allow any person other than an authorized supervisor of the test to have access to, look at or be informed about any examination materials. Under no circumstances will I examine or discuss the test contents with the test taker during the ELP Test. I will not offer any hints, suggestions, definitions or clues to the answer of a test item. I know of no past breach of confidentiality and agree to report any breach to Aerosolutions that I witness or come to know about. I declare that I am not related to, nor a teacher of, a test taker who will take the ELP Test. I hereby agree to perform, and will be held responsible for, all the necessary duties described above and those described in the Test Administration procedures set out in this Manual.

Name:

Position: Organisation:

Signature: Date:

Appendix 4 – Confidentiality Agreement


CONFIDENTIALITY AGREEMENT (Raters, TC Administrators and Aviation Authorities) Please email the signed confidentiality agreements to: [email protected] Note that ELP Tests may not be issued and the EDMS may not be used until this signed Confidentiality Agreement form has been returned to Aerosolutions. Please retain the original copy of the confidentiality agreements. Training Centers: Please copy this page, and ensure that all Raters complete an agreement. _________________________________________________________________________ In signing this Confidentiality Agreement I undertake: to maintain the security of the ELP Test and all test items at all times and

to allow no copies or transcripts of the tests to be made. to allow NO persons to access any examination materials under any

circumstances; not to share my EDMS login/password with anyone;

I understand the need for the maintenance of the strictest test security and confidentiality. I promise that I will not, directly or indirectly, in any way compromise the security of any examination materials. I will not allow any person other than an authorized supervisor of the test to have access to, look at or be informed about any examination materials or test results. I know of no past breach of confidentiality and agree to report any breach to Aerosolutions that I witness or come to know about. I hereby agree to perform, and will be held responsible for, all the necessary duties described above and those described in the Test Administration procedures set out in this Manual.

Name:

Position: Organisation:

Signature: Date:

Appendix 5 – Data Privacy


DATA PRIVACY Aerosolutions considers the use and management of all personal information to be of the utmost importance. All employees involved in the conduct of testing at Aerosolutions are committed to the most stringent of privacy policies. Further to this management philosophy and our commitment to the proper use and administration of personal information, our privacy policy, subject to further improvement, is as follows:

1. We adhere to all applicable legal statutes pertaining to the handling, management and use of personal information;

2. No personal information shall ever be used for any purpose other than written in the language testing basic agreement;

3. We adhere to proper administration and information management practices at all times, while establishing policies directly related to the protection of personal information, including:

a) Implementation of strict measures designed to ensure the

security of personal information and to prevent the disclosure, loss, alteration or misuse of personal information; and

b) Implementation of safeguards to restrict access to our database of personal information, in order to maintain a secure environment for governing the management of personal information.

4. Aerosolutions shall under no circumstance release or provide to a third party any personal information, with the exception of the following circumstances:

a) We are compelled to do so under a pertinent legal statute; b) We have obtained the consent of the user in question prior to the

release of such information; c) We have processed information which cannot be specified by

the individual in question. .

Appendix 6 – ELP Test Procedure


ELP Test Procedure – Check List

TEST ROOM SETUP

Separate workstations

Adequate workspace

Noise level

Temperature

Lighting

ELP Testing Center Home Page on screen

Audio equipment working properly

Video equipment working properly

Internet connection working properly

‘Testing in progress’ sign placed

Provide pencils

REGISTRATION

Registration email

Verify User account

Verify Payment

CHECK IDENTITY

Ask for Passport/ ID / License

Ask test taker to fill-in test sign-off sheet

PREPARE TEST SETUP

Schedule ELP Practice Test

Appendix 6 – ELP Test Procedure


Schedule ELP Test

Print Test Book

GIVE BRIEFING

Introduction to the ELP Test

Test security

Cheating

Personal belongings

Duration of the Test

ELP Practice Test

Incorrect test items

Leaving the test room

Emergencies (fire etc.)

Appeal procedure

TEST PHASE

Show test taker how to log on

Start Practice Test

Start ELP Test

AFTER THE TEST

Collect Test Book

Fill-in Test Sign-off Sheet

Store Test Book

Documents

English Language Proficiency Test - Aerosolutions · The English Language Proficiency Test, or ELP Test, has been designed by Aerosolutions to validate the English proficiency level