T HE A IRMAN C ERTIFICATION S TANDARDS T HE A IRMAN C ERTIFICATION S TANDARDS

THE AIRMANCERTIFICATION STANDARDS

THE AIRMANCERTIFICATION STANDARDS

Certification RequirementsCertification Requirements

For each airman certificate or rating, 14 CFR lists required areas of:

•Aeronautical Knowledge•Flight Proficiency

There is nothing that currently links aeronautical knowledge and flight proficiency in a cohesive manner.


Flight Proficiency:

•Practical Test Standards (PTS) provide metrics to evaluate flight proficiency in each Area of Operation and Task.

•“Special Emphasis” areas have no defined guidance metrics to evaluate “soft skills.”


Aeronautical Knowledge:

•There has never been a corresponding set of “Knowledge Test Standards” (KTS) metrics to evaluate aeronautical knowledge elements tested via “the written” exam.

Rationale for ChangeRationale for Change

As an instructor or DPE, do you believe the knowledge test accurately determines an applicant’s knowledge?

Do you think the knowledge test has shortcomings? If so, what are they?

Do you thinks the knowledge test determines real skills necessary to operate safely in the NAS?

If you had the opportunity to change the knowledge test, what would you change?


Lack of a KTS has allowed the accumulation of too many FAA knowledge test questions that are:

•Out of date (lots of NDB*, not much RNAV)

•Overly complex (multiple interpolations required to calculate a 2 knot difference in wind or landing distance within 3 feet)

•Irrelevant (number of satellites in the GPS constellation)

•Disconnected from “real” skills and knowledge required for safe operation in today’s NAS

* Note: as of February 9, 2015, the FAA has deleted ADF/NDB questions.


Until now:

•There has been a disconnect between the knowledge test and the practical test.

•There has been limited guidance on ways to evaluate “soft skills.”

•There has been no way to pinpoint an applicant’s areas of deficiency in knowledge, skills and/or risk management.


“If it ain’t broke, why fix it?”

Many in the training industry believed that the way airman

were tested was indeed broken.


How do we quantify “Special Emphasis” areas?

PTS states that “examiners shall place special emphasis upon areas of aircraft operations considered critical to flight safety.” Among these are:

1. Positive aircraft control 9. ADM and risk management2. Positive exchange of the flight controls 10. Wire strike avoidance3. Stall/spin awareness, 11. Checklist usage4. Collision avoidance 12. Temporary flight restrictions5. Wake turbulence avoidance 13. Special use airspace6. LAHSO 14. Aviation security7. Runway incursion avoidance 15. Single-Pilot resource management8. CFIT 16. Other areas deemed appropriate

In today’s structure, a given special emphasis area may not be specifically addressed under any particular Task. All areas are essential to flight and will be evaluated during the Practical Test.


How do we determine airman knowledge deficiencies?

Learning Statement Codes:• PLT 116 Recall aircraft general knowledge/ publications/

AIM/ navigational aids• PLT 208 Recall emergency conditions/procedures• PLT 235 Recall human factors (ADM) – judgment• PLT 381 Recall regulations – altimeter settings• PLT 383 Recall regulations – basic flight rules• PLT 395 Recall regulations – definitions• PLT 415 Recall regulations – IFR flying

PTSACS

What is the ACS?What is the ACS?


Aeronautical knowledge

Flight proficiency

Aeronautical decision-making and

special emphasis

ACS also includes unique codes for each element of knowledge, skill, and risk management.

PA = Private Pilot Airplane (defines applicable ACS)V = Performance Maneuvers (defines Area of Operation)A = Steep Turns (defines Task)K5 = Accelerated Stalls (defines element)


• ACS codes will replace Learning Statement Codes (LSCs).

• ACS codes are anchored in the standard, not in references like LSCs.

• ACS codes enable FAA to align standards to handbooks and test questions, to maintain that alignment, and to develop better test questions.

• ACS codes provide sharper, more focused feedback to applicants, instructors, and evaluators.


Doug Stewart

PA = Private Pilot Airplane (Applicable ACS)III = Airport & Seaplane Base (Area of Operation)B = Traffic Patterns (Task)K4 = Right of Way Rules (Element)

Today’s test report lists reference-based LSCs that are often very broad – creates an obstacle to effective remedial training and retesting.

CODE COMPARISONCODE COMPARISON

A Brief HistoryA Brief History

Project HistoryPhase I – 2011-2012

•The ARC (Aviation Rulemaking Committee)

• FAA chartered the Airman Testing Standards and Training ARC to obtain expert industry and stakeholder advice and recommendations for improving content, process, methodology, and priorities for improving airman knowledge testing.

• The ARC developed the ACS concept and

recommended that the FAA pursue it.

A Brief HistoryA Brief HistoryProject HistoryPhase II – 2012-2013

•The FAA tasked the industry Aviation Rulemaking Council (ARAC) to develop the ACS concept.•ARAC formed the Airman Testing Standards and Training Workgroup (ATSTWG) to perform this work.•The ATSTWG developed ACS documents for Private, Commercial and Instructor certificates & Instrument Rating Plus “baseline” proposal for an ATP ACS•PVT, IFR & CFI ACS submitted for public comment•Report and recommendations submitted to FAA in September 2013

A Brief HistoryA Brief History

Project HistoryPhase III – 2014-2015

•The FAA tasked the industry ARAC to further advance the ACS concept.•In March 2014, ARAC formed the Airman Certification System Working Group (ACS WG) to perform this work.•Ongoing tasks:

• Refine and Complete ACS for COM, ATP & CFI• Help FAA map standards to guidance• Prototype use of PVT (PAR) ACS in Orlando area, with strong

support from AFS-800 (policy memo) and ORL FSDO (assisting with ACS WG efforts).

A Brief HistoryA Brief HistoryProject HistoryPhase III – 2014-2015Prototyping:

• Initial effort with ERAU Summer Academy – June 2014: positive feedback from instructor, evaluators, and ORL FSDO inspectors (still attempting to gather student feedback).

• ACS WG & ORL FSDO are collaborating on a larger prototype to include part 61 instructors, applicants, and DPEs. Goal is to develop foundation for future 8900 guidance and ACS job aids.

• Exploring prototype expansion in 2015, with a view to “real” implementation of at least one ACS (likely PVT)

in late 2015 / early 2016

FAQsFAQs

What’s wrong with the PTS?

The PTS provides metrics to define acceptable performance of the “flight proficiency” skills listed in 14 CFR part 61 for a given certificate or rating. Most people believe that the PTS generally serves its intended purpose but, like all such documents, it has become bloated over the years with an ever-expanding list of “special emphasis” items, repetitive or overlapping Areas of Operation/tasks, and poorly-defined additional requirements (e.g., evaluation of the applicant’s risk management and aeronautical decision-making skills).

FAQsFAQs

How does the ACS improve the PTS?

The ACS approach does not increase or expand any of the skill evaluation requirements in the existing PTS, but it significantly improves the PTS. The ACS: •Provides integrated guidance that defines performance metrics for aeronautical knowledge as well as flight proficiency (skill).

•Strengthens the PTS by explicitly defining the aeronautical knowledge needed to support each Area of Operation/task. This linkage enhances the relevance of the testing/training process for adult learners by clearly answering the “why do I need to know that?!” question.

•Enhances safety by using the risk management section in each ACS Area of Operation to translate abstract terms like “aeronautical decision-making” into specific safety behaviors relevant to each task.

•Eliminates “bloat” by consolidating duplicative or overlapping PTS tasks.

FAQsFAQs

Why does the ACS have a separate section for risk management? Isn’t that just the latest buzz word?

The PTS already requires evaluation of the applicant’s risk management abilities, but the existing document doesn’t offer the kind of concrete “what do I have to do?” guidance that users need and deserve. The rationale for including a risk management section in the ACS is to enhance safety by translating abstract terms into specific safety behaviors relevant to each task.

The ACS is also intended to communicate and demonstrate that risk management is a continuous process that includes identification, assessment, and mitigation of task-specific hazards that create risk. The risk management element identifies the circumstantial issues that aviators must consider in association with a particular task.

FAQsFAQs

How does the ACS approach improve the knowledge test?

Accepted best practices for any certification process stipulate that training and testing be based on a job/task analysis. The ACS documents function as the required job/task analysis, because they define the knowledge and skills needed to perform at the level of the target certificate or rating. In this way, the ACS approach better serves the applicant, the instructor, and the evaluator.

Because the process of developing the ACS requires a thorough review and update of knowledge and skills for airman certification, it aligns with certification industry standards for periodic review and revision of the job/task analysis. In addition, the ACS approach will enable the FAA to create and maintain a clear link between the regulations, knowledge/skill performance standards, guidance, and test materials.

FAQsFAQs

How will you provide the “clear link” connecting knowledge/skill performance standards, guidance, and test materials?

The ACS coding system provides the mechanism for establishing and maintain alignment of standards with guidance and test questions. ACS-based codes:

•Clearly align guidance and test questions to the ACS;

•Make the airman test report meaningful to stakeholders (applicant, instructor, evaluator);

•Provide a means for automated generation of tests, whether using the existing test forms or future randomized selections; and

•Eliminate subjectivity and vastly simplify system management requirements for the FAA.

FAQsFAQs

How will the use of the ACS approach change airman training?

With clearly defined standards for knowledge, skill and risk management, airman training can be conducted more effectively to ensure that applicants who complete flight and ground training are safe, competent aviators as well as successful in passing the FAA knowledge test.

Training and testing will be aligned, which means that “test prep” will be a review of the ground school curriculum rather than a separate, unrelated step to learn questions for the sole purpose of passing a test.

FAQsFAQs

Won’t the ACS make the knowledge test a lot harder?

No. In fact, applicants who have been training properly all along are likely to find the FAA knowledge test a lot easier, for several reasons.

•First, the knowledge test will be a reflection of the training applicants receive in ground school. Instead of having to “learn” or memorize information that isn’t relevant to knowledge and skills actually needed for operation in today’s National Airspace System, the test will consist of questions coded (via that ACS) to specific Areas of Operation/Tasks.

•Second, ACS-coded questions will make it much easier for applicants, instructors, and evaluators to specifically identify and retrain on weak areas.

•Overall, the ACS-enabled correlation of training and testing will make the process easier because knowledge testing will be meaningful and relevant – not an exercise in memorizing information you will never need in real-world operations.

FAQsFAQs

Won’t the ACS approach dramatically increase the length (and expense) of the practical test?

No. In fact, a more integrated and efficient presentation of the material to be tested could even shorten the test, especially if the evaluator has more confidence in the quality and meaning of the applicant’s knowledge test score. Evaluators will be able to effectively and efficiently re-test any deficient knowledge identified on the airman knowledge test report to ensure the applicant has trained to proficiency in all areas.

FAQsFAQs

How will the ACS change the way I teach ground and flight training?The ACS should help you teach more effectively, and more efficiently, in both activities:

•The integrated ACS approach clearly shows how knowledge, skills, and risk management are connected for any given Area of Operation/Task. With clearly defined standards, training can be conducted more effectively to ensure that applicants are safe, competent aviators as well as successful in passing the FAA knowledge test.

•Training and testing will be aligned, which means that “test prep” will be a review of the ground school curriculum rather than a separate, unrelated step to learn questions for the sole purpose of passing a test.

•Defining the terms and concepts now presented in a list of “special emphasis” items in the PTS introduction, and placing them in the right context, will simplify teaching these topics. The presentation of risk management enhances safety, and it can also contribute to much greater standardization in teaching and testing these concepts.

FAQsFAQs

How often will the ACS be updated?

The FAA intends to establish a regular, predictable schedule for updates to the ACS and all other components of the airman certification system. These include the ACS itself, the H-series handbooks, and knowledge test materials. The agency will establish this update schedule in consultation with industry members of the ARAC Airman Certification System Working Group.

Industry members of the ACS WG will play an active role in the writing and review of knowledge test questions as well as the H-series handbooks.

FAQsFAQs

How will the ACS affect the way I conduct a practical test?

The ACS does not change the practical test, and it does not make the practical any longer than it takes to conduct a PTS checkride today. In fact, the ACS could expedite the practical test because it streamlines and combines overlapping and redundant tasks in today’s PTS, and because it gives the evaluator more focused information on knowledge and risk management elements associated with each skill task.

In addition, the ACS codes (which will appear on the Airman Knowledge Test Report in place of today’s LSC codes) give the evaluator much more specific information on items the applicant missed on the knowledge test.

FAQsFAQs

How will the ACS affect the way I conduct a practical test?

The FAA does NOT expect the evaluator to cover every single knowledge and risk management element for each task. Rather, the expectation is for the evaluator to address ONE knowledge element and ONE risk management element in each Task.

As is the case today, however, the evaluator has discretion to test additional knowledge and risk management elements if the Airman Knowledge Test Report and/or the applicant’s response to questions suggests the need for more thorough coverage.

FAQsFAQs

What document should we use to retrain and evaluate an applicant’s areas of deficient or weak knowledge?

One of the strongest tools that the industry team developed for the ACS framework is a new coding system. When the ACS approach is implemented, ACS codes will replace the Learning Statement Codes (LSCs) that are used right now.

The ACS codes have several very strong advantages over the Learning Statement Codes. The ACS codes are anchored in the standard – not in handbooks and other reference documents, like today’s Learning Statement Codes. ACS codes provide sharper, more focused feedback to applicants, instructors, and evaluators.

FAQsFAQs

Will the ACS codes point me to the exact text in the handbooks and other references?

No, because the ACS codes will point you to the specific area of missed knowledge in the standard, rather than to a handbook chapter or section. Because they are anchored in the standard – not in handbooks and other reference documents, like today’s Learning Statement Codes – ACS codes provide sharper, more focused feedback to applicants, instructors, and evaluators.

In SummaryIn Summary

The ACS does not change the checkride, but it gives the evaluator more focused information on:

• Knowledge and risk management and special emphasis elements associated with each skill task.

• Specific information (via ACS codes) on items the applicant missed on the knowledge test.

Except to streamline and combine overlapping and duplicative tasks (e.g., runway safety), the ACS does not change PTS elements or performance metrics.

The FAA reviews each industry-developed ACS to validate its content and ensure that all PTS elements are included.


By offering an integrated, holistic, and systematic approach to standards, guidance, and testing, the ACS:

•Provides better guidance to applicants, instructors, evaluators, and inspectors.

•Allows the FAA to develop better test questions that are clearly tied to standards (“need to know”) and supported by guidance (handbooks).

•Reduces subjectivity and increases system-wide standardization.

•Enhances safety by ensuring that standards, guidance and testing for airman certification all work together effectively.


In this presentation we discussed:

•Rationale for change to the ACS•The differences between PTS and ACS•A history of the process•Frequently asked questions

For More InformationFor More Information

For more information:

https://www.faa.gov/training_testing/testing/

FAASafety Teamwww.FAASafety.gov

SAFEwww.SafePilots.org

This presentation would not have been possible without the generous help and

support of the following:

Bob Newell, Manager, Airman Testing Standards Branch (AFS-630)

[email protected]/training_testing/testing/

Susan Parson, Special Technical Assistant, Flight Standards Service (AFS-3A)

[email protected]

Documents

T HE A IRMAN C ERTIFICATION S TANDARDS T HE A IRMAN C ERTIFICATION S TANDARDS