Embed Size (px)
Citation preview
Evaluating New Hire Pilot Attitude and
Behavior with Probationary
Assessments
Arianna Hoffmann
Human Capital Management & Performance
29 August, 2017
Why is the probationary period becoming
more critical?
Pilot Supply < Pilot Demand
Regulatory Changes
US Pressures: Age 65
Retirements at Majors
As it becomes more challenging to bring pilots through the
front end of the selection process, evaluating new hire pilots for
safety and success becomes increasingly significant
Pilots will be increasingly recruited from non-traditional sources
with varied backgrounds and experience
Probation is the most extended element of the selection process
3
Recruitment/Application
Screening
In person screening
(Testing, Interview,
Applied Exercises)
Job
Offer
ProbationTraining-IOE-Performance on the line
Drug Test
Background
Check
Pilot Selection
Selection Process Philosophy
KSAPsCapabilities:
Pre-hire
Performance:
Post-hire
Job Content
Knowledge
Personality
Characteristics
Cognitive
Skills
Training
Performance
Attendance
Disciplinary
Actions
Predictive Validity for Pilot Selection
Actual
problems * * * * * * * * * * * * * *
Predicted
problems
.
0
1
.
0
2
.
0
5
.
1
0
.
1
7
.
2
2
.
2
4
.
2
8
.
3
0
.
3
1
.
3
3
.
3
6
.
4
1
.
4
4
.
4
7
.
4
8
.
5
0
.
5
1
.
5
4
.
5
5
.
5
7
.
5
8
.
6
0
.
6
2
.
6
5
.
6
7
.
6
8
.
7
0
.
7
5
.
7
6
.
7
9
.
8
0
.
8
6
.
8
7
.
8
8
.
9
6
.
9
7
Attorney Client Privileged communication 5
1. The Predictive Statistical ModelPredicted Problems = A(“X” Cognitive ability measures) + B(“Y” Aviation
knowledge/problem solving measures) + C(“Z” personality measures)
2. Comparison of of the predicted performance versus actual
Red Predicted
Problems No problems
Real Problems 11 3
No Real Problems 6 17
BlackPredicted
Problems No problems
Real Problems 8 6
No Real Problems 1 22
3. Optimizing the cost benefit
But are we hiring safe pilots?• Engagement
• Understanding of
principles
• Ability to multi-
task
• Teamwork
• Leadership
• Accountability
• Dedication to
continuous
learning
All of these behaviors and capabilities should be demonstrated during probation and can be
measured, but many are not
Traditional Monitoring During ProbationData available during this time period typically falls
into two categories:◦ Approved and monitored with specific regulatory
reporting criteria
◦ Reports from Chief Pilots, Captain’s evaluations or other disciplinary letters
Although this information serves an important role for specific stakeholders, it is often treated as the basis for evaluation of probationary pilots, but this misses an important part of the story
By necessity, training is focused on proficiency
In the US, programs developed to meet FAA standards have some reference to CRM and
HF, but evaluate these things strictly from a procedural basis, i.e. properly following
checklists and leading debriefs
Performance Dimensions Reason Codes
Human Factors
Airmanship
Automation
Procedures
Knowledge
Systems
KnowledgeStick & Rudder
Procedures
Automation
Proficiency
CRM/TEM
The role of attitude in aviation trainingA typical refrain from aviation instructors;
“ I would rather have a student with a good attitude who needed skills improvement than a student with good skills who needed attitude improvement”
This is true for a company making the investment in an employee, too
But conventional methods for evaluating newly hired pilots focus primarily on procedural execution and technical proficiency
Probation assessments broaden the scope
of new hire evaluation Prepared
Undistracted
Shows Dedication
Participates
Exhibits Confidence
Exhibits Passion
Low Entitlement
Takes Critique Well
Collaborates
CommunicatesThese are the desired characteristics a new hire will bring to training and the
cockpit throughout their career
Design recommendations Link to proficiency scale
◦ Proficiency problems won’t cloud behavior evaluation
Use a scale that diverges from the typical training scale
◦ Greater range than error free, error captured
Provide behavioral anchors for scale
◦ Standardization
Focus on type of student, not just type of practitioner
◦ Will they be a lifelong learner?
Evaluate non-validation events
◦ Allows for more interaction, teaching environment
Get repeated measures
◦ Mitigate any personality clashes
Base evaluations on the job description
◦ Hold the pilot accountable for the displaying the capabilities required of the job
Sample Assessment:Student Focus
Does not allow outside distractions to disrupt the training environment (e.g.,
business, family)
1 Far short of Expectations: E.g. Preoccupied with trip home before training even starts. Clearly distracted during training without communicating reason, or withdrawing from session. Attitude that checking social media during training is acceptable. Lack of focus on training disruptive to other students.
2 Does not Meet Expectations: E.g. Personal problems or distractions not managed effectively.
3-Barely Meets Expectations
3 Reasonably Meets Expectations: E.g. Outside distractions not communicated, but have only minor impact on concentration. Engaged with material and not disruptive to other students.
3+ Completely Meets Expectations
4 Exceeds Expectations: E.g. No outside distractions, or aware of distractions, communicates them appropriately, conduct appropriate. Has made arrangements so that outside distractions will not interfere with training.
5 Far Exceeds Expectations: E.g. Intensely focused on training
Deployment recommendations Training is key
◦ Inter-rater reliability and standardization
If the tool can be integrated into IT systems, automated alerts can be built in
◦ Early recognition of potential problems
Performance on this tool can be linked to selection performance
◦ Identify characteristics in candidates before hire, find strengths and weaknesses of selection program
Can compare aircraft types as well as training and IOE
◦ Gives insight into differences across Training and Standards
Linking Performance Back:
NEO PI-R Normal Personality
14
Neuroticism ExtraversionOpenness to
experienceAgreeableness Conscientious-ness
Anxiety Warmth Fantasy Trust Competence
Hostility Gregariousness Aesthetics Straightforwardness Order
Depression Assertiveness Feelings Altruism Dutifulness
Self-consciousness Activity Actions Compliance Achievement Striving
Impulsiveness Excitement Seeking Ideas Modesty Self-Discipline
Vulnerability to Stress Positive Emotion Values Tendermindedness Deliberation
Comparison of IOE outcomes to Training
3.43.53.63.73.83.9
Training Mapped to IOE
3.4
3.5
3.6
3.7
3.8
3.9
Type of Skill: IOE Combined
Please feel free to contact me with
questions
01 919 602 2439
