Title Page HeliStat™ An Automated Risk Assessment Tool for Helicopter Safety Management Systems...

HeliStat™

An Automated Risk Assessment Tool for Helicopter Safety Management Systems

Alex Richman

AlgoPlus Consulting Limited

Halifax NS Canada

Arichman@algoplusaviation.com

Leonard MacLean

Dalhousie University

Halifax NS Canada

T SEPT 28 Copyright©2005 AlgoPlus Consulting

AbstractSafety Management System (SMS) developers have challenges in:Becoming proactive

Assessing risks systematically

Prioritizing

This presentation describes HeliStat™, a computerized system which automatically performs these functions for mechanical malfunctions (SDR and MMIR) and operational incidents (CADORS)

The Evolution of Safety Culture(and Safety Management Systems)

1. PATHOLOGICALwho cares as long as we’re not

caught

2. REACTIVESafety is important, we do a lot every time we have an accident

3. CALCULATIVEwe have systems in place to

manage all hazards

4. PROACTIVEwe work on the problems that

we still find

5. GENERATIVEsafety is how we do business

round here

Increasing vigilance for

potential hazards

Systematic knowledge about operation of system as a whole.

Adapted from Hudson , Leiden Univ, 2004

SMS requires Data

SMS ImplementationThere are multiple approaches to SMS

Transport Canada’s Guide is more detailed than others in specifying the process (TP14343)

Phase 1: Plans and procedures

Phase 2: Reactive system (3.1)

Capture internal data

Monitor/analyze trends

Phase 3- Proactive hazard identification from external sources(3.2)

Risk assessment (3.4)

1. Because of their larger size, system-wide databases are far more likely to yield information about events which might occur later in a smaller fleet

2. Larger data bases are more likely to reveal trends

3. Benchmark comparisons are possible

Advantages of Proactive Analysis

Reactive/ Proactive Dichotomy

Reactive ProactivePost facto Identify potential hazards“Tombstone mentality”Investigation of event

Microscopic

Wariness General analysis of systemMacroscopic

“Smoking gun” Chain of risk factorsForensic cause Probabilistic

Richman & Murphy 2005

Sources for Proactively Identifying Potential Hazards

•Company experience: reports, minutes•Managers’ perceptions•Workforce opinions•Audit reports•Previous hazard analyses•Hazards identified by other organizations

Safety and reliability data recording systems:

CADORS, BASIS

SDR, MMIR

TC TP 1381E

Service Difficulty Reports: •Federal Aviation Administration and Transport

Canada•Mandated•“Mechanical problems, defects, malfunctions affecting safety”

Mechanical Malfunction Information Reports:•Helicopter Association International •Voluntary•Broad definition

Web sites are useful for locating specific reports but are less useful for risk assessment and trend assessment

SDR and MMIR Reports:Mechanical and Maintenance

Questions about Safety-Significant Events (SSE)

• Are there significant increases over time? • Are SSEs “consistent” over time (SPC)?• Which SSEs are more severe and more likely to

occur (risk assessment)?• Are SSEs more likely in one model or operator?• Are there historically unusual SSEs?• Which SSEs should have more priority for further

attention?• How can we identify SSE precursors ?

GAIN Initial Survey of 15 Safety Officers

– Too much time spent on routine tasks

– Little time for proactive safety efforts

– Follow-up is not always well tracked

– Need outside benchmarking

– Need better analytical tools

– Clear safety strategy infrequent (1/15)

Problems in SMS Analyses

•Complexity of data preparation and updating

•Difficulties in integrating data and software into IT infrastructure

•Need for computer proficiency

•Acquiring analytic skills

•Lack of time

Lots to Do: Too Little Analysis

• Easy to use

• Contain proactive data bases

• Identify hazards and assess risks(Likelihood, severity, trends)

• Indicate priorities

• Generate documentation

• Monitor system continually

SMS Software Requirements

Continuous Monitoring and Automated Analysis

• (Morningstar is a financial example)– Continuous monitoring of updated databases

– Near-real-time reporting of updates to benchmarks, trends, rankings, priorities

– – “Pushes” analytic results to user

– Very user-friendly for SMEs

HeliStat™ uses this model•

What is HeliStat™?

A subscription service

for web-based analyses

of mechanical malfunctions

reported to the FAA- Service Difficulty Reports

and to Helicopter Association International’s

Mechanical Malfunction Information Reports

TM

HeliStat™ is Designed for Aviation Specialists Without

Data Analytic Skills

HeliStat™

• Incorporates multiple databases SDR/MMIR• Near-real-time data• Uses census denominators • Compares models/operators with industry wide

benchmarks • Model specific • Source data accompanies analyses• Prioritizes problems• Shares and tracks reports

HeliStat™ is a System

E-mail list formodel by JASC

1

Threat Matrix2

Sourcereport3

Share Report4

Out mail list5

Priority ALERTS in E-mail

A. Risk assessment traffic light summarizes Threat MatrixB. PrioritizedC. Recent reports, past 5 weeksD. Reports not yet viewed

A

BD

C

Systematically DefinedRisk Matrix

RelativeLikelihood

Significantly aboveaverage

Below average

HazardMinimal Increased

Reliable and ObjectiveRisk Matrix Calculation:

•Relative likelihood of occurrence:

Rate in the specified model Benchmark rate for all the other models

•Hazard is derived from the FAA Severity score for specific ATA/JASC codes and the Flight Safety Foundation’s CARE project for classifying “Nature of Condition” and “Precautionary Procedures” in SDRs

3 Graphs Combined to Show Time Trends

Time Trends in One Risk Assessment Matrix

Risk Matrix Summarized in Traffic Lights

•Relative likelihood•Hazard level•Time Trends•Recent Increase

Risk Matrices forIndividual Parts

• No limit on number of parts

Individual Narrative Reportsfor 6 Months

Historically Unusual Events:Unique, First, Rare, Cluster

HeliStat™ Overcomes Many Pitfalls

in Risk Assessment

• Objective, quantitative assessment • Specified time span • Indicates trends over time - trajectory • States nature of the hazard • Shows source narratives• Automated without specialist assessment• Sensitive to short-term changes• Continuous monitoring

HeliStat™ System for Sharing Reports

The user can e-mail the JASC threat matrix to otherswithin the organization.

The recipient has full access to all linked data for this JASC only

HeliStat™ Share Message

The sender can state why s/he is sending this report, ask a specific question or request a general comment.

ALERT keeps track for the sender:•Recipient •ATA/JASC code•Model•Date sent•Date opened

Unopened link

HeliStat™ Maintains Out-Mail List and Tracks Recipients’ Opening Dates

HeliStat™ Enables Early Decision Making

1. For the model and each ATA/JASC component HeliStat™ gives the user :

• Priority traffic light

• Risk assessment matrix

• Time trends

• Historically unusual occurrences

• Source narratives

2. Users have more time to target their activity

and to explore source data

HeliStat™ Analyses Give More Time for SMEs

to Use Expertise

Possible Reactions to HeliStat™ Information

• Problem already known and taken care of• Check fleet now/soon• Consider preventive suggestion• Modify procedure: installation/inspection• Modify inspection: method or frequency• Recommend as audit topic• Place on MEL • Await further information, tickle

HeliStat™- Features• Point and click – no programming required• Push technology- subscriber specifies model,

frequency of reports• Mechanical problems affecting safety in 2 data

bases(SDR, MMIR)• Near real time- reports daily, weekly, monthly• E-mail for newly reported events, • Risk matrices for ATA/JASC system – 488 codes• Trajectory: likelihood and severity over 3 time periods• Likelihood rates compared to system-wide benchmarks• Likelihood and hazards quantified• Past history since 1/1/1996• Narratives of reports for past 182 days

SMS becomes more effective when HeliStat™ overcomes SMS challenges for :

• Becoming proactive

• Risk assessment

• Prioritization

Making SMS more effective helps move the organization towards achieving a safety culture

HeliStat™ Helps a Safety Culture Evolve Faster

HeliStat™ References

1. Rotor magazine (Winter 2004-2005, 42-43

2. Air Beat (January-February 2005), 42-43

3. Aviation Maintenance, December 2004, 34

4. Air Safety Week, September 27, 2004

5. www.helistat.com

AlgoPlus Consulting Limited

Suite 3085670 Spring Garden RoadHalifax, NS, Canada B3J 1H6http://www.helistat.com

fax: 1-902-484-7061email: arichman@algoplusaviation.com

telephone: 1-902-423-5155 1-888-ALGOPLUS

Questions ?