Upload
estella-moody
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
FAA EDR Standards Project
RTCA Plenary Sub-Group 4 & 6
September 16, 2014
Presented by: Sal Catapano - Exelis
Outline
• Background / FAA EDR Standards Project Report • Current In Situ EDR Algorithms / Aircraft Equipped• FAA EDR Standards Project Overview• 1-Minute Mean In Situ EDR Report Performance and
Recommendations• 1-Minute Peak In Situ EDR Report Performance and
Recommendations• Variability Analysis Findings• Follow-on Recommendations• Open Question / Discussion Period
Background &
Current EDR Algorithms / Aircraft Equipped
Background
• In 2001, ICAO made EDR the turbulence metric standard• In 2011, ADS-B ARC recommended the FAA “establish
performance standards for EDR”• In 2012, RTCA SC-206, developed an Operational Services
and Environmental Definition (OSED) identifying the necessity for:
An international effort to develop performance standards for aircraft (in situ) EDR values, independent of computation approach
• In response, the FAA sponsored an EDR Standards Project from July, 2012 to September, 2014 that:
Provides the analysis, inputs, and recommendations required to adopt
in situ EDR performance standards
FAA EDR Standards Final Report
• Delivered to FAA August 29, 2014• Documents research, analysis,
and findings of project• Includes performance tolerance
recommendations for 1 minute mean and peak in situ EDR reports
• Report and appendices submitted to RTCA on September 10, 2014
6
Panasonic Longitudinal Wind-Based
ATR Accelerometer-Based Input: TAS, Altitude, Vertical Acceleration, Weight, Frequency Response
Users: American Airlines, others
Windowing: 5 sec running window
Average Calc: N/A
Peak Calc: Largest EDR in 30 seconds
NCAR Vertical Acceleration-Based Input: TAS, Altitude, Vertical Acceleration, Weight, Frequency Response, Mach, Flap Angle, Autopilot Status, QC Parameters
Users: United Airlines
Windowing: 10 sec window every 5 sec
Average Calc: Arithmetic mean over 1 min
Peak Calc: 95th percentile over 1 minute
Input: TAS, Roll Angle for QC, TAMDAR Icing for QC (if using TAMDAR Sensor)
Users: TAMDAR - Regional Airlines
Windowing: 9 sec window
Average Calc: 1, 3, 7min; 300, 1500ft
Peak Calc: Largest EDR in 1, 3, 7min; 300, 1500ft
NCAR Vertical Wind-BasedInput: TAS, Altitude, Inertial Vertical Velocity, Body Axis AoA, Pitch Rate, Pitch, Roll Angle, QC, Filter Parameters
Users: Delta and Southwest Airlines
Windowing: 10 sec running
Average Calc: Median over 1 min
Peak Calc: Largest EDR in 1 Minute
Current Operational In Situ EDR Algorithms
Aircraft Equipped
Airline Type Method Count
American and
others
B737-800, B757-200 B767-
300, A320, A321, A330-300 Vertical Acceleration 500+
Delta B737NG, B767 Vertical Wind 167
Southwest B737-700, B737NG Vertical Wind 156
UnitedB757 (EDR equipped B737
no longer in fleet)Vertical Acceleration
54
(reducing to 15 by Dec
31, 2015)
Regional Airlines
via TAMDAR
SAAB 340, ERJ-145, ERJ-
190, ERJ-195, Beech
1900C, Dash 8 (Q-100, Q-
300, Q-400)
Longitudinal Wind
(via TAS)256
Total: 1133
7
FAA EDR Standards Project Overview
Project Team and Key Stakeholders
Standards Research Process
Commercial Output
Simulator Output
Research Quality Data
Raw Vertical Winds
Input Winds Development
• Homogenous – exercise mean EDR
Maintains single EDR on average throughout wind dataset (e.g. 0.5 EDR)
• Non-Homogenous – exercise peak EDR Simulate “burst” of turbulence embedded in
background field of ambient turbulence
X =
Minute 1 Minute2 Minute 3
.5 EDR
Input Winds Datasets
X =
Case Title Case DescriptionInput Wind Datasets Specifications
Turbulence Intensity or
Modulation ShapeLength Scales
Random Phenomena(Homogenous)
Turbulence is a random process, therefore datasets were created to
replicate
5 von Karman
0.1, 0.2, 0.3, 0.5, 0.7 EDR 500 Meters
Varying Length Scale Nonconformance w/
Algorithm Assumptions(Homogenous)
Operational In situ EDR algorithms all assume a 500 meter length scale,
however in real-world conditions the length scale varies, therefore datasets were created with varying length scales
20 von Karman (including the 5 for random)
0.1 0.2, 0.3, 0.5, 0.7 EDR 200 Meters
0.1, 0.2, 0.3, 0.5, 0.7 EDR 500 Meters
0.1, 0.2, 0.3, 0.5, 0.7 EDR 750 Meters
0.1, 0.2, 0.3, 0.5, 0.7 EDR 1000 Meters
NoiseFloor
(Homogenous)
Aircraft sensors and avionics introduce noise to EDR calculations, so datasets
at low EDR severity levels were generated to determine impacts to EDR
calculations
15 von Karman
0.01, 0.02, 0.03, 0.04, 0.06 EDR 200 Meters
0.01, 0.02, 0.03, 0.04, 0.06 EDR 500 Meters
0.01, 0.02, 0.03, 0.04, 0.06 EDR 750 Meters
Nonconformance w/ Algorithm Assumptions(Non-Homogeneous)
In real-world conditions not all turbulence is homogenous, so
modulation was applied to von Karman datasets to simulate convective and
mountain wave turbulence
von Karman at 500 meters and 0.06 EDR background
Narrow/Tall Pulse Shape (SPIKE)A1 = 20
L_Sigma = 500M
Mid/Mid Pulse Shape (MID)A1 = 15
L_Sigma =1000M
Wide/Short Pulse Shape (FLAT)A1 = 5
L_Sigma =1500M
‘Expected’ EDR Value
• Homogeneous Wind Datasets – Mean EDR Single ‘expected’ EDR value corresponding to each
dataset (i.e., .5 EDR dataset has ‘expected’ EDR value of .5 EDR)
• Non-homogeneous Wind Dataset – Peak EDR Spectral Scaling Method developed by the project
used to calculated EDR ‘expected’ values based on individual implementation window length and modulation characteristics
.5 EDR
Expected Mean vs. Sample Mean
Statistical Sets
Sample Mean
Sample Mean
Sample Mean
1-Minute Mean EDR
• Implementations performed well compared with EDR ‘expected’ value
Consistency across turbulence severity levels and length scales, as well as across implementations
• Noise floor for the project’s simulation determined to be at or below 0.02 EDR
Below noise floor bias is high for all implementations Above noise floor bias quickly improves, as signal-to-noise ratio
increases
In Situ EDR Mean Report Findings
Real-world operational noise floors are dependant on avionics and sensor characteristics
.01 .02 .03 .04 .06 0.1 0.2 0.3 0.5 0.70%
5%
10%
15%
20%
25%
30%
35%32.2%
19.4%18.3% 17.6% 17.9% 18.7% 18.9% 19.5%
21.1%22.8%
14.3%
7.5% 7.2% 7.3% 7.3% 7.5% 7.4% 7.5% 8.3% 9.0%
19.1%
4.7%3.2% 2.4% 1.7% 2.1% 2.2% 2.3% 2.3% 2.3%
1-Minute Mean EDR Performance
99%-Band70%-BandBias
Expected EDR Value
P
erfo
rman
ce (
% o
f E
xpec
ted
ED
R
Val
ue)
Performance Category EDR Range Bias 70%-band 99%-band
Supplemental 0.01 - 0.02 +5% +10% +20%
Minimum >0.02 – 0.20 +5% +10 +20%
Minimum >0.20 – 0.70 +5% +10 +25%
Note: Statistics normalized to expected value
1-Minute Mean EDR Performance & Recommendations
.01 .02 .03 .04 .06 0.1 0.2 0.3 0.5 0.70%
5%
10%
15%
20%
25%
30%
35%32.2%
19.4%18.3% 17.6% 17.9% 18.7% 18.9% 19.5%
21.1%22.8%
14.3%
7.5% 7.2% 7.3% 7.3% 7.5% 7.4% 7.5% 8.3% 9.0%
19.1%
4.7%3.2% 2.4% 1.7% 2.1% 2.2% 2.3% 2.3% 2.3%
1-Minute Mean EDR Performance
99%-Band70%-BandBias
Expected EDR Value
P
erfo
rman
ce (
% o
f E
xpec
ted
ED
R
Val
ue)
1-Minute Mean EDR Performance Tolerance Threshold Example
Example: 0.1 Mean EDR Performance Tolerance Thresholds
Bias +5%Expected Value
0.1 EDRBias must be between
0.095 – 0.105 EDR
70%-band +10% Sample Mean At least 70% of results must be
Sample Mean + 0.01 EDR
99%-band +20% Sample Mean At least 99% of results must be
Sample Mean + 0.02 EDR
Note: Statistics normalized to expected value
.01 .02 .03 .04 .06 0.1 0.2 0.3 0.5 0.70%
10%
20%
30%
40%
50%
60%
70%
80%
32.2%
19.4% 18.3% 17.6% 17.9% 18.7% 18.9% 19.5% 21.1% 22.8%
75.0%
60.0%
45.0%
30.0%
20.0%
99%-Band
Wake 99%-Band
Expected EDR Value
P
erfo
rman
ce (
% o
f E
xpec
ted
E
DR
Val
ue)
Wake Vortex Decay Modeling Performance Objectives (99%-Band)
Note: Wake decay modeling community has performance objectives for null to light turbulence
1-Minute Peak EDR
5 Sec Window
10 Sec Window
Window Length (Meters)
ED
R E
xpec
ted
Val
ue
Spike
Mid
Flat
Standard Deviation Increases
.7
.5.6
Window Length Flat Mid Spike
5 Second Window
0.32 0.51 0.70
8 Second Window
0.28 0.42 0.56
10 Second Window
0.26 0.38 0.50
Bias IncreasesExpected Value
Decreases
1-Minute Peak EDR Expected Value
‘Representative’ Expected Value
Performance Recommendations for1-Minute Peak In Situ EDR Reports
Statistic Set Performance (FLT) Performance (MID) Performance (SPIKE)
1Bias +15.6% +18.3% +23.2%
270%-band +23.8% +26.9% +35.7%
299%-band +65.6% +69.2% +82.9%
Statistic SetRecommended Peak
Standard (FLT)Recommended Peak
Standard (MID)
Recommended Peak Standard
(SPIKE)
1Bias +20% +20% +25%
270%-band +25% +30% +40%
299%-band +70% +70% +85%
1Bias is normalized to the “representative” expected value2 70% and 99% bands are normalized to the “window length specific” expected value
.51Bias Increases
Bias Increases
1-Minute Peak EDR Example
Window Length (Meters)
ED
R E
xpec
ted
Val
ue
Spike
Mid
Flat
5 Sec Window
10 Sec Window
.38
.445.51
Example: MID Dataset Peak EDR Performance Tolerance Thresholds
Bias +20%‘Representative’ Expected Value
0.445 EDR
Bias must be “Representative’ Expected
Value + 0.089 EDR
70%-band +30%
9 Second Window Specific Expected Value
0.40 EDR
At least 70% of results must be Window Specific
Expected Value + 0.12 EDR
99%-band +70%
9 SecondWindow Specific Expected Value
0.40 EDR
At least 99% of results must be Window Specific
Expected Value + 0.28 EDR
‘Representative’ Expected Value
Today • Current user applications employ peak EDR data only for
strategic forecasting and planning Interested in receiving numerous peak EDR reports User objectives for peak EDR are subjective and require further
sensitivity analysis
Future• EDR used in tactical decision making (e.g., cross-linking
between aircraft) Improved inter-implementation consistency may be required across
EDR implementations Today’s peak EDR performance may not meet future use
performance requirements
Peak EDR Performance Objectives
In Situ EDR Peak Report Findings
• All implementations performed well compared with EDR ‘expected’ value
However, all existing in situ EDR implementations employ different window lengths, parameter settings, and calculation methodologies
Some of these component differences lead to inconsistent peak EDR values across implementations in non-homogeneous turbulence (i.e., convective, mountain wave)
• Project elected to perform variability analysis on a set of EDR algorithm components to discover sources of variability
Variability Analysis
Scatter Plot of Results Consistency Performance Curve
Algorithm Parameters included in Variability Analysis
Input Type Window Length Window Function
Window Overlap Lower Cutoff Wavenumber Upper Cutoff Wavenumber
Consistency Improvement Potential
Follow-on Recommendations
• Performance standard adoption Validate in situ recommendations Determine how compliance will be enforced
• Define operational requirements Pursue broad ConOps for EDR Perform application specific sensitivity analyses
• Continue variability analyses Research additional algorithm components Define parameter values for all components
• Pursue additional research into the science of EDR Analyze impact of distorting assumptions Define an approach to develop vertical EDR profiles
• Consider non-in situ EDR performance standards
Leverage momentum of Project
Team’s Success
Follow-on activities MUST have operational significance and benefit
Questions / Discussions
Name Phone Email
Michael Emanuel 609-485-4873 [email protected]
Joe Sherry 571-224-8926 [email protected]
Sal Catapano 301-867-2879 [email protected]
Back-up Slides
Inertial Sub-Range
Work Element Relationship
EDR Standards Process
Algorithm Input Data Development
Modulation Pulse
Notional Depiction of Relationships Associated with Differing Window
Length
Window Length (Seconds)
1 2 3 4 5 6 7 8 9 10
SPIKE Dataset Expected
Values1.19 1.05 0.90 0.79 0.70 0.64 0.60 0.56 0.53 0.50
MID Dataset
Expected Values0.65 0.63 0.59 0.55 0.51 0.48 0.45 0.42 0.40 0.38
FLT Dataset
Expected Values- - - - 0.32 - - 0.28 - 0.26
Spike, Mid, and Flat Expected Values
ISE Comparison of Project Generated Non-homogenous Wind Data Sets
99%-band
+49.5%-49.5%
Example: 0.1 Mean EDRBias
Sample Mean
70%-band
-35% +35%
Sample Mean
Expected Value = 0.1 EDRResults Center = 0.1021 EDRPerformance Stnd = 0.105 EDR
Sample Mean = 0.1 EDRRight Tolerance Band = 0.1075 EDRLeft Tolerance Band = 0.0925 EDRPerformance Stnd = At least 70% of results must be between 0.09 EDR and 0.11 EDR
Sample Mean = 0.1 EDRRight Tolerance Band = 0.1187 EDRLeft Tolerance Band = 0.0813 EDRPerformance Stnd = At least 99% of results must be between 0.08 EDR and 0.12 EDR