GAES Advanced Emissions Model v1 - · PDF file3.4 Fuel flow analysis ... (A310-325, B737-300 and B737-700), was published in 2004. Both appendices (namely AEM3 validation exercise#2

GAESAdvanced Emissions Model v1.5

Validation Exercise 3

EEC/SEE/2006/005

GAES - Advanced Emissions Model v1.5 Validation Exercise 3

EEC/SEE/2006/005 ii

GAES: Advanced Emissions Model v1.5 Validation Exercise 3 Issue 1.2 Prepared for EUROCONTROL Experimental Centre by: GAES project team under contract with ‘ISA Software’: Authors: ENVISA: Sandrine Carlier, [email protected] James Smith, [email protected] EUROCONTROL Project Manager: Frank Jelinek: [email protected]

© European Organisation for the Safety of Air Navigation EUROCONTROL 2006 This document is published by EUROCONTROL in the interest of the exchange of information. It may be copied in whole or in part providing that the copyright notice and disclaimer are included. The information contained in this document may not be modified without prior written permission from EUROCONTROL. EUROCONTROL makes no warranty, either implied or express, for the information contained in this document, neither does it assume any legal liability or responsibility for the accuracy, completeness or usefulness of this information.


EEC/SEE/2006/005 iii

EXECUTIVE SUMMARY

AEM3 validation exercise#3 builds on the AEM3 validation exercise#1 completed in early 2004, as did validation exercise#2 late in the year 2004. It addresses four additional aircraft types, namely B737-300, B737-500, B757-200 and B767-300. The data set used for this supplementary validation exercise consists of flight data recordings collected from a European airline. The data granularity was depending on portions of flight:. Predefined intervals between two flight points were of 5 seconds at low altitude and around 30 seconds during cruise. Two flight completion options were addressed during AEM3 execution: no completion and full completion. Like the first and second AEM3 validation exercise, the current exercise indicates a very good ability to estimate fuel burn. The resulting average fuel ratio for the whole traffic sample (i.e. four aircraft types) indicates that AEM3 underestimates fuel burn by only 3% when no flight profile completion is required. In case of flight profile completion, the resulting AEM3's error is an overestimation of fuel burn by 10%. Nevertheless, if the error on duration due to flight completion is isolated, AEM3 seems to underestimate fuel burn by only 0.81%. Results obtained during previous AEM3 validation exercises with FDR data are thus perfectly confirmed and the necessity of using as complete flight profiles as possible in input is once again demonstrated. This validation exercise#3 also corroborated the influence of TOW on the accuracy of fuel burn estimation through AEM3. Results confirms findings from validation exercise#1, that is to say AEM3 slightly underestimates fuel burn for flights with a high take-off weight. The results for the 2708 flights of the data set are visualized below (no flight completion.

Figure 1: AEM3 fuel burn vs. operational fuel burn for 2708 flights (no flight completion)

During AEM3 validation exercise#1 and #2, significant differences were highlighted for low and high thrust fuel flows compared to fuel flows documented in the ICAO Engine Exhaust Emissions Data Bank ([Ref 1.]) and BADA data. Similar differences were observed for the four supplementary aircraft types under study, leading to the same errors.


iv EEC/SEE/2006/005

The influence of many parameters on emissions for the specific data set exploited for this third validation exercise led to an increase of the proportion of CO and HC regarding NOx for all aircraft but B767-300. This was the case for FDR data during the first validation exercise as well, which was based on similar data. As a conclusion of the whole AEM3 validation exercise, AEM3 fuel burn estimation offers a high level of realism. Emissions estimation compares with the results published by NASA and ANCAT; the quality of the results depends on the granularity of the AEM3 input data. This extra validation exercise thus did not bring supplementary indications but confirmed results obtained for other aircraft types. As a result, based on 7350 flights, AEM3 is now validated for 14 different aircraft types as summarized in the conclusion.


EEC/SEE/2006/005 v

REPORT DOCUMENTATION PAGE

Reference: SEE Note No. EEC/SEE/2006/005

Security Classification: Unclassified

Originator: Society, Environment, Economy Research Area

Originator (Corporate Author) Name/Location: EUROCONTROL Experimental Centre Centre de Bois des Bordes B.P.15 91222 BRETIGNY SUR ORGE CEDEX France Telephone: +33 1 69 88 75 00

Sponsor: EUROCONTROL project EEC/SEE/GAES

Sponsor (Contract Authority) Name/Location: EUROCONTROL Agency Rue de la Fusée, 96 B –1130 BRUXELLES Telephone: +32 2 729 90 11

TITLE: Advanced Emissions Model v1.5 Validation Exercise 3

Authors : ENVISA Sandrine Carlier James Smith EEC Frank Jelinek

Date 03/06

Pages 40

Figures 17

Tables 14

Appendix -

References 7

EATMP Task Specification -

Project AEM Validation (GAES)

Task No. Sponsor -

Period 2005-2006

Distribution Statement: (a) Controlled by: EUROCONTROL Project Manager (b) Special Limitations: None (c) Copy to NTIS: YES / NO Descriptors (keywords): Global Aviation Emissions – Aircraft Emissions – AEM – TEA – NOx – CO – HC – CO2 – H2O – SOx – VOC – TOG – PM – EEC – ICAO/CAEP – ANCAT – EMEP – etc Abstract: This report is an appendix to the AEM3 validation report EEC/SEE/2004/004 dealing with statistics and validation measures concerning four supplementary aircraft types. Three supplementary aircraft types were already addressed in a first appendix (EEC/SEE/2004/012).


vi EEC/SEE/2006/005

(This page intentionally blank)


EEC/SEE/2006/005 vii

TABLE OF CONTENTS

Table of Contents.................................................................................................... vii

List Of Tables ......................................................................................................... viii

List of Figures ........................................................................................................ viii

ABBREVIATIONS ..................................................................................................... ix

1 INTRODUCTION .................................................................................................. 1

2 Data collection and preparation ........................................................................ 2 2.1 General Approach .................................................................................................................... 2 2.2 FDR#3 Data Preparation.......................................................................................................... 2 2.3 FDR#3 Data set........................................................................................................................ 4 2.4 AEM3 execution ....................................................................................................................... 5

3 Output data analysis and results – Fuel burn estimation with AEM3 ............ 6 3.1 Data reliability ........................................................................................................................... 6 3.2 Flight duration........................................................................................................................... 7

3.2.1 Duration with NoAdd option .............................................................................................. 7

3.2.2 Duration with AddAll option............................................................................................... 7

3.2.3 Duration of LTO cycles ..................................................................................................... 9

3.3 Fuel burn ................................................................................................................................ 11

3.3.1 Fuel burn by aircraft type ................................................................................................ 11

3.3.2 Fuel Burn by Take-Off Weight ........................................................................................ 11

3.3.3 FDR#3 results versus SITA, FDR#1 and FDR#2. .......................................................... 14

3.4 Fuel flow analysis ................................................................................................................... 15

3.4.1 Fuel flow evolution .......................................................................................................... 15

3.4.2 Fuel flow limits ................................................................................................................ 18

3.5 Emissions estimation with AEM3 ........................................................................................... 20

3.5.1 NOx, CO and HC distribution .......................................................................................... 20

3.5.2 NOx average emission indices from ANCAT and NASA ................................................ 22

3.5.3 Emissions along the flight profiles .................................................................................. 23

4 CONCLUSIONS ................................................................................................. 26

5 REFERENCES ................................................................................................... 28


viii EEC/SEE/2006/005

LIST OF FIGURES

FIGURE 1: AEM3 FUEL BURN VS. OPERATIONAL FUEL BURN FOR 2708 FLIGHTS (NO FLIGHT COMPLETION) ........................................................................................................................................... 3

FIGURE 2: EVOLUTION OF FUEL FLOW – B733 – 2000 KM MISSION .............................................................. 15 FIGURE 3: EVOLUTION OF FUEL FLOW – B735 – 1500 KM MISSION .............................................................. 16 FIGURE 4: EVOLUTION OF FUEL FLOW – B752 – 3000 KM MISSION .............................................................. 16 FIGURE 5: EVOLUTION OF FUEL FLOW – B763 – 8000 KM MISSION .............................................................. 17 FIGURE 6: FUEL FLOW LIMITS – B733................................................................................................................... 18 FIGURE 7: FUEL FLOW LIMITS – B735................................................................................................................... 19 FIGURE 8: FUEL FLOW LIMITS – B752................................................................................................................... 19 FIGURE 9: FUEL FLOW LIMITS – B763................................................................................................................... 20 FIGURE 10: EMISSIONS COMPARISON OF 757-200 FOR A 750 KM AND 5500 KM MISSION [REF 5].......... 21 FIGURE 11: EMISSION DISTRIBUTION FOR THE WHOLE FDR#3 TRAFFIC SAMPLE................................... 22 FIGURE 12: EVOLUTION OF NOX, CO AND HC EMISSION INDICES – B733 – 2000 KM MISSION ............... 23 FIGURE 13: EVOLUTION OF NOX, CO AND HC EMISSION INDICES – B735 – 1500 KM MISSION ............... 24 FIGURE 14: EVOLUTION OF NOX, CO AND HC EMISSION INDICES – B752 – 3000 KM MISSION ............... 24 FIGURE 15: EVOLUTION OF NOX, CO AND HC EMISSION INDICES – B763 – 8000 KM MISSION ............... 25 FIGURE 16: THE AEM3 FUEL BURN VS. OPERATIONAL FUEL BURN FOR 7350 FLIGHTS (NO FLIGHT

COMPLETION) ......................................................................................................................................... 26 FIGURE 17: THE AEM3 FUEL BURN VS. OPERATIONAL FUEL BURN FOR 6558 FLIGHTS (FULL FLIGHT

COMPLETION) ......................................................................................................................................... 27

LIST OF TABLES

TABLE 1: FDR#3 FLEET............................................................................................................................................... 4 TABLE 2: NUMBER OF FDR#3 MOVEMENTS ......................................................................................................... 4 TABLE 3: DURATION RATIO PER AIRCRAFT TYPE.............................................................................................. 8 TABLE 4: DURATION RATIO PER AIRCRAFT TYPE AND TAKE-OFF WEIGHT ............................................... 8 TABLE 5: DURATION RATIO PER LTO FLIGHT PHASE........................................................................................ 9 TABLE 6: DURATION RATIO PER LTO FLIGHT PHASE – BREAK DOWN PER AIRCRAFT TYPE ............... 10 TABLE 7: FUEL RATIO PER AIRCRAFT TYPE ...................................................................................................... 11 TABLE 8: WEIGHT CATEGORY FOR FDR#3 DATA (KG) .................................................................................... 12 TABLE 9: TRAFFIC SAMPLE WEIGHT LIMITS ..................................................................................................... 12 TABLE 10: FUEL RATIO PER TAKE-OFF WEIGHT ............................................................................................... 13 TABLE 11: GENERAL FUEL RATIOS FOR AEM3 VALIDATION EXERCISES .................................................. 14 TABLE 12: B737-300 & B737-500 OVER THE THREE VALIDATION EXERCISES ............................................ 14 TABLE 13: EMISSION DISTRIBUTION FOR THE WHOLE STUDY..................................................................... 21 TABLE 14: PUBLISHED AVERAGE EINOX (G/KG) OF REFERENCE PROJECT [REF 6] ................................. 22


EEC/SEE/2006/005 ix

ABBREVIATIONS

AEM Advanced Emission Model AEM3 Advanced Emission Model, 3rd version ANCAT Abatement of Nuisances Caused by Air Transport BADA Base of Aircraft Data BEN Benzene BM2 The Boeing Method 2 EEC-BM2 The EUROCONTROL modified Boeing Method 2 CDRate Climb/Descent rate CO Carbon Monoxide CO2 Carbon Dioxide EEC EUROCONTROL Experimental Center EEC-BM2 EEC corrected BM2 EI Emission Index FDR Flight Data Recordings FL Flight Level H2O Water HC Hydrocarbon ICAO International Civil Aviation Organisation Lat Latitude Long Longitude LTO Landing- and Take-Off cycle Max Maximum Min Minimum MS Microsoft MTOW Maximum Take-Off Weight NASA National Aeronautics and Space Administration NOx Oxides of Nitrogen OEW Operational Empty Weight SEE Society, Environment, Economy SOx Oxides of Sulphur TEA Toolset for Emission Analysis TOG Total Organic Gases TOW Take-Off Weight VOC Volatile Organic Compounds


EEC/SEE/2006/005 1

1 INTRODUCTION

The aim of the AEM3 validation exercise #3 is to build on the AEM3 validation work already carried out previously ([Ref 2.]). The result of the supplementary analysis therefore, is to produce a new set of validation tables presented in this appendix to the existing report ([Ref 2.]) which deals only with statistics and validation measures concerning the specific aircraft types contained in new FDR data (B737-300, B737-500, B757-200 and B767-300). Another appendix (AEM3 validation exercise#2 [Ref 7.]) to the main validation report, addressing three additional aircraft types (A310-325, B737-300 and B737-700), was published in 2004. Both appendices (namely AEM3 validation exercise#2 and #3) have the same focus. Therefore, the forth validation exercise developed in this report will follow the same structure as validation exercise#2 ([Ref 7.]), unless specific features which were not assessable in validation exercise#2. Since the desire for a supplementary validation exercise is to focus on new FDR data and corresponding additional aircraft types, it is not necessary to repeat many of the fundamental (non-aircraft specific) validation tests, focusing only on aircraft specific validation figures. Results for CO2, H2O and SOx are not repeated either as they follow exactly the evolution of fuel burn. Similarly VOC/TOG are proportional to HC and thus do not need to be validated in the current validation exercise. To avoid a confusion between FDR data used in the main ([Ref 2.]) and second ([Ref 7.]) validation exercises and the current validation exercise, FDR data concerning the current validation exercise are called "FDR#3" in the continuation of this report.


2 EEC/SEE/2006/005

2 DATA COLLECTION AND PREPARATION

2.1 General Approach

Operational flight data recordings in the form of FDR data were acquired from one European airline for five aircraft types. FDR data provide detailed information on routes taken by aircraft and different flight parameters among which fuel consumption all along the flight profile. Detailed information is also provided for a certain number of significant flight profile positions such as 'Start of Taxi-out', 'Start of Take-Off', 35ft, 400ft, 1000ft, 1500ft, 2000ft, 3000ft and Touchdown for each flight, even when only a portion of the flight has been recorded (i.e., transatlantic flights often have incomplete profiles in FDR data). A flight leg by flight leg comparison was thus possible, as well as more global assessments based on averages. After a first data review, it was established that some information was missing for one aircraft type (namely ERJ145) making impossible the use of this data in the context of AEM3 validation. Indeed, fields indicating latitude and longitude were empty, while no precise information was available on actual fuel burnt. Reconstitution of flight profiles was simply impossible, preventing from any AEM3 run. May missing information on ERJ145 become available, a release of this document would be issued. As a consequence, the third validation exercise concentrates on four aircraft types:

• B737-300: 270 flights

• B737-500: 213 flights

• B757-200: 1552 flights

• B767-300: 673 flights

2.2 FDR#3 Data Preparation

Data preparation consists of two main activities: • Reconstitution of flight profiles and estimation of the amount of fuel burnt and emissions

due to the data set, using AEM3.

• Extraction of fuel flow or fuel burn information indicated by the airline all along each flight profile, in order to compare actual fuel burn with AEM3 estimation.

Before processing each of these activities, data needs to be "cleaned" in order to eliminate every obvious error which would make AEM3 misinterpreting flight profiles. Then data has to be formatted to fit AEM3 input format. Given the content and format of the data, and in recognition that much of the modification/cleaning process did inevitably involve manual intervention even if an automation was set up when possible, this part was the most time consuming part of the supplementary exercise.


EEC/SEE/2006/005 3

FDR#3 data were of good quality. The conversion to AEM3 format was quite easy despite the appearance of a few problems:

• Altitudes were in some FDR#3 data set indicated as negative figures. As only 8 flight points out of 284,781 were impacted and altitudes were close to zero (minimum value = -200ft), negative figures were set to zero.

• When different information was indicated for the same flight exactly at the same time, the most accurate information regarding the whole flight profile was kept. This step was performed manually.

• Several flights taking off at a given day did land in terms of date one day later. Since AEM3 does not provide the notion of days or dates, such differences have to be expressed by translating day and time information in a virtual simulation time information, e.g. day one is expressed by simulation time 0 to 24, day two from 24 – 48 hours and so on. Where data sets include such cases any time information along all flight profiles of all flights had to be translated from real date and time to AEM3 simulation time.

• When no latitude and longitude were indicated, the leg was deleted. This was the case for the whole ERJ145 aircraft types, making flight profile reconstitution impossible for this aircraft type.

• Similarly, when no ground speed was indicated in the FDR#3 data1 (e.g. ground speed equal to 0 all along a flight), latitude, longitude and time information were used to calculate a ground speed (mostly ERJ145 is concerned by this problem). When no latitude and longitude were indicated either, the leg was deleted.

• When no climb/descent rate was available in the FDR#3 data (i.e. several B737-300's flights), climb/descent rate was calculated based on altitude and time information.

Fuel information was kept in line with other information of the flight profile all along the process.

1 Several ground speeds indicated in the FDR#3 data were obviously wrong (e.g.: Values strictly constant or equal to zero all over a flight). Therefore, a damper has to put on FDR#3 ground speed data quality. This has an impact on the accuracy of flight profile completion in AEM3, as detailed in section "3.1 Data reliability".


4 EEC/SEE/2006/005

2.3 FDR#3 Data set

Useful information on FDR#3 fleet is reported in table below.

ACType BADA aircraft

Number of engines AEM3 Engine Real engine2

B737-300 B733 2 CFM56-3-B1 30%CFMI CFM56-3C1 70% CFMI CFM56-3B2

B737-500 B735 2 CFM56-3C-1 CFMI CFM56-3C1 B757-200 B752 2 RB211-535C Rolls Royce RB211-535E4 B767-300 B763 2 PW4060 Pratt&Whitney PW4060

Table 1: FDR#3 fleet

Aircraft performances, and in particular fuel flows for the four aircraft types in this assessment are directly modelled by BADA. The engine used by AEM3 is close to the actual engine installed on FDR#3 aircraft. AEM3 aircraft/engine assessment is thus consistent with FDR#3. The following table indicates the number of movements available for each aircraft type3.

Aircraft Type Number of movements

B737-300 270 B737-500 213 B757-200 1,552 B767-300 673 Total 2,708

Table 2: Number of FDR#3 movements

No flight was totally deleted from the data set. Therefore 2708 movements were available for validation exercise#3 after data preparation. Nevertheless, specific information on duration or fuel burn was occasionally missing in FDR#3 data, which led to the exclusion of 2 to 3 flights in some averages presented in this report. The granularity of data depends on portions of flight: predefined intervals between two flight points are of +/-5 seconds during take-off, climb, descent and landing and +/-30 seconds during cruise.

2 Real engines for the airline providing the FDR data were determined using JPFleets database [Ref 3.]. 3 The 620 ERJ145 's flights do not appear in Table 2 since they cannot be part of this validation exercise.


EEC/SEE/2006/005 5

2.4 AEM3 execution

AEM3 available options were discussed during validation exercise#1 ([Ref 2.]) and do not need to be repeated here. This section only presents AEM3 user options used for validation exercise#3. To profile completion options were run:

• No flight completion: only portions of flights described in the input profiles were considered. This option is called "NoAdd" in this document.

• Flights were completed by AEM3 when flight profiles retrieved from FDR#3 recordings were not complete. This option is called "AddAll" in this document.

"First leg start" option was selected as "traffic sample entry time" since, similarly to FDR data used during validation exercise#1, FDR#3 take-off and off-block times were not always correct. Other user options, discussed in [Ref 2.] section "AEM3 User Options for Validation", were chosen similarly to the AEM3 validation exercise#1.


6 EEC/SEE/2006/005

3 OUTPUT DATA ANALYSIS AND RESULTS – FUEL BURN ESTIMATION WITH AEM3

Similarly to AEM3 validation exercises#1 ([Ref 2.]) and #2 ([Ref 7.]), results of the comparisons between airline and AEM3 fuel burn estimation and flight duration are presented as follows:

=

=

ninformatio burn fuel Airlineestimation burn fuel AEM3Ratio Fuel

duration flight Airlineduration flight AEM3Ratio Duration

In an ideal case, duration ratio and fuel ratio would be equal to one. A value greater (resp. lower) than one indicates that AEM3 overestimates (resp. underestimates) the real duration or fuel burn.

3.1 Data reliability

The validation exercise #3 totally relies on FDR#3 data quality. However, random manual checks were performed on the data and significant obvious problems were found in the data. Example on a specific flight: according to FDR#3 data, taxi-out is supposed to start 3.4 hours after the aircraft crosses 400ft altitude. The flight mentioned above would indicate 663% overestimation of AEM3 while the actual overestimation lies around 13%. GMT times indicated for this flight are obviously wrong but no recording error warning appears in the data files. Similarly, with reference to other flights, ground speeds equal to 0 during cruise phases were detected. As a consequence, other inaccuracies may no have been identified during data preparation. Consequences of errors in the FDR#3 data set are dual:

• Impact on the quality of AEM3 results: AEM3 flight completion is based on ground speeds as well as rates of climb and descent indicated in the input files. As FDR#3 horizontal and vertical speeds were used to create AEM3 input files, the impact of erroneous values is really significant in AEM3's flight profile completion and may even lead to unrealistic flight profiles.

• Impact on AEM3 versus FDR#3 comparison: When "actual" duration and fuel burn extracted from FDR#3 data set are not accurate, duration and fuel ratios based on such errors are not correct, which hangs over the quality of the whole validation exercise. Indeed duration is the key item for fuel burn and emission calculation.


EEC/SEE/2006/005 7

When obvious for individual flights, errors in the FDR#3 data were corrected before any comparison with AEM3 results. However, as FDR data is supposed to be actual and thus reliable data, most of FDR#3 information had to be used "as is". This section, and thus duration and fuel ratios obtained for the validation exercise #3, relies on FDR#3 data quality and would read significantly different with another data set.

3.2 Flight duration

Similarly to validation exercise#1 ([Ref 2.]), flight duration can be compared to AEM3 calculation. This test aims at addressing the quality of estimated duration of legs added by the tool when input flight profiles are not complete. Legs duration estimation by AEM3 and it's importance are discussed in [Ref 2.]. This section thus only provides duration ratios for FDR#3 data set and focuses on the influence of various criteria.

3.2.1 Duration with NoAdd option

When NoAdd option is applied, AEM3 uses FDR GMT times. Duration ratios with NoAdd option thus are always equal to 1. Nevertheless a range of duration ratios between 0.82 and 1.33 (i.e. -18% to +33% difference) was highlighted although the overall average is close to 1. This reflects the discrepancy between GMT information indicated for each flight point and general information for each flight in FDR#3 data set. This discrepancy is not linked to AEM3 and thus not further investigated in this document. However the whole validation exercise is subject to be affected by equivalent inconsistencies in the FDR#3 data set, as detailed in the "Data reliability" section.

3.2.2 Duration with AddAll option

The duration ratio for the total FDR#3 traffic sample is 1.10. This means that the total flight duration indicated by AEM3 for 2708 flights under study is 10% higher than the real total flight duration. The breakdown by aircraft type presented in Table 3 below highlights disparities between aircraft, even if an overestimation of flight duration is observed for all the aircraft types under study. The completion of the raw input flight profiles leads to a slight overestimation of the mission duration which varies between +7% and +21%. B737 show high duration overestimation regarding other aircraft from both #1 and #3 validations. This may partially be due to the fact that no actual rate of climb/descent was indicated in the FDR#3 for 84 flights operated with B737. The completion of these flight profiles by AEM3 is thus based on estimated climb and descent rates determined during data preparation. Another explanation may lie the completion of flight profiles by AEM3, which may not properly correspond to the actual route taken before and after flight points indicated in the FDR#3 recordings. The reason why aircraft type has an influence on flight duration is that climb/descent rates and ground speed values vary with the aircraft type, its actual take-off weight and meteorological conditions. As a consequence, the completion of flight profile by AEM3 differs depending on the aircraft performances.


8 EEC/SEE/2006/005

Aircraft Type Number of movements

AddAll Duration Ratio

B737-300 270 1.18 B737-500 212 1.21 B757-200 1552 1.09 B767-300 672 1.07

Table 3: Duration ratio per aircraft type

For this reason, the breakdown to address take-off weight influence is also performed by aircraft type. Table 4 shows the same tendency: duration ratio is higher for lighter aircraft regardless of the aircraft type. This is a logical result: with the same engine setting, LTO operations (especially take-off and climb-out) and the climb phase are longer for heavy aircraft. AEM3 duration overestimation is thus less important for heavy aircrafts.

Aircraft Type

Bada Nominal Weight

(kg)

Weight Category

No of movements

Average Take-off Weight

(kg)

AddAll Duration

Ratio

Low 26 43,158 1.36 Medium 102 51,248 1.18 B737-300 54,000

High 100 57,261 1.14 Low 16 43,272 1.31 Medium 131 49,719 1.18 B737-500 52,000

High 65 55,368 1.23 Low 195 80,545 1.14 Medium 1,333 93,512 1.09 B757-200 95,000

High 24 104,025 1.03 Low 131 124,233 1.14 Medium 541 146,609 1.05 B767-300 150,000

High 0 - -

Table 4: Duration ratio per aircraft type and take-off weight


EEC/SEE/2006/005 9

3.2.3 Duration of LTO cycles

Similarly to FDR#1, FDR#3 data hold information on LTO cycles duration. Large disparities in terms of durations can be observed between the certification times of the ICAO Engine Exhaust Emission Data Bank ([Ref 1.]) used in AEM3 and the duration information extracted from the FDR#3 data, especially for climb-out and taxi-out phases. Table 5 shows FDR#3 duration ratio for four LTO phases. Duration ratios obtained with FDR#1 data set are also indicated.

FDR#1 FDR#3 AEM3 Phase

Number of flights for which actual duration is

available

Duration Ratio

Number of flights for which actual duration is

available

Duration Ratio

Taxi-out 3,827 1.62 1,552 2.12 Take-off 3,840 1.08 2,707 1.14

Climb-out 3,840 1.64 2,707 1.66 Approach 3,837 1.04 2,706 1.08

Table 5: Duration ratio per LTO flight phase

The rough estimate of duration ratios is equivalent for both #1 and #3 validations although #3rd exercise shows an higher overestimation of AEM3 for all item of Table 5. Take-off and approach for the whole FDR#3 traffic sample is close to ICAO/AEM3 values. The duration ratio for taxi-out and climb-out indicates an important overestimation of duration for these phases compared to average FDR#3 duration. Climb-out overestimation seems partially due to climb-out geographical constraints at the most represented airport of the data set. Taxi-out seems to be particularly affected by AEM3 overestimation. However taxi-out is the longest LTO phase and is very dependent on factors like the airport or the traffic, which can explain the percentage of variation. Moreover FDR#3 quality of duration information may partially explain high figures in Table 5, since a manual check of LTO durations, and especially taxi-out, highlighted weaknesses for a consequent number of flights (see section "3.1 Data reliability"). A break down per aircraft type (Table 6) confirms the weakness of taxi-out results: taxi-out was incorrect or equal to zero for all the FDR#3 data set but one aircraft type. Similarly to FDR#1 results, the aircraft type dependency for take-off is verified. Considering both first and third validations, the range of take-off duration ratios is spread from 0.74 (A340) to 1.30 (B735). If overall duration ratios are comparable for climb-out in Table 5, the range of figures seems to be more limited for the Boeing family (i.e. #3) than Airbus family (i.e. #1). Indeed Boeing's duration ratios vary from 1.51 to 1.72 while Airbus shows a 0.97 to 1.95 duration ratio variation. Approach figures are very similar with FDR#1 and FDR#3. This tends to demonstrate than approach phase is not specifically aircraft dependant.


10 EEC/SEE/2006/005

AEM3 Phase Aircraft type Number of movements for whichactual duration is available Duration Ratio

Taxi-out B752 1552 2.12 B733 270 1.09 B735 212 1.30 B752 1552 1.15

Take-off

B763 673 1.09 B733 270 1.68 B735 212 1.70 B752 1552 1.72

Climb-out

B763 673 1.51 B733 269 1.03 B735 212 1.03 B752 1552 1.11

Approach

B763 673 1.05

Table 6: Duration ratio per LTO flight phase – Break down per aircraft type

These figures thus confirm an overestimation of AEM3 for taxi-out, take-off, climb-out and approach phases, but the magnitude of AEM3's overestimation may be lower than suggested by Table 5. The consequences of such an overestimation are discussed in [Ref 2.]. Anyway the use of actual LTO durations based on aircraft or airport is an option in AEM3 although very few information is available in the current version of AEM3. This feature will be enhanced as and when aircraft and/or airport specific LTO durations will become available. Duration ratios will thus gradually get closer to 1 as new versions of the tool will be issued.


EEC/SEE/2006/005 11

3.3 Fuel burn

3.3.1 Fuel burn by aircraft type

The fuel ratio for the overall traffic sample is 0.97 with NoAdd option and 1.10 with AddAll option. Seeing that the duration error for AddAll option is around +10%, the resulting fuel error can be considered as an underestimation of fuel of less than 1%4. Table 7 below shows fuel burn ratios as a function of aircraft type.

Fuel Ratio Duration Ratio Aircraft Type Number of movements

NoAdd AddAll AddAll B737-300 270 0.93 1.07 1.18 B737-500 212 1.00 1.17 1.21 B757-200 1,552 0.96 1.10 1.09 B767-300 672 1.01 1.07 1.07

Table 7: Fuel ratio per Aircraft Type

With NoAdd option, fuel burn estimation error of AEM3 lies between -7 and +1%. When AddAll option is used, the completion of the raw input flight profiles combined with the underestimation of the fuel flow as indicated above leads to a final fuel burn estimation error of AEM3 varying between 7 and 17%. With a duration ratio of 1, the fuel ratio for this data set would read -11 to +1%, which confirms that AEM3 underestimates the fuel burn during all flight phases except the LTOs (see [Ref 2.]). Indeed the high percentage of B757-200 in the data sample pulls up the overall AEM3 fuel underestimation, since B757-200 is the only aircraft type showing an overestimation of AEM3 around 1% with a duration ratio of 1. The overall fuel ratio with AddAll option, if duration ratio equals 1, would read lower.

3.3.2 Fuel Burn by Take-Off Weight

An important factor in fuel consumption is the actual weight of the aircraft. AEM3 validation exercise#1 ([Ref 2.]) presented a detailed study of fuel ratio depending on take-off weight. Similarly, 'gross weight' information was used for validation exercise#3 to consider the influence of weight variation of real flights. Three categories of take-off weight have been defined to group the results. The difference (MTOW – 1.2 × OEW) has been divided into equal 3 parts called in below table Low, Medium and High:

• Low = 0 to 33% of the difference

• Medium = 33 to 66% of the difference

4 AddAll duration overestimation=10.47% and AddAll fuel overestimation=9.67%, the resulting fuel underestimation thus lies around 0.81%.


12 EEC/SEE/2006/005

• High = 66 to 100% of the difference

The categories have been estimated based on BADA weights, as discussed in the main validation report ([Ref 2.]), section "Fuel burn by take-off weight".

<---Low---> <---Medium---> <---High--->

Aircraft Type BADA Low

Weight 0%

Limit Inf

33%

BADA Nominal Weight

Limit Sup 66%

BADA High

Weight 100%

B737-300 38,280 46,453 54,000 54,627 62,800 B737-500 37,896 45,491 52,000 53,085 60,680 B757-200 71,520 86,213 95,000 100,907 115,600 B767-300 107,880 132,387 150,000 156,893 181,400

Table 8: Weight category for FDR#3 data (kg)

The table below shows the minimum and maximum take-off weight provided with FDR#3 data for the fleet in the study.

Aircraft Type Minimum TOW Maximum TOW B737-300 36,977 61,035 B737-500 38,719 61,362 B757-200 66,478 107,229 B767-300 102,185 148,597

Table 9: Traffic sample Weight Limits

Note that the weight information found in the FDR#3 data exceeds the weight limit envelope as defined in the BADA aircraft performance tables for all aircraft. B737-300, B757-200 and B767-300 show lighter aircrafts than supposed by BADA while some B737-500 from the data set are heavier than BADA's supposition. As detailed in [Ref 2.], calculations in AEM3 are performed with BADA "nominal" mass. Heavy aircraft are expected to burn more fuel than AEM3 indicates, as opposed to light aircraft which should use less fuel than estimated by AEM3. In the context of global emission studies, this variation should not have significant impact if the “nominal” weight and fuel burn corresponds to a high extent to the average real operating weight of the flights under analysis. In Table 10, weight information for each flight has been considered to evaluate the influence of real weight against AEM3 normalised reference weight.


EEC/SEE/2006/005 13

Fuel Ratio Duration Ratio Aircraft

Type

BADA Nominal Weight

(kg)

Weight Category

No of movements

Average Take-off Weight

(kg) NoAdd AddAll AddAll

Low 26 43,158 0.93 1.24 1.36 Medium 102 51,248 0.93 1.06 1.18 B737-300 54,000

High 100 57,261 0.92 1.03 1.14 Low 16 43,272 0.94 1.26 1.31 Medium 131 49,719 1.02 1.18 1.18 B737-500 52,000

High 65 55,368 0.99 1.11 1.23 Low 195 80,545 0.96 1.18 1.14 Medium 1,333 93,512 0.96 1.09 1.09 B757-200 95,000

High 24 104,025 0.98 1.03 1.03 Low 131 124,233 1.01 1.15 1.14 Medium 541 146,609 1.00 1.05 1.05 B767-300 150,000

High 0 - - - -

Table 10: Fuel ratio per take-off weight

The tendency is respected for all aircraft when AddAll option is used. This is the case for B737-300 and B767-30 only when NoAdd option is used but this does not highlight any inconsistency. Indeed, the portion of flight profile actually in the AEM3 input data also impacts fuel ratios. The influence of the use of a nominal weight assumption as in AEM3 compared to real aircraft TOW becomes visible in a result variation between 3% and 26% around the real observed fuel burn as a function of the aircraft type.


14 EEC/SEE/2006/005

3.3.3 FDR#3 results versus SITA, FDR#1 and FDR#2.

Table 11 below repeats average fuel ratios obtained for the whole AEM3 validation exercise.

AEM3 validation exercise

Data set NoAdd

Fuel Ratio AddAll

Fuel Ratio

AddAll Corrected5 Fuel Ratio

SITA - 1.24 1.08 #1

FDR#1 0.97 1.01 0.99 #2 FDR#2 0.94 - - #3 FDR#3 0.97 1.10 0.996

Table 11: General fuel ratios for AEM3 validation exercises

Note that both FDR#1 and FDR#3 validation exercises lead to the same fuel ratios ("corrected fuel ratio" with AddAll option). One of the reasons for this lies on the fact that both data sets were acquired from the same provider, and thus have the same level of quality and granularity. B737-300 and B737-500 were already addressed during previous validation exercises. Results obtained were the following:

Fuel Ratio Aircraft type Data set Number of

movements NoAdd AddAll SITA 942 - 1.01

FDR#2 483 0.98 - B737-300 FDR#3 270 0.93 1.07 SITA 824 - 1.11

B737-500 FDR#3 212 1.00 1.17

Table 12: B737-300 & B737-500 over the three validation exercises

Results from the third validation exercise are consistent with previous results. Indeed Table 3 indicates an important duration overestimation for both aircraft types, which explains that AddAll fuel ratios are slightly higher than observed for other validation exercises. Moreover, for B737-300 and B737-500, data providers for each validation exercise were different. The high granularity of FDR#2 data helped obtaining particularly accurate fuel ratios (0.98 vs. 0.93 with FDR#3) since:

• flight profiles description were very detailed,

• actual fuel information was easy to handle and lead to particularly precise actual fuel burn.

5 "AddAll corrected" column corresponds to the resulting fuel ratio after compensation of the error due to AEM3 duration estimation. For more details, see [Ref 2.] section "Flight Profile Analysis: Fuel burn". 6 See "3.3.1 Fuel burn by aircraft type"


EEC/SEE/2006/005 15

It can thus be extrapolated that AEM3 quality remains constant whatever data type is used as input.

3.4 Fuel flow analysis

3.4.1 Fuel flow evolution

The evolution of AEM3 fuel flow versus time regarding actual FDR#3 fuel flow was plotted for the four aircraft types under study. A different distance range flight was picked out for each aircraft type in order to have a representative view of the sample. The plots are presented in Figure 2 to Figure 5. In each figure, the blue curve represents fuel flow estimated by BADA in AEM3; the pink curve represents the real fuel flow provided with FDR data; and the dotted curve shows the flight profile. The scale used for this third curve refers to the secondary axis on the right of the plot. For more readability, the four plots were represented using the same scale.

Figure 2: Evolution of fuel flow – B733 – 2000 km mission


16 EEC/SEE/2006/005




EEC/SEE/2006/005 17


Blue and pink curves for the four plots have the same trends. Fuel flows calculated by AEM3 are thus close to reality. Plots are very similar to figures previously obtained for validation exercise #1 ([Ref 2.]) and comments in "Evolution of Fuel Flow" section of [Ref 2.] remain valid here. Several further observations are nevertheless coming out with these new plots:

• If concentrating on the different aircraft types validated so far, it appears that over- and underestimations of AEM3 are dependent on aircraft type and flight phase. Indeed during climb, plots in [Ref 2.] globally showed an overestimation of AEM3 regarding FDR, while an underestimation is visible for all aircraft types on this third validation exercise. This phenomenon comes from many factors among which the way aircraft are used regarding BADA assumptions.

• AEM3 remains sensible to every single change of altitude. This is noticeable on Figure 5 around 20000 seconds: a climb from flight level 280 to 360 leads to a temporarily overestimation of fuel flow. This illustrates the importance of data granularity. Indeed, if applied during a long leg (low granularity data), such a climb would lead to a significant over estimation of fuel for this flight. In contrast, if data granularity is high, this will represent a real physical phenomenon and the fuel estimation will be correct.

• The evolution of aircraft weight is visible for FDR#3 data (pink curve decreasing during cruise) but not directly considered in the current version of AEM3. Indeed an average distance flown by aircraft type, and thus an average weight all along a typical mission, are considered in BADA and this constant mass is used in AEM3. This has a small impact when aircraft actually perform shorter or longer missions. Tiny discrepancies would also appear in a leg by leg study; but it is clear from the plots that AEM3 uses an average value all along the cruise phase which lead to a correct fuel estimation if all the main cruise phase is considered.


18 EEC/SEE/2006/005

3.4.2 Fuel flow limits

Similarly to two previous validation exercises, limits of actual fuel flow as absolute values were compared to the limits of modelled fuel flow determined with BADA and ICAO. Graphical representations are shown for each aircraft type addressed in this document on figures Figure 6 to Figure 9. Items plotted are the following:

• Real limits in the input data. Values are directly extracted from FDR#3 recordings.

• BADA limits. This item corresponds to the minimum and maximum fuel flows indicated in the BADA files used in AEM3.

• ICAO7 limits for engine used by AEM3. This bar shows the range of fuel flows corresponding to the engine used in AEM3.

• ICAO limits for real engine. This bar shows the range of fuel flows corresponding to the actual engine installed on the aircraft. In the case of B733, the fleet is supposed to corresponds for 70% to CFM56-3B2 engines and for 30% to CFM56-3C1 engines (see Table 1). Therefore, fuel flows from both engines were considered in the fuel flow range.

Figure 6: Fuel flow limits – B733

7 ICAO Engine Exhaust Emissions Data bank ([Ref 1.])


EEC/SEE/2006/005 19




20 EEC/SEE/2006/005


Conclusions of the plots are similar to findings from other aircraft: real FDR#3 fuel flows do not fit either BADA nor ICAO fuel flow envelopes. Discussions included in section "Fuel Flow Limits" of [Ref 2.] and [Ref 7.] remain valid for the four new aircraft types.

3.5 Emissions estimation with AEM3

The estimation of level of realism for the NOx, CO and HC emissions is based on ANCAT and NASA projects, as detailed during the AEM3 validation exercise#1 [Ref 2.]. As a reminder, a brief summary of ANCAT and NASA findings can be found at the beginning of the following sections.

3.5.1 NOx, CO and HC distribution

The NASA Scheduled Civil Aircraft Emission Inventories for 1992 [Ref 4.] indicates that the internal distribution between the three above pollutants should vary between 72.5 and 90% for Oxides of Nitrogen, 25 and < 10% for Carbon Monoxide and <1 - 2.5% for Hydrocarbon, dependent on the mission length. The estimation is based on Boeing standard mission profiles, for a mission range between 750 and 5500 km (400 NM and 3000 NM) for a B757-200.


EEC/SEE/2006/005 21

Figure 10: Emissions Comparison of 757-200 for a 750 km and 5500 km Mission [Ref 5]

Previous sections of the report foresaw a consequent amount of CO and HC regarding NOx emission for the FDR#3 data set. Table 6 shows that this prediction is respected.

NoAdd AddAll Aircraft Type

Number of movements % NOx % CO % HC % NOx % CO % HC

B737-300 270 54.54% 43.62% 1.84% 52.73% 45.20% 2.07%B737-500 213 56.17% 42.28% 1.55% 56.08% 42.29% 1.63%B757-200 1,552 63.28% 29.17% 7.55% 64.26% 28.99% 6.75%B767-300 673 73.77% 24.31% 1.91% 73.70% 24.38% 1.92%

Table 13: Emission distribution for the whole study

Figure 11 provides the same information in a graphic.


22 EEC/SEE/2006/005

Figure 11: Emission distribution for the whole FDR#3 traffic sample

NOx proportion evolutes between 52.7 and 73.8%, while CO varies from 24.3 to 45.2% and HC from 1.5 to 7.5%. Percentages obtained for B767-300 perfectly enter NASA range of values obtained with a B757-200. B757-200 shows a high proportion of HC emissions while high CO are observed for B373. The reason for such a high CO and HC percentage lies in the extrapolation of emissions indices outside the ICAO fuel flows, as observed on Figure 6 to Figure 9. This extrapolation to further low fuel flows leads to very high emission indices for CO and HC.

3.5.2 NOx average emission indices from ANCAT and NASA

NASA and ANCAT researchers have analyzed the NOx emission estimates for larger traffic inventories and calculated the amount of NOx emissions in relation to the estimated amount of fuel burn. These calculated values are called Average NOx Emission Indices (EINO). This analysis led to the following estimations for average NOx Emissions Indices in g per kg fuel burn:

ANCAT 1A ANCAT2 NASA NASA 1992 1992 1990 1992

Horizontal resolution (°) 2.8 × 2.8 1 × 1 1 × 1 1 × 1

Vertical resolution (°) 1 1 1 1

EINOx (g/kg) 16.8 13.7 10.9 11.1

Table 14: Published average EINOx (g/kg) of reference project [Ref 6.]


EEC/SEE/2006/005 23

NASA results are based on Boeing Method 1 and Method 2. ANCAT 1A results have been obtained using a thermo-dynamic NOx emission model which has been replaced during ANCAT 2 by the DLR NOx estimation method. The ANCAT/EC2 inventory in the base year 1991/1992 (published in 1998) estimates that the civil subsonic fleet average emissions index (EINOx: g NOx/kg fuel) is 13.7. This estimation is significantly lower than the previous ANCAT/EC1A study published in 1995 which indicated an average of 16.8 g NOx/kg fuel. This difference is due to revisions to the movement data base and use of a different methodology for the prediction of NOx at cruise altitudes. The average NOx emission indices for the 2708 flights under study lie at 9.76g/kg fuel burn with NoAdd option and 10.13g/kg fuel burn with AddAll option. These results are respectively about 0.12% and 0.09% lower than NASA results ([Ref 4.]) from 1992 and 0.29% and 0.26% lower than ANCAT2 results ([Ref 5.]). Average EINOx for the third AEM3 validation exercise are in accordance with ANCAT and NASA findings, although slightly lower than results from other validation exercises.

3.5.3 Emissions along the flight profiles

Figure 12 to Figure 15 show the evolution of NOx, CO and HC emission indices. Four typical flights are plotted as done for fuel flow in section "Fuel flow analysis: Fuel flow evolution". The flight profiles (flight level versus time) are also plotted and refer to the secondary axis on the right. EINOx is expected to be highest during climb phase, remain high during cruise phase and decrease during descent phase. On the other hand, EICO and EIHC are supposed to be high during descent phase, where the combustion process is less complete.

Figure 12: Evolution of NOx, CO and HC emission indices – B733 – 2000 km mission


24 EEC/SEE/2006/005




EEC/SEE/2006/005 25


The tendencies are verified in above figures and comments in "Emission along the flight profiles" section of [Ref 2.] remain valid here. Once again, AEM3 appears as very sensible to climb or descent phases. Every single short climb leads to a peak of EINOx while descents lead to peaks of EICO and EIHC.


26 EEC/SEE/2006/005

4 CONCLUSIONS

The AEM3 validation exercise#3 confirmed results obtained during exercise#1 and #2. As a conclusion of these three AEM3 validation exercises, AEM3 fuel burn estimation offers a high level of realism. AEM3 fuel burn is globally underestimated by -6 to -3% when no flight completion is required. In case of flight completion, the error on fuel burn lies around -1% underestimation if the error on duration is isolated. The error on duration is highly dependent on input data, and especially the percentage of flight profile actually described as well as horizontal and vertical speeds. It is obvious that the better a real aircraft compares to the underlying aircraft model assumptions (BADA) in AEM3, the higher is the level of realism obtained by AEM3 for fuel burn and emissions estimation. The influence of take-off weight is significant as well as mission length for which aircrafts are designed. The following figures compare AEM3 fuel burn estimation to actual FDR operational data for NoAdd and AddAll option, i.e. for the whole AEM3 validation exercise.

Figure 16: The AEM3 fuel burn vs. operational fuel burn for 7350 flights (no flight completion)


EEC/SEE/2006/005 27

Figure 17: The AEM3 fuel burn vs. operational fuel burn for 6558 flights (full flight completion)

The correlation between AEM3 fuel burn and actual fuel burn is of high level for more than 6500 flights corresponding to 14 different aircraft types. As for previous validation exercises, emissions cannot be compared to actual data since such information was not available. Nevertheless they compare to NASA and ANCAT estimations which allows concluding that emissions indicated by AEM3 are realistic. The three different AEM3 validation exercises exploited different kinds of data granularity leading to comparable positive results. Based on the good results for the 148 aircraft types validated so far, it is assumed that the AEM3 fuel burn and emission estimation for other aircraft types is of similar quality. AEM3 validation for additional aircraft-types will continue with availability of FDR data.

8 11 different aircraft types validated with FDR data and 3 aircraft types validated with SITA data.


28 EEC/SEE/2006/005

5 REFERENCES

[Ref 1.] ICAO Engine Exhaust Emissions Data Bank; ICAO; Doc 9646-AN/943: First Edition – 1995; Internet Issue 1 (10/03/1998); Internet Issue 2 (08/02/1999), Internet Issue 10(19/05/2003)

[Ref 2.] The Advanced Emission Model (AEM3) Version 1.5 – Validation Report; EUROCONTROL Experimental Centre; Society, Environment and Economics Business Area; Jelinek, Carlier, Smith; EEC/SEE/2004/004

[Ref 3.] JP airline-fleets international Aviation Database – BUCHair UK ltd

[Ref 4.] Scheduled Civil Aircraft Emission Inventories for 1992: Database Development and Analysis; April 1996; NASA LRC; Contractor Report 4700; Steven L. Baughcum, Terrance G. Tritz, Stephen C. Henderson, David C. Picket

[Ref 5.] ANCAT/EC2: Global Aircraft Emissions Inventories for 1991/92 and 2015 – Report by the ECAC/ANCAT and EC working group – Editor R.M. Gardner

[Ref 6.] Impact de la flotte aérienne sur l'environnement atmosphérique et le climat; Rapport no.40, Décembre 1997, Institut de France, Académie des sciences – Académie Nationale de l'air et de l'espace

[Ref 7.] Advanced Emission Model (AEM3) v1.5 – Validation Exercise#2; EUROCONTROL Experimental Centre; Society, Environment and Economics Business Area; Jelinek, Carlier, Smith; EEC/SEE/2004/004

(This page intentionally blank)

For more information about the EEC Society, Environment and Economy Research Area please contact: Ted Elliff SEE Research Area Manager, EUROCONTROL Experimental Centre BP15, Centre de Bois des Bordes 91222 BRETIGNY SUR ORGE CEDEX France Tel: +33 1 69 88 73 36 Fax: +33 1 69 88 72 11 E-Mail: [email protected] or visit : http://www.eurocontrol.int/

Documents

GAES Advanced Emissions Model v1 - · PDF file3.4 Fuel flow analysis ... (A310-325, B737-300 and B737-700), was published in 2004. Both appendices (namely AEM3 validation exercise#2