Upload
eunice-pope
View
219
Download
1
Embed Size (px)
Citation preview
Pierre Jaccard 1
MyOcean Quality Control for Ferryboxes
10.06.2013 - MyOcean Tutorial, NERSC, Bergen
Pierre Jaccard 2
Quality Controlled Data
• How good are they?• Is a good data good enough?• Definitions
10.06.2013 - MyOcean Tutorial, NERSC, Bergen
NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013
323.04.2013
MyOcean In-Situ Quality Control
• Real Time Quality Control Procedures– Based on ARGO QC procedures
• Argo, 2009: Argo quality control management, Version 2.4, Argo data management
– T and S QC manual– BioGeoChem QC manual
• Dealyed Mode Quality Control Procedures– T and S QC manual– BioGeoChem QC manual– Interaction with SDN2
• Parameters– Temperature, Salinity– Chla Fluorescence, Dissolved Oxygen
NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013
4
Infrastructure Types
• Vertical Profiles– ARGO, XBT, CTD– Gliders, AUV
• Time Series–Moorings– Ferrybox
23.04.2013
Pierre Jaccard 5
netCDF File Format
Variables
• TIME, TIME_QC• LONGITUDE, LATITUDE, POSITION_QC• DEPH, DEPH_QC• TEMP, TEMP_QC• PSAL, PSAL_QC• FLU2, FLU2_QC• DOXY, DOXY_QC
10.06.2013 - MyOcean Tutorial, NERSC, Bergen
Pierre Jaccard 6
Variable AttributesExample for variable TEMP:
long_name = sea water temperature in-situ ITS-90 scale standard_name = sea_water_temperature units = degrees_Celsius valid_min = -2.0 valid_max = 40.0 comment = QC_indicator = QC_procedure = 7 sensor_depth = ancillary_variables = TEMP_QC uncertainty = 0 accuracy = 0 precision = 0 resolution = 0 cell_methods = TIME:point DEPTH:point LATITUDE:point LONGITUDE:point DM_indicator = R ferrybox_sensor = <name of sensor installation on ferrybox>
10.06.2013 - MyOcean Tutorial, NERSC, Bergen
Pierre Jaccard 7
QC Variables Attributes
10.06.2013 - MyOcean Tutorial, NERSC, Bergen
long_name = quality flag conventions = OceanSITES reference table 2 valid_min = numpy.byte: 0 valid_max = numpy.byte: 9 flag_values = func:MyO.Export.Formats.MyOcean.flags_values flag_meanings = no_qc_performed
good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used nominal_value interpolated_value missing_value
Pierre Jaccard 8
Flags
10.06.2013 - MyOcean Tutorial, NERSC, Bergen
0 No QC was performed Should not be used without control
1 Good data Can be used safely
2 Probably good data Can be used safely for some applications
3 Bad data, but potentially correctable Should not be used, but maight be improved later
4 Bad data Should not be used
5 Value changed
6 Below detection limit
7 In excess of quoted value
8 Interpolated value
9 Missing value
A Incomplete information
NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013
9
ARGO Specific Tests
• Platform identification (GTS)• Impossible speed test (GTS)
– Check drift speed of floats
• Digit rollover test– When number of digits is too small
• Grey list (ARGO)– Stop assimiliation and dissemination of data from sensors not
working properly
• Gross salinity or temperature drift (ARGO)– Detect significant sensor drift
• Deepest pressure test (ARGO)– Some kind of pressure range test
• Instrument range test (Gliders, AUV)– Detect if instrument is within oceanographic limits
23.04.2013
NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013
10
Vertical Profiles Tests
• Frozen profile test– Detect instruments that reproduce the
same profile (with too little variations)
• Density inversion• Bottom spike test (XBT)– Detect sudden temperature change
• Pressure increasing test
23.04.2013
NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013
11
Common Tests• Impossible date test• Impossible location test• Position on land test• Global range test• Regional range test• Spike test• Gradient test• Stuck value test• Rate of change in time test (Moorings)• Calibration status check
– Move data from good to probably good after a certain period of time
– Delayed mode
• Parameter relationship test
23.04.2013
NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013
12
Ferrybox Specific Tests
• Speed range test (Ferrybox)• Pump test (Ferrybox)• Pump history test (Ferrybox)• Instrument comparison test– Detect if 2 measurements of same
parameter are close
• Subsequent trip test– Detects and corrects sensor cleaning events
(biofouling) – Delayed mode
23.04.2013
NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013
13
Location Tests
23.04.2013
[TIME][[FROZEN]]
consecutive measurements at identical time[[VALUE]]
1950-01-01 00:00:00 < t < now[[LENGTH]]
require a minimum number of consecutive measurements
minimum = 900s
[SPEED][[RANGE]]
min = 0.5max = 15.0
[[FROZEN]]Consecutive measurements with identical speed
[[HISTORY]]Require sustained speed for a minimum periode of timeduration = 300s
[POSITION][[FROZEN]]
consecutive measurements at identical position
[[RANGE]]geographic placement on earth
NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013
14
Flow Tests
23.04.2013
[PUMP][[VALUE]]
System must be pumping[[HISTORY]]
Require flow through system for a minimal period of time
Duration = 600s
[OBSTRUCTION][[VALUE]]
Obstruction signal must be inactive[[HISTORY]]
Require inactive signal for a minimum period of time
NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013
15
Parameter Tests
23.04.2013
[PARAMETER][[GLOBAL_RANGE]][[REGIONAL_RANGE]]
Special range tests by geographical range and time range in year
[[GRADIENT]]ARGO gradient test on T&SModified ARGO Test on BGCWork in progress to support all parameters and
platforms[[SPIKE]]ARGO gradient test on T&SModified ARGO Test on BGC Work in progress to support all parameters and
platforms
| V2 - (V3 +V1)/2 | - | (V3 - V1) / 2 |
| V2 - (V3 + V1)/2 |