GEOTRACES Data Assembly Centre (GDAC) ... GEOTRACES Data Assembly Centre (GDAC) Activities GEOTRACES

  • View
    1

  • Download
    0

Embed Size (px)

Text of GEOTRACES Data Assembly Centre (GDAC) ... GEOTRACES Data Assembly Centre (GDAC) Activities GEOTRACES

  • GEOTRACES Data Assembly Centre (GDAC) Activities

    GEOTRACES project team: Donna Cockwell, Gwen Moncoiffe, Helen Snaith, Richenda

    Houseago-Stokes, Graham Allen

    Mohamed Adjou Data manager

    British Oceanographic Data Centre, NOC, Liverpool, UK

    JULY 2018: SSC – TAWAIN

  • Outline

     IDP download from GDAC server

     Catch-up on tasks inherited from the prioritized work of IDP2017 delivery

     Improved Reporting of GEOTRACES Data Ingestion QC

     Improving data workflow

     Online management of GEOTRACES parameter names via the NERC vocabulary server

  • IDP download from GDAC server

    IDP2014 and IDP2017 total downloads (to 10 July)

    440

    392

    622

    551

    172 211 195

    282

    205

    92

    0

    100

    200

    300

    400

    500

    600

    IDP2014 IDP2014_V2 IDP2014_V3 IDP2017_V1 IDP2017_V2

    All Downloads Unique Users

  • IDP download from GDAC server IDP2014 and IDP2017

    Total downloads between 10 April-10 July

    0 0 3

    32

    88

    0 0 2

    13

    47

    0

    10

    20

    30

    40

    50

    60

    70

    80

    90

    100

    IDP2014 IDP2014_V2 IDP2014_V3 IDP2017_V1 IDP2017_V2

    All Downloads Unique Users

  • IDP download from GDAC server IDP2014 and IDP2017 downloads

    by format (to 10 July)

    233

    144

    249 231

    75

    207

    42

    76 90

    20 0

    47

    96 106

    23

    0

    159

    201

    124

    54

    0

    50

    100

    150

    200

    250

    300

    IDP2014 IDP2014_V2 IDP2014_V3 IDP2017_V1 IDP2017_V2

    Total IDP downloads by format

    ODV ASCII NETCDF Excel

  • IDP download from GDAC server IDP2014 and IDP2017 downloads

    by format (to 10 July)

    312 282

    449

    290

    136128 110

    173 178

    00 0 0

    83

    36

    0

    50

    100

    150

    200

    250

    300

    350

    400

    450

    500

    IDP2014 IDP2014_V2 IDP2014_V3 IDP2017_V1 IDP2017_V2

    Total IDP downloads by data type

    Sample CTD Aerosol/Rain

  • 0

    50

    100

    150

    200

    250

    IDP downloads by Country

    IDP2014 IDP2014_V2 IDP2014_V3 IDP2017_V1 IDP2017_V2

    IDP2014 and IDP2017 downloads by Country (to 10 July)

  • IDP2014 and IDP2017 downloads by Country (to 10 July without USA & UK)

    0

    10

    20

    30

    40

    50

    60

    IDP downloads by Country

    IDP2014 IDP2014_V2 IDP2014_V3 IDP2017_V1 IDP2017_V2

    Presenter Presentation Notes Data as of 10 July

  • Catch-up on tasks inherited from the prioritized work of IDP2017 delivery

     Insuring up-to-date gdac correspondence (geotraces.dac@bodc.ac.uk).

     Updating cruise inventory: https://www.bodc.ac.uk/geotraces/data/inventories/

    https://www.bodc.ac.uk/geotraces/data/inventories/

  • Catch-up on tasks inherited from the prioritized work of IDP2017 delivery

     Updating section maps: https://www.bodc.ac.uk/geotraces/cruises/section_maps/

    Section cruises per Ocean basin

    https://www.bodc.ac.uk/geotraces/cruises/section_maps/

  • Catch-up on tasks inherited from the prioritized work of IDP2017 delivery

     Updating section maps: https://www.bodc.ac.uk/geotraces/cruises/section_maps/

    Process studies and Compliant data

    https://www.bodc.ac.uk/geotraces/cruises/section_maps/

  • Catch-up on tasks inherited from the prioritized work of IDP2017 delivery

     Diverse information update:

    1. A dedicated IDP2017 web page indicating that IDP2014 and previous versions of the IDP are still available with a link to the new web page;

    2. Updating the IDP2017 bibliographic reference;

    3. Updating the download agreement;

  • Improved Reporting of GEOTRACES Data Ingestion QC

     Increasing data QC standards was presented as a priority task during the last DMC meeting.

     A working document on a QC strategy was submitted to the DMC (end of May)

     GDAC propose to deliver a standardised data QC report for each dataset, together with a final document summarizing the QC analyses.

  • ORIGINATOR

    BODC

    Improving data workflow How do we store samples’ data

    Parameter name & Unit Value

    From information submitted in the “Data metadata” file

    Sample’s (x,y,z,t) geographical coordinates, depth and time

    From information submitted in the “Data” file

    DB

  • ORIGINATOR

    BODC

    Improving data workflow

    Actions for more process fluidity and decreasing the chances of human errors

    Parameter name & Unit Value

    From information submitted in the “Data metadata” file

    Sample’s (x,y,z,t) geographical coordinates, depth and time

    From information submitted in the “Data” file

    DB

    2) Systematically supply data with event and bottle identifiers.

    1) Provide GEOTRACES parameter names in data file headers (DMC proposal via the data portal)

  • For more fluid data processing at GDAC

    1) Provide GEOTRACES parameter names in data file headers: DMC proposal via the data portal.

    2) Systematically supply data with event and bottle identifiers: this will be done using cruise events/samples information that could be supplied by the chief scientist after the cruise.

    Event Number

    Gear (CTD, FISH, Net etc.)

    Time begin

    Time end

    Long Lat Bottle number

    Depth Comment

    Here an example of the headers of the cruise logs master file:

    Once submitted by the chief scientist we can make it available for data submitters through BODC stfp location.

    Actions for more process fluidity and decreasing the chances of human errors

  • Online management of GEOTRACES parameter names via the NERC vocabulary server

    Work load for this task submitted by GDAC to GIPO

    When approved, we need up-to-date parameters’ lists

    Would replace stand-alone spreadsheets

    Would make the GEOTRACES parameter names searchable and referenceable

    Would enable GDAC to improve data workflow

    Why?

    Current status

  • GEOTRACES Data Managers

    geotraces.dac@bodc.ac.uk

  • IDP2017 (V2) Access

     www.bodc.ac.uk/geotraces/data/idp2017

    Presenter Presentation Notes IDP2017 is available (www.bodc.ac.uk/geotraces/data/idp2017 )

    http://www.bodc.ac.uk/geotraces/data/idp2017

  • To download files, need to:  Log in to the BODC system  Agree to the download agreement

    IDP2017 (V2) Access

    Presenter Presentation Notes 3 datasets: CTD, Sample and aerosol data Available in 4 formats: ODV, ASCII, NetCFD, excel

  • Metadata form update

     We recently added two mentions on the GDAC metadata submission template :

    1) listing the contributors, their respective ranks and affiliations

    2) Error calculation (1 or 2 times Standard deviation…etc.).

  • Flags

    Flag Description 0 no quality control 1 good value 2 probably good value 3 probably bad value 4 bad value 5 changed value 6 value below detection 7 value in excess 8 interpolated value 9 missing value

    A value phenomenon uncertain

    Q value below limit of quantification

    SeaDataNet Quality Control Flags The following single character qualifying flags may be associated with one or more individual parameters with a data cycle:

    Flag Description Blank Unqualified < Below detection limit > In excess of quoted value A Taxonomic flag for affinis (aff.) B Beginning of CTD Down/Up Cast C Taxonomic flag for confer (cf.) D Thermometric depth E End of CTD Down/Up Cast

    G Non-taxonomic biological characteristic uncertainty

    H Extrapolated value

    I Taxonomic flag for single species (sp.)

    K Improbable value - unknown quality control source

    L Improbable value - originator's quality control

    M Improbable value - BODC quality control

    N Null value

    O Improbable value - user quality control

    P Trace/calm Q Indeterminate R Replacement value S Estimated value T Interpolated value U Uncalibrated W Control value X Excessive difference

    BODC Quality Control Flags The following single character qualifying flags may be associated with one or more individual parameters with a data cycle:

  • Flags

    All the SeaDataNet data quality flags have a correspondent flag in BODC except (“probably good value”).

    Source: https://odv.awi.de/fileadmin/user_upload/odv/misc/ODV4_QualityFlagSets.pdf

    https://odv.awi.de/fileadmin/user_upload/odv/misc/ODV4_QualityFlagSets.pdf

  • Work to budget/income for past and future IDP

    Scope of IDP increases each version (new parameters, new media)

    Effort for GDAC increases

    Effort for whole IDP>GDAC income

    NERC funding pays for different activities

    We will work with funded budget

    BODC has still worked more than funded, so reduced FTE this year