Embed Size (px)
Citation preview
1
CESD SAGES Scottish Alliance for Geoscience, Environment & Society
The challenges of geo-simulation data
Centre For Earth System Dynamics
2
CESDCESD
This talk: perspectives from CESD’s climate modelling
• How climate modelling is done– Why model the climate?– NetCDF– CF – climate and forecast – Archives and metadata
• Current challenges
• Imminent challenges
3
CESDCESD
What is “the climate”?
• Statistical concepts such as:– Typical seasonal rainfall distribution– Global mean annual outgoing shortwave
radiation– Monthly mean surface temperature
• …arising from physical processes– Fluid dynamics on rotating sphere– Interactions of radiation – ….
4
CESDCESD
Why use a computer model of the climate?
1. Explore the climate:– Test hypotheses about how the climate works– Interpret observations – Express scientific community understanding– Generate possible past and future climates
2. Use climate model output data – To drive other models– To inform mitigation/adaptation– Where observations are sparse at best…
e.g. the future
5
CESDCESD
Karl and Trenberth 2003
Modelling the Climate System
Main Message: Lots of things going on!
6
CESDCESD
A climate model
δ that/δ other = something else
δ this/δ that = something
Initial state Ancillary data can be time series
Files of means: 6hr, daily…decadal
Modelled processes
New process
New “diagnostic”
Toolbox – not a black box!
7
CESDCESD
Data volumes and typical analyses
• Typically we make 1-5GB/model year– 40 model years/day (coarse coupled model (HadCM3)
using 40 cores)
• Our biggest project: 14TB• Researcher selects/slices data• Does
– Global/regional analyses – global means– Comparisons with related runs and observation,….,
….,…– NCL, IDL, NCO,… tools built on data standards
8
CESDCESD
NetCDF
• “NetCDF is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data”.
• File contains dimensions, variables, and attributes.
Ed Hartnett’s talk at: http://www.unidata.ucar.edu/software/netcdf/papers/nasa_data_workshop_2010.pdf
9
CESDCESD
Climate Forecast conventions
• http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.4/cf-conventions.html
• define metadata that provide a definitive description of what the data in each variable represents – E.g. A variable called temp
• Long name (ad hoc): near-surface daily mean• Standard name: air_temperature• Units: K
10
CESDCESD
CF: time – two examples
double time(time) ; time:long_name = "time" ; time:units = "days since 1990-1-1 0:0:0" ;
Days; Hours; Min; Sec
time:units = "days since 1-7-15 0:0:0”time:calendar = "none" ; data: time = 0., 1., 2., ...;
All data are for same date:
11
CESDCESD
How are data made accessible?
• publish data in data centres:– Provide “experiment” metadata– Upload NetCDF data – Metadata are harvested from files into
catalogue
• Web services– E.g ncWMS
12
CESDCESD
Some challenges
13
CESDCESD
Current trends
DataDiversity
Volume
Computation
Legacy analyses (IDL, …,..,..,..)
Collaboration
Cooperation across groups
EnsemblesGlobal + Regional
Publish more than papersBuild research ecosystem
14
CESDCESD
Future Lifecycle of research data
Researcher
Project
Research communityArchives:
BADC
ECDF Tools to capture metadata: instrument current codes + workflow
Easy transitions personal-project-world
Provenance: re-use/modify analyses
PublicWeb services
15
CESDCESD
• Wrap/instrument tools to give Metadata + Provenance in post-model analyses, impact modelling… learn from– SYSMO (Univ. of Manchester)– e-Science Central (Univ. of Newcastle)– Steve!
• Workflow with wrapped legacy tools?
Current challenges
16
CESDCESD
Imminent challenges: impact / adaptation
Socio-economics
Climate
landcrops
……
ecologiesflood
urban
data
NetCDF
Census – sociopolitical area
Regular/nested grids
Triangulated irregular nwks
Dat
a sy
nth
esis
Cli
mat
e d
ow
nsc
alin
gp
oin
t->
area
mo
del
lin
gp
rob
abil
isti
c d
ata
biodiversity
