35
Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX workshop on Arctic climate modeling results and needs, March 20-21, 2012

Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

Embed Size (px)

Citation preview

Page 1: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

Chris A. MattmannSenior Computer ScientistJet Propulsion Laboratory

D. Waliser (JPL)C. Goodale (JPL)

J. Kim (UCLA/JIFRESSE

Many others

ADSIMNOR-CORDEX workshop on Arctic climate modeling results and needs, March 20-21, 2012

Page 2: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

Using Satellite & Other Observations For Regional Model Evaluation

Page 3: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 4: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 5: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 6: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 7: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

We started with MySQL…and then broke it at 6B data points…and then federated it, and broke it

again

Then we experimented with Hadoop/HIVE

Query times scaledwith the amount of nodes weadded

Currently evaluating

Page 8: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

The RCMED API:- Search the entire database

- Space/Time box- Simple RESTful URL- Simple ASCII result format

Page 9: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

• Use web service from RCMED

• Regrid onto obs or model grid

• Seasonal, monthly or annual temporal compositing

• Perform metric (bias, RMSE, etc.)

• Output plot using NCAR NCL

• Bottle Python web services• PHP Wizard Front End

Page 10: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 11: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

• Config ACentralized RCMEDVM-based RCMET

• Config BCentralized RCMET and RCMED

Page 12: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

ftp://ftp.cccma.ec.gc.ca/pub/yjiao/CanRCM4/

Worked with Melanie Cookie/Paul Kushner

Compared Near-Surface Air Temperature with ERA 2 Metre Temp

Selected time range:  2008/01/01 00:00 2008/02/01 00:00•Use Model grid•Calculate time mean for full period.•Bias: mean bias across full time range

Page 13: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 14: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 15: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 16: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 17: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 18: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 19: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 20: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 21: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 22: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 23: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 24: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 25: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 26: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 27: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 28: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 29: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 30: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 31: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX
Page 32: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

[K]

Page 33: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

[K]

Page 34: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

[K]

Page 35: Chris A. Mattmann Senior Computer Scientist Jet Propulsion Laboratory D. Waliser (JPL) C. Goodale (JPL) J. Kim (UCLA/JIFRESSE Many others ADSIMNOR-CORDEX

Thank you!

Contact information:Chris A. Mattmann – [email protected]

Duane Waliser – [email protected]

Jinwon Kim – [email protected]