ascat Documentation - Read the Docs · conda install-c conda-forge pygrib=2.0.1 # for reading GRIB files ... For details and usage information on PyScaffold seehttp:// ... ascat Documentation,

ascat DocumentationRelease 0.7.post0.dev39+ngc937809.dirty

TU Wien

Aug 14, 2017

Contents

1 Supported Products 31.1 Time Series Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Image products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Installation 5

3 Contribute 73.1 Development setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4 Citation 9

5 Note 11

6 Contents 136.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136.2 ASCAT BUFR format table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386.3 License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406.4 Developers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416.5 Changelog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416.6 ascat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

7 Indices and tables 63

Python Module Index 65

i

ii

ascat Documentation, Release 0.7.post0.dev39+ngc937809.dirty

Read and convert data acquired by ASCAT on-board the series of Metop satellites. Written in Python.

Works great in combination with pytesmo.

Contents 1

https://travis-ci.org/TUW-GEO/ascat

https://coveralls.io/github/TUW-GEO/ascat?branch=master

http://badge.fury.io/py/ascat

https://zenodo.org/badge/latestdoi/12761/TUW-GEO/ascat

https://github.com/TUW-GEO/pytesmo


2 Contents

CHAPTER 1

Supported Products

This gives a short overview over the supported products. Please see the documentation for detailed examples of howto work with a product.

Read ASCAT data from different sources into a common format supported by pytesmo.

1.1 Time Series Products

• Metop ASCAT Surface Soil Moisture (SSM) Climate Data Record (CDR) in time series format

Available in netCDF format from H SAF (H25, H108-H112)

• CGLS SWI(Soil Water Index) Time Series (SWI_TS)

Available from the Copernicus Global Land Service (CGLS)

• ASCAT SWI(Soil Water Index) Time Series

Available in binary format from TU Wien

1.2 Image products

1.2.1 H SAF

H SAF provides several different image products:

• H16 - SSM ASCAT-B NRT R : Metop-B ASCAT soil moisture 12.5km sampling NRT

• H103 - SSM ASCAT-B NRT O : Metop-B ASCAT soil moisture 25km sampling NRT

• H101 - SSM ASCAT-A NRT R : Metop-A ASCAT soil moisture 12.5km sampling NRT

• H102 - SSM ASCAT-A NRT O : Metop-A ASCAT soil moisture 25km sampling NRT

• SM OBS 2 - H08 - Small scale surface soil moisture by radar scatterometer in BUFR format over Europe

3

http://hsaf.meteoam.it/soil-moisture.php

http://land.copernicus.eu/global/products/swi

http://rs.geo.tuwien.ac.at/products/



• SM DAS 2 - H14 - Profile index in the roots region by scatterometer data assimilation in GRIB format, global

The products H16, H103, H101, H102 come in the same BUFR format. Since the default filenames are slightlydifferent the following readers should be used:

• H16 - ascat.h_saf.H16img




They are available after registration from the H SAF Website

The H07 (SM OBS 1) is discontinued and replaced by H101, H102, H16 and H103.


1.2.2 EUMETSAT

EUMETSAT provides ASCAT Level 1 and Level 2 data among others through the EUMETSAT Data Centre. At themoment this package supports the following products:

• ASCAT Soil Moisture at 12.5 km Swath Grid - Metop in BUFR format ascat.eumetsat.AscatAL2Ssm125 and ascat.eumetsat.AscatBL2Ssm125.

• ASCAT Soil Moisture at 12.5 km Swath Grid - Metop in BUFR format - 3 Minute PDUfiles. ascat.eumetsat.AscatAL2Ssm125PDU and ascat.eumetsat.AscatBL2Ssm125PDU . Forreading half orbits use ascat.eumetsat.AscatAL2Ssm125PDUChunked and ascat.eumetsat.AscatBL2Ssm125PDUChunked

• ASCAT Soil Moisture at 12.5 km Swath Grid - Metop in netCDF format ascat.eumetsat.AscatAL2Ssm125Nc and ascat.eumetsat.AscatBL2Ssm125Nc.

• ASCAT Soil Moisture at 25.0 km Swath Grid - Metop in BUFR format ascat.eumetsat.AscatAL2Ssm250 and ascat.eumetsat.AscatBL2Ssm250.

• ASCAT Soil Moisture at 25.0 km Swath Grid - Metop in BUFR format - 3 Minute PDUfiles. ascat.eumetsat.AscatAL2Ssm250PDU and ascat.eumetsat.AscatBL2Ssm250PDU . Forreading half orbits use ascat.eumetsat.AscatAL2Ssm250PDUChunked and ascat.eumetsat.AscatBL2Ssm250PDUChunked

• ASCAT Soil Moisture at 25.0 km Swath Grid - Metop in netCDF format ascat.eumetsat.AscatAL2Ssm250Nc and ascat.eumetsat.AscatBL2Ssm250Nc.

4 Chapter 1. Supported Products


http://www.eumetsat.int/website/home/Data/DataDelivery/EUMETSATDataCentre/index.html

CHAPTER 2

Installation

The packages you have to install depend on the features you want to use. The H SAF soil moisture NRT productsare disseminated in BUFR (H16, H103, H101, H102, H08) or GRIB (H14) format. So to read them you will have toinstall the appropriate packages which will be explained shortly. Unfortunately neither BUFR nor GRIB readers workon Windows so if you need these formats then Linux or OS X are your only options.

For installation we recommend Miniconda. So please install it according to the official installation instructions. Assoon as you have the conda command in your shell you can continue.

The following script will download and install all the needed packages.

conda create -q -n ascat python=2 numpy pandas netCDF4 pytest pip pyprojsource activate ascatconda install -c conda-forge pybufr-ecmwf # for reading BUFR filesconda install -c conda-forge pygrib=2.0.1 # for reading GRIB filespip install ascat

This script should work on Windows, Linux or OSX but on Windows you will get errors for the installation commandsof pybufr-ecmwf and pygrib.

5

http://conda.pydata.org/miniconda.html


6 Chapter 2. Installation

CHAPTER 3

Contribute

We are happy if you want to contribute. Please raise an issue explaining what is missing or if you find a bug. We willalso gladly accept pull requests against our master branch for new features or bug fixes.

3.1 Development setup

For Development we also recommend a conda environment. You can create one including test dependencies anddebugger by running conda env create -f environment.yml. This will create a new ascat-dev envi-ronment which you can activate by using source activate ascat-dev.

3.2 Guidelines

If you want to contribute please follow these steps:

• Fork the ascat repository to your account

• Clone the repository, make sure you use git clone --recursive to also get the test data repository.

• make a new feature branch from the ascat master branch

• Add your feature

• Please include tests for your contributions in one of the test directories. We use py.test so a simple functioncalled test_my_feature is enough

• submit a pull request to our master branch

7


8 Chapter 3. Contribute

CHAPTER 4

Citation

If you use the software in a publication then please cite it using the Zenodo DOI:

9

https://zenodo.org/badge/latestdoi/12761/TUW-GEO/ascat


10 Chapter 4. Citation

CHAPTER 5

Note

This project has been set up using PyScaffold 2.5.6. For details and usage information on PyScaffold see http://pyscaffold.readthedocs.org/.

11

http://pyscaffold.readthedocs.org/

http://pyscaffold.readthedocs.org/


12 Chapter 5. Note

CHAPTER 6

Contents

6.1 Examples

6.1.1 Reading and plotting Metop ASCAT Surface Soil Moisture CDR (NetCDF)

H SAF provides the following Metop ASCAT Surface Soil Moisture (SSM) Climate Data Record (CDR) products:

• H25 - Metop ASCAT SSM CDR2014 : Metop ASCAT Surface Soil Moisture CDR2014 time series 12.5 kmsampling



The following CDR extensions are also provided by H SAF:

• H108 - Metop ASCAT SSM CDR2014-EXT : Metop ASCAT Surface Soil Moisture CDR2014-EXT time series12.5 km sampling



Example H SAF SSM CDR products

In this Example we will read and plot the H SAF SSM CDR products H109, H110 and H111 using the test dataincluded in the ascat package.

import osimport ascat.h_saf as h_saf

13



test_data_path = os.path.join('..', 'tests','test-data', 'hsaf')h109_path = os.path.join(test_data_path, 'h109')h110_path = os.path.join(test_data_path, 'h110')h111_path = os.path.join(test_data_path, 'h111')grid_path = os.path.join(test_data_path, 'grid')static_layer_path = os.path.join(test_data_path, 'static_layer')

h109_reader = h_saf.H109Ts(h109_path, grid_path, static_layer_path=static_layer_path)h110_reader = h_saf.H110Ts(h110_path, grid_path, static_layer_path=static_layer_path)h111_reader = h_saf.H111Ts(h111_path, grid_path, static_layer_path=static_layer_path)

A soil moisture time series is read for a specific grid point. The data attribute contains a pandas.DataFrameobject.

gpi = 2501225h109_ts = h109_reader.read(gpi)print(h109_ts.data)

corr_flag dir proc_flag sat_id sm sm_noise→˓2007-01-01 21:06:01.843200 0 0 0 3 92 32007-01-02 09:18:16.848000 0 1 0 3 91 32007-01-03 10:37:43.104000 0 1 0 3 93 32007-01-04 20:03:56.275200 0 0 0 3 91 32007-01-05 21:23:26.246400 0 0 0 3 97 32007-01-06 09:35:41.251200 0 1 0 3 92 32007-01-06 21:02:37.507200 0 0 0 3 99 32007-01-07 09:14:52.512000 0 1 0 3 88 32007-01-08 10:34:20.582400 0 1 0 3 91 32007-01-09 20:00:33.753600 0 0 0 3 92 32007-01-10 21:20:01.910400 0 0 0 3 90 32007-01-11 09:32:16.915200 0 1 0 3 87 32007-01-11 20:59:13.084800 0 0 0 3 99 32007-01-12 09:11:28.089600 0 1 0 3 87 32007-01-12 10:51:33.753600 0 1 0 3 87 32007-01-13 10:30:58.147200 0 1 0 3 87 32007-01-14 19:57:13.132800 0 0 0 3 85 32007-01-15 21:16:37.488000 0 0 0 3 85 32007-01-16 09:28:52.492800 0 1 0 3 86 32007-01-17 10:48:11.232000 0 1 0 3 82 32007-01-18 10:27:35.625600 0 1 0 3 98 32007-01-18 20:14:26.217600 0 0 0 3 93 22007-01-19 19:53:50.611200 0 0 0 3 89 32007-01-20 21:13:13.152000 0 0 0 3 93 32007-01-21 09:25:28.156800 0 1 0 3 89 32007-01-22 10:44:48.710400 0 1 0 3 89 32007-01-23 10:24:13.104000 0 1 0 3 46 32007-01-23 20:11:03.782400 0 0 0 3 43 32007-01-24 19:50:28.089600 0 0 0 3 51 32007-01-25 21:09:48.729600 0 0 0 3 43 3... ... ... ... ... ... ...2015-12-19 21:20:37.507200 0 0 0 4 97 32015-12-20 09:32:50.611200 0 1 0 4 88 32015-12-20 20:05:52.483200 0 0 0 3 90 32015-12-20 20:59:46.867200 0 0 0 4 97 32015-12-21 09:11:59.971200 0 1 0 4 89 3

14 Chapter 6. Contents


2015-12-21 10:52:05.635200 0 1 0 4 86 32015-12-21 19:45:16.876800 0 0 0 3 89 32015-12-22 09:37:37.459200 0 1 0 3 90 22015-12-22 10:31:30.028800 0 1 0 4 90 32015-12-22 21:04:35.616000 0 0 0 3 87 32015-12-23 09:16:50.620800 0 1 0 3 89 32015-12-23 19:57:43.113600 0 0 0 4 90 32015-12-24 10:36:14.976000 0 1 0 3 89 32015-12-24 21:17:05.654400 2 0 0 4 100 32015-12-25 09:29:20.659200 2 1 0 4 100 32015-12-25 20:02:29.961600 0 0 0 3 94 32015-12-26 10:48:37.497600 0 1 0 4 90 32015-12-26 21:21:58.118400 0 0 0 3 91 32015-12-27 09:34:13.123200 0 1 0 3 86 32015-12-27 10:28:01.891200 0 1 0 4 88 32015-12-27 20:14:52.483200 0 0 0 4 86 22015-12-27 21:01:09.379200 0 0 0 3 88 32015-12-28 09:13:24.384000 0 1 0 3 85 32015-12-28 19:54:14.976000 0 0 0 4 88 32015-12-29 10:32:52.540800 0 1 0 3 86 32015-12-29 21:13:35.616000 0 0 0 4 90 32015-12-30 09:25:50.620800 0 1 0 4 88 32015-12-30 19:59:05.625600 0 0 0 3 86 32015-12-31 10:45:09.360000 0 1 0 4 91 32015-12-31 21:18:31.881600 0 0 0 3 89 3

ssf snow_prob frozen_prob abs_sm_gldas 2007-→˓01-01 21:06:01.843200 1 0 29 NaN2007-01-02 09:18:16.848000 1 0 21 NaN2007-01-03 10:37:43.104000 1 11 29 NaN2007-01-04 20:03:56.275200 1 22 29 NaN2007-01-05 21:23:26.246400 1 11 36 NaN2007-01-06 09:35:41.251200 1 0 32 NaN2007-01-06 21:02:37.507200 1 0 32 NaN2007-01-07 09:14:52.512000 1 0 18 NaN2007-01-08 10:34:20.582400 1 0 18 NaN2007-01-09 20:00:33.753600 1 0 14 NaN2007-01-10 21:20:01.910400 1 0 32 NaN2007-01-11 09:32:16.915200 1 0 36 NaN2007-01-11 20:59:13.084800 1 0 36 NaN2007-01-12 09:11:28.089600 1 0 39 NaN2007-01-12 10:51:33.753600 1 0 39 NaN2007-01-13 10:30:58.147200 1 0 21 NaN2007-01-14 19:57:13.132800 1 0 18 NaN2007-01-15 21:16:37.488000 1 0 21 NaN2007-01-16 09:28:52.492800 1 0 14 NaN2007-01-17 10:48:11.232000 1 0 21 NaN2007-01-18 10:27:35.625600 1 0 14 NaN2007-01-18 20:14:26.217600 1 0 14 NaN2007-01-19 19:53:50.611200 1 0 18 NaN2007-01-20 21:13:13.152000 1 0 29 NaN2007-01-21 09:25:28.156800 1 0 36 NaN2007-01-22 10:44:48.710400 1 0 29 NaN2007-01-23 10:24:13.104000 2 0 39 NaN

6.1. Examples 15


2007-01-23 20:11:03.782400 2 0 39 NaN2007-01-24 19:50:28.089600 2 0 25 NaN2007-01-25 21:09:48.729600 2 0 18 NaN... ... ... ... ...2015-12-19 21:20:37.507200 1 0 7 NaN2015-12-20 09:32:50.611200 1 0 11 NaN2015-12-20 20:05:52.483200 1 0 11 NaN2015-12-20 20:59:46.867200 1 0 11 NaN2015-12-21 09:11:59.971200 1 0 7 NaN2015-12-21 10:52:05.635200 1 0 7 NaN2015-12-21 19:45:16.876800 1 0 7 NaN2015-12-22 09:37:37.459200 1 0 7 NaN2015-12-22 10:31:30.028800 1 0 7 NaN2015-12-22 21:04:35.616000 1 0 7 NaN2015-12-23 09:16:50.620800 1 0 25 NaN2015-12-23 19:57:43.113600 1 0 25 NaN2015-12-24 10:36:14.976000 1 0 32 NaN2015-12-24 21:17:05.654400 1 0 32 NaN2015-12-25 09:29:20.659200 1 0 25 NaN2015-12-25 20:02:29.961600 1 0 25 NaN2015-12-26 10:48:37.497600 1 0 29 NaN2015-12-26 21:21:58.118400 1 0 29 NaN2015-12-27 09:34:13.123200 1 0 50 NaN2015-12-27 10:28:01.891200 1 0 50 NaN2015-12-27 20:14:52.483200 1 0 50 NaN2015-12-27 21:01:09.379200 1 0 50 NaN2015-12-28 09:13:24.384000 1 0 43 NaN2015-12-28 19:54:14.976000 1 0 43 NaN2015-12-29 10:32:52.540800 1 0 43 NaN2015-12-29 21:13:35.616000 1 0 43 NaN2015-12-30 09:25:50.620800 1 0 43 NaN2015-12-30 19:59:05.625600 1 0 43 NaN2015-12-31 10:45:09.360000 1 0 54 NaN2015-12-31 21:18:31.881600 1 0 54 NaN

abs_sm_noise_gldas abs_sm_hwsd abs_sm_noise_hwsd2007-01-01 21:06:01.843200 NaN NaN NaN2007-01-02 09:18:16.848000 NaN NaN NaN2007-01-03 10:37:43.104000 NaN NaN NaN2007-01-04 20:03:56.275200 NaN NaN NaN2007-01-05 21:23:26.246400 NaN NaN NaN2007-01-06 09:35:41.251200 NaN NaN NaN2007-01-06 21:02:37.507200 NaN NaN NaN2007-01-07 09:14:52.512000 NaN NaN NaN2007-01-08 10:34:20.582400 NaN NaN NaN2007-01-09 20:00:33.753600 NaN NaN NaN2007-01-10 21:20:01.910400 NaN NaN NaN2007-01-11 09:32:16.915200 NaN NaN NaN2007-01-11 20:59:13.084800 NaN NaN NaN2007-01-12 09:11:28.089600 NaN NaN NaN2007-01-12 10:51:33.753600 NaN NaN NaN2007-01-13 10:30:58.147200 NaN NaN NaN2007-01-14 19:57:13.132800 NaN NaN NaN2007-01-15 21:16:37.488000 NaN NaN NaN



2007-01-16 09:28:52.492800 NaN NaN NaN2007-01-17 10:48:11.232000 NaN NaN NaN2007-01-18 10:27:35.625600 NaN NaN NaN2007-01-18 20:14:26.217600 NaN NaN NaN2007-01-19 19:53:50.611200 NaN NaN NaN2007-01-20 21:13:13.152000 NaN NaN NaN2007-01-21 09:25:28.156800 NaN NaN NaN2007-01-22 10:44:48.710400 NaN NaN NaN2007-01-23 10:24:13.104000 NaN NaN NaN2007-01-23 20:11:03.782400 NaN NaN NaN2007-01-24 19:50:28.089600 NaN NaN NaN2007-01-25 21:09:48.729600 NaN NaN NaN... ... ... ...2015-12-19 21:20:37.507200 NaN NaN NaN2015-12-20 09:32:50.611200 NaN NaN NaN2015-12-20 20:05:52.483200 NaN NaN NaN2015-12-20 20:59:46.867200 NaN NaN NaN2015-12-21 09:11:59.971200 NaN NaN NaN2015-12-21 10:52:05.635200 NaN NaN NaN2015-12-21 19:45:16.876800 NaN NaN NaN2015-12-22 09:37:37.459200 NaN NaN NaN2015-12-22 10:31:30.028800 NaN NaN NaN2015-12-22 21:04:35.616000 NaN NaN NaN2015-12-23 09:16:50.620800 NaN NaN NaN2015-12-23 19:57:43.113600 NaN NaN NaN2015-12-24 10:36:14.976000 NaN NaN NaN2015-12-24 21:17:05.654400 NaN NaN NaN2015-12-25 09:29:20.659200 NaN NaN NaN2015-12-25 20:02:29.961600 NaN NaN NaN2015-12-26 10:48:37.497600 NaN NaN NaN2015-12-26 21:21:58.118400 NaN NaN NaN2015-12-27 09:34:13.123200 NaN NaN NaN2015-12-27 10:28:01.891200 NaN NaN NaN2015-12-27 20:14:52.483200 NaN NaN NaN2015-12-27 21:01:09.379200 NaN NaN NaN2015-12-28 09:13:24.384000 NaN NaN NaN2015-12-28 19:54:14.976000 NaN NaN NaN2015-12-29 10:32:52.540800 NaN NaN NaN2015-12-29 21:13:35.616000 NaN NaN NaN2015-12-30 09:25:50.620800 NaN NaN NaN2015-12-30 19:59:05.625600 NaN NaN NaN2015-12-31 10:45:09.360000 NaN NaN NaN2015-12-31 21:18:31.881600 NaN NaN NaN

[5194 rows x 13 columns]

Time series plots of grid points

A simple time series plot of surface soil moisture can be created using matplotlib.

import matplotlib.pyplot as plt

fig, ax = plt.subplots(1, 1, figsize=(15, 5))

6.1. Examples 17


ax.plot(h109_ts.data.index, h109_ts.data['sm'], label='H109 SSM')ax.set_ylabel('Degree of Saturation (%)')ax.legend()plt.show()

The SSM CDR H109 can be extended using H110, representing a consistent continuation of the data set

h110_ts = h110_reader.read(gpi)

fig, ax = plt.subplots(1, 1, figsize=(15, 5))ax.plot(h109_ts.data.index, h109_ts.data['sm'], label='H109')ax.plot(h110_ts.data.index, h110_ts.data['sm'], label='H110')ax.set_ylabel('Degree of Saturation (%)')ax.legend()plt.show()

A soil moisture time series can also be plotted using the plot function provided by the pandas.DataFrame objectstored in the .data attribute. The following example displays several variables stored in the time series.

h111_ts = h111_reader.read(gpi)

fields = ['sm', 'sm_noise', 'ssf', 'snow_prob', 'frozen_prob']

fig, ax = plt.subplots(1, 1, figsize=(15, 5))h111_ts.data[fields].plot(ax=ax)ax.legend()plt.show()



Masking invalid soil moisture measurements

In order to mask invalid/suspicious soil moisture measurements, the confidence flag can be used. It masks soil moisturemeasurements with a frozen or snow cover probability > 50% and using th eSurface State Flag (SSF) information.

conf_flag_ok = h111_ts.data['conf_flag'] == 0

fig, ax = plt.subplots(1, 1, figsize=(15, 5))h111_ts.data[conf_flag_ok][fields].plot(ax=ax)ax.legend()plt.show()

Differentiate between Metop-A and Metop-B soil moisture

The sat_id field can be used to differentiate between Metop-A (sat_id=3) and Metop-B (sat_id=4) measure-ments.

metop_a = h111_ts.data[conf_flag_ok]['sat_id'] == 3metop_b = h111_ts.data[conf_flag_ok]['sat_id'] == 4

fig, ax = plt.subplots(1, 1, figsize=(15, 5))h111_ts.data[conf_flag_ok]['sm'][metop_a].plot(ax=ax, ls='none', marker='s', color='C1→˓', label='Metop-A SSM')h111_ts.data[conf_flag_ok]['sm'][metop_b].plot(ax=ax, ls='none', marker='o', color='C0→˓', label='Metop-B SSM')

6.1. Examples 19


ax.set_ylabel('Degree of Saturation (%)')ax.legend()plt.show()

Convert to absolute surface soil moisture

It is possible to convert relative surface soil moisture given in degree of saturation into absolute soil moisture (𝑚3𝑚−3)using the absolute_sm keyword during reading. Porosity information provided by NOAH GLDAS and pre-computed porosity from the Harmonized World Soil Database (HWSD) using the formulas of Saxton and Rawls(2006) is used to produce volumetric surface soil moisture expressed in 𝑚3𝑚−3.

h111_ts = h111_reader.read(gpi, absolute_sm=True)conf_flag_ok = h111_ts.data['conf_flag'] == 0

fig, ax = plt.subplots(1, 1, figsize=(15, 5))h111_ts.data[conf_flag_ok]['abs_sm_gldas'].plot(ax=ax, label='Absolute SSM using→˓porosity from NOAH GLDAS')h111_ts.data[conf_flag_ok]['abs_sm_hwsd'].plot(ax=ax, label='Absolute SSM using→˓porosity from HWSD')ax.set_ylabel('Vol. soil moisture ($m^3 m^{-3}$)')ax.legend()plt.show()


https://ldas.gsfc.nasa.gov/gldas/GLDASsoils.php

http://www.fao.org/soils-portal/soil-survey/soil-maps-and-databases/harmonized-world-soil-database-v12/en/

https://dl.sciencesocieties.org/publications/sssaj/abstracts/70/5/1569

https://dl.sciencesocieties.org/publications/sssaj/abstracts/70/5/1569


6.1.2 Reading and plotting CGLS SWI_TS data (NetCDF)

This example script reads the SWI_TS product of the Copernicus Global Land Service.

If the standard file names assumed by the script have changed this can be specified during initialization of the SWI_TSobject. Please see the documentation of ascat.cgls.SWI_TS.

Example CGLS SWI time series

import osfrom ascat.cgls import SWI_TSimport matplotlib.pyplot as plt%matplotlib inline

By default we should have the grid file from the SWI-STATIC collection and the unzipped SWI-TS products in onefolder like so:

ls ../tests/test-data/cglops/swi_ts

c_gls_SWI-STATIC-DGG_201501010000_GLOBE_ASCAT_V3.0.1.ncc_gls_SWI-TS_201612310000_C0375_ASCAT_V3.0.1.nc

Now we can initialize the data reader

data_path = os.path.join('..', 'tests', 'test-data', 'cglops', 'swi_ts')rd = SWI_TS(data_path)

With this object we can read the data

data = rd.read_ts(3002621)print(data)

SSF SWI_001 SWI_005 SWI_010 SWI_015 SWI_020 2007-→˓05-10 12:00:00 1.0 70.0 71.0 71.0 71.0 71.02007-05-14 12:00:00 1.0 71.5 71.5 71.5 71.5 71.52007-05-15 12:00:00 1.0 82.0 78.0 77.0 76.5 76.52007-05-16 12:00:00 1.0 82.5 79.0 78.0 77.5 77.02007-05-17 12:00:00 1.0 77.5 77.5 77.0 76.5 76.52007-05-18 12:00:00 1.0 75.0 76.5 76.0 76.0 76.02007-05-19 12:00:00 1.0 75.5 76.0 76.0 76.0 76.02007-05-20 12:00:00 1.0 86.0 81.0 79.5 79.5 79.02007-05-21 12:00:00 1.0 84.5 81.0 80.0 79.5 79.52007-05-22 12:00:00 1.0 81.5 80.5 80.0 79.5 79.02007-05-23 12:00:00 1.0 81.5 80.5 80.0 79.5 79.52007-05-24 12:00:00 1.0 81.0 81.0 80.0 80.0 79.52007-05-25 12:00:00 1.0 79.5 80.0 79.5 79.5 79.52007-05-26 12:00:00 1.0 81.5 80.5 80.0 79.5 79.52007-05-27 12:00:00 1.0 76.5 79.5 79.5 79.0 79.02007-05-28 12:00:00 1.0 80.5 79.5 79.5 79.5 79.52007-05-29 12:00:00 1.0 82.0 80.5 80.0 79.5 79.52007-05-30 12:00:00 1.0 83.0 81.0 80.5 80.0 80.02007-05-31 12:00:00 1.0 79.0 80.5 80.0 80.0 80.02007-06-01 12:00:00 1.0 75.5 79.5 79.5 79.5 79.52007-06-02 12:00:00 1.0 80.5 79.5 79.5 79.5 79.52007-06-03 12:00:00 1.0 85.0 81.5 80.5 80.5 80.0

6.1. Examples 21


2007-06-04 12:00:00 1.0 85.0 82.0 81.0 81.0 80.52007-06-05 12:00:00 1.0 84.5 82.0 81.5 81.0 81.02007-06-06 12:00:00 1.0 79.5 81.5 81.0 80.5 80.52007-06-07 12:00:00 1.0 77.5 80.0 80.0 80.0 80.02007-06-08 12:00:00 1.0 77.0 79.5 80.0 80.0 80.02007-06-09 12:00:00 1.0 77.0 79.5 79.5 80.0 80.02007-06-10 12:00:00 1.0 77.0 79.0 79.5 79.5 79.52007-06-11 12:00:00 1.0 76.5 78.0 79.0 79.0 79.5... ... ... ... ... ... ...2016-09-19 12:00:00 1.0 73.5 76.5 78.5 80.0 81.02016-09-20 12:00:00 1.0 74.5 76.0 78.5 80.0 81.02016-09-21 12:00:00 1.0 75.5 76.0 78.0 79.5 80.52016-09-22 12:00:00 1.0 73.5 75.5 77.5 79.0 80.52016-09-23 12:00:00 1.0 72.0 75.0 77.5 79.0 80.02016-09-24 12:00:00 1.0 69.5 73.5 76.5 78.0 79.52016-09-25 12:00:00 1.0 71.0 73.0 75.5 77.5 79.02016-09-26 12:00:00 1.0 71.5 73.0 75.5 77.0 78.52016-09-27 12:00:00 1.0 77.0 74.0 76.0 77.5 78.52016-09-28 12:00:00 1.0 83.5 76.0 76.5 78.0 79.02016-09-29 12:00:00 1.0 92.0 81.0 79.0 79.5 80.02016-09-30 12:00:00 1.0 93.5 82.5 80.0 80.0 80.52016-10-01 12:00:00 1.0 90.0 84.0 81.0 81.0 81.02016-10-02 12:00:00 1.0 84.0 83.5 81.0 80.5 81.02016-10-03 12:00:00 1.0 72.0 80.0 79.5 80.0 80.02016-10-04 12:00:00 1.0 64.5 76.0 77.5 78.5 79.02016-10-05 12:00:00 1.0 57.5 71.5 75.0 76.5 78.02016-10-06 12:00:00 1.0 65.0 70.5 74.0 75.5 77.02016-10-07 12:00:00 1.0 66.0 70.0 73.5 75.0 76.52016-10-08 12:00:00 1.0 68.0 69.5 73.0 75.0 76.02016-10-09 12:00:00 1.0 67.5 69.0 72.5 74.5 76.02016-10-10 12:00:00 1.0 65.5 68.5 71.5 74.0 75.52016-10-11 12:00:00 1.0 60.0 66.0 70.0 72.5 74.02016-10-12 12:00:00 1.0 54.0 63.5 68.5 71.0 73.02016-10-13 12:00:00 1.0 52.5 61.0 66.5 69.5 72.02016-10-14 12:00:00 1.0 55.0 60.0 65.5 69.0 71.02016-10-15 12:00:00 1.0 56.5 59.0 64.5 68.0 70.52016-10-16 12:00:00 1.0 61.5 60.0 64.5 67.5 70.02016-10-26 12:00:00 1.0 12.5 42.5 59.0 64.5 68.02016-10-28 12:00:00 1.0 69.5 54.0 60.0 64.5 67.5

SWI_040 SWI_060 SWI_1002007-05-10 12:00:00 71.0 NaN NaN2007-05-14 12:00:00 71.5 71.5 NaN2007-05-15 12:00:00 76.0 76.0 76.02007-05-16 12:00:00 77.0 77.0 76.52007-05-17 12:00:00 76.0 76.0 76.02007-05-18 12:00:00 76.0 76.0 76.02007-05-19 12:00:00 76.0 75.5 75.52007-05-20 12:00:00 79.0 78.5 78.52007-05-21 12:00:00 79.0 79.0 79.02007-05-22 12:00:00 79.0 79.0 78.52007-05-23 12:00:00 79.0 79.0 79.02007-05-24 12:00:00 79.5 79.5 79.02007-05-25 12:00:00 79.0 79.0 79.0



2007-05-26 12:00:00 79.5 79.0 79.02007-05-27 12:00:00 79.0 79.0 79.02007-05-28 12:00:00 79.0 79.0 79.02007-05-29 12:00:00 79.5 79.5 79.02007-05-30 12:00:00 80.0 79.5 79.52007-05-31 12:00:00 79.5 79.5 79.52007-06-01 12:00:00 79.5 79.0 79.02007-06-02 12:00:00 79.5 79.0 79.02007-06-03 12:00:00 80.0 80.0 80.02007-06-04 12:00:00 80.5 80.0 80.02007-06-05 12:00:00 80.5 80.5 80.02007-06-06 12:00:00 80.5 80.0 80.02007-06-07 12:00:00 80.0 80.0 79.52007-06-08 12:00:00 80.0 79.5 79.52007-06-09 12:00:00 79.5 79.5 79.52007-06-10 12:00:00 79.5 79.5 79.52007-06-11 12:00:00 79.5 79.5 79.5... ... ... ...2016-09-19 12:00:00 83.0 84.0 84.02016-09-20 12:00:00 83.0 83.5 84.02016-09-21 12:00:00 83.0 83.5 84.02016-09-22 12:00:00 82.5 83.5 83.52016-09-23 12:00:00 82.5 83.0 83.52016-09-24 12:00:00 82.0 83.0 83.52016-09-25 12:00:00 81.5 82.5 83.02016-09-26 12:00:00 81.5 82.5 83.02016-09-27 12:00:00 81.5 82.5 83.02016-09-28 12:00:00 81.5 82.5 83.02016-09-29 12:00:00 82.0 83.0 83.52016-09-30 12:00:00 82.0 83.0 83.52016-10-01 12:00:00 82.5 83.0 83.52016-10-02 12:00:00 82.5 83.0 83.52016-10-03 12:00:00 82.0 82.5 83.02016-10-04 12:00:00 81.5 82.5 83.02016-10-05 12:00:00 80.5 81.5 82.52016-10-06 12:00:00 80.0 81.5 82.02016-10-07 12:00:00 79.5 81.0 82.02016-10-08 12:00:00 79.5 81.0 82.02016-10-09 12:00:00 79.0 80.5 81.52016-10-10 12:00:00 79.0 80.5 81.52016-10-11 12:00:00 78.0 80.0 81.02016-10-12 12:00:00 77.5 79.5 81.02016-10-13 12:00:00 77.0 79.0 80.52016-10-14 12:00:00 76.5 78.5 80.02016-10-15 12:00:00 75.5 78.0 80.02016-10-16 12:00:00 75.5 78.0 79.52016-10-26 12:00:00 74.5 77.0 79.02016-10-28 12:00:00 74.0 76.5 79.0

[1603 rows x 9 columns]

Since the returned value is a pandas.DataFrame we can plot the data easily.

6.1. Examples 23


fig, ax = plt.subplots(1, 1, figsize=(15, 5))data[['SWI_001', 'SWI_010']].plot(ax=ax)ax.set_ylabel('Soil Water Index (%)')plt.show()

6.1.3 Reading and plotting H SAF NRT Surface Soil Moisture products (BUFR)

H SAF provides the following NRT surface soil moisture products:

• (H07 - SM OBS 1 : Large scale surface soil moisture by radar scatterometer in BUFR format over Europe) -discontinued

• H08 - SSM ASCAT NRT DIS : Disaggregated Metop ASCAT NRT Surface Soil Moisture at 1 km

• H14 - SM DAS 2 : Profile index in the roots region by scatterometer data assimilation in GRIB format

• H101 - SSM ASCAT-A NRT O12.5 : Metop-A ASCAT NRT Surface Soil Moisture 12.5km sampling

• H102 - SSM ASCAT-A NRT O25 : Metop-A ASCAT NRT Surface Soil Moisture 25 km sampling

• H16 - SSM ASCAT-B NRT O12.5 : Metop-B ASCAT NRT Surface Soil Moisture 12.5 km sampling

• H103 - SSM ASCAT-B NRT O25 : Metop-B ASCAT NRT Surface Soil Moisture 25 km sampling

The products H101, H102, H16, H103 come in BUFR format and can be read by the same reader. So examples for theH16 product are equally valid for the other products.

The product H07 is discontinued and replaced by Metop-A (H101, H102) and Metop-B (H103, H16), both availablein two different resolutions.

The following example will show how to read and plot each of them.

Example H SAF NRT SSM products

In this Example we will read and plot images of the H SAF NRT products H08, H14 and H16 using the test imagesincluded in the ascat package.

import osfrom datetime import datetime

import pytesmo.colormaps.load_cmap as smcolormapsimport ascat.h_saf as h_saf

test_data_path = os.path.join('..', 'tests','test-data', 'hsaf')




h08_path = os.path.join(test_data_path, 'h08')h14_path = os.path.join(test_data_path, 'h14')h16_path = os.path.join(test_data_path, 'h16')

h08_reader = h_saf.H08img(h08_path)h14_reader = h_saf.H14img(h14_path)h16_reader = h_saf.H16img(h16_path, month_path_str='')

Reading H08 product

H08 data has a much higher resolution and comes on a 0.00416 degree grid.

The sample data included in the ascat package was observed on the same time as the included H16 product.

Instead of read you can also use the daily_images iterator.

You just specify a day and it will read all the images that are in your folder for this day.

This also works for the H16, H101, H102, H103 and H14 reader.

for h08_data, metadata, timestamp, lons, lats, time_var in h08_reader.daily_→˓images(datetime(2010, 5, 1)):

# this tells you the exact timestamp of the read imageprint(timestamp.isoformat())print(type(h08_data))

# the data is a dictionary, each dictionary key contains the array of one variableprint("The following variables are in this image", h08_data.keys())print(h08_data['ssm'].shape)print(lons.shape)print(lats.shape)

2010-05-01T08:33:01<type 'dict'>('The following variables are in this image', ['ssm', 'proc_flag', 'ssm_noise', 'corr_→˓flag'])(3120, 7680)(3120, 7680)(3120, 7680)

In our case only one image is in the folder so the loop exits after this image is read.

The data has higher resolution but it already comes as a 2D image.

Let’s plot it.

import numpy as npimport cartopyimport matplotlib.pyplot as plt%matplotlib inline

plot_crs = cartopy.crs.Mercator()data_crs = cartopy.crs.PlateCarree()

fig = plt.figure(figsize=(8, 10))

6.1. Examples 25


ax = fig.add_axes([0.05, 0.2, 0.6, 0.7], projection=plot_crs)

ax.set_title('H08 example')

blue = '#4b92db'ax.background_patch.set_facecolor(blue)ax.add_feature(cartopy.feature.GSHHSFeature(scale='low', facecolor='#efefdb'))ax.add_feature(cartopy.feature.BORDERS, linestyle=':')ax.add_feature(cartopy.feature.LAKES)ax.add_feature(cartopy.feature.RIVERS)

ax.set_extent([10, 47, 50, 75])

data = np.ma.masked_greater(np.flipud(h08_data['ssm']), 100)sc = ax.pcolormesh(lons, np.flipud(lats), data, zorder=3,

transform=data_crs, cmap=smcolormaps.load('SWI_ASCAT'),vmin=0, vmax=100)

cax = fig.add_axes([0.65, 0.2, 0.025, 0.7])cbar = fig.colorbar(sc, cax=cax)cbar.set_label('Degree of Saturation (%)')plt.show()



Reading a Region Of Interest (ROI) of H08

H08 has a very high resolution and users might want to read only data for their area of interest. This can be done usingthe lat_lon_bbox keyword

6.1. Examples 27


# the reader returns not only the data but also metadata and the longitudes and→˓latitudesh08_roi, metadata, timestamp, lons, lats, time_var = \

h08_reader.read(datetime(2010, 5, 1, 8, 33, 1), lat_lon_bbox=[60, 70, 15, 25])


fig = plt.figure(figsize=(8, 10))ax = fig.add_axes([0.05, 0.2, 0.6, 0.7], projection=plot_crs)# initialize the readers with the pathax.set_title('H08 ROI example')

blue = '#4b92db'ax.background_patch.set_facecolor(blue)ax.add_feature(cartopy.feature.GSHHSFeature(scale='low', facecolor='#efefdb'))ax.add_feature(cartopy.feature.BORDERS, linestyle=':')ax.add_feature(cartopy.feature.LAKES)ax.add_feature(cartopy.feature.RIVERS)

ax.set_extent([10, 47, 50, 75])

data = np.ma.masked_greater(np.flipud(h08_roi['ssm']), 100)sc = ax.pcolormesh(lons, np.flipud(lats), data, zorder=3,

transform=data_crs, cmap=smcolormaps.load('SWI_ASCAT'),vmin=0, vmax=100)




Reading H14 product

The H14 product is a global product on a reduced gaussian grid with a resolution of approx. 25km.

# the reader returns not only the data but also metadata and the longitudes and→˓latitudesh14_data, metadata, timestamp, lons, lats, time_var = h14_reader.read(datetime(2014,→˓5, 15))

6.1. Examples 29


print(type(h14_data))

# the data is a dictionary, each dictionary key contains the array of one variableprint("The following variables are in this image", h14_data.keys())print(h14_data['SM_layer1_0-7cm'].shape)print(lons.shape)print(lats.shape)

<type 'dict'>('The following variables are in this image', ['SM_layer1_0-7cm', 'SM_layer2_7-28cm',→˓'SM_layer3_28-100cm', 'SM_layer4_100-289cm'])(800, 1600)(800, 1600)(800, 1600)

The data comes as a 2D array. If the keyword expand_grid is set to False during reader initialization then only 1Darrays would be returned.

This can be good for working with the data but for plotting the expanded grid is easier to handle.

Let’s plot all layers in the H14 product

plot_crs = cartopy.crs.Robinson()data_crs = cartopy.crs.PlateCarree()

for layer in h14_data:fig = plt.figure(figsize=(12, 6))ax = fig.add_axes([0.1, 0.1, 0.8, 0.8], projection=plot_crs)ax.set_title('H14 {:}'.format(layer))

ax.add_feature(cartopy.feature.LAND)ax.add_feature(cartopy.feature.OCEAN)ax.add_feature(cartopy.feature.COASTLINE)ax.add_feature(cartopy.feature.BORDERS, linestyle=':')ax.add_feature(cartopy.feature.LAKES, alpha=0.5)ax.add_feature(cartopy.feature.RIVERS)

sc = ax.pcolormesh(lons, lats, h14_data[layer], zorder=3,transform=data_crs, cmap=smcolormaps.load('SWI_ASCAT'))

cax = fig.add_axes([0.92, 0.1, 0.025, 0.8])cbar = fig.colorbar(sc, cax=cax)cbar.set_label('Liquid Root Zone Soil Moister')plt.show()



6.1. Examples 31


Reading H16 product

The products H16, H101, H102, H103 come in the same BUFR format. Since the default filenames are slightlydifferent the following readers should be used:





• H07 - ascat.h_saf.H07img - discontinued product

Otherwise the format and API is identical.



The ascat repository includes one H16 file with the timestamp 2017-02-20 11:00:00. We can either read this file aloneif we know the exact timestamp or iterate over all images on 2017-02-20.

# the reader returns not only the data but also metadata, longitudes and latitudesh16_data, metadata, timestamp, lons, lats, time_var = h16_reader.read(datetime(2017,→˓2, 20, 11, 15, 0))

print(type(h16_data))

# the data is a dictionary, each dictionary key contains the array of one variableprint("The following variables are in this image", h16_data.keys())print h16_data['Surface Soil Moisture (Ms)'].shape

# it is only a 1D array to plot the data we also need latitude and logitude→˓informationprint(lons.shape)print(lats.shape)

<type 'dict'>('The following variables are in this image', ['Estimated Error In Sigma0 At 40 Deg→˓Incidence Angle', 'Slope At 40 Deg Incidence Angle', 'Topographic Complexity',→˓'Surface Soil Moisture (Ms)', 'Inundation And Wetland Fraction', 'Wet Backscatter',→˓'Snow Cover', 'Rain Fall Detection', 'Frozen Land Surface Fraction', 'Mean Surface→˓Soil Moisture', 'Estimated Error In Slope At 40 Deg Incidence Angle', 'Soil→˓Moisture Sensitivity', 'Soil Moisture Processing Flag', 'jd', 'Dry Backscatter',→˓'Soil Moisture Correction Flag', 'Soil Moisture Quality', 'Orbit Number',→˓'Estimated Error In Surface Soil Moisture', 'Backscatter', 'Direction Of Motion Of→˓Moving Observing Platform'])(637,)(637,)(637,)

Plot the SSM data as individual points


fig = plt.figure(figsize=(7, 6))ax = fig.add_axes([0.1, 0.1, 0.8, 0.8], projection=plot_crs)ax.set_title('H16 example - Original data')

blue = '#4b92db'ax.background_patch.set_facecolor(blue)ax.add_feature(cartopy.feature.GSHHSFeature(scale='low', facecolor='#efefdb'))ax.add_feature(cartopy.feature.BORDERS, linestyle=':')ax.add_feature(cartopy.feature.LAKES, alpha=0.5)ax.add_feature(cartopy.feature.RIVERS)ax.set_extent([130, 175, -10, -42])

data = h16_data['Surface Soil Moisture (Ms)']

sc = ax.scatter(lons, lats, c=data, zorder=3, marker='s', s=2,transform=data_crs, cmap=smcolormaps.load('SWI_ASCAT'),vmin=0, vmax=100)

cax = fig.add_axes([0.92, 0.1, 0.025, 0.8])cbar = fig.colorbar(sc, cax=cax)cbar.set_label('Degree of Saturation (%)')

6.1. Examples 33


plt.show()

Or resample orbit geometry to a regular 0.1 deg x 0.1 deg grid for plotting

import pytesmo.grid.resample as resample

# lets resample to a 0.1 degree grid# define the grid points in latitude and logitudelats_dim = np.arange(-80, 80, 0.1)lons_dim = np.arange(-160, 170, 0.1)

# make 2d grid out the 1D grid spacingslons_grid, lats_grid = np.meshgrid(lons_dim, lats_dim)

resampled_data = resample.resample_to_grid(h16_data, lons, lats,lons_grid, lats_grid)

fig = plt.figure(figsize=(7, 6))ax = fig.add_axes([0.1, 0.1, 0.8, 0.8], projection=plot_crs)ax.set_title('H16 example - Resampled to 0.1 x 0.1 grid')

blue = '#4b92db'ax.background_patch.set_facecolor(blue)



ax.add_feature(cartopy.feature.GSHHSFeature(scale='low', facecolor='#efefdb'))ax.add_feature(cartopy.feature.BORDERS, linestyle=':')ax.add_feature(cartopy.feature.LAKES, alpha=0.5)ax.add_feature(cartopy.feature.RIVERS)ax.set_extent([130, 175, -10, -42])

data = resampled_data['Surface Soil Moisture (Ms)']sc = ax.pcolormesh(lons_grid, lats_grid, data, zorder=3,

cmap=smcolormaps.load('SWI_ASCAT'),vmin=0, vmax=100, transform=data_crs)


6.1.4 Reading and plotting TU Wien Metop ASCAT Surface Soil Moisture (Binary)

This example program reads and plots Metop ASCAT SSM and SWI data with different masking options. The readersare only provided for the sake of completeness, because the data sets are outdated and superseded by the H SAFSurface Soil Moisture Climate Data Records (e.g. H109, H111). The SWI data sets are replaced by the CGLS SWIproduct.

6.1. Examples 35


Example TU Wien Soil Moisture time series

In this Example we will read and plot TU Wien soil moisture time series using the test data included in the ascatpackage.

import osimport matplotlib.pyplot as pltimport ascat.tuw as tuw

test_data_path = os.path.join('..', 'tests','test-data', 'tuw')sm_path = os.path.join(test_data_path, 'ascat', 'ssm')grid_path = os.path.join(test_data_path, 'grid')advisory_flags_path = os.path.join(test_data_path, 'advisory_flags')

Read surface soil moisture data nearest to this lon/lat coordinates and plot the data using pandas builtin plot function-ality.

ascat_ssm_reader = tuw.Ascat_SSM(sm_path, grid_path,advisory_flags_path=advisory_flags_path)

lon, lat = 14.3, 45.7ssm_data_raw = ascat_ssm_reader.read_ssm(lon, lat)ssm_data_raw.plot()plt.show()

Read the same data but mask observations where the SSF shows frozen and where frozen and snow probabilty aregreater than 20% and plot the data using pandas builtin plot functionality.

ssm_data_masked = ascat_ssm_reader.read_ssm(lon, lat, mask_ssf=True,mask_frozen_prob=20, mask_snow_prob=20)

ssm_data_masked.plot()plt.show()



Plot raw and masked SSM data in one plot to compare them using pandas plotting functions.

fig, ax = plt.subplots(1, 1, figsize=(15, 5))ssm_data_raw.data['SSM'].plot(ax=ax, label='raw SSM data')ssm_data_masked.data['SSM'].plot(ax=ax, label='masked SSM data')ax.legend()plt.show()

6.1.5 Reading and plotting TU Wien Metop ASCAT Vegetation Optical Depth (VOD)

This example program reads and plots Metop ASCAT VOD data.

Example Vegetation Optical Depth (VOD)

In this Example we will read and plot Metop ASCAT Vegetation Optical Depth (VOD) using the test data included inthe ascat package.

import osimport matplotlib.pyplot as pltimport ascat.tuw as tuw

test_data_path = os.path.join('..', 'tests','test-data')sm_path = os.path.join(test_data_path, 'tuw', 'ascat', 'vod')grid_path = os.path.join(test_data_path, 'hsaf', 'grid')

6.1. Examples 37


Read VOD from grid point and plot the data using pandas builtin plot functionality.

ascat_vod_reader = tuw.AscatVodTs(sm_path, grid_path)

gpi = 2199945vod = ascat_vod_reader.read(gpi)

fig, ax = plt.subplots(1, 1, figsize=(15, 5))vod.plot(ax=ax)ax.set_ylabel('VOD (-)')ax.legend()plt.show()

6.2 ASCAT BUFR format table

Taken from http://projects.knmi.nl/scatterometer/publications/pdf/ASCAT_Product_Manual.pdf

Number Descriptor Parameter Unit1 001033 Identification Of Originating/Generating Centre Code Table2 001034 Identification Of Originating/Generating Sub-Centre Code Table3 025060 Software Identification Numeric4 001007 Satellite Identifier Code Table5 002019 Satellite Instruments Code Table6 001012 Direction Of Motion Of Moving Observing Platform Degree True7 004001 Year Year8 004002 Month Month9 004003 Day Day10 004004 Hour Hour11 004005 Minute Minute12 004006 Second Second13 005001 Latitude (High Accuracy) Degree14 006001 Longitude (High Accuracy) Degree15 005033 Pixel Size On Horizontal-1 m16 005040 Orbit Number Numeric17 006034 Cross Track Cell Number Numeric18 010095 Height Of Atmosphere Used m19 021157 Loss Per Unit Length Of Atmosphere Used dB/m20 021150 Beam Collocation Flag Table


http://projects.knmi.nl/scatterometer/publications/pdf/ASCAT_Product_Manual.pdf


Number Descriptor Parameter Unit21 008085 Beam Identifier Code Table22 002111 Radar Incidence Angle Degree23 002134 Antenna Beam Azimuth Degree24 021062 Backscatter dB25 021063 Radiometric Resolution (Noise Value) %26 021158 ASCAT Kp Estimate Quality Code Table27 021159 ASCAT Sigma-0 Usability Code Table28 021160 ASCAT Use Of Synthetic Data Numeric29 021161 ASCAT Synthetic Data Quality Numeric30 021162 ASCAT Satellite Orbit And Attitude Quality Numeric31 021163 ASCAT Solar Array Reflection Contamination Numeric32 021164 ASCAT Telemetry Presence And Quality Numeric33 021165 ASCAT Extrapolated Reference Function Numeric34 021166 ASCAT Land Fraction Numeric



6.2. ASCAT BUFR format table 39


Number Descriptor Parameter Unit63 025060 Software Identification Numeric64 025062 Database Identification Numeric65 040001 Surface Soil Moisture (Ms) %66 040002 Estimated Error In Surface Soil Moisture %67 021062 Backscatter dB68 021151 Estimated Error In Sigma0 At 40 Deg Incidence Angle dB69 021152 Slope At 40 Deg Incidence Angle dB/Degree70 021153 Estimated Error In Slope At 40 Deg Incidence Angle dB/Degree71 021154 Soil Moisture Sensitivity dB72 021062 Dry Backscatter dB73 021088 Wet Backscatter dB74 040003 Mean Surface Soil Moisture Numeric75 040004 Rain Fall Detection Numeric76 040005 Soil Moisture Correction Flag Flag Table77 040006 Soil Moisture Processing Flag Flag Table78 040007 Soil Moisture Quality %79 020065 Snow Cover %80 040008 Frozen Land Surface Fraction %81 040009 Inundation And Wetland Fraction %82 040010 Topographic Complexity %

Number Descriptor Parameter Unit83 025060 Software Identification Numeric84 001032 Generating Application Code Table85 011082 Model Wind Speed At 10 m m/s86 011081 Model Wind Direction At 10 m Degree True87 020095 Ice Probability Numeric88 020096 Ice Age (A-Parameter) dB89 021155 Wind Vector Cell Quality Flag Table90 021101 Number Of Vector Ambiguities Numeric91 021102 Index Of Selected Wind Vector Numeric92 031001 Delayed Descriptor Replication Factor Numeric93 011012 Wind Speed At 10 m m/s94 011011 Wind Direction At 10 m Degree True95 021156 Backscatter Distance Numeric96 021104 Likelihood Computed For Solution Numeric97 011012 Wind Speed At 10 m m/s98 011011 Wind Direction At 10 m Degree True99 021156 Backscatter Distance Numeric100 021104 Likelihood Computed For Solution Numeric

Note that descriptor numbers 93-96 can be repeated 1 to 144 times, depending on the value of the Delayed DescriptorReplication Factor (descriptor number 92)

6.3 License

Copyright (c) 2016, TU WienAll rights reserved.



Redistribution and use in source and binary forms, with or withoutmodification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, thislist of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,this list of conditions and the following disclaimer in the documentationand/or other materials provided with the distribution.

* Neither the name of ascat nor the names of itscontributors may be used to endorse or promote products derived fromthis software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THEIMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AREDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLEFOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIALDAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS ORSERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVERCAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USEOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

6.4 Developers

• Christoph Paulik <[email protected]>

• Sebastian Hahn <[email protected]>

6.5 Changelog

6.5.1 Version 0.8

• Add reader for ASCAT VOD time series data.

• Add readers for all H-SAF time series products.

6.5.2 Version 0.7

• Fix bugs in BUFR reading with newer numpy versions.

6.5.3 Version 0.6

• Fix bug when reading CGLS SWI QFLAG values.

• Add chunked half-orbit readers for the three minute PDU BUFR files.

6.4. Developers 41

mailto:[email protected]



6.5.4 Version 0.5

• Include resample interface for Level 2 BUFR data.

6.5.5 Version 0.4

• Restructure ASCAT swath readers and add support for NetCDF, BUFR and BUFR PDU files from EUMETSAT.

• Fix read_ts function of CGLS SWI_TS reader.

6.5.6 Version 0.3

• Add reader for Copernicus Global Land SWI_TS products.

6.5.7 Version 0.2

• Fix pygrib support for pygrib 2.x for H14 products.

• Internal changes. Readers now based on pynetCF and pygeobase.

6.5.8 Version 0.1

• Initial version with readers migrated from the pytesmo package.

6.6 ascat

6.6.1 ascat package

Submodules

ascat.bufr module

Created on May 21, 2014

@author: Christoph Paulik [email protected]

class ascat.bufr.BUFRReader(filename, kelem_guess=500, max_tries=10)Bases: object

BUFR reader based on the pybufr-ecmwf package but faster

Parameters

• filename (string) – filename of the bufr file

• kelem_guess (int, optional) – if the elements per variable in as message areknown please specify here. Otherwise the elements will be found out via trial and errorThis works most of the time but is not 100 percent failsafe Default: 500

• max_tries (int, optional) – the Reader will try max_tries times to unpack a bufrmessage. Some messages can not be read even if the array sizes are ok. Most of the timethese files are corrupt.



https://docs.python.org/3.6/library/functions.html#object

https://docs.python.org/3.6/library/string.html#module-string

https://docs.python.org/3.6/library/functions.html#int



messages()

Raises IOError: – if a message cannot be unpacked after max_tries tries

Returns data

Return type yield results of messages

ascat.cgls module

Module for reading CGLOPS SWI TS products

class ascat.cgls.SWI_TS(data_path, parameters=[‘SWI_001’, ‘SWI_005’, ‘SWI_010’,‘SWI_015’, ‘SWI_020’, ‘SWI_040’, ‘SWI_060’, ‘SWI_100’, ‘SSF’],dt=‘201612310000’, version=‘3.0.1’, grid_fname=None, read_bulk=True,fname_template=’c_gls_SWI-TS_{dt}_C{{:04d}}_ASCAT_V{version}’)

Bases: pynetcf.time_series.GriddedNcOrthoMultiTs

SWI TS reader for timeseries data from CGLOPS

Parameters

• data_path (string) – path to the netCDF files

• parameters (list) – list of parameters to read from netCDF file

• dt (string, optional) – datetime in the filenames of the cells

• version (string, optional) – version number of the files

• grid_fname (string, optional) – filename + path of the gridnetCDF file, default is the standard grid file (c_gls_SWI-STATIC-DGG_201501010000_GLOBE_ASCAT_V3.0.1.nc) in the same folder as the data

• read_bulk (boolean, optional) – if set to true then a complete 5x5 degree cell willbe read at once providing speedup if the complete data is needed.

• fname_template (string, optional) – Filename template. Has to have two slotsfor {dt} and {version} and a slot for the cell number that is available for further formatting.Because of this the cell number location has to be written as ‘{{:04d}}’. The has to bewithout the .nc ending since this is added during reading.

ascat.eumetsat module

Readers for data downloaded from EUMETSAT data centre (UMARF)

class ascat.eumetsat.AscatAL2Ssm125(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmBufr

ASCAT A Level2 Soil Moisture at 12.5 km Swath Grid BUFR files from EUMETSAT

Parameters

• path (string) – path where the data is stored

• month_path_str (string, optional) – If the data is stored in subpaths per yearor month then specify the string that should be used in datetime.datetime.strftime to get thesubpath for a file. Default: ‘’

class ascat.eumetsat.AscatAL2Ssm125Nc(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmNc

ASCAT A Level2 Soil Moisture at 12.5 km Swath Grid NetCDF files from EUMETSAT

6.6. ascat 43


https://docs.python.org/3.6/library/stdtypes.html#list








Parameters



class ascat.eumetsat.AscatAL2Ssm125PDU(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmBufr

ASCAT A Level2 Soil Moisture at 12.5 km Swath Grid PDU BUFR files from EUMETSAT

Parameters



class ascat.eumetsat.AscatAL2Ssm125PDUChunked(path, month_path_str=’‘,chunk_minutes=100)

Bases: ascat.level2.AscatL2SsmBufrChunked

ASCAT A Level2 Soil Moisture at 12.5 km Swath Grid PDU BUFR files from EUMETSAT in 50 minutechunks.

Parameters



• chunk_minutes (int, optional) – How many minutes should a chunk of datacover.

class ascat.eumetsat.AscatAL2Ssm250(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmBufr

ASCAT A Level2 Soil Moisture at 25.0 km Swath Grid BUFR files from EUMETSAT

Parameters



class ascat.eumetsat.AscatAL2Ssm250Nc(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmNc

ASCAT A Level2 Soil Moisture at 25 km Swath Grid NetCDF files from EUMETSAT

Parameters
















class ascat.eumetsat.AscatAL2Ssm250PDU(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmBufr

ASCAT A Level2 Soil Moisture at 25 km Swath Grid PDU BUFR files from EUMETSAT

Parameters



class ascat.eumetsat.AscatAL2Ssm250PDUChunked(path, month_path_str=’‘,chunk_minutes=100)


ASCAT A Level2 Soil Moisture at 25 km Swath Grid PDU BUFR files from EUMETSAT

Parameters




class ascat.eumetsat.AscatBL2Ssm125(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmBufr

ASCAT B Level2 Soil Moisture at 12.5 km Swath Grid BUFR files from EUMETSAT

Parameters



class ascat.eumetsat.AscatBL2Ssm125Nc(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmNc

ASCAT B Level2 Soil Moisture at 12.5 km Swath Grid NetCDF files from EUMETSAT

Parameters



class ascat.eumetsat.AscatBL2Ssm125PDU(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmBufr

ASCAT B Level2 Soil Moisture at 12.5 km Swath Grid PDU BUFR files from EUMETSAT

Parameters


6.6. ascat 45













class ascat.eumetsat.AscatBL2Ssm125PDUChunked(path, month_path_str=’‘,chunk_minutes=100)


ASCAT B Level2 Soil Moisture at 12.5 km Swath Grid PDU BUFR files from EUMETSAT in 50 minute chunks.

Parameters




class ascat.eumetsat.AscatBL2Ssm250(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmBufr

ASCAT B Level2 Soil Moisture at 25.0 km Swath Grid BUFR files from EUMETSAT

Parameters



class ascat.eumetsat.AscatBL2Ssm250Nc(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmNc

ASCAT B Level2 Soil Moisture at 25 km Swath Grid NetCDF files from EUMETSAT

Parameters



class ascat.eumetsat.AscatBL2Ssm250PDU(path, month_path_str=’‘)Bases: ascat.level2.AscatL2SsmBufr

ASCAT B Level2 Soil Moisture at 25 km Swath Grid PDU BUFR files from EUMETSAT

Parameters



class ascat.eumetsat.AscatBL2Ssm250PDUChunked(path, month_path_str=’‘,chunk_minutes=100)


ASCAT B Level2 Soil Moisture at 25 km Swath Grid PDU BUFR files from EUMETSAT













Parameters




ascat.h_saf module

class ascat.h_saf.H07img(path, month_path_str=’h07_%Y%m_buf’,day_search_str=’h07_%Y%m%d_*.buf’, file_search_str=’h07_{datetime}*.buf’,datetime_format=’%Y%m%d_%H%M%S’, filename_datetime_format=(4,19, ’%Y%m%d_%H%M%S’), msg_name_lookup=None)

Bases: ascat.level2.AscatL2SsmBufr

class ascat.h_saf.H08Single(filename, mode=’r’, **kwargs)Bases: pygeobase.io_base.ImageBase

close()

flush()

read(timestamp=None, lat_lon_bbox=None)Read specific image for given datetime timestamp.

Parameters

• filename (string) – filename

• timestamp (datetime.datetime) – exact observation timestamp of the image thatshould be read

• lat_lon_bbox (list, optional) – list of lat,lon cooridnates of bounding box[lat_min, lat_max, lon_min, lon_max]

Returns

• data (dict or None) – dictionary of numpy arrays that hold the image data for each variableof the dataset, if no data was found None is returned

• metadata (dict) – dictionary of numpy arrays that hold the metadata

• timestamp (datetime.datetime) – exact timestamp of the image

• lon (numpy.array or None) – array of longitudes, if None self.grid will be assumed

• lat (numpy.array or None) – array of latitudes, if None self.grid will be assumed

• time_var (string or None) – variable name of observation times in the data dict, if Noneall observations have the same timestamp

write(data)

class ascat.h_saf.H08img(path, month_path_str=’h08_%Y%m_buf’,day_search_str=’h08_%Y%m%d_*.buf’, file_search_str=’h08_{datetime}*.buf’,datetime_format=’%Y%m%d_%H%M%S’, filename_datetime_format=(4,19, ’%Y%m%d_%H%M%S’))

Bases: pygeobase.io_base.MultiTemporalImageBase

6.6. ascat 47





https://docs.python.org/3.6/library/datetime.html#datetime.datetime



Reads H SAF H08 images. The images have to be uncompressed in the following folder structure path -month_path_str (default ‘h08_%Y%m_buf’)

For example if path is set to /home/user/hsaf08 and month_path_str is left to the default ‘h08_%Y%m_buf’ thenthe images for March 2012 have to be in the folder /home/user/hsaf08/h08_201203_buf/

Parameters


• month_path_str (string, optional) – if the files are stored in folders by monthas is the standard on the H SAF FTP Server then please specify the string that should beused in datetime.datetime.strftime Default: ‘h08_%Y%m_buf’

• day_search_str (string, optional) – to provide an iterator over all images ofa day the method _get_possible_timestamps looks for all available images on a day on theharddisk. This string is used in datetime.datetime.strftime and in glob.glob to search for allfiles on a day. Default : ‘h08_%Y%m%d_*.buf’

• file_search_str (string, optional) – this string is used in date-time.datetime.strftime and glob.glob to find a 3 minute bufr file by the exact date.Default: ‘h08_{datetime}*.buf’

• datetime_format (string, optional) – datetime format by which {datetime}will be replaced in file_search_str Default: %Y%m%d_%H%M%S

tstamps_for_daterange(startdate, enddate)Get the timestamps as datetime array that are possible for the given day, if the timestamps are

For this product it is not fixed but has to be looked up from the hard disk since bufr files are not regularspaced and only europe is in this product. For a global product a 3 minute spacing could be used as a fistapproximation

Parameters

• start_date (datetime.date or datetime.datetime) – start date

• end_date (datetime.date or datetime.datetime) – end date

Returns dates – list of datetimes

Return type list

class ascat.h_saf.H101img(path, month_path_str=’h101_%Y%m_buf’)Bases: ascat.level2.AscatL2SsmBufr

Parameters




Parameters









https://docs.python.org/3.6/library/datetime.html#datetime.date











Class reading H103 Metop-A ASCAT soil moisture in BUFR format.

Parameters



class ascat.h_saf.H108Ts(cdr_path, grid_path, grid_filename=’TUW_WARP5_grid_info_2_1.nc’,static_layer_path=None, **kwargs)

Bases: ascat.timeseries.AscatNc

Class reading H108 soil moisture climate data record (CDR) extension.



Class reading H109 soil moisture climate data record (CDR).



Class reading H110 soil moisture climate data record (CDR) extension.




class ascat.h_saf.H14Single(filename, mode=’r’, expand_grid=True, metadata_fields=[‘units’,‘name’])

Bases: pygeobase.io_base.ImageBase

Class reading H14 soil moisture in GRIB format.

Parameters

• expand_grid (boolean, optional) – if set the images will be expanded to a 2D im-age during reading if false the images will be returned as 1D arrays on the reduced gaussiangrid Default: True

• metadata_fields (list, optional) – fields of the message to put into the meta-data dictionary.

close()

flush()

read(timestamp=None)Read specific image for given datetime timestamp.

Parameters timestamp (datetime.datetime) – exact observation timestamp of the im-age that should be read

Returns

• data (dict) – dictionary of numpy arrays that hold the image data for each variable of thedataset

6.6. ascat 49











write(data)

class ascat.h_saf.H14img(path, month_path_str=’h14_%Y%m_grib’,file_str=’H14_{datetime}00.grib’, datetime_format=’%Y%m%d’, ex-pand_grid=True)


Class for reading H SAF H14 SM DAS 2 products in grib format.

The images have to be uncompressed in the following folder structure path - month_path_str (default‘h14_%Y%m_grib’)

For example if path is set to /home/user/hsaf14 and month_path_str is left to the default ‘h14_%Y%m_grib’then the images for March 2012 have to be in the folder /home/user/hsaf14/h14_201203_grib/

Parameters


• month_path_str (string, optional) – if the files are stored in folders by monthas is the standard on the H SAF FTP Server then please specify the string that should beused in datetime.datetime.strftime Default: ‘h14_%Y%m_grib’

• file_str (string, optional) – this string is used in datetime.datetime.strftime toget the filename of a H14 daily grib file Default: ‘H14_%Y%m%d00.grib’

• datetime_format (string, optional) – datetime format by which {datetime}will be replaced in file_str Default: %Y%m%d


Parameters






ascat.level2 module

General Level 2 data readers for ASCAT data in all formats. Not specific to distributor.









class ascat.level2.AscatL2SsmBufr(path, month_path_str=’h07_%Y%m_buf’,day_search_str=’h07_%Y%m%d_*.buf’,file_search_str=’h07_{datetime}*.buf’, date-time_format=’%Y%m%d_%H%M%S’, file-name_datetime_format=(4, 19, ’%Y%m%d_%H%M%S’),msg_name_lookup=None)


Class for reading HSAF ASCAt SSM images in bufr format. The images have the same structure as the ASCAT3 minute pdu files and these 2 readers could be merged in the future The images have to be uncompressed in thefollowing folder structure path - month_path_str (default ‘h07_%Y%m_buf’)

For example if path is set to /home/user/hsaf07 and month_path_str is left to the default ‘h07_%Y%m_buf’ thenthe images for March 2012 have to be in the folder /home/user/hsaf07/h07_201203_buf/

Parameters


• month_path_str (string, optional) – if the files are stored in folders by monthas is the standard on the HSAF FTP Server then please specify the string that should be usedin datetime.datetime.strftime Default: ‘h07_%Y%m_buf’

• day_search_str (string, optional) – to provide an iterator over all images ofa day the method _get_possible_timestamps looks for all available images on a day on theharddisk. This string is used in datetime.datetime.strftime and in glob.glob to search for allfiles on a day. Default : ‘h07_%Y%m%d_*.buf’

• file_search_str (string, optional) – this string is used in date-time.datetime.strftime and glob.glob to find a 3 minute bufr file by the exact date.Default: ‘h07_{datetime}*.buf’

• datetime_format (string, optional) – datetime format by which {datetime}will be replaced in file_search_str Default: %Y%m%d_%H%M%S

• msg_name_lookup (dict, optional) – Dictionary mapping bufr msg number toparameter name. See ASCAT BUFR format table.

Default:

6.6. ascat 51






https://docs.python.org/3.6/library/stdtypes.html#dict


Key Value6 ’Direction Of Motion Of Moving Observing Platform’,16 ’Orbit Number’,65 ’Surface Soil Moisture (Ms)’,66 ’Estimated Error In Surface Soil Moisture’,67 ’Backscatter’,68 ’Estimated Error In Sigma0 At 40 Deg Incidence Angle’,69 ’Slope At 40 Deg Incidence Angle’,70 ’Estimated Error In Slope At 40 Deg Incidence Angle’,71 ’Soil Moisture Sensitivity’,72 ’Dry Backscatter’,73 ’Wet Backscatter’,74 ’Mean Surface Soil Moisture’,75 ’Rain Fall Detection’,76 ’Soil Moisture Correction Flag’,77 ’Soil Moisture Processing Flag’,78 ’Soil Moisture Quality’,79 ’Snow Cover’,80 ’Frozen Land Surface Fraction’,81 ’Inundation And Wetland Fraction’,82 ’Topographic Complexity’



Parameters




Return type list

class ascat.level2.AscatL2SsmBufrChunked(path, month_path_str=’h07_%Y%m_buf’,day_search_str=’h07_%Y%m%d_*.buf’,file_search_str=’h07_{datetime}*.buf’, date-time_format=’%Y%m%d_%H%M%S’,filename_datetime_format=(4,19, ’%Y%m%d_%H%M%S’),msg_name_lookup=None, chunk_minutes=50)

Bases: pygeobase.io_base.IntervalReadingMixin, ascat.level2.AscatL2SsmBufr

Reads BUFR files but does not return them on a file by file basis but in bigger chunks. For example it allows toread multiple 3 minute PDU’s in half orbit chunks of 50 minutes. This speeds up operations like e.g. resamplingof the data.

Parameters chunk_minutes (int, optional) – How many minutes should a chunk of datacover.

class ascat.level2.AscatL2SsmBufrFile(filename, mode=’r’, msg_name_lookup=None,**kwargs)

Bases: pygeobase.io_base.ImageBase









Reads ASCAT SSM swath files in BUFR format. There are the following products:

• H101 SSM ASCAT-A NRT O 12.5 Metop-A ASCAT NRT SSM orbit geometry 12.5 km sampling

• H102 SSM ASCAT-A NRT O 25.0 Metop-A ASCAT NRT SSM orbit geometry 25 km sampling

• H16 SSM ASCAT-B NRT O 12.5 Metop-B ASCAT NRT SSM orbit geometry 12.5 km sampling

• H103 SSM ASCAT-B NRT O 25.0 Metop-B ASCAT NRT SSM orbit geometry 25 km sampling

• H104 SSM ASCAT-C NRT O 12.5 Metop-C ASCAT NRT SSM orbit geometry 12.5 km sampling

• H105 SSM ASCAT-C NRT O 25.0 Metop-C ASCAT NRT SSM orbit geometry 25 km sampling

• EUMETSAT ASCAT Soil Moisture at 12.5 km Swath Grid - Metop in BUFR format

• EUMETSAT ASCAT Soil Moisture at 25.0 km Swath Grid - Metop in BUFR format

Parameters

• filename (str) – Filename path.

• mode (str, optional) – Opening mode. Default: r

• msg_name_lookup (dict, optional) – Dictionary mapping bufr msg number toparameter name. See ASCAT BUFR format table.

Default:

Key Value6 ’Direction Of Motion Of Moving Observing Platform’,16 ’Orbit Number’,65 ’Surface Soil Moisture (Ms)’,66 ’Estimated Error In Surface Soil Moisture’,67 ’Backscatter’,68 ’Estimated Error In Sigma0 At 40 Deg Incidence Angle’,69 ’Slope At 40 Deg Incidence Angle’,70 ’Estimated Error In Slope At 40 Deg Incidence Angle’,71 ’Soil Moisture Sensitivity’,72 ’Dry Backscatter’,73 ’Wet Backscatter’,74 ’Mean Surface Soil Moisture’,75 ’Rain Fall Detection’,76 ’Soil Moisture Correction Flag’,77 ’Soil Moisture Processing Flag’,78 ’Soil Moisture Quality’,79 ’Snow Cover’,80 ’Frozen Land Surface Fraction’,81 ’Inundation And Wetland Fraction’,82 ’Topographic Complexity’

close()

flush()

read(timestamp=None)Read specific image for given datetime timestamp.

Parameters timestamp (datetime.datetime) – exact observation timestamp of the im-age that should be read

6.6. ascat 53

https://docs.python.org/3.6/library/stdtypes.html#str





Returns

• data (dict) – dictionary of numpy arrays that hold the image data for each variable of thedataset






read_masked_data(**kwargs)It does not make sense to read a orbit file unmasked so we only have a masked implementation.

resample_data(image, index, distance, weights, **kwargs)Takes an image and resample (interpolate) the image data to arbitrary defined locations given by index anddistance.

Parameters

• image (object) – pygeobase.object_base.Image object

• index (np.array) – Index into image data defining a look-up table for data elementsused in the interpolation process for each defined target location.

• distance (np.array) – Array representing the distances of the image data to thearbitrary defined locations.

• weights (np.array) – Array representing the weights of the image data that shouldbe used during resampling. The weights of points not to use are set to np.nan This array isof shape (x, max_neighbors)

Returns image – pygeobase.object_base.Image object

Return type object

write(data)

class ascat.level2.AscatL2SsmNc(path, month_path_str=’‘, day_search_str=’W_XX-EUMETSAT-Darmstadt, SURFACE+SATELLITE,METOPA+ASCAT_C_EUMP_%Y%m%d*_125_ssm_l2.nc’,file_search_str=’W_XX-EUMETSAT-Darmstadt, SURFACE+SATELLITE,METOPA+ASCAT_C_EUMP_{datetime}*_125_ssm_l2.nc’,datetime_format=’%Y%m%d%H%M%S’, file-name_datetime_format=(62, 76, ’%Y%m%d%H%M%S’),nc_variables=None)


Class for reading HSAF ASCAT SSM images in netCDF format. The images have to be uncompressed in thefollowing folder structure

Parameters


• month_path_str (string, optional) – if the files are stored in folders by monthas is the standard on the HSAF FTP Server then please specify the string that should be usedin datetime.datetime.strftime Default: ‘’







• day_search_str (string, optional) – to provide an iterator over all images ofa day the method _get_possible_timestamps looks for all available images on a day on theharddisk. This string is used in datetime.datetime.strftime and in glob.glob to search for allfiles on a day.

• file_search_str (string, optional) – this string is used in date-time.datetime.strftime and glob.glob to find a 3 minute bufr file by the exact date.

• datetime_format (string, optional) – datetime format by which {datetime}will be replaced in file_search_str

• nc_variables (list, optional) – list of variables to read from netCDF. Default:read all available variables



Parameters




Return type list

class ascat.level2.AscatL2SsmNcFile(filename, mode=’r’, nc_variables=None, **kwargs)Bases: pygeobase.io_base.ImageBase

Read ASCAT L2 SSM File in netCDF format, as downloaded from EUMETSAT

Parameters

• filename (str) – Filename path.

• mode (str, optional) – Opening mode. Default: r

• nc_variables (list, optional) – list of variables to read from netCDF. Default:read all available variables

close()

flush()

read(timestamp=None)reads from the netCDF file given by the filename

Returns data

Return type pygeobase.object_base.Image

read_masked_data(**kwargs)It does not make sense to read a orbit file unmasked so we only have a masked implementation.

write(data)

6.6. ascat 55














ascat.timeseries module

class ascat.timeseries.AscatNc(path, fn_format, grid_filename, static_layer_path, thresh-olds=None, **kwargs)

Bases: pynetcf.time_series.GriddedNcContiguousRaggedTs

Class reading Metop ASCAT soil moisture Climate Data Record (CDR).

Parameters

• path (str) – Path to Climate Data Record (CDR) data set.

• fn_format (str) – Filename format string, typical ‘<prefix>_{:04d}’

• grid_filename (str) – Grid filename.

• static_layer_path (str) – Path to static layer files.

• thresholds (dict, optional) – Thresholds for topographic complexity (default 50)and wetland fraction (default 50).

gridpygeogrids.CellGrid – Cell grid.

thresholdsdict – Thresholds for topographic complexity (default 50) and wetland fraction (default 50).

slayerstr – StaticLayer object

close()Close file-like objects.

class ascat.timeseries.AscatSsmCdr(cdr_path, grid_path, grid_filename=’TUW_WARP5_grid_info_2_1.nc’,static_layer_path=None, **kwargs)


Class reading Metop ASCAT soil moisture Climate Data Record (CDR).

Parameters

• cdr_path (str) – Path to Climate Data Record (CDR) data set.

• grid_path (str) – Path to grid file.

• grid_filename (str) – Name of grid file.

• static_layer_path (str) – Path to static layer files.

gridpygeogrids.CellGrid – Cell grid.

class ascat.timeseries.AscatTimeSeries(gpi, lon, lat, cell, data, topo_complex=None,wetland_frac=None, porosity_gldas=None, poros-ity_hwsd=None)

Bases: pygeobase.object_base.TS

Container class for ASCAT time series.

Parameters

• gpi (int) – Grid point index

• lon (float) – Longitude of grid point

• lat (float) – Latitude of grid point












https://docs.python.org/3.6/library/functions.html#float



• cell (int) – Cell number of grid point

• data (pandas.DataFrame) – DataFrame which contains the data

• topo_complex (int, optional) – Topographic complexity at the grid point

• wetland_frac (int, optional) – Wetland fraction at the grid point

• porosity_gldas (float, optional) – Porosity taken from GLDAS model

• porosity_hwsd (float, optional) – Porosity calculated from Harmonised WorldSoil Database

gpiint – Grid point index

lonfloat – Longitude of grid point

latfloat – Latitude of grid point

cellint – Cell number of grid point

datapandas.DataFrame – DataFrame which contains the data

topo_complexint – Topographic complexity at the grid point

wetland_fracint – Wetland fraction at the grid point

porosity_gldasfloat – Porosity taken from GLDAS model

porosity_hwsdfloat – Porosity calculated from Harmonised World Soil Database

class ascat.timeseries.StaticLayers(path, grid_filename)Bases: object

Class to read static layer files.

Parameters

• path (str) – Path of static layer files.

• grid_filename (str) – Grid filename.

topo_complexpynetcf.point_data.GriddedPointData – Topographic complexity.

wetland_fracpynetcf.point_data.GriddedPointData – Inundation and wetland fraction.

frozen_probpynetcf.time_series.GriddedNcOrthoMultiTs – Frozen soil/canopy probability.

snow_probpynetcf.time_series.GriddedNcOrthoMultiTs – Snow cover probability.

porositypynetcf.time_series.GriddedNcOrthoMultiTs – Soil porosity information.

6.6. ascat 57


http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html#pandas.DataFrame









close()Close all file objects.

ascat.timeseries.load_grid(grid_filename)Load grid file.

Parameters grid_filename (str) – Grid filename.

Returns grid – Grid.

Return type pygeogrids.CellGrid

ascat.tuw module

class ascat.tuw.AscatVodTs(path, grid_path, grid_info_filename=’TUW_WARP5_grid_info_2_1.nc’,variables=None)

Bases: pynetcf.time_series.GriddedNcContiguousRaggedTs

Class that provides access to ASCAT VOD data stored in netCDF format.

Parameters

• path (string) – Path to data folder which contains the zip files from the FTP server

• grid_path (string) – Path to grid_info folder which contains a netcdf file with infor-mation about grid point index,latitude, longitude and cell

• grid_info_filename (string, optional) – Name of the grid info netCDF filein grid_path default ‘TUW_WARP5_grid_info_2_1.nc’

pathstring – Path to data folder which contains the zip files from the FTP server

grid_pathstring – Path to grid_info folder which contains txt files with information about grid point index,latitude,longitude and cell

grid_info_filenamestring, optional – Name of the grid info netCDF file in grid_path default‘TUW_WARP5_grid_info_2_1.nc’

gridgrids.CellGrid object – CellGrid object, which provides nearest neighbor search and other features

variableslist of string

class ascat.tuw.Ascat_SSM(*args, **kwargs)Bases: ascat.tuw.Ascat_data

Class for reading ASCAT SSM data. It extends Ascat_data and provides the information necessary for readingSSM data

Parameters


• grid_path (string) – Path to grid_info folder which contains txt files with informationabout grid point index,latitude, longitude and cell

• grid_info_filename (string, optional) – Name of the grid info txt file ingrid_path










• advisory_flags_path (string, optional) – Path to advisory flags .dat files, ifnot provided they will not be used

• topo_threshold (int, optional) – If topographic complexity of read grid point isabove this threshold a warning is output during reading

• wetland_threshold (int, optional) – If wetland fraction of read grid point isabove this threshold a warning is output during reading

gp_filename_templatestring – Defines how the gpi is put into the template string to make the filename

gp_filestructnumpy.dtype – Structure template of the SSM .dat file

scale_factordict – Factor by which to multiply the raw data to get the correct values for each field in the gp_filestruct

include_in_dflist – List of fields that should be returned to the user after reading

nan_valuesdict – Nan value saved in the file which will be replaced by numpy.nan values during reading

datatypedict – Datatype of the fields that the return data should have

read_ssm(*args, **kwargs)read surface soil moisture

read_ssm(*args, **kwargs)Function to read SSM takes either 1 or 2 arguments. It can be called as read_ssm(gpi, **kwargs) orread_ssm(lon, lat, **kwargs)

Parameters

• gpi (int) – Grid point index

• lon (float) – Longitude of point

• lat (float) – Latitude of point

• mask_ssf (boolean, optional) – Default False, if True only SSF values of 1 willbe allowed, all others are removed

• mask_frozen_prob (int,optional) – If included in kwargs then all observationstaken when frozen probability > mask_frozen_prob are removed from the result

• mask_snow_prob (int,optional) – If included in kwargs then all observationstaken when snow probability > mask_snow_prob are removed from the result

Returns AscatTimeSeries – ascat.AscatTimeSeries instance

Return type object

class ascat.tuw.Ascat_SWI(*args, **kwargs)Bases: ascat.tuw.Ascat_data

Class for reading ASCAT SWI data. It extends Ascat_data and provides the information necessary for readingSWI data

Parameters


6.6. ascat 59

















gp_filename_templatestring – Defines how the gpi is put into the template string to make the filename

gp_filestructnumpy.dtype – Structure template of the SSM .dat file

scale_factordict – Factor by which to multiply the raw data to get the correct values for each field in the gp_filestruct

include_in_dflist – List of fields that should be returned to the user after reading

nan_valuesdict – Nan value saved in the file which will be replaced by numpy.nan values during reading

datatypedict – Datatype of the fields that the return data should have

T_SWIdict – Information about which numerical T-Value maps to which entry in the datastructure

T_QFLAGdict – Information about which numerical T-Value maps to which entry in the datastructure

read_swi(*args, **kwargs)read soil water index

read_swi(*args, **kwargs)Function to read SWI takes either 1 or 2 arguments being. It can be called as read_swi(gpi, **kwargs) orread_swi(lon, lat, **kwargs)

Parameters

• gpi (int) – grid point index

• lon (float) – longitude of point

• lat (float) – latitude of point

• T (int, optional) – if set only the SWI and QFLAG of this T-Value will be returned

• mask_qf (int, optional) – if set, SWI values with a QFLAG value lower than themask_qf value will be masked. This is done for each T value independently

• mask_frozen_prob (int,optional) – if included in kwargs then all observationstaken when frozen probability > mask_frozen_prob are removed from the result

• mask_snow_prob (int,optional) – if included in kwargs then all observationstaken when snow probability > mask_snow_prob are removed from the result















Returns df – containing all fields in self.include_in_df plus frozen_prob and snow_prob if ad-visory_flags_path was set. If T was set then only SWI and QFLAG values for the selected Tvalue are included plut frozen_prob and snow_prob if applicable

Return type pandas.DataFrame

class ascat.tuw.Ascat_data(path, grid_path, grid_info_filename=’TUW_W54_01_lonlat-ld-land.txt’,advisory_flags_path=None, topo_threshold=50, wetland_threshold=50)

Bases: object

Class that provides access to ASCAT data stored in userformat which is available from the TU Wien FTP Serverafter registration at http://rs.geo.tuwien.ac.at.

Parameters







pathstring – Path to data folder which contains the zip files from the FTP server

grid_pathstring – Path to grid_info folder which contains txt files with information about grid point index,latitude,longitude and cell

grid_info_filenamestring – Name of the grid info txt file in grid_path

grid_info_np_filenamestring – Name of the numpy save file to the grid information

topo_thresholdint – If topographic complexity of read grid point is above this threshold a warning is output during reading

wetland_thresholdint – If wetland fraction of read grid point is above this threshold a warning is output during reading

grid_info_loadedboolean – True if the grid information has already been loaded

gridpygeogrids.grids.CellGrid object – CellGrid object, which provides nearest neighbor searchand other features

advisory_flags_pathstring – Path to advisory flags .dat files, if not provided they will not be used

include_advflagsboolean – True if advisory flags are available

6.6. ascat 61

http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html#pandas.DataFrame


http://rs.geo.tuwien.ac.at








unzip_cell(cell)unzips zipped grid point files into subdirectory

read_advisory_flags(gpi)reads the advisory flags for a given grid point index

read_advisory_flags(gpi)Read the advisory flags located in the self.advisory_flags_path Advisory flags include frozen probability,snow cover probability topographic complexity and wetland fraction.

Parameters gpi (long) – grid point index

Returns

• df (pandas.DataFrame) – containing the columns frozen_prob and snow_prob. lenght 366with one entry for every day of the year, including February 29th

• topo (numpy.uint8) – topographic complexity ranging from 0-100

• wetland (numpy.uint8) – wetland fraction of pixel in percent

unzip_cell(cell)Unzips the downloaded .zip cell file into the directory of os.path.join(self.path, cell).

Parameters cell (int) – cell number

Module contents



CHAPTER 7

Indices and tables

• genindex

• modindex

• search

63


64 Chapter 7. Indices and tables

Python Module Index

aascat, 62ascat.bufr, 42ascat.cgls, 43ascat.eumetsat, 43ascat.h_saf, 47ascat.level2, 50ascat.timeseries, 56ascat.tuw, 58

65


66 Python Module Index

Index

Aadvisory_flags_path (ascat.tuw.Ascat_data attribute), 61ascat (module), 62ascat.bufr (module), 42ascat.cgls (module), 43ascat.eumetsat (module), 43ascat.h_saf (module), 47ascat.level2 (module), 50ascat.timeseries (module), 56ascat.tuw (module), 58Ascat_data (class in ascat.tuw), 61Ascat_SSM (class in ascat.tuw), 58Ascat_SWI (class in ascat.tuw), 59AscatAL2Ssm125 (class in ascat.eumetsat), 43AscatAL2Ssm125Nc (class in ascat.eumetsat), 43AscatAL2Ssm125PDU (class in ascat.eumetsat), 44AscatAL2Ssm125PDUChunked (class in ascat.eumetsat),

44AscatAL2Ssm250 (class in ascat.eumetsat), 44AscatAL2Ssm250Nc (class in ascat.eumetsat), 44AscatAL2Ssm250PDU (class in ascat.eumetsat), 44AscatAL2Ssm250PDUChunked (class in ascat.eumetsat),

45AscatBL2Ssm125 (class in ascat.eumetsat), 45AscatBL2Ssm125Nc (class in ascat.eumetsat), 45AscatBL2Ssm125PDU (class in ascat.eumetsat), 45AscatBL2Ssm125PDUChunked (class in ascat.eumetsat),

46AscatBL2Ssm250 (class in ascat.eumetsat), 46AscatBL2Ssm250Nc (class in ascat.eumetsat), 46AscatBL2Ssm250PDU (class in ascat.eumetsat), 46AscatBL2Ssm250PDUChunked (class in ascat.eumetsat),

46AscatL2SsmBufr (class in ascat.level2), 50AscatL2SsmBufrChunked (class in ascat.level2), 52AscatL2SsmBufrFile (class in ascat.level2), 52AscatL2SsmNc (class in ascat.level2), 54AscatL2SsmNcFile (class in ascat.level2), 55AscatNc (class in ascat.timeseries), 56

AscatSsmCdr (class in ascat.timeseries), 56AscatTimeSeries (class in ascat.timeseries), 56AscatVodTs (class in ascat.tuw), 58

BBUFRReader (class in ascat.bufr), 42

Ccell (ascat.timeseries.AscatTimeSeries attribute), 57close() (ascat.h_saf.H08Single method), 47close() (ascat.h_saf.H14Single method), 49close() (ascat.level2.AscatL2SsmBufrFile method), 53close() (ascat.level2.AscatL2SsmNcFile method), 55close() (ascat.timeseries.AscatNc method), 56close() (ascat.timeseries.StaticLayers method), 57

Ddata (ascat.timeseries.AscatTimeSeries attribute), 57datatype (ascat.tuw.Ascat_SSM attribute), 59datatype (ascat.tuw.Ascat_SWI attribute), 60

Fflush() (ascat.h_saf.H08Single method), 47flush() (ascat.h_saf.H14Single method), 49flush() (ascat.level2.AscatL2SsmBufrFile method), 53flush() (ascat.level2.AscatL2SsmNcFile method), 55frozen_prob (ascat.timeseries.StaticLayers attribute), 57

Ggp_filename_template (ascat.tuw.Ascat_SSM attribute),

59gp_filename_template (ascat.tuw.Ascat_SWI attribute),

60gp_filestruct (ascat.tuw.Ascat_SSM attribute), 59gp_filestruct (ascat.tuw.Ascat_SWI attribute), 60gpi (ascat.timeseries.AscatTimeSeries attribute), 57grid (ascat.timeseries.AscatNc attribute), 56grid (ascat.timeseries.AscatSsmCdr attribute), 56grid (ascat.tuw.Ascat_data attribute), 61

67


grid (ascat.tuw.AscatVodTs attribute), 58grid_info_filename (ascat.tuw.Ascat_data attribute), 61grid_info_filename (ascat.tuw.AscatVodTs attribute), 58grid_info_loaded (ascat.tuw.Ascat_data attribute), 61grid_info_np_filename (ascat.tuw.Ascat_data attribute),

61grid_path (ascat.tuw.Ascat_data attribute), 61grid_path (ascat.tuw.AscatVodTs attribute), 58

HH07img (class in ascat.h_saf), 47H08img (class in ascat.h_saf), 47H08Single (class in ascat.h_saf), 47H101img (class in ascat.h_saf), 48H102img (class in ascat.h_saf), 48H103img (class in ascat.h_saf), 48H108Ts (class in ascat.h_saf), 49H109Ts (class in ascat.h_saf), 49H110Ts (class in ascat.h_saf), 49H111Ts (class in ascat.h_saf), 49H14img (class in ascat.h_saf), 50H14Single (class in ascat.h_saf), 49H16img (class in ascat.h_saf), 50H25Ts (class in ascat.h_saf), 50

Iinclude_advflags (ascat.tuw.Ascat_data attribute), 61include_in_df (ascat.tuw.Ascat_SSM attribute), 59include_in_df (ascat.tuw.Ascat_SWI attribute), 60

Llat (ascat.timeseries.AscatTimeSeries attribute), 57load_grid() (in module ascat.timeseries), 58lon (ascat.timeseries.AscatTimeSeries attribute), 57

Mmessages() (ascat.bufr.BUFRReader method), 42

Nnan_values (ascat.tuw.Ascat_SSM attribute), 59nan_values (ascat.tuw.Ascat_SWI attribute), 60

Ppath (ascat.tuw.Ascat_data attribute), 61path (ascat.tuw.AscatVodTs attribute), 58porosity (ascat.timeseries.StaticLayers attribute), 57porosity_gldas (ascat.timeseries.AscatTimeSeries at-

tribute), 57porosity_hwsd (ascat.timeseries.AscatTimeSeries at-

tribute), 57

Rread() (ascat.h_saf.H08Single method), 47

read() (ascat.h_saf.H14Single method), 49read() (ascat.level2.AscatL2SsmBufrFile method), 53read() (ascat.level2.AscatL2SsmNcFile method), 55read_advisory_flags() (ascat.tuw.Ascat_data method), 62read_masked_data() (ascat.level2.AscatL2SsmBufrFile

method), 54read_masked_data() (ascat.level2.AscatL2SsmNcFile

method), 55read_ssm() (ascat.tuw.Ascat_SSM method), 59read_swi() (ascat.tuw.Ascat_SWI method), 60resample_data() (ascat.level2.AscatL2SsmBufrFile

method), 54

Sscale_factor (ascat.tuw.Ascat_SSM attribute), 59scale_factor (ascat.tuw.Ascat_SWI attribute), 60slayer (ascat.timeseries.AscatNc attribute), 56snow_prob (ascat.timeseries.StaticLayers attribute), 57StaticLayers (class in ascat.timeseries), 57SWI_TS (class in ascat.cgls), 43

TT_QFLAG (ascat.tuw.Ascat_SWI attribute), 60T_SWI (ascat.tuw.Ascat_SWI attribute), 60thresholds (ascat.timeseries.AscatNc attribute), 56topo_complex (ascat.timeseries.AscatTimeSeries at-

tribute), 57topo_complex (ascat.timeseries.StaticLayers attribute),

57topo_threshold (ascat.tuw.Ascat_data attribute), 61tstamps_for_daterange() (ascat.h_saf.H08img method),

48tstamps_for_daterange() (ascat.level2.AscatL2SsmBufr

method), 52tstamps_for_daterange() (ascat.level2.AscatL2SsmNc

method), 55

Uunzip_cell() (ascat.tuw.Ascat_data method), 61, 62

Vvariables (ascat.tuw.AscatVodTs attribute), 58

Wwetland_frac (ascat.timeseries.AscatTimeSeries at-

tribute), 57wetland_frac (ascat.timeseries.StaticLayers attribute), 57wetland_threshold (ascat.tuw.Ascat_data attribute), 61write() (ascat.h_saf.H08Single method), 47write() (ascat.h_saf.H14Single method), 50write() (ascat.level2.AscatL2SsmBufrFile method), 54write() (ascat.level2.AscatL2SsmNcFile method), 55

68 Index

Documents

ascat Documentation - Read the Docs · conda install-c conda-forge pygrib=2.0.1 # for reading GRIB files ... For details and usage information on PyScaffold seehttp:// ... ascat Documentation,