Soil Moisture Data Assimilation in the SHEELS Land Surface Model
Clay BlankenshipUSRA
Special thanks to: Bill Crosson, Jon Case
Top ofcanopy
Bare soil energy fluxes
Surface runoff Groundheat flux
Sensible LatentShortwaveLongwave Precipitation
Interceptionby canopy
Canopy energy fluxes
Radiative Fluxes
Upper zone
Root zone
Bottom zone
Diffusion/drainage Heat exchange
SoilLayers
Wind
ThroughfallSensible Latent
Infiltration
Sub-surfacelateral flow
Top ofcanopy
Bare soil energy fluxes
Surface runoff Groundheat flux
Sensible LatentShortwaveLongwave Precipitation
Interceptionby canopy
Canopy energy fluxes
Radiative Fluxes
Upper zone
Root zone
Bottom zone
Diffusion/drainage Heat exchange
SoilLayers
Wind
ThroughfallSensible Latent
Infiltration
Sub-surfacelateral flow
Overview
•Data assimilation and retrieval background
•1dvar, 3dvar, and Kalman Filters
•Soil Moisture Data Assimilation
•SHEELS LSM
•AMSR-E data
•Results
•Some results from profile retrievals with MIRS
Data Assimilation (and Variational Retrievals)
Given observations {y} and background state {xb}, estimate most likely state {x} taking into account errors in both xb and y.
€
P(x | y) =P(y | x)P(x)
P(y)~
exp −1
2y − H (x)( )
TR−1 y − H (x)( )
⎧ ⎨ ⎩
⎫ ⎬ ⎭exp −
1
2x − xb( )
TPb
−1 x − xb( ) ⎧ ⎨ ⎩
⎫ ⎬ ⎭
Obs-Calc Obs
{ {
Retrieved-Bkgd State
x: geophysical statexb: background (a priori) of statey: observationsH(x): forward operator converting x into observation spacePb: background error covarianceR: obs+forward model error covariance
•In “data assimilation” you are using observations to improve a model. In “profile retrievals” you always have some assumed background state (maybe climatology).
P(UT fan|orange shirt)=P(orange shirt|UT fan)P(UT fan) P(orange shirt)
Profile Retrieval (1d)
Soil Moisture (1d)
NWP
(3d or 4d)
Model Variable Column vector of T, q, hydrometeors
Column vector of soil q [and T]
3D (or 4D) grid of T, q, u, v, clouds…
Observations {TB} from satellite instrument(s)
{TB} or retrieved near-surface water content
{TB} from multiple satellites, raobs, surface obs, etc.
Typical size of state vector
500 (or 5 to 15 in EOF space)
5-20 10^6 to 10^8
Typical size of observation vector
5-20 1-3 up to 10^6
Retrieval and Data Assimilation Applications
Data Assimilation (and Variational Retrievals)
Given observations {y} and background state {xb}, estimate most likely state {x} taking into account errors in both xb and y.
To maximixe this probability, minimize -2 times its log:
€
P(x | y) =P(y | x)P(x)
P(y)~
exp −1
2y − H (x)( )
TR−1 y − H (x)( )
⎧ ⎨ ⎩
⎫ ⎬ ⎭exp −
1
2x − xb( )
TPb
−1 x − xb( ) ⎧ ⎨ ⎩
⎫ ⎬ ⎭
O+F Error Cov Bkgd Error Cov
Obs-Calc TB
{ {
Retrieved-Bkgd Atmosphere
€
J(x) = −2* ln P({x} |{y})( ) = y − H (x)( )TR −1 y − H (x)( ) + x − xb( )
TPb
−1 x − xb( )
Minimizing the cost function:
Given initial guess x=xb, linearize and solve for new estimate of x. Skipping the math, the solution can be expressed as:
The “Gain” represents a weighted average of bk and ob errors
The Jacobian K relates the y terms to x termsand can include interpolation, averaging or integration,radiative transfer, etc.
€
xa = xb + PbKT (KPbK
T + R)−1(y − H (xb ))
DA Solution
€
J(x) = y − H (x)( )TR −1 y − H (x)( ) + x − xb( )
TPb
−1 x − xb( )
€
K ij ≡∂y i∂x j
Bkgd Gain Innovation€
Pb(Pb + R)
simplified:
1DVar/3DVar 4DVar Kalman Filter Ensemble Kalman Filter
Background Error
Covariance Matrix
(normally fixed)
Defined at t0. Flow dependent based on model adjoint.
Computed based on linearized model dynamics.
Estimated from ensemble covariance
Gain Calculation
Equivalent to KF with fixed Gaussian background errors
Equivalent to EnKF assuming perfect model, Gaussian bk error
Most general method in terms of error covariance
Applies innovations at proper time
Data Assimilation Methods
Hybrid methods use ensemble spread to define background covariance in 3dvar.
€
PbKT(KPbK
T + R)−1
€
PbKT(KPbK
T + R)−1
€
QK T(KQK T + R)−1
€
PbMTK T(KMPbM
TK T + R)−1
Soil Moisture Data Assimilation in the SHEELS Land Surface Model
SHEELS – Simulator for Hydrology and Energy Exchange at the Land Surface
SHEELS
• Distributed land surface hydrology model provides soil state(T,q), fluxes of energy and moisture• Heritage: 1980’s Biosphere-Atmosphere Transfer Scheme (BATS)• Can run off-line or coupled with meteorological model• Flexible vertical layer configuration designed to facilitate microwave data assimilation• Described in Martinez et al. (2001), Crosson et al. (2002)
Top ofcanopy
Bare soil energy fluxes
Surface runoff Groundheat flux
Sensible LatentShortwaveLongwave Precipitation
Interceptionby canopy
Canopy energy fluxes
Radiative Fluxes
Upper zone
Root zone
Bottom zone
Diffusion/drainage Heat exchange
SoilLayers
Wind
ThroughfallSensible Latent
Infiltration
Sub-surfacelateral flow
Top ofcanopy
Bare soil energy fluxes
Surface runoff Groundheat flux
Sensible LatentShortwaveLongwave Precipitation
Interceptionby canopy
Canopy energy fluxes
Radiative Fluxes
Upper zone
Root zone
Bottom zone
Diffusion/drainage Heat exchange
SoilLayers
Wind
ThroughfallSensible Latent
Infiltration
Sub-surfacelateral flow
SHEELS InputSHEELS Input
Required static variables:Soil type (STATSGO): Landcover (U of Md):Saturated hydraulic conductivity Canopy heightSaturated matric potential Fractional vegetation coverSoil wilting point Minimum stomatal resistanceSoil porosity Root depth
Reflectance propertiesSeasonal:Leaf area index Topography (GTOPO30):
Surface elevation and slopeTime-dependent input (forcing):• Wind speed (NLDAS)• Air temperature• Relative humidity• Atmospheric pressure• Downwelling solar radiation• Downwelling longwave radiation
• Rainfall (Stage IV)
SHEELS Input
STATESSoil surface and canopy temperaturesSoil temperature Soil water/ice content Depth of water on canopy Ponded waterSnow temperature, depth, and density
FLUXESSurface latent and sensible heat fluxesGround heat fluxNet radiation flux
Evapotranspiration InfiltrationRunoff
SHEELS Output
Land Information System (LIS)A modeling and data assimilation system with the capability to run several different LSMs, from GSFC’s Hydrological Sciences Branch. It is very customizable with the ability to swap out LSMs, forcing datasets, etc.
LSMS VIC, Noah, CLM, Catchment,SiB2, Hyssib Base Forcings ECMWF, GDAS, NLDAS... Supplemental Forcings TRMM 3B42, Agrrad, Cmap, Cmorph, Stg4... Parameters Landcover, soils, greenness, albedo, LAI, topography, tbot Data Assimilation algorithm, observation, perturbation method
Fractional soil moisture (water+ice) Soil Temperature
NebraskaJAN-JUL 2003
Example Depth-Time Sections
14
SHEELS output time seriesVolumetric soil moisture, 1 Mar 2011 - 21 Apr 2011
Salinas, California
Root zone
0-10cm
Total column (10m)
SHEELS output time series
Precip, Soil Water, ET
AMSR-E (Advanced Microwave Scanning Radiometer for EOS)
NASA Aqua satellite with AMSR-E instrument
AMSR-E retrieved soil moisture for
August 2, 2008 over the SE US
AMSR-EConically scanning passive microwave radiometer on NASA Aqua polar orbiterMeasures polarized brightness temperatures at 6 frequencies from 6.9 to 89.0 GHzWe use the Level 3 retrieved soil moisture product (Njoku et al. 2003) resampled to a 25-km grid. (Obs twice daily.)Stated accuracy is .06 m3/m3. (Typical range is .05 to .40)Algorithm minimizes differences between the observed brightness temperatures and those generated using a forward radiative transfer model. Due to extensive radio frequency interference in the 6.9 GHZ channel, 10.7 and 18.7 GHz observations are used for soil moisture estimation.
Background and Observed Variables
SHEELS state variables (x) include temperature and fractional water content in each of 14 layers •6 in the top 10 cm•6 root zone (up to 1.5m) •2 deep layers to 10m.
The observation* (y) (retrieved soil moisture) is the volumetric water content (cm3/cm3) near the surface (exponentially weighted by depth)
Mostly top layer but must account for porosity to convert from FWC to VWC
*Actually a retrieval or estimate but in the context of data assimilation it’s an observation.
q1
q2q3
q4
q5
q6
q7
q8
q9
€
J(x) = y − H (x)( )TR −1 y − H (x)( ) + x − xb( )
TPb
−1 x − xb( )
•The ensemble consists of N model state fields.
•The mean of the N ensemble states is used to define the state vector estimate.
•The spread of an ensemble of N model ‘trajectories’ is used to estimate the error covariances. The full non-linear dynamic equations are used to propagate each ensemble member forward in time, thus determining the trajectories. This is in contrast to the traditional Kalman filter in which linearized model dynamics are used to propagate error covariances.
•When observations are available, each ensemble member is updated based on the difference between the observation and the model state, weighted by the Kalman gain (as in the EKF).
•Random error is added to the observation based on assumed noise characteristics; this ensures that the variance of the updated ensemble matches the true estimation error covariances (Burgers et al., 1998, Mon. Wea. Rev.)
•Propagation of error covariance matrix is more stable than in the traditional Kalman filter, especially if there are strong non-linearities in the model.
Ensemble Kalman Filter in LIS
Domain
Micronet Validation•First validation attempt vs. ARS Micronet (Little Washita River, OK)
•Large bias and variability due to sampling error and soil properties
•Ran artificial experiments with 0.5x and 1.5x rain forcing, validated against Micronet and “truth” run.
•Large errors remain, but DA runs (dashed) tend to converge
•Square-wave error maybe due to day-night bias differences
QuickTime™ and a decompressor
are needed to see this picture.
Bias Correction (CDF Matching)
The dynamic range of AMSR-E observed soil moisture is small relative to that of the model.A correction (right) is applied to convert the observation into a model-equivalent value. A Cumulative Distribution Function (CDF)-matching technique is used here. This is similar in purpose to the bias corrections usually applied to satellite observations in NWP models. Simulations made without the proper correction showed a pronounced dry bias.
0.0 0.1 0.2 0.3 0.4 AMSR-E Observed Soil Moisture
CD
F-C
orr
ecte
d S
oil
Mo
istu
re
0.5
0.4
0.3
0.2
0.1
0.0
AMSR-E Bias Correction
Landcover-dependent CDF Correction
Impact of Land Use CDF Correction
Difference in fractional soil moisture immediately following
assimilation of AMSR-E data at 8 UTC on June 7, 2003: Land Use
CDF minus Uniform CDF simulation. The spatial pattern
reflects the land use type distribution and illustrates the impact of the Land Use CDF
correction on soil moisture data
assimilation.
“Truth”Stage IV
Precipitation Forcing ControlNo Data
Assimilation
Data Assimilation (Combined Bias
Correction)
22Z 24 Jun 2003
The data assimilation adds soil water, particularly in the eastern part of the domain and the Texas panhandle.
No Rain Experiment
“Truth”Stage IV
Precipitation Forcing ControlNo Data
Assimilation
Data Assimilation (Combined Bias
Correction)
09Z 22 Jun 2003
The data assimilation is able to reduce the water content from the false rainfall in Kansas while increasing it in the Texas panhandle.
False Rain Experiment
Time series of top layer soil water in SE Nebraska (point marked by diamond at left). The 3 DA runs are able to match the “true” run closely throughout much of the experiment, although they overestimate soil moisture from about June 7-14.
No Rain Experiment
False Rain Experiment
Time series of top layer (1.6 cm) soil water fraction from 5 runs (false rain scenario) in Texas panhandle (100.5W, 35N). The DA run with the combined bias correction (red line) shows transitions that are less abrupt (reduced amplitude) during data assimilation steps, compared to the other DA runs (blue/green).
No Rain Run
Control (No DA)
Uniform BC Vegetation BC
Combined BC
Bias -0.161 -0.042 -0.041 -0.042
Std. Dev. 0.128 0.130 0.129 0.130
RMS 0.206 0.137 0.136 0.137
Correlation (r2)
0.381 0.444 0.454 0.454
False Rain Run
Control (No DA)
Uniform BC Vegetation BC
Combined BC
Bias -0.015 -0.013 -0.013 -0.014
Std. Dev. 0.171 0.149 0.148 0.148
RMS 0.172 0.150 0.149 0.149
Correlation (r2)
0.090 0.375 0.384 0.391
Results (Statistics)
Summary• AMSR-E Soil Moisture Estimates assimilated into SHEELS LSM using EnKF• Bias correction is necessary for good results.
– Must be regionally and seasonally appropriate. – Landcover- and day/night-dependent corrections implemented.
• Day/night correction alleviates square wave pattern in biases• With hiqh quality forcing data, validation vs. ground truth is difficult
– Variability of rainfall and soil properties within a FOV confound this.– Could use anomaly correlations for verification
• DA improves modeled soil moisture in simulations with intentionally poor rain forcing
• Potentially of greatest use in areas without high-quality continuous rainfall data (radar, gauge networks).– An advantage over microwave (polar orbiting) rainfall observations: It
can see the effects of rainfall after it happens (alleviates sampling issue)
Possible Future Work
• Validation against ground stations by anomaly correlation
• Test impact on a coupled weather forecast model.
• Implement for SMOS and SMAP.– AMSR-E .06 cm3/cm3 accuracy – SMOS (ESA, 2009) .04 cm3/cm3
– SMAP (NASA, 2014) .04 cm3/cm3
1DVAR in MIRS
NOAA STAR algorithm/software package
•1DVAR physically-based retrieval algorithm based on OI theory
•Simultaneous retrieval of temperature, humidity, and hydrometeor profiles in EOF space
•Water vapor and hydrometeors are in logarithmic coordinates
•Assumes local linearity and gaussian pdf of state variable
•Applicable to any sensor combining imaging /sounding capabilities.
•Experimental products of cloud / precipitation liquid and ice have potential for refinement into contributions to GPM.
•Emissivity spectrum is part of the retrieved state vector enabling more direct response to precipitation over land.
Microwave Integrated Retrieval System (MIRS)
MIRS VariablesBackground+3 EOF’s
GraupelWater Vapor
Cloud Liquid
Rain
covariance error model forward : E
covariance error background : B
Xs on based estimates model forward : Y(X)
radiances channel measured : Y
quantity retrieved of background : X
state) sfc or atmos r,hydrometeo (e.g.
quantity lgeophysica retrieved : X
XYYEXYY2
1XXBXX
2
1 J(x)
m
o
m1Tmo
1To
priori a
bservedmeasured/omatchingnotradiancesmodeled
forpenalty
estimatebackgroundfromdeparting forpenalty
4444444 34444444 2144444 344444 21
⎥⎦⎤
⎢⎣⎡ −−+⎥⎦
⎤⎢⎣⎡ −−= −− ))((xx))(()(xx)(
MIRS 1DVAR
TS Lee (Sep 2, 2011)
•Retrievals from AMSU have consistently smaller Rain/Ice amounts compared to TMI
•Are these differences due to sensors or algorithms?
•Test whether improved first guess and background/covariance constraints can give more consistent results
•TRMM and GPM (will) have high resolution measurements but AMSU’s are important for sampling
Intercomparison of AMSU and TRMM Rain Retrievals
MIRS Experiments
Sep 2009 Study
MIRS Graupel
W.Pac. SE US C. Africa
MIRS Rain
• More frequent large ice consistent with high lightning frequency over Africa.
• Strong convection, dry environment.
MIRS Results (Graupel and Rain)