New Approaches to New Approaches to Tidal AnalysisTidal Analysis

David A. JayDavid A. JayDepartment of Civil & Environmental EngineeringDepartment of Civil & Environmental Engineering

Portland State UniversityPortland State UniversityPortland, OR 97207 USAPortland, OR 97207 USA

Thanks to Ed Thanks to Ed ZaronZaron and Keith and Keith LefflerLeffler (PSU), (PSU), Justin Romberg (Georgia Tech) and Rich Justin Romberg (Georgia Tech) and Rich PawlowiczPawlowicz (UBC)(UBC)Research supported by the National Science Foundation, Research supported by the National Science Foundation,

Bonneville Power Administration, and NOAABonneville Power Administration, and NOAA--FisheriesFisheries

Quotes for the Day Quotes for the Day ––

• “We are attempting to improve the one area of geophysical prediction that actually works tolerably well already; to this charge we plead guilty. But predicting and learning are in a sense orthogonal, and the most interesting effects are those that cause the most trouble with forecasting.” (Munk and Cartwright, 1966)

• “The credo of smoothness: We do not believe, nor will we tolerate, the existence of very sharp resonance peaks." (Munk and Cartwright, 1966)

Why do we Need to Improve Tidal Analysis? Why do we Need to Improve Tidal Analysis? ––

•• Existing methods are outdated (formulated <1930)Existing methods are outdated (formulated <1930)–– Poor frequency resolutionPoor frequency resolution–– No tidal sampling theory that is f[ LOR, noise, No tidal sampling theory that is f[ LOR, noise, ∆∆t]t]–– DonDon’’t use modern inverse methodst use modern inverse methods–– DonDon’’t include modern astronomical knowledget include modern astronomical knowledge

•• Urgent need to ask more sophisticated questions:Urgent need to ask more sophisticated questions:–– Tides provide a integrated view of climate effects on the oceanTides provide a integrated view of climate effects on the ocean–– Internal tides are complex and nonInternal tides are complex and non--stationary, and play an stationary, and play an

important role in tidal dissipation in the global oceanimportant role in tidal dissipation in the global ocean–– Need to deNeed to de--tide large, spatially distributed data sets that lack tide large, spatially distributed data sets that lack

wellwell--defined time series, especially from OOSdefined time series, especially from OOS–– Unconventional analyses needed (biological data, images, etc.)Unconventional analyses needed (biological data, images, etc.)

•• Analysis of nonAnalysis of non--stationary tides is important:stationary tides is important:–– Can expand knowledge of tidal theoryCan expand knowledge of tidal theory–– Tell us about the processes modulating tidesTell us about the processes modulating tides

What are the Goals of What are the Goals of ““Tidal AnalysisTidal Analysis””? ? ––•• A tidal analysis A tidal analysis shouldshould::

–– Optimally reconstructs the dataOptimally reconstructs the data–– Allow predictions outside the observation periodAllow predictions outside the observation period–– Provide dynamical fidelity and insightProvide dynamical fidelity and insight

•• Comments:Comments:–– With a noiseless, stationary deltaWith a noiseless, stationary delta--function tidal spectrum and a function tidal spectrum and a

long record (LOR large), these goals are consistent; with real long record (LOR large), these goals are consistent; with real data, there are tradedata, there are trade--offsoffs

–– All means are fair in love, war and tidal analysis All means are fair in love, war and tidal analysis ––•• We can use astronomical forcing, reference stations or model resWe can use astronomical forcing, reference stations or model resultsults•• Need to use better inverse methodsNeed to use better inverse methods•• Borrow sampling theory from image analysisBorrow sampling theory from image analysis

–– We need to expand the realm of We need to expand the realm of ““routinely tractableroutinely tractable”” tidal tidal problems problems

Present Tidal Sampling Theory Present Tidal Sampling Theory ––•• Rayleigh Criterion is usually used:Rayleigh Criterion is usually used:

–– Rayleigh criterion is unduly conservative, but ignores the effecRayleigh criterion is unduly conservative, but ignores the effects ts of noiseof noise

•• MunkMunk and and HasselmanHasselman (1964) modified Rayleigh Criterion:(1964) modified Rayleigh Criterion:

–– Should be able to violate Rayleigh by a factor of (N/S)Should be able to violate Rayleigh by a factor of (N/S)–– Suggests infinite resolution for no noise (hmmSuggests infinite resolution for no noise (hmm……))

•• ShannonShannon--NyquistNyquist theory theory –– two samples per cycle requiredtwo samples per cycle required•• Remarkable that this is all the tidal sampling theory Remarkable that this is all the tidal sampling theory

there isthere is…

1≤×∆ LORω

( )( )

SignalSNoiseNdttS

dttNLOR LOR

LOR

LOR

LOR ===×∆∫∫−

−2/

2/

2

2/

2/

2

ω

…

Compressive Sampling Compressive Sampling ––•• From medical image analysis (From medical image analysis (CandCandèèss, Romberg and others) :, Romberg and others) :

–– Optimal reconstruction from limited sampling of an imageOptimal reconstruction from limited sampling of an image–– How little is enough?How little is enough?

•• Key result for frequency resolution; 3 pts per frequency: Key result for frequency resolution; 3 pts per frequency:

Where: N = # of data pts, Where: N = # of data pts, ΩΩ = # of frequencies resolvable, = # of frequencies resolvable, and and nnωω = total # of frequencies= total # of frequencies

•• The leastThe least--squares (L2) inversion result obeys:squares (L2) inversion result obeys:

Where: Where: ρρ22 is the is the rmsrms noise level, noise level, ζζ is the actual amplitude, is the actual amplitude, ζζLSLS is the is the estimated amplitude and estimated amplitude and kk is a condition number:is a condition number:

–– ΩΩ/N is the /N is the ““oversamplingoversampling”” factor for limiting the frequencies resolvedfactor for limiting the frequencies resolved–– nnωω/N is the /N is the ““undersamplingundersampling”” factor for discrete factor for discrete ∆∆t (interesting that it is t (interesting that it is

linear)linear)

[ ] Ω≅Ω≥ 31 ωnLogCN

κρρςς ω 22

2=

Ω≈−

Nn

NLLS

Compressive Sampling (More) Compressive Sampling (More) ––•• NOT necessary to know the frequencies NOT necessary to know the frequencies ωωii at which energy residesat which energy resides

–– Only that the support is limited Only that the support is limited •• For unknown For unknown ωωii, using an L1 minimum energy constraint with an L2 , using an L1 minimum energy constraint with an L2

solution is as good as an analysis where the solution is as good as an analysis where the ωωii are known; minimize: are known; minimize:

•• Sounds great, BUT lets look at a practical example:Sounds great, BUT lets look at a practical example:–– Take 253 Take 253 constitsconstits, 761 hrs of data, & , 761 hrs of data, & nnωω assuming 1 cy/mo to 4 cy/dayassuming 1 cy/mo to 4 cy/day–– Condition number Condition number kk ~0.05; i.e., an uncertainty of 5% for uniform ~0.05; i.e., an uncertainty of 5% for uniform constitsconstits–– In reality, only a few In reality, only a few constitsconstits are within 5% of Mare within 5% of M22

–– We get many poorly determined constituentsWe get many poorly determined constituents•• Why does compressive sampling fail for tides? Why does compressive sampling fail for tides? ––

–– We want dynamic fidelity and predictive capabilityWe want dynamic fidelity and predictive capability–– Compressive sampling just optimizes reconstruction with minimal Compressive sampling just optimizes reconstruction with minimal datadata

•• We need:We need:–– A sampling theory specifically for tidesA sampling theory specifically for tides–– To use an admittance function to shape the tidal spectrum

12 LLSLjLS withdF ςς −

To use an admittance function to shape the tidal spectrum

Compressive Sampling (Summary) Compressive Sampling (Summary) ––

•• NyquistNyquist and RC criteria are too conservative, IF and RC criteria are too conservative, IF frequency domain support is limited frequency domain support is limited

•• Constituent selection can (should?) be left to the Constituent selection can (should?) be left to the inversion routineinversion routine

•• A minimum energy constraint is very powerfulA minimum energy constraint is very powerful•• More samples are better (but must be More samples are better (but must be ““independentindependent””))

–– Hourly data are obsolete!Hourly data are obsolete!–– DonDon’’t even need to be evenly spacedt even need to be evenly spaced–– Why not put in the Why not put in the extremaextrema, along with , along with extremalextremal constraints?constraints?

•• Meeting all three goals of tidal analysis requires codes Meeting all three goals of tidal analysis requires codes based on new knowledge (a better sampling theorybased on new knowledge (a better sampling theory……))

Robust L1/L2 Inverse Solutions Robust L1/L2 Inverse Solutions ––•• Robust Solutions or IRLS: Robust Solutions or IRLS:

–– A.K.A. A.K.A. ““iteratively reiteratively re--weighted leastweighted least--squares (IRLS)squares (IRLS)””–– For lowFor low--noise data, approach the frequency resolution of L2 (OLS)noise data, approach the frequency resolution of L2 (OLS)–– For highFor high--noise data, approach the noise rejection of L1noise data, approach the noise rejection of L1–– Use weighted L2 to Use weighted L2 to downweightdownweight ““improbableimprobable”” datadata

•• Example: fitting a straight line to data with Example: fitting a straight line to data with ““spikesspikes””::–– IRLS IRLS downweightsdownweights the outliers; almost as good as w/o noisethe outliers; almost as good as w/o noise–– Better confidenceBetter confidence

limits than OLS, limits than OLS, even if solutions are even if solutions are similar similar

Fit to dataFit to data IRLSIRLS

OLSOLSCauchy Cauchy weighting weighting functionfunction

IRLS (L1/L2) Rejection of Noisy Data IRLS (L1/L2) Rejection of Noisy Data ––

•• Astoria (Tongue Pt) Astoria (Tongue Pt) 1999 tidal heights1999 tidal heights

•• IRLS IRLS downweightsdownweightsdata during stormy data during stormy periodsperiods

•• This appears to This appears to produce better resultsproduce better results

Astoria (Tongue Pt) 1999 tidal heightsAstoria (Tongue Pt) 1999 tidal heights

IRLS weightsIRLS weights

|Atmospheric pressure deviation||Atmospheric pressure deviation|

IRLS ResidualIRLS Residual

IRLS Reduction of Uncertainty IRLS Reduction of Uncertainty ––•• Changes in constituent estimates are small relative to Changes in constituent estimates are small relative to t_tidet_tide, BUT, BUT•• The Cauchy (IRLS) weighting reduces constituent uncertainty The Cauchy (IRLS) weighting reduces constituent uncertainty •• Overlapping 190 d analyses Overlapping 190 d analyses

using using t_tidet_tide with an with an IRLS solverIRLS solver

•• You can download a You can download a ββ--version from version from http://http://www.cee.pdx.edu/~jaylabwww.cee.pdx.edu/~jaylab//

•• The version we will useThe version we will usethis PM only does tides,this PM only does tides,not currentsnot currents

•• Note: a 10% reduction in Note: a 10% reduction in MAD = a 20% reduction inMAD = a 20% reduction in95% confidence limits

Constituent estimatesConstituent estimates Median Abs DeviationMedian Abs Deviation= MAD= MAD

95% confidence limits

The The ““StandardStandard”” Failure Mode Failure Mode ––•• Rayleigh Criterion: RC = LOR Rayleigh Criterion: RC = LOR XX ∆ω∆ω §§ 1 limits number of constituents1 limits number of constituents

–– Intuitive Intuitive –– ““you canyou can’’t get something for nothingt get something for nothing””,,–– Heisenberg uncertainty principle: Heisenberg uncertainty principle: σσ∆ω∆ω X X σσLORLOR §§ 1 doesn1 doesn’’t apply; t apply; σσ∆ω∆ω ~0~0

•• As length of record (LOR) decreases, harmonic analysis (HA) failAs length of record (LOR) decreases, harmonic analysis (HA) fails s –––– Constituent pairs growConstituent pairs grow–– Results useless for dynamics, predictionResults useless for dynamics, prediction–– HA reduces number constituentsHA reduces number constituents

•• This failure occurs for both L2 and This failure occurs for both L2 and Robust (L1/L2) solutionsRobust (L1/L2) solutions

•• Compressive sampling theory suggestsCompressive sampling theory suggestsa minimum energy constrainta minimum energy constraint

•• But this is NOT sufficient for But this is NOT sufficient for maximal frequency resolution, maximal frequency resolution,

•• becausebecause……

The Standard Failure Mode (more) The Standard Failure Mode (more) ––•• As LOR increase, sideband leakage occursAs LOR increase, sideband leakage occurs•• Can be curtailed using an admittance function relative to Can be curtailed using an admittance function relative to

a reference time series (or tidal potential):a reference time series (or tidal potential):–– MunkMunk and Cartwright: the and Cartwright: the ““credo of smoothnesscredo of smoothness””–– The responses at neighboring frequencies have similar valuesThe responses at neighboring frequencies have similar values

Left: an L2 analysis Left: an L2 analysis w/o w/o an energy an energy constraintconstraint

Right: an L2 analysis Right: an L2 analysis with with an energy an energy constraintconstraint

Still need an Still need an admittance admittance function!function!

Other Things we are Working on Other Things we are Working on ––•• L2 (or L1/L2) with energy constraint (done)L2 (or L1/L2) with energy constraint (done)•• Implement admittance calculation for long recordsImplement admittance calculation for long records•• Implement an admittance constraint for short recordsImplement an admittance constraint for short records•• Test an L1 norm + energy constraintTest an L1 norm + energy constraint•• Analysis of historic highAnalysis of historic high--low data with low data with extremalextremal and other and other

constraintsconstraints•• Incorporate river flow forcing in analysis as a basis Incorporate river flow forcing in analysis as a basis fctfct; e.g.:; e.g.:

•• This allows estimation of a This allows estimation of a ““generalized admittancegeneralized admittance”” relative to relative to stationary tidal and nonstationary tidal and non--tidal (nontidal (non--stationary) forcing

][)(][)( 22

002 tSinQbtCosQaM

nn

R

m

nnR

m

nn ωω ∑∑

==

+=

stationary) forcing

Columbia River Tides Columbia River Tides ––

The Columbia: a system where largeThe Columbia: a system where large--scale process are scale process are overwhelmed by human alterationoverwhelmed by human alteration

River Tides River Tides ––•• Governed by geometry, friction, wave Governed by geometry, friction, wave

steepeningsteepening•• Tide propagates upriver; DTide propagates upriver; D11 powerpower--peaking peaking

(pseudo(pseudo--tide) propagates to oceantide) propagates to ocean•• Friction damps tides; river flow QFriction damps tides; river flow QRR increases increases

damping; tidal range ~1/Qdamping; tidal range ~1/QRR

•• Ocean tide is mostly semidiurnal (DOcean tide is mostly semidiurnal (D22), some ), some diurnal (Ddiurnal (D11))

•• Very nonVery non--stationary stationary –– use wavelet analysisuse wavelet analysis

CR tides, from near the ocean (top) to CR tides, from near the ocean (top) to Bonneville dam (bottom)Bonneville dam (bottom)

Astoria/Tongue Pt

The Tidal Spectrum Evolves Upriver The Tidal Spectrum Evolves Upriver ––•• Mostly DMostly D11 and Dand D22 at at

Astoria (bandAstoria (band--limited)limited)•• OvertidesOvertides develop develop

upriver by frictional upriver by frictional distortiondistortion

•• QQR R effects also grow effects also grow (non(non--stationary, broadstationary, broad--band signal)band signal)

•• CWT methods are CWT methods are useful for analysisuseful for analysis

Time and space evolution of Time and space evolution of tidal amplitude, as tidal amplitude, as determined by wavelet determined by wavelet transform

Horizontal band = a stationary wave vertical cone = an event

transform

TodayToday’’s Experimental Setup s Experimental Setup ––

•• Keith Keith LefflerLeffler has put together a has put together a MatlabMatlab Object that Object that includes:includes:–– Modified Modified t_tidet_tide with robust fit options. You can compare L2 with robust fit options. You can compare L2

(method = (method = ‘‘olsols’’) with Robust fit (method =) with Robust fit (method =‘‘bisquarebisquare’’) ) –– Simple plotting and result extraction optionsSimple plotting and result extraction options–– 19961996--2007 tide data for six stations along the Columbia2007 tide data for six stations along the Columbia–– Daily river flow for 1996Daily river flow for 1996--20072007

•• Points about the data:Points about the data:–– River flow disturbance of tide increases drastically upriverRiver flow disturbance of tide increases drastically upriver–– So tides at upriver points are very nonSo tides at upriver points are very non--stationary stationary –– amplitudes amplitudes

may vary by 10X over the annual river flow cyclemay vary by 10X over the annual river flow cycle–– Highest river flows were in 1996, 1997 and 1999Highest river flows were in 1996, 1997 and 1999–– Lowest sustained flows were in 2001 and 2003 (I think)Lowest sustained flows were in 2001 and 2003 (I think)–– Data quality is poor until March 2002, except at AstoriaData quality is poor until March 2002, except at Astoria

The Stations The Stations ––

••

Suggested Exercises Suggested Exercises ––

•• Compare error limits and SNR for L2 (Compare error limits and SNR for L2 (olsols) and Robust Fit ) and Robust Fit ((bisquarebisquare) for different stations) for different stations

•• Try to resolve the seasonal cycle of tides at the more Try to resolve the seasonal cycle of tides at the more landward stations:landward stations:–– How big is the seasonal variability as How big is the seasonal variability as f(xf(x)?)?–– How does How does overtideovertide variability compare to variability of M2 and variability compare to variability of M2 and

K1? Why?K1? Why?–– How many frequencies can you resolve while still resolving the How many frequencies can you resolve while still resolving the

seasonal cycle as seasonal cycle as f(xf(x)?)?•• What is the balance of What is the balance of riverflowriverflow & atmospheric effects?& atmospheric effects?

–– Which stations show significant atmospheric effects? Which stations show significant atmospheric effects? •• How does Robust fit deal with freshets and storms? Is How does Robust fit deal with freshets and storms? Is

this good or bad?this good or bad?

How to get Started How to get Started ––

•• Copy and unzip Copy and unzip PASI_lab.zipPASI_lab.zip•• Read the documentation file Read the documentation file PASI_Lab.pdfPASI_Lab.pdf (or .doc) in (or .doc) in

\\PASIPASI\\documentationdocumentation•• Try the Try the PASI_example.mPASI_example.m file in file in \\PASIPASI\\ExamplesExamples•• Modify as needed!Modify as needed!•• Other useful things to do:Other useful things to do:

–– Read Read PawlowiczPawlowicz et al. on et al. on t_tidet_tide–– Read Read LefflerLeffler and Jay about modifications to and Jay about modifications to t_tidet_tide