Realization of Linear Wave-Propagation Models from HPC Simulations

Stephen A. Ketcham and Michael W. Parker USACE Engineer Research and Development Center,

Cold Regions Research and Engineering Laboratory (ERDC-CRREL), Hanover, NH

{stephen.a.ketcham, michael.w.parker}@usace.army.mil

Minh Q. Phan Thayer School of Engineering,

Dartmouth College, Hanover, NH mqphan@dartmouth.edu

Abstract

Modeling of sound propagation in complex environments requires high performance computing (HPC) to simulate three-dimensional wave fields with realistic fidelity. This is especially true for urban areas, where sound waves reflect and diffract due to the built-up infrastructure. HPC can predict these wave fields with desired fidelity, but the computational investment would have greater return if reduced-size models that operate with considerably less computational resources could be produced from the results. The objective of this work is to develop such models. The work applies a modified version of the Eigensystem Realization Algorithm, using Markov parameters from HPC input-output response functions, to generate state-space models that simulate hundreds of thousands of output signals of the HPC wave field. The results include predicted acoustic signals and signatures from realized models, using a source with a different time series than the source used to generate the Markov parameters. We compare wave-field signals from reduced-order models with HPC model signals over a large urban domain, adjusting the model order and accuracy by singular-value cutoff. We conclude that the method produces efficient high-fidelity models of sound propagation in complex environments. 1. Introduction Modeling of outdoor sound propagation in complex environments requires high performance computing (HPC) to simulate three-dimensional (3D) wave fields with realistic fidelity. This is especially true for urban environments, where sound waves are reflected and diffracted by buildings and other structures or objects. As a result, energy from a source typically travels along multiple paths to reach a sensing location, with constructive or destructive phase interference observed in continuous signals. Finite difference and finite element

calculations using HPC can predict these wave fields with desired fidelity, but the computational investment would have far greater return if accurate yet smaller wave-field models, which operate with considerably less computational resources, could be produced from the results. The objective of this work is to develop reduced-order models (ROMs) of 3D HPC outdoor-sound-propagation systems that operate efficiently and accurately on desktop or mobile computers. A reduced-order model is a low-order model derived from the degrees of freedom of a dynamic system[1]. The ROM is smaller yet reproduces the system dynamics within a desired error level. Our focuses are: 1) linear-time-invariant (LTI) large-wave-field acoustic models, because this applies to sound sensing and sensor-location planning; 2) post-processed ROMs using calculated pulse responses (Markov parameters), because this creates reusable site-specific models; 3) subset fields of the full three-dimensional HPC model domain, because this produces wave-field ROMs with practical application while maintaining three-dimensional accuracy; and 4) a modified state-space system-identification algorithm, because state-space algorithms are efficient and well-developed for model reduction, yet existing algorithms based on singular-value decomposition (SVD) are unrealistic for the number of outputs in our wave-field ROMs. The work develops and applies a modified Eigensystem Realization Algorithm (ERA)[2] as the state-space identification algorithm. It uses Markov parameters of input-output response functions and generates ROMs that simulate hundreds of thousands of output signals of the HPC wave-field. The results include predicted acoustic wave-field signals from realized models, using a source with a different time series than the source used to generate the Markov parameters. We compare ROM wave-field signals and source signatures with HPC model results over a large and highly resolved subset domain, adjusting the model order and accuracy by singular-value

2009 DoD High Performance Computing Modernization Program Users Group Conference

978-0-7695-3946-1 2010

U.S. Government Work Not Protected by U.S. Copyright

DOI 10.1109/HPCMP-UGC.2009.57

350

cutoffs. We conclude that the method produces efficient and accurate reduced-order models for sound propagation. 2. Methodology 2.2. High-Performance Computations The HPC analyses were finite-difference time-domain computations in a linear time-invariant acoustic medium[3–5]. The computations solved first-order partial differential equations for pressure and particle velocity responses to a dilatation-rate source. We used an urban geometry model derived from a city-core modeling template[6]. Figure 1 shows this model. It has x×y×z dimensions of 750 × 748 × 179-m using 2,336 × 2,328 ×508 nodes. The source location was 1.93-m above the ground surface at the x-y center. We used two time series to define dilatation-rate sources. The first was a filtered pulse. The simulation using this source generated a broadband acoustic response to just beyond 100-Hz. The duration was 4.4-s. The saved wave field contained 1.26M outputs from a contour 0.5-m above the ground and rooftops. The spatial resolution was 0.643-m. Each output signal had 1,024 time steps with a 118-Hz Nyquist frequency. To generate Markov parameters for the system identification, we processed the source and wave-field time series by Fourier analysis[7], dividing the wave field into eleven sections, each having 114,240 signals. The second source contained superimposed impulsive and continuous signals designed to produce an independent HPC result. Figure 2 plots the initial part of this source. We generated ROMs of each wave-field section from the first source, and tested their accuracy using the ROM and HPC responses to the second source. The Cray XT3 at the US Army Engineer Research and Development Center performed the simulations with 256 cores. The simulations with the first and second sources used 6.4-h and 10.2-h wall time, respectively.

Figure 1. Geometry of propagation model CC750_01, derived from city-core urban template. The grey blocks are buildings

contained wholly or partly within the model. The black surface spans the ground-level extents of the model.

Figure 2. Dilatation-rate source made up of superimposed 30-

and 45-Hz Gaussian pulses and 22.5-, 37.5-, and 52.5-Hz harmonics. The time series terminated at 4 s in a filtered

transient. 2.2. Reduced-Order-Model Calculation State-space equations for a discrete LTI system are: ( 1) ( ) ( ); ( ) ( )x k Ax k Bu k y k Cx k+ = + = (1)

where x is an n-dimensional state vector; u is an m-dimensional input vector; and y is a q-dimensional output vector. A is the n×n system matrix, B the n×m input matrix, and C the q×n output matrix. k is the discrete-time index. The Markov parameters of this system, i.e., the impulse responses, are

( ) 1 ; 1,2,kh k CA B k−= = … , (2)

where each h(k) has size q×m. Eigensystem Realization Algorithm. Using Markov parameters calculated from input and output data, ERA can identify a state-space model [ABC] such that Eqn. 1 holds and the state-space dimension is minimal[2]. ERA begins by forming Hankel and shifted Hankel matrices with I row blocks and J column blocks. The Hankel matrix, H(0), is:

(1) (2) ( )(2) (3) ( 1)

(0) ,

( ) ( 1) ( 1) qI Jm

h h … h Jh h … h J

H

h I h I … h I J

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦ ×

+=

+ + −

(3)

The Hankel matrix shifted by one index, H(1), is

(2) (3) ( 1)(3) (4) ( 2)

(1) .

( 1) ( 2) ( ) qI Jm

h h … h Jh h … h J

H

h I h I … h I J

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦ ×

++

=

+ + +

(4)

351

Singular value decomposition of H(0) produces left and right singular vectors U and V and a matrix of singular values Σ, i.e.,

( )0 TH U V= Σ . (5)

Using this result, a discrete-time minimum-order realization of the system can be shown to be

( )1/2 1/2

1/2 1/2

1 ;

; ,

T

T Tm q

A U H V

B V E C E U

− −= Σ Σ

= Σ = Σ (6)

Where Em and Eq are utility matrices defined by

( ) ( ) 0 ; 0T Tm m m q q qJm m m qI q qE I E I× ×− × − ×

⎡ ⎤ ⎡ ⎤= =⎣ ⎦ ⎣ ⎦ . (7)

The realized state-space model differs only with [ABC] in Eqn. 1 by an equivalence transform, i.e., both have identical input-output responses h(k). With perfect Markov parameters, the singular value decomposition in Eqn. 5 reveals the true minimum order n of the system if I and J are chosen to be sufficiently large so that the rank n of H(0) can be revealed. In the presence of noise, or when the Markov parameters are imperfect, the Hankel matrix H(0) will contain more than n non-zero singular values, rendering the determination of the true minimum order of the system not obvious. In that case, the number of singular values that the user decides to retain in Eqn. 5 determines the order of the realization in Eqn. 6. Modified Eigensystem Realization Algorithm. For wave-fields with hundreds of thousands or even millions of outputs, the calculation in Eqn. 5 is unrealistic in terms of both computational time and random access memory (RAM) requirements. Our modified ERA formulation avoids this calculation. Let the Eqn. 1 system have a number of outputs many orders of magnitudes larger than the number of inputs, i.e., q m. Similar to the data correlation matrices in Reference 8, we define matrices H(0)T H(0) and H(1)T H(0) containing auto- and cross-correlation sequences of the q×m pulse response matrices h(k). Applying Eqn. 5 and the orthonormality of the right singular vectors, which yields UTU=I, results in

( ) ( ) ( ) 20 0TT T T TH H U V U V V V= Σ Σ = Σ . (8)

Thus, singular value decomposition of H(0)T H(0) produces the squares of the singular values of H(0) and its matrix of right singular vectors V. The corresponding matrix of left singular vectors U derives from post-multiplying H(0) in Eqn. 5 with VΣ−1 and applying the orthonormality of the right singular vectors:

( ) 1 10 TH V U V V U− −Σ = Σ Σ = . (9)

Using the transpose of the Eqn. 6 realization and substituting the above expression for U, a memory-

efficient and smaller singular-value-decomposition solution for the q m problem is:

( )( )

( )

1/2 1/2

1/2 1 1/2

1/2

1/2

1/2 1

1/2

[ ] [ ] [ (1)] [ ] [ ]

(1) (0)

(0)

(0)

[ ] ; [ ] ; [ ]

T T T T T T T

T T

T Tm

T Tq

T Tq

T Tq

T T T T T T

A V H U

V H H V

B E V

C U E

V H E

V H E

A A B B C C

− −

− − −

−

−

= Σ Σ

= Σ Σ Σ

= Σ

= Σ

= Σ Σ

= Σ

= = =

(10)

As indicated, the transposes of AT, BT, and CT provide the desired minimum realization [ABC]. An implementation can begin by computing H(1)T H(0) and H(0)T H(0) using the Markov parameters from the input-output data, and by computing the first block row of the Hankel matrix H(0)T Eq. Both H(1)T H(0) and H(0)T H(0) are matrices of summations. They can be formed in the same operation without the full Hankel matrix in memory. Processing only the unique block multiplications reduces the computations in this operation considerably. Method for Outdoor Sound Propagation. Equation 10 will produce a realization that reproduces the data-generated Markov parameters h(k) almost exactly. The prediction error of a realized model will therefore derive primarily from inaccuracies in the generated Markov parameters, plus inaccuracies from user-selected model order reduction. There are two principal sources of error in the Markov parameters. The first is Fourier-analysis error, which we minimize by techniques in Reference 7. The second is due to the limited duration of the HPC data; i.e., the Markov-parameter sequence is limited to the finite-time interval k=1,2,…p, where p is the final time step used in the Fourier analysis of the HPC wave field. Because outdoor propagation cannot be described with classical modal decomposition, setting h(k)=0 for k>p is appropriate for identifying the propagation dynamics during k=1,2,…p. However, for simulations in a multi-path environment beyond time step p, this produces error by neglecting trapped reflected energy. To mitigate this error, we designed the HPC urban-acoustic analyses to have acceptable sound decay throughout a model domain before time step p. When setting h(k)=0 for k>p, we operate with p=I+J Markov parameters, such that H(0) and H(1) in Eqns. 3 and 4 become:

352

(0)(1) (2) ( 2) ( 1)(2) (3) ( 1) ( )(3) (4) ( ) 0(4) (5) ( ) 0

( 1) ( ) 0 0 0

Hh h h p h ph h h p h ph h h ph h h p

h p h p

=

− −⎡ ⎤⎢ ⎥−⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥

−⎢ ⎥⎣ ⎦

(11)

( )

(1)(2) (3) ( 1) ( )(3) (4) ( ) 0(4) (5) 0 0

.(5) (6) 0 0

( ) 0 0 0 0

Hh h h p h ph h h ph h h ph h

h p

=

−⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

(12)

These Hankel matrices produce a state-space model consisting of n=(p−1)m states. In contrast to conventional ERA, implementation of Eqn. 10 is practical for wave-field applications. The size of H(0)T H(0) required for SVD is Jm×Jm , which is smaller than the Hankel matrix H(0) in the Eqn. 5 SVD by the factor qI/Jm. The size of V is Jm×n, of which the first term in BT uses only the first m rows. The size of H(0)T Eq in the expression for CT is Jm×q, and thus only the first q rows of the Hankel matrix H(0), i.e., its first block row, need be formed. U, the qI×n matrix of left singular vectors, is not calculated. We have operated with pulse-response lengths p=1,024 in this work, which produces a model order that is far less than the number of degrees of freedom in the HPC model. For our single input system, m=1, and the models have n=1,023-states. H(0)T H(0), required for SVD, is 1,023×1,023. Finally, the order of the realizations can be reduced as described in Reference 2, from the order n to an order n′ that captures the system within a selected dominant subspace. This allows the user to trade between ROM simulation efficiency and accuracy. 3. Results The results include performance of the modified ERA to identify and create a state-space model [ABC], and performance and results of simulations using the identified model. We coded both the identification and simulation routines in MATLAB. Figure 3 illustrates a snapshot of the HPC-model response to the Figure 2 source.

Figure 3. HPC model CC750_01: map of sound pressure response (Pa) above ground and rooftops at t=1.085 s to

source of Figure 2. Circles are locations where signals were extracted.

3.1. Identification of State-Space Model We performed the identification on a desktop computer. The tasks included: 1) dividing the HPC wave-field data into sections; 2) calculating the Markov parameters for each section; and 3) applying the modified Eigensystem Realization Algorithm. As mentioned previously, we created eleven sections, each with 114,240 signals. The modified ERA processing, which currently operates sequentially over data sections, took approximately 1-h to create a 1,023-state [ABC] model for each section. 3.2. Simulations and Model-Order Reduction With the identification process complete, we performed state-space simulations using the Figure 2 source. This source had the sampling rate required by the identified model, but, as mentioned above, its time series was independent of the source used to generate the model. The objective of the simulations was: 1) to quantify the efficiency of the reduced-order models and 2) to quantify and qualify their accuracy. We applied SVD-based model truncation to a balanced realization of the 1,023-state [ABC] model, creating a suite of models having 1,023, 767, 511, and 255 states. Table 1 quantifies the performance of simulations using these models. The simulation software processes Eqn. 1 using the MATLAB linear time-invariant time response kernel LTITR. The software performs the simulation sequentially for each data section, predicting the model response over sets of time steps. It accesses the [ABC] model directly from disk using virtual memory. By this approach, small-memory computers do not overly restrict the simulation.

353

The Table 1 data reveal very efficient ROM simulations relative to the requirements of the reference HPC simulation. Although the ROMs simulated a wave field that is a fraction of the size of the HPC model, the 1.26M output field, illustrated by Figure 3, is highly practical for sensor-location planning due to its very large size. In this context, the efficiency ratios in Table 1 are impressive. For the 1,023-state model on a laptop, the HPC-to-ROM speedup ratio was 23. Considering memory and CPU requirements, the ratio was over 1 million, which is what made the laptop processing possible. On a desktop these values improved, with model-order reduction showing improvements in computational efficiency. Tables 2 and 3 present error measures of the ROM wave fields relative to the HPC wave fields. Table 2 focuses on different stages of the 1,023-state-model simulation to calculate error measures attributable to (1) Fourier analysis of the Markov parameters and (2) late-arriving reflected energy neglected in the Markov parameters. Because the Figure 2 source has 4-s duration, and the simulation is 6.36-s, which is longer than the 4.36-s duration of the Markov parameter sequence, the simulation provided a test of both error types. The median relative RMS error in the early part of the simulation was less than 2 %, while in the latter part of the simulation, beyond the number of Markov parameters, this error jumped to nearly 15%. Over the entire simulation duration, the median relative RMS error was 6.3 %.

Table 1. Efficiency ratios: reduced-order models compared to HPC analyses

Order

Write-step duration, sHPC/sROM

Number of cores×duration

(CPU-s)HPC/(CPU-s)ROM

Number of cores×duration×RAM used

(CPU-s-GB)HPC/(CPU-s-

GB)ROM

1,0231 23 5.9E+03 1.2E+06 1,0232 55 1.4E+04 1.5E+06 7672 67 1.7E+04 1.8E+06 5112 75 1.9E+04 2.0E+06 2552 92 2.4E+04 2.5E+06

1Single-core analysis on laptop using maximum of 2.6GB RAM and I/O to external drive 2Single-core analysis on desktop using maximum of 4.8GB RAM Table 2. Statistics of the relative root-mean-squared-error fields of the response to the source in Figure 2:

1,023-state model, different stages of simulation

Time steps Median Mean Standard Deviation

257–768 0.016 0.019 0.013 1,025–1,536 0.147 0.165 0.077 1–1,536 0.063 0.073 0.044

Table 3. Statistics of the relative root-mean-squared-error fields of the response to the source in Figure 2: full simulation duration, time steps 1–1.536, in four

reduced-order models

Order Median Mean Standard Deviation

1,023 0.063 0.073 0.044 767 0.081 0.096 0.061 511 0.265 0.293 0.162 255 N/A N/A N/A

Table 3 illustrates the increasing error with model-order reduction. The measure is the relative RMS error for the full simulation duration. Both the 1,023- and 767-state models had a median error less than 10 %. This error increased to beyond 20 % in the lower-order models. Figure 3 depicts two locations where signals were extracted from the HPC and 1,023-state-model simulations for time-series and spectrogram comparisons. One is relatively close to the source in a line-of-sight location, while the other is farther away in a non-line-of-sight urban-canyon setting. Figure 4 graphs these results. The signals and the spectra both show very good agreement, qualifying the error measures in Table 2. The graphs reveal the responses to the early Gaussian pulses, the responses to the longer harmonic sources, and the transient and decaying responses after the harmonic sources drop to zero. Figure 5 graphs a zoomed-in view of the quantity 20 log10 (|Y/Y0|) , for the frequency 22.5-Hz, where Y/Y0 is the ratio of the Fourier transform of the HPC wave field with buildings divided by the Fourier transform of a reference HPC model without buildings[3]. The figure illustrates the relative power, in dB, between scattered signals at the 22.5-Hz propagation frequency and corresponding reference signals over the wave field. The images are greenish in areas of constructive multi-path interference, and reddish in obstructed zones and areas of destructive multi-path interference. We chose a very low frequency for this example because the different areas are easily distinguished. The white star and the black circles locate the source and two signal-extraction locations, respectively. Both signal locations are 25-m from the source. Without buildings the amplitudes would be identical at the two locations, but with buildings one signal is in an area of constructive interference, and the other is in an area of destructive interference. In Figures 6 and 7, we used these locations to examine the signature-propagation capabilities of the reduced-order models. Each of these figures show band-filtered signals, where the pass band was 17.5–27.5-Hz, in order to isolate the 22.5-Hz signature characteristic of the Figure 2 source. Figure 6 contains data from the destructive-interference location, while Figure 7 contains

354

the corresponding data from the constructive-interference location. The expectation from Figure 5 is that a 22.5-Hz source would produce an approximately 15-dB amplitude difference at these locations. Although there are subtle differences in the spectrograms with changes in model order, this is what Figures 6 and 7 show for all of the model orders. Thus, even though the relative error measure may have been excessive in the lower-order models, they still propagated realistic signature characteristics and amplitudes at the example frequency, accounting for the scattering interference inherent in the HPC model. 4. Conclusion We have developed a computationally efficient version of the Eigensystem Realization Algorithm to derive reduced-order models from HPC-model wave-field data. The method uses Markov parameters of the HPC input-output response functions, and identifies ROMs that simulate hundreds of thousands of output signals of the HPC wave field. Simulations using the ROMs were highly efficient and able to operate on laptop and desktop computers when simulating an urban-acoustic wave field containing more than 1 million output signals. Relative RMS errors over the wave field indicated acceptably high accuracy in higher-order models and decreasing accuracy with the lower-order models. However, even the lower-order simulations predicted a 15-dB interference effect of the urban-acoustic scattering environment on a low-frequency harmonic signature. The method thus produces efficient and accurate reduced-order models for sound and sound-signature propagation. As a result, when the wave-field response to a different source input is needed, the ERA-derived reduced-order model can be used in place of the original HPC simulation code, resulting in many orders of magnitude savings in overall computational requirements. Acknowledgments Funding support is from DoD High Performance Computing Modernization Program (HPCMP) Software Applications Institute I-01: Institute for Maneuverability and Terrain Physics Simulation; and USAERDC ILIR Program “Reduced-Order High-Fidelity Models for Signature Propagation.” Computational support is from HPCMP Challenge Project C2X, “Decision Support for Seismic and Acoustic Sensors in Urban Terrain.”

References 1. Lucia, D.J., P.S. Beran, and W.A. Silva, “Reduced–Order Modeling: New Approaches for Computational Physics.” Prog. in Aerospace Sci., 40, pp. 51–117, 2004. 2. Juang J.-N. and R.S. Pappa, “An Eigensystem Realization Algorithm for Modal Parameter Identification and Model Reduction.” Journal of Guidance, Control, and Dynamics, 8, pp. 620–627, 1985. 3. Ketcham, S.A., M.W. Parker, H.H. Cudney, and D.K. Wilson, “Scattering of Urban Sound Energy from High-Performance Computations.” Proc. 2008 Users Group Conference, DoD High Performance Computing Modernization Program, IEEE Computer Society, pp. 341–348, 2008. 4. Cudney, H.H., S.A. Ketcham, and M.W. Parker, “Verification and Validation of Acoustic Propagation Over Natural and Synthetic Terrain.” Proc. 2007 Users Group Conference, DoD High Performance Computing Modernization Program, pp. 247–252, 2007. 5. Parker, M.W., S.A. Ketcham, and H.H. Cudney, “Acoustic Wave Propagation in Urban Environments.” Proc. 2007 Users Group Conference, DoD High Performance Computing Modernization Program, pp. 233–237, 2007. 6. Fordyce, D.F., “Standardized Urban Terrain Templates Based on UTZs.” Presentation at 71st MORS Symposium, Military Operations Research Society, Quantico, VA, 2003. 7. Bendat, J.S. and A.G. Piersol, Random Data, Analysis and Measurement Procedures, 2nd ed., John Wiley, New York, 1986. 8. Juang, J.-N., J.E. Cooper, and J.R. Wright, “An Eigensystem Realization Algorithm Using Data Correlations (ERA/DC) for Modal Parameter Identification.” Control Theory and Advanced Technology, 4(1), pp. 5–14, 1988.

355

Figure 4. Signals and spectrograms. (a) HPC model,

[x,y]=[389,338] m; (b) 1,023-state model, [389,338]m; (c) HPC model, [239,163] m; (d) 1,023-state model, [239,163] m.

Figure 5. Zoomed view of model CC750_01: map of

constructive and destructive interference (dB) at 22.5-Hz above ground and rooftops. The white star is at the source

position. The black circles are at two signal-extraction locations, each 25-m from the source.

356

Figure 6. Signals and spectrograms, location [x,y]=[389,413] m, for models with (a) 1,023-, (b) 767-, (c) 511-, and (d) 255-

states

Figure 7. Signals and spectrograms, location [x,y]=[412,378] m, for models with (a) 1,023-, (b) 767-, (c) 511-, and (d) 255-

states

357