THE INTERNATIONAL HYDROGRAPHIC REVIEW - iho.int · international hydrographic review may 2015 the international hydrographic review international hydrographic organization monaco

INTERNATIONAL HYDROGRAPHIC REVIEW MAY 2015

THE

INTERNATIONAL

HYDROGRAPHIC

REVIEW

INTERNATIONAL HYDROGRAPHIC ORGANIZATION

MONACO

Publication P-1

No. 13

2


© Copyright International Hydrographic Organization [2015]

This work is copyright. Apart from any use permitted in accordance with the Berne Conven-tion for the Protection of Literary and Artistic Works (1886), and except in the circumstances described below, no part may be translated, reproduced by any process, adapted, communi-cated or commercially exploited without prior written permission from the International Hydrographic Bureau (IHB). Copyright in some of the material in this publication may be owned by another party and permission for the translation and/or reproduction of that material must be obtained from the owner.

This document or partial material from this document may be translated, reproduced or distributed for general information, on no more than a cost recovery basis. Copies may not be sold or distributed for profit or gain without prior written agreement of the IHB and any other copyright holders.

In the event that this document or partial material from this document is reproduced, translated or distributed under the terms described above, the following statements are to be included:

“Material from IHO publication [reference to extract: Title, Edition] is reproduced with the permission of the International Hydrographic Bureau (IHB) (Permission No ……./…) acting for the International Hydrographic Organization (IHO), which does not accept responsibility for the correctness of the material as reproduced: in case of doubt, the IHO’s authentic text shall prevail. The incorporation of material sourced from IHO shall not be construed as constitut‐ing an endorsement by IHO of this product.”

“This [document/publication] is a translation of IHO [document/publication] [name]. The IHO has not checked this translation and therefore takes no responsibility for its accuracy. In case of doubt the source version of [name] in [language] should be consulted.”

The IHO Logo or other identifiers shall not be used in any derived product without prior written permission from the IHB.

3


By Ian HALLS, Editor

- Sound Velocity Profile (SVP) inversion through correcting the terrain distortion.

by: JIN Shaohua, SUN Wenchuan, BAO Jingyang, LIU Min, CUI Yang (China)

□ Editorial

□ Articles

- Operation Tirúa: Hydrographic Vision.

by: Nicolás A. GUZMÁN MONTESINOS (Chile)

□ Notes

555

777

171717

333333

474747

- HUDDL: the Hydrographic Universal Data Description Language.

by: Giuseppe MASETTI, Brian CALDER (USA)

- An enterprise approach to the next generation systems environment for the Australian Hydrographic Service (AHS).

by: Ian HALLS (Australia)

□ General Information

- † Obituary: Richard Michael (Mike) EATON 535353

4


Page intentionally left blank

5


Editorial Despite my plans that the November 2014 edition would be my last as Editor, as it turns out, my role as Editor continues. This edition comprises several articles and notes. The first article from China provides a technical solution to improving multi-beam bathymetry by using seafloor terrain distortion and modelling to correct sound velocity measurements. As multibeam technology is in wide use, following proper processes should minimise errors during field collection. However, this is some-times not possible and this paper provides a thoughtful process to improve survey results when field operations don’t quite proceed according to plan. The second paper describes a new approach to the problem of handling the multi-tude of hydrographic survey data formats. Anyone who has dealt with digital survey formats will appreciate the difficulties with managing various data struc-tures, version control, metadata, data re-formatting and so on. Staff from the University of New Hampshire, USA, have developed a XML-based Hydrographic Universal Data Description Language (HUDDL) and a set of utilities to assist with data access, cataloguing, version control, etc. The authors intend this suite of tools to be an open, community-led initiative. Prior to 2000, the Australian Hydrographic Service (AHS) was a leader in the evolutionary development of digital technology to improve charting and data man-agement. With the introduction of a Digital Hydrographic Database (DHDB) solution in 2000, developed under a more structured major Defence capability process, the reality of the technology capability and the organisation re-structure did not meet the expectations of the total grand solution. In 2011, the AHS commenced evolutionary sustainment processes to refresh and incrementally re-build the technology capability in alignment with the AHS business using an Enterprise Architecture (EA) approach. The third article describes this approach and its impact across the whole of the organization. Included in our Notes section are two papers. The first paper is an account from Chile on their experiences with trying to find a small airplane that went missing and assumed to have crashed into the sea. Whilst their search was un-successful, they identify key processes and capabilities required to undertake a search opera-tion. Our final note is a tribute to an esteemed colleague, Mike Eaton. I had the pleasure to be involved with Mike’s work in the early 1990s through the IHO ECDIS working groups. Australia along with several other HOs and private com-panies contributed to the S-52 Presentation Library work led by Mike. The current ECDIS capability is in part, a direct result of his professionalism and steadfast resolve to ensure electronic chart information is presented efficiently and effectively to the mariner. He was a true pioneer of the ECDIS technology and his legacy will continue for a long time. On behalf of the Editorial Board, I hope that this edition is of interest to you. Thank you to the authors for your contributions and to my colleagues who provided peer reviews for the Articles in this edition.

Ian W. Halls Editor

6



7


SOUND VELOCITY PROFILE (SVP) INVERSION THROUGH CORRECTING THE TERRAIN DISTORTION

S. JIN, W. SUN, J. BAO, M. LIU, Y. CUI Department of Hydrography and Cartography, Dalian Naval Academy, China

Abstract

Résumé

Resumen

In this paper, mode vectors are obtained via the Empirical Orthogonal Function (EOF) based on real Sound Velocity Profiles (SVP) measurements. Through correcting the terrain distortion, reconstructed coefficients of SVPs are determined by Genetic Algorithm (GA) and then the inversion result of the SVP is obtained. The conclusions show that the terrain distortion caused by sound velocity errors can be effectively corrected by the inversion result of the SVP. Using this process, the accuracy and processing efficiency of multi-beam bathymetry data can be significantly improved. Key Words: sound velocity profile; SVP; EOF; GA; inversion; accuracy

Cet article décrit la décomposition en vecteurs propres via les fonctions empiriques orthogonales (EOF) des mesures des profils de vitesse du son (SVP). La correc-tion des distorsions de terrain permet de déterminer des coefficients reconstruits des SVP à l’aide d’un algorithme génétique (AG), puis le résultat de l’inversion du SVP est obtenu. Les conclusions montrent que les distorsions de terrain causées par les erreurs de vitesse du son peuvent être effectivement corrigées par le résultat de l’inversion du SVP. A l’aide de ce processus, la précision et l’efficacité du traitement des données bathymétriques multifaisceaux peuvent être améliorées de façon importante. Mots clés: profil de vitesse du son, SVP, EOF, AG, inversion, précision.

En este artículo, se obtienen los vectores de forma a través de la Función Empírica Ortogonal (EOF) basada en mediciones reales de los Perfiles de la Velocidad del Sonido (SVP). Mediante la corrección de la distorsión del terreno, los coeficientes reconstruidos de los SVPs son determinados por el Algoritmo Genético (AG) y posteriormente se obtiene el resultado de la inversión del SVP. Las conclusiones muestran que la distorsión del terreno causada por errores de velocidad del sonido puede corregirse eficazmente mediante el resultado de la inversión del SVP. Utilizando este proceso, la exactitud y la eficiencia del procesa-do de los datos de batimetría multihaz pueden mejorarse significativamente. Palabras clave: perfil de la velocidad del sonido; SVP; EOF; GA; inversión; exactitud.

8


1. Introduction During multi-beam bathymetric data process-ing, the speed of sound varies with depth and can cause the oblique sounding ray paths to bend, introducing significant and systematic biases in soundings which leads to terrain distortion. The accuracy of bathymetric survey can be greatly improved by using corrected Sound Velocity Profile (SVP) data. SVP sampling is the major method of obtain-ing sound velocity during a multi-beam bathy-metric survey. (Herman, et al. 1998). Current survey practice requires that sound velocity measurements are taken near the extreme times of water temperature changes during the day, being 08:00 am, 02:00 pm and 08:00 pm. Rather, it is preferred that sound velocity measurements are taken nearer the time of the bathymetric data collection. (ZHU, 2011). Due to the variability in time and space, the ray trace solutions of multi-beam bathymetry collected using current methods won’t necessarily meet the accuracy requirements for all bathymetric data. Thus, the distortion in the resultant seafloor terrain can be the result of sound velocity differences (errors) between the SVPs. During the multi-beam bathymetric data processing, the terrain distortion caused by sound velocity differences, can be corrected, by adjusting the Coefficients parameter of Refraction in commercial processing software.

Based on the measurement characteristics discussed, mode vectors can be obtained by EOF using real SVPs obtained in the survey area. Correcting the terrain distortion, the inversion result of the SVP can be determined by Genetic Algorithm (GA). Finally, these methods have been tested using observed data. 2. Terrain Distortion Caused by Sound Ve-locity Errors Changes in temperature and salinity through-out the water column affect sound velocity. Changes in the physical properties of the medium causes deflection of the acoustic direction. If the ray traced solutions are computed using the incorrect sound velocity

profile, the incorrect ray paths will lead to a lower accuracy in the bathymetry measure-ment and increased terrain distortion. (LI, et al. 1999, DONG, et al. 2007, LI, et al. 2001, ZHAO, LIU 2009). When the sound velocity at the surface is less than the sound velocity at lower depths, the terrain distortions are concave. Where the above is reversed, the terrain distortions are convex. (DONG, et al. 2007, DONG, et al. 2011). If the adjacent two swaths are both distorted, there is a “ridge” or “groove” in the along-track direction. As shown in Figure 1, both of the multi-beam swaths are concave and there is a “ridge” in the along-track direc-tion. Hence it can be determined that the SVP used to compute the ray traced solutions of the multi-beam bathymetry is wrong in this area.

3. Sound Velocity Profiles (SVPs) by EOF Modeling A SVP is the vertical distribution of a velocity structure and can be estimated by some regularity in coastal and ocean areas. (Cartwright, 2003, Kammerer, 2000, ZHU, 2011). They can also be described by a para-metric model. (ZHANG, et al. 2011). Currently, there are two parametric models for determin-ing SVPs: Analytic Function (AF) and Empiri-cal Orthogonal Function (EOF).

Figure 1. Terrain Distortion caused by the Sound Velocity Errors

9


According to LeBlanc and Middleton (1980), it is a valid method to describe the SVP using an EOF parametric model based on the spatio-temporal correlation of the profiles. (DING, et al. 2007, LeBlanc and Middleton, 1980, SHEN, et al. 1999, PENG, et al. 2003, HE, et al. 2006). First, mode vectors via EOF are obtained based on real, measured SVPs. When combined with the sample data, recon-structed modeled SVPs can then be obtained.

The modeling process follows: As is shown in Figure 2, is the number of SVPs in the survey area. Interpolating equally spaced vertical measurements to a standard floor, the SVP standardization matrix with a size of can be obtained:

(1)

The average SVP can be obtained after each row of the standardization matrix is averaged. The SVP disturbance matrix is obtained by subtracting each column and and is the covariance matrix of the SVPs disturbance matrix . As shown be-low, the covariance matrix eigenvalue decomposition:

(2)

is the matrix consisting of characteristic values and is determined from the feature vectors from the matrix .

(3)

is the eigenvalue corresponding to the feature vector (EOF mode). Thus, each SVP in the survey area can be represented with dimension vectors:

(4)

Where are the coef-ficients of SVP reconstruction. Once obtained they are used for inversion of the SVP. Moving to the left side of the equation（4）:

(5)

(6) According to least squares theory:

(7)

4. Inversion of the SVP by correcting the terrain distortion The inversion of the SVP is a method whereby the acoustic properties of the ocean can be estimated by using observed data. Many scholars have proposed methods associated with this problem (HE, et al. 2011, JIN, et al.

K

iii zfazczc

1

)()()(

NKKMNMNMNM AFCC C

)()()(

)1()2()2(

)1()1()1(

)(

21

21

21

1

KaKaKa

aaa

aaa

CPFPFFA

N

N

N

TT

Figure 2. Regular Intervals of the Sound Velocity Profiles (SVPs)

NMMM

N

N

ccc

ccc

ccc

21

22212

12111

C

NM

C NM

CC C

TCCR C

R

DFRF

nD 21diagF

R

TM zfzfzfF )()()( 21

)(zfi

K

i

K

iii zfazczc

1

)()()(

)(zc

)()2()1( Kaaaa iiii

10


1996, Lindsay and Chapman, 1993, Vaccaro, 1998, ZHANG, et al. 2002). This section discusses how mode vectors obtained via EOF are used for the inversion of the SVP. By correcting the terrain distortion, reconstruct coefficients of the SVP are determined by Ge-netic Algorithm (GA) using MATLAB (2005) software to compute the SVP inversion result.

4.1 The fitness function based on the terrain distortion As discussed in section, the terrain distortion appears when the incorrect SVP is used for the ray traced solutions of the seabed bathym-etry. As shown in Figure 3, depth gradually increases from the edge to the center when the terrain distortion is concave.

Figure 3. Composition Principle of the Fitness Function In order to measure the extent of the terrain distortion, a fitness function should be constructed as follows:

(8)

(9)

(10)

Where : is the depth of beam ; is the across-track distance of beam ;

, are the slope of the left measured depths respectively from the mid-point Pmid; is a constant; When the degree of deformation decreases, the value of fitness function becomes larger while the becomes smaller. The actual observed measurements, provide more rigor and are retained when determining the solution. 4.2 Process of SVP Inversion Figure 4 shows the SVP inversion process by GA.

First, the echo times (travel-times) of the terrain distortion are collected while the mode vectors and reconstruct coefficients ranges of the profiles are obtained by EOF in the survey area. Second, the SVP is reconstructed based on the above two elements and is used for the ray traced solutions for the bathymetry. By evolving with GA, the best individual measure-ments are preserved. Finally, the best recon-stitution coefficient of the SVP is computed and the inversion result of the SVP is obtained.

rl kkCafit max)(

1

1

__ acrossxacrossx

depthdepthk

mid

midl

midn

midnr acrossxacrossx

depthdepthk

__

idepth

iiacrossx _

i

lk rk

maxC

)(afit

rl kk

11


5. Experiments 5.1 Inversion of the Sound Velocity Profile (SVP) In this experiment, 11 SVP measurements were taken over 3 days are selected in during the survey area (Figure 5).period. The data of 70 pings arewere selected. One of the 11 measured SVPs is selected and inverted by the others. The process is measured SVPs and the EOF modeling results are shown in Figures 6 and 75 and Table 1 shows the 6. The computed ranges of the reconstruct coef-ficients are shown in Table 1.

Based on the vectors and the 22nd ping of the bathymetry measurement, the SVP can be obtained by the method described in Section 4 (The energy of the SVP represented by the first six-dimensional vectors can reach above 96%).(HE, et al. 2011, ZHANG, et al. 2010). According to the coefficient ranges in Table 1, the population can be initialized. The parame-ters for the GA method are: Population Size: 50, Hybrid Probability: 0.8, Mutation Probability: 0.15,

Figure 5. The measured SVPs

Figure 6. Mode Vectors via EOF Based on measured SVP’s

Coefficients of EOF ID

of Coefficient Lower Bound Upper Bound

Coefficient 1 1a -85.232068 87.875102

Coefficient 2 2a -27.131524 28.083683

Coefficient 3 3a -14.213039 13.785649

Coefficient 4 4a -11.402809 11.203382

Coefficient 5 5a -9.080608 11.533016

Coefficient 6 6a -8.664668 7.817749

Table 1. The Ranges of Reconstruct Coefficients

100max C

12


After an evolution process of 2000 iterations, the optimal reconstruct coefficients are com-puted and listed in Table 2.

In Figure 7, the real, measured SVP is shown as a black line whereas the inversion result for the SVP is shown by the red line. The blue line shows the SVP which is nearest to the real SVP in time.

Comparing the error statistics (listed in Table 3) and the graphical results (Figure 7), it appears that the SVP obtained by inversion (red line), more closely estimates the real SVP (black line).

In Figure 8, the black line shows the actual seabed topography and this corresponds closely with the red line that shows the seabed topography derived from the inversion

result of the SVP. The blue line shows the seabed topography which is nearest to the real SVP in time.

Statistical results shown in Table 4, indicates centimeter-level errors in the bathymetry based on the inversion SVP modeling.

5.2 Correction of Terrain Distortion by the Inversion SVP In Figure 9, the inversion SVP is used forleft hand image shows the ray traced solutions of two adjacent swaths:

Coefficient 1a 2a

3a 4a

5a 6a

Value 16.901 9.145 11.350 -6.023 -5.350 -5.319

Table 2. Computed Reconstruct Coefficients

Figure 7. Comparison of Sound Velocity Profiles

Max

（m） Min (m)

Means (m)

RMS (m)

SVP (time nearest)

11.1000 0.0222 -3.1179 3.6447

SVP (inversion)

6.0699 0.0337 -0.6938 2.1904

Table 3. Velocity Error Statistics

Figure 8. Comparison of Bathymetries

Max

（m） Min (m)

Means (m)

RMS (m)

SVP (time nearest)

11.1000 0.0222 -3.1179 3.6447

SVP (inversion)

6.0699 0.0337 -0.6938 2.1904

Table 4. Bathymetry Error Statistics

13


6. Conclusions In this paper, the influence of sound velocity error in determining seabed terrain is ana-lyzed. According to the degree of the terrain distortion, a method for inverting the sound velocity profile (SVP) is proposed by combin-ing EOF and GA methods. The design of a fitness function based on the terrain distortion provides the contents of the kernel to be used in the inversion process. The test results demonstrate that the terrain distortion caused by sound velocity errors can be effectively corrected by the SVP inversion results. Mean-while, the accuracy of the multibeam bathym-etry data has been significantly improved. 7. References

1. Cartwright, D.S, . (2003, Multibeam). Multi-beam Bathymetric Surveys in the Fraser River Delta, Managing Severe Acoustic Refraction Issues. M.Eng Re-port, University of New Brunswick, Can-ada.

2. DING Jisheng, ZHOU Xinghua, TANG Qiuhua, CHEN Yilan, . (2007,). Expres-sion of MultibeamMulti-beam Echo Sounding Sound Velocity Profile with Empirical Orthogonal Functions, Geo-matics and Information Science of Wu-han University 32(5): 446-449.

3. DONG Qingliang, CUI Minxun, ZHOU Junhua, WANG Jiguo, XU Yan. (2011). Analysis and Processing of Transform Geography of Convex and Concave in Multi-beam Sounding System, Hydro-graphic Surveying and Charting 31(1): 32-35.

4. DONG Qingliang, HAN Hongqi, FANG Zhaobao. (2007). The Influence of Sound Speed Profiles Correction on Multi-beam Survey, Hydrographic Sur-veying and Charting (2): 56-58.

5. HE Li, LI Zhenglin, PENG Zhaohui, WU LiXin, LIU JianJun. (2011). Inversion for sound speed profiles in the northern of South China Sea, SCIENTIA SINICA Phys, Mech & Astron 41(1): 49-57.

6. Herman, A.W, B. Beanlands, Chin-Yee M, Furlong A, Snow J, Young S, Phillips T. (1998). The Moving Vessel Profiler (MVP): in-situ Sampling of Plankton and Physical Parameters at 12 kts and inte-gration of a new laser/optical plankton counter. Oceanology 98, 102: 123-135.

7. JIN G, Lynch J.F, CHIU C.S, Miller J.H, . (1996,). A theoretical and simulation study of acoustic normal mode coupling effects due to the Barents Sea polar front, with applications to acoustic to-mography and matched field process-ing, J. Acoust. Soc. Am. 101(1): 193-

Figure 9. Original distorted terrain (left) and the corrected terrain (right)

14


205.

8. Kammerer E. (2000). New Method for the Removal of Refraction Artifacts in Multi-beam Echosounder Systems. PhD thesis, University of New Brunswick, Canada.

9. LeBlanc L.R & Middleton F.H. (1980). An underwater acoustic sound velocity data model, J Acoust Soc Am 67: 2055-2062.

10. LI Jiabiao, et al. (1999). Principles of multi-beam survey techniques and me-thods, Ocean Press, Beijing.

11. LI Jiabiao, Zheng Yulong, Wang Xiao-bo. (2001). The Main Affecting Factors of Multi-beam Bathymetry Accuracy, Hydrographic Surveying and Charting (1): 26-32.

12. Lindsay C.E & Chapman N.R, . (1993,). Matched field inversion study of geoacoustic model parameter using adaptive simulated annealing, IEEE Journal of Oceanic Engineering 18(3): 224-231.

13. MATLAB Genetic Algorithm Toolbox and Application. (2005). XiAn University of Electronic Science and Technology Press, XiAn.

14. PENG LH , WANG L, QIU XF, TIAN J. (2003). Modal wave number tomogra-phy for South China Sea front, China Ocean Engineering 17(2): 289-294.

15. SHEN Yuanhai, MA Yuanliang, TU Qingping, et al. (1999). Feasibility of description of the sound speed profile in shallow water via empirical orthogonal function (EOF), Applied Acoustics 18(2): 21-25.

16. Vaccaro R.J, . (1998,). The past, pre-sent and future of underwater acoustic signal processing, IEEE Signal Proc-essing Magazine 7: 21-51.

17. ZHANG Xu, ZHANG Yonggang, ZHANG Jianxu, DONG Nan. (2011). A new model for calculating sound speed profile structure, ACTA OCEANOLOGI-CA SINICA 33(5): 54-60.

18. ZHANG Xu, ZHANG Yonggang , ZHANG Jianxue, NIE Bangsheng, YAO

Zhongshan. (2010). EOF Analysis of Sound Speed Profile in East Water of Taiwan, Advances in Marine Science 28(4): 498-506.

19. ZHANG Zhongbing, MA Yuanliang, NI Jinping, et al, . (2002,). A New and Practical Method for Inverting Sound Speed Profile in Shallow Water, Journal of Northwestern Polytechnical Univer-sity 20(1): 36-39.

20. ZHAO Jianhu, LIU Jingnan. (2009). Multi-beam Bathymetric Survey and Image Processing, Wuhan University Press, Wuhan.

21. ZHU Xiaochen. (2011). Research on data processing critical modeling and application on multi-beam echo sound-ing. Dalian: Dalian Naval Academy, China.

Author Biographies JIN Shaohua is currently a lecturer at the Department of Hydrography & Cartography of Dalian Naval Academy. He worked in the Naval Survey Troop from 2004 to 2005 and was appointed as the substitute captain of the survey unit. He has been lecturing since 2006 and was engaged in hydrographic surveying theory and data processing. He received his Doctor’s degree in Geodesy and Surveying Engineering in 2011. He has extensive experi-ence in marine magnetic survey practice and data processing. Email: [email protected] SUN Wenchuan is currently a Ph.D. student in the Department of Hydrography & Cartogra-phy of the Dalian Naval Academy. He is engaged in hydrographic surveying theory and data processing. He obtained his Master’s degree in Geodesy and Surveying Engineer-ing in 2012. He has extensive experience in marine magnetic survey practice and data processing. Email: [email protected] BAO Jingyang is a Professor in the Depart-ment of Hydrography & Cartography of the Dalian Naval Academy. He worked in the Naval Survey Troop from 1987 to 1990 where

15


he was appointed as the substitute captain of the survey unit. After receiving his Master’s degree from Dalian Naval Academy in 1995, he became a lecturer in the Department of Hydrography & Cartography in the same college. Following research and teaching in marine geodesy, he received his Doctor’s degree in Geodesy and Surveying Engineer-ing in 2002 from Wuhan University. He was appointed Professor and supervisor of the Ph.D. program in 2005. He has extensive experience in tidal surveying practice and data processing. Email: [email protected] LIU Min worked in the Naval Survey Troop from 2001 to 2005. He undertook postgraduate studies at Huazhong University of Science and Technology and received his Master’s degree in 2008. He is now a Ph.D. student of Information Engineering University, majoring in Geodesy. He has extensive experience in gravity theory and data processing. CUI Yang is currently a lecturer in the Depart-ment of Hydrography & Cartography of the Dalian Naval Academy. She worked in the Naval Survey Troop from 2004 to 2005 and was appointed as the substitute captain of the survey unit. She has been lecturing since 2006 in marine geodesy, survey theory and data processing. She obtained her Doctor’s degree in Geodesy and Surveying Engineer-ing in 2013. She has extensive experience in marine geodesy survey practice and data processing. Email: [email protected].

16



17


HUDDL: THE HYDROGRAPHIC UNIVERSAL DATA DESCRIPTION LANGUAGE

G. MASETTI and B. CALDER Center for Coastal and Ocean Mapping & Joint Hydrographic Center

University of New Hampshire, Durham, New Hampshire, USA

Abstract

Résumé

Resumen

Since many of the attempts to introduce a universal hydrographic data format have failed or have been only partially successful, a different approach is proposed. Our solution is the Hydrographic Universal Data Description Language (HUDDL), a descriptive XML-based language that permits the creation of a standardized description of (past, present, and future) data formats, and allows for applications like HUDDLER, a compiler that automatically creates drivers for data access and manipulation. HUDDL also represents a powerful solution for archiving data along with their structural description, as well as for cataloguing existing format specifica-tions and their version control. HUDDL is intended to be an open, community-led initiative to simplify the issues involved in hydrographic data access.

Etant donné que de nombreuses tentatives d’introduction d’un format universel de données hydrographiques ont échoué ou n’ont que partiellement été couronnées de succès, une approche différente est proposée. Notre solution est le langage hydrographique universel de description des données (HUDDL), un langage descriptif basé sur la norme XML qui permet une description normalisée des formats de données (passés, actuels et futurs) et à partir duquel peuvent être développées des applications comme HUDDLER, un compilateur qui crée automati-quement des pilotes pour l’accès et la manipulation des données. HUDDL constitue également une solution puissante pour l’archivage des données avec leur descrip-tion structurelle, ainsi que pour le catalogage des spécifications de format actuelles et le contrôle de version. HUDDL se veut une initiative communautaire ouverte pour résoudre les difficultés d’accès aux données hydrographiques.

Dado que muchos de los intentos de introducir un formato universal de datos hidrográficos han fracasado o han sido sólo un éxito parcial, se propone un enfo-que diferente. Nuestra solución es el Lenguaje Universal de la Descripción de los Datos Hidrográficos (HUDDL), un lenguaje descriptivo basado en el XML, que permite la creación de una descripción normalizada de formatos de datos (pasados, presentes y futuros), y que permite aplicaciones como HUDDLER, un compilador que crea automáticamente controladores para el acceso a y la manipu-lación de datos. HUDDL también representa una solución muy potente para el archivo de datos, junto con su descripción estructural, así como para la cataloga-ción de las especificaciones de formato existentes y el control de sus versiones. HUDDL pretende ser una iniciativa abierta, dirigida por la comunidad para simplifi-car las cuestiones relacionadas con el acceso a los datos hidrográficos.

18


1. Introduction

Data acquired during a hydrographic survey may be stored in a number of different formats. Essentially, every manufacturer has developed their own format specification. Ongoing development during the lifetime of existing or new acquisition systems usually requires a sequence of file format releases. Trying to keep abreast of all of the different formats, their change-points, and idiosyncra-sies, can be a time-consuming and problem-atic endeavor for anyone who has to read multiple different data formats, or deals with archival data.

One potential solution to this problem is to convert each data type into a ‘universal’ format for archive or processing. Many of the attempts to introduce such a format for hydro-graphic data have however failed or have been only partially successful. This is because such formats either have to simplify the data to such an extent that they only support the lowest common subset of all the formats covered, or they attempt to be a superset of all formats and quickly become cumbersome. Neither choice works well in practice.

This issue is exacerbated by a lack of a com-mon repository for hydrographic data formats. Each manufacturer documents their own format in a different way, and often in different locations, with different release schedules, and, often, only partially consistent release announcements. To find details of a particular data format requires a user to navigate many different websites - often driven by having at-tempted and failed, to read survey lines in a new variant of the format. This also means that data conversion parsers or tools for differ-ent data formats are only available for a lim-ited number of format pairs (some for free, the largest part with a cost).

One of the biggest (and negative) conse-quences of the current situation is that each data handling application has its own data parsers (coded mostly from scratch) for every supported data format. These parsers must be kept up to date. This is a significant resource soak that could be reduced, and entails the danger of allowing variant data content inter-pretations in different software packages. Sur-

vey data access becomes more complicated if the source files are stored in legacy data for-mats, where negotiation of multiple versions of even one format may be required if historical trend analysis is the primary goal. A useful solution to survey data access should offer access to mixed-format historical data, provid-ing a mechanism to describe data collected and archived in sometimes ‘exotic’ data for-mats (e.g., developed by defunct manufactur-ers). Solving this issue implies the definition of a reliable way to access the data collected today by our descendants, with obvious ad-vantages in the adoption of these methods by hydrographic data archiving centers.

Our long-term solution to this issue is a descriptive language flexible enough to describe past, existing, and likely future hydro-graphic data formats: the Hydrographic Universal Data Description Language (HUDDL). This can also be readily extended to convert new data format concepts that might appear in the future. The key point of the HUDDL approach, is to describe the existing formats as they are, rather than define another chimeric format able to encapsulate the information present in all the existing data formats (with all the related semantic issues in case a conversion is attempted) (Masetti and Calder, 2014).

A HUDDL File Description (HFD) is a machine-readable description of the content of a data format that can be used in multiple ways. For example, it is possible to use an HFD for auto-matic generation of data drivers, validation of the content of survey lines claiming to be con-sistent with a particular format release, recov-ery of partial information from corrupted data, storage and reference of the description of how data are organized in a given data for-mat, or for incremental update of data format specifications. Since HFDs are implemented as XML files, they can also be uniformly and consistently converted to produce documenta-tion in different formats (e.g. HTML). A simple metadata link to an online repository of HFDS represents a robust way to uniquely identify the data organization.

A uniform collection of data format descrip-tions represents a powerful resource for data

19


format conversion, a step that has been described as “the soft underbelly of process-ing scientific data,” consuming immense amounts of time for those who work in a heterogeneous software environment (Georgieva et al., 2009). At the same time, the collection of information in one place, and the simplicity of the descriptive mechanism, provides a tool for inspiration, definition, and testing of new data formats and updated releases before being made public to the hydrographic community.

Given the many different fields of interest, we believe that the final overall result of HUDDL will be to drastically reduce the resource bur-den focused on accessing the information stored in hydrographic data formats.

2. The Description Language

Language requirements and features develop-ment

A number of different solutions have been de-veloped to describe data files over the last 30-40 years. None of them provides the full set of requirements of a hydrographic format de-scription language (Masetti and Calder, 2014). In essence, the language must be:

- Readily adoptable (e.g., using XML-based syntax, familiar to a large number of poten-tial users);

- Well-maintained (some languages, e.g. ESML (Ramachandran et al., 2004), do not have any time schedule for standard development);

- Widely accepted (there is a common lack of this requirement in any of the existing solutions, which may represent a weak-ness of available methods);

- Flexible, with a low-cost implementation (e.g., JSON requires data conversion in different structures (Nurseitov et al., 2009), while manufacturers likely want to maintain their own data formats);

- Based on a simple syntax, while still re-taining enough expressivity to describe hydrographic binary data formats (the in-tent, for example, of DFDL (Powell et al., 2011; Westhead and Bull, 2003) to be uni-

versal increases the overall complexity); and

- Available with an open-source and open-community implementation (the hydro-graphic community is narrower than the communities targeted by each existing solution, which may speed up the adoption and the contribution to develop a working approach). For instance, Protocol Buffers, while open source, is not open in the development process (Kaur and Fuad, 2010; Varda, 2008).

None of the existing proposed data descrip-tion methods explicitly focus on hydrographic use cases, which are dominated by data streams of sensor data, and arrays and lists of floating point numbers. The more structured nature of these data streams allows for some simplification in the implementation of a data description. Useful features from each of the existing solutions have, however, been adopted into HUDDL.

The HUDDL development was focused on a language that can:

- Provide a common set of many basic validation and computation functionalities;

- Explain the structure of binary files to users (readability);

- Automatically generate a parser directly from a schema;

- Provide a convenient basis for building ar-bitrary transformations between binary data formats (data conversion/transformation) and data file indexing; and

- Extend applications with content-aware functionalities (e.g., tools that can inspect any binary file given a schema, file com-parison, etc.).

HUDDL describes the physical representation, the overall structure and the semantics of various existing data formats used in the hydrographic field. The language relies on a set of core schemas that make available various description tools such as array data structures and primitive data types. New elements may be added each time a new un-known structure is encountered. This solution

20


was preferred to an attempt to a priori define all the possible required elements (e.g., rare middle-endianesses or some uncommon IEEE formats for representing floating point num-bers) and their exponentially growing combi-nation. Doing otherwise might make HUDDL-aware technology too complex and difficult to adopt.

HUDDL is focused on describing most types of hydrographic data formats in a simple syntax, rather than attempting to be a generic and ar-bitrary spatial acquisition format. The main reason for this is the desire to maintain imple-mentation libraries that are lightweight and as simple as possible, providing order rather than adding complexity to the existing scenario of hydrographic data formats.

This solution also provides an inexpensive but robust way to deal with many legacy data for-mats. When required to access old datasets in an arbitrary binary format, an ad hoc HFD may be created that can deal with the particu-lar vagueness of some legacy formats or some rare variant implementations.

Conceptual and Physical Data Modelling

HUDDL was developed as a community-specific, format-oriented data description lan-guage. These characteristics provide a certain level of simplification since existing data for-mats are different answers to the same prob-lem: fast storage of data acquired in real-time. All of the data format specifications targeted

by HUDDL have three components which formed the requirements for the abstract con-ceptual model and physical implementation reported here:

- The semantic: what a given value col-lected in the data format actually means (e.g., the unit of measure);

- The physical description: how the bits and the bytes are stored on disk (e.g., endianess, memory alignment); and

- The logical structure: what data struc-tures are used to organize the data (e.g., an array)

The analysis of existing data formats suggested a natively tree-structured model: a top-level container, called a ‘Schema’ that may hold several different descriptions of data formats, each of which has both a ‘Prolog’ and a ‘Content’ element (Figure 1). The ‘Prolog’ represents a collection of meta-data related to the described data format, such as the organization that created it, the personnel responsible for its maintenance, or the history of releases (Figure 2). This information is required to create a homoge-neous and consistent documentation for different data formats, although the extent to which it is implemented can vary between formats – the better the information, the more complete, and useful, the documenta-tion.

Figure 1 : Top-level elements of the HUDDL format model. A Schema provides the ability to host more than one format, each of which contains a prolog to provide metadata on the format, and then a content description providing the details of data’s format.

Figure 1

21


The ‘Content’ branch is used to describe both the structure and the format of a binary data file in a platform-independent way. This branch has three main containers (Figure 3):

- Blocks: which may contain any number of fields and available data structures (e.g., 2D array) (Figure 4). A ‘Block’ represents a logically related group of information elements committed to file at the same

Figure 2 : Example of elements present in the 'Prolog' branch of the HUDDL format model. Any amount of metadata on the data format can be provided. This infor-mation is not strictly required for some uses of the HFD (e.g., to generate source code to access the data), but has significant benefits when documentation is being generated.

Figure 3 : Four sections of the 'Content' branch of the HUDDL format model. Blocks represent a group of data elements written to file as a group (e.g., a single ping’s worth of bathymetric data), while Streams represent a collection of Blocks that can be read together to provide in composite the description of a single version of a data format. Maps provide the means to link semantic representation to the data, for example, by providing physical units or the means to translate encoded values into physical units.

Figure 3

Figure 2

22


time instant. A ‘Field’ is used as a basic value container (e.g., bytes, two- or four-byte integers, floating point numbers, etc.). In order to allow for reading of com-plex data structures, blocks can also con-tain other blocks, optionally as one- and two-dimensional arrays with either fixed or variable sizes defined in the preceding data block.

- Streams: which lists all the releases of a data format. Each release is represented by a ‘Stream’ containing the overall com-position of a data file (Figure 5): a ‘Header’ which describes initial shared data fields present in all top-level blocks, a ‘TopBlocks’ list with all the blocks that can be encountered at the top level of the format, and an optional ‘Tail’ description, representing a common data element found at the end of all top-level blocks (e.g., a checksum). Thus, it is possible to have, in the same document, multiple streams that reflect different releases of the same data format, and hence updates are only required to be incremental (this characteristic makes them smaller, sim-pler, and faster than would be required for a single-release format description approach).

- Maps: for explicitly describing relations among fields and other available data structures for a given ‘Stream’ such as, adding sensor-specific semantic context.

Among the wide range of possible solutions for the physical implementation, the Extensi-ble Markup Language (XML), with the support of strictly linked XML Schemas, was selected. XML provides a representation standard that is convenient because it is both human- and machine-readable, easily and quickly extensi-ble, has wide adoption, and is a mature technology.

In HUDDL, XML is used to give a structural description of the contents of a file format (rather than the content of a particular file). Coupling one or more of the proposed descriptive XML schemas with a given hydro-graphic dataset, as metadata, provides a detailed definition on how data have been actually stored. For the structural representa-

tions HUDDL follows the main data structures (e.g., data streams) present in the most used hydrographic data formats (eXtended Triton Format, Generic Sensor Format, Kongsberg EM series, etc.) as well as the work done for XDR and BinX (variable and fixed length ar-rays, simple structures, strings, unions, etc.) (Eisler, 2006; Kongsberg Maritime AS, 2013; SAIC, 2012; Triton, 2013; Westhead and Bull, 2003).

XML already has a key role in the representa-tion of metadata associated with a hydro-graphic dataset since it represents the accepted means to describe information re-lated to the data collector, acquisition parame-ters, meteorological conditions, etc. At the same time, hydrographic applications that once were tightly-coupled and monolithic are now becoming more modular, with collaborat-ing components spread across diverse computational elements (Calder, 2013). In such a distributed environment, open meta-data systems are increasingly important and useful to communicate substantial amounts of structured data (Widener et al., 2001). The increasing popularity of XML in the field of ma-rine science and engineering is also driven by its role in the ISO 19000 series metadata standard (Georgieva et al., 2009; Hua and Weiss, 2011; ISO, 2008; Yongguo et al., 2009).

While relatively simple, the implementation of the HUDDL Format Model is quite expressive. For example, since blocks can contain other blocks and arrays of blocks, a data unit which contains a header segment (e.g., the parame-ters for a given ping’s depth detections), along with a record of the depths detected per beam, can be easily represented by, block for each detection, and a block that contains the header information as elemental fields, with an embedded 1D array of the detection blocks. The HUDDL Core Schemas also allow for variable length arrays (e.g., if the number of beams reported is variable per ping), for two dimensional arrays of fields or blocks, and other common features of typical hydrographic data formats. It is therefore typically a fairly simple matter to translate a given data format into a HUDDL description given the appropriate documentation.

23


Figure 4 : 'Blocks' internal structure. Each block may contain any number of basic data objects (e.g., integers of different signedness and sizes, floating point values, etc.) as Fields, other Blocks to provide for composite and complex data types, and 1D and 2D fixed and variable arrays of Fields or Blocks.

Figure 5 : 'Streams' inter-nal structure. Each Stream consists of a special Block that appears at the start of each data object in the file (typically, this contains a length and identification integer to indicate what data is being stored), along with a list of all of the Blocks that can occur at the outer-most level of the data file (TopBlocks), and an optional Tail block for a common data struc-ture that appears at the end of each Block (e.g., a checksum).

24


The HUDDL Format Descriptions are based on the HUDDL Core Schemas (which are also XML schemas) so that they describe the physical and logical implementation of a data format. They can also be used to create docu-mentation to explain the semantic meaning to a human through the use of a suitable XSLT (Extensible Stylesheet Language Transforma-tions) translator, which are widely available. The latest generation of web browsers is able to use XSLT stylesheets directly, so that XML documents can be viewed easily by a human (e.g., HTML pages, PDF files), as well as be-ing understandable by machines. Extant tools for XML creation can also be used to structure HFDs. These tools are intelligent, disallowing invalid data entry, and suggesting that which is valid. Programs can also read HFDs through any of a variety of parsers. Some XML parsers are already built into program-ming languages (e.g., Java, Python), and there are a variety of external parsers (e.g., Xerces, libxml). HUDDL schemas can also be easily translated to other formats using XSLT.

There are also advantages in using a data-description language such as HUDDL versus using diagrams (e.g., UML). HUDDL is more formal than diagrams (leading to less ambigu-ous descriptions of data formats) and easier to understand (allowing software developers to focus on other issues instead of the low-level details of bit encoding). Also, there is a close analogy between the types used by HUDDL and a high-level language such as C/C++ or Python. Finally, the language specifications themselves are XML files that can be passed from machine to machine to perform on-the-fly data interpretation.

A web repository for HFDs was created at the Center for Coastal and Ocean Mapping (CCOM) to provide an initial safe and easy-to-check common point for data format specifica-tions. Widely used systems (e.g., RSS, or an open-subscription mailing list) could assist in staying current with the last release of data formats for all of the interested players. The repository is part of a community-oriented website to access, catalogue, and dissemi-nate hydrographic data formats resources and HUDDL-specific information that has been developed and is now publicly available

3. The Format Driver Compiler

HUDDLER is an implementation of one of the many advantages of having available machine-readable HUDDL Format Descriptions: a com-piler that automatically creates drivers for data access and manipulation (Calder and Masetti, 2015).

HUDDLER implements the HUDDL-philosophy of constraining the description of the data format to the schema, so that the user has to touch the minimal amount of code to reflect any change in the data format specification (Masetti and Calder, 2014). Instead of having to change the user’s application code directly to reflect the format changes, changes to the schema are translated automatically by HUDDLER into the library that represents the data format, and this can be readily auto-mated in most software build systems. In practice, updating the software to support a new data format version is as simple as changing the schema and then recompiling the library or application, as appropriate, leaving the programmer to work on the appli-cation logic to use the new facilities added by the new version of the format.

The compiler is based on an XML parsing library that loads into memory the format description (frontend), and a code generator (backend) that creates code able to access the data in three different types of computer languages: procedural ANSI C, object-oriented C++, and multi-paradigm Python. The system is designed to admit other languages readily (e.g., Matlab).

The creation of a new format driver is structured in four steps (Figure 6):

- HFD validation: which automatically checks that the description follows the HUDDL Core Schemas;

- HFD parsing: which loads the format de-scription into memory;

- Format processing: which performs addi-tional checks on the format description and solves internal block and field cross-references; and

- Code generation in one of more of the available code generators.

25


Generating code directly in the target language allows the code generator to take advantage of particular language features that would simplify the generated code, or better express the idiomatic nature of the target language usage. However, particularly for some interpreted languages, performance issues dictate that it is preferable to automati-cally generate a language-specific wrapper around a C/C++ library. This was the approach followed for the Python backend. It would be possible to build a pure Python backend driver if required, for pedagogical purposes, but the performance would gener-ally be sufficiently constrained as to make its practical application limited. As a common factor, the output code from the language-specific generators attempts to provide data types that are as transparent as possible in order to reduce the complexity of manipulating routines in the master application.

To better illustrate the simplicity and the potential of this approach, an example that

accesses and plots attitude data from a real file is shown in Figure 7. The left pane shows the part of a HUDDL Format Description used to describe the specific blocks containing atti-tude data, and the stream that provides access to them as top-blocks. The right pane displays the code snippets specifically created by HUDDLER to read the format version, which internally calls a helper function to retrieve the top-block containing the attitude measure-ments. Once the generated code is compiled, a simple script (Figure 8, left pane) can be used to import the HUDDLER-generated library and use the generated methods to open the data file, access the attitude data, and ma-nipulate the data (e.g., to plot roll, pitch, heave and heading as shown in the right pane of Figure ). The Python script is a simple dem-onstration of the many advantages of HUDDL, since it provides a means to easily access hy-drographic data taking advantage both of the flexibility and ease-of-use of Python and the speed of C code for data reading. The full working code for this example, and the con-

Figure 6 : HUDDLER steps to create a new format driver: HFD validation, checking that the description follows the HUDDL Core Schemas; HFD parsing, loading the format description into memory; Format processing, performing additional checks on the format description and solving internal Block and Field references; Code generation in one of more of the available code generators.

26


Figure 7 : On the left pane, the part of a HUDDL Format Description that describes the specific blocks containing attitude data and the stream that provides access to them as top-blocks. On the right pane, the code snippets created by HUDDLER to read the specific format release and the top-block containing the attitude measurements.

27


Figure 8 : On the left pane, a simple script that imports the HUDDLER-generated library that provides all the methods to open and access the attitude data. On the right pane, the output generated by the Python script that can be used to quickly inspect the attitude data before manipulation and/or use in processing algorithms.

28


version software to compile it, is available on t h e p r o j e c t w e b s i t e ( h t t p s : / /huddl.ccom.unh.edu).

The compiler is accessible via the command line or through a GUI application (using Qt for cross-platform support), named HUSH (HUDDL Schema Handler), which provides additional tools and information to the user. The com-piler has been demonstrated with a variety of data formats from sonar manufacturers (e.g., Kongsberg EM Series) and acquisition soft-ware companies (e.g., HyPack) both legacy and in active development, both binary and ASCII (Calder and Masetti, 2015).

4. Discussion

The HUDDL framework provides a simple and relatively low-effort solution to harmonize and catalogue the wide (and sometimes wild) range of hydrographic data formats, with their multiple revisions and releases. It also provides the opportunity to generate a cata-logue of HUDDL Format Descriptions written in XML, each containing the description for a given data format (with its subsequent upgrades) that can be used as a set of in-structions for an application on how to manipulate a data file in a specific format/version. The automation inherent in HUDDLER provides a low cost means of adding new data formats to an application, at least at the basic syntactic level of data access, leaving the coder to focus on the higher-level semantics of what to do with the data after the syntax problem is resolved.

The code in the HUDDLER project is only one means to translate an HFD into source code for use in a data reader: the HUDDL Core Schemas are available directly from the HUDDL community website, and can be used by anyone to develop additional services for HFDs. The code currently generated by HUDDLER is already relatively efficient for data handling, having derived in part from a cruder code generator that has been in use for over a decade. Many optimizations can still be made to improve the performance, and there is significant benefit to doing this in a community supporting a common code generator infra-structure. For example, if code to generate an index for files on first-read were to be added

to HUDDLER, or if the frontend reader were multi-threaded, it would then be automatically available to every data format for the cost of a re-compile of the application software.

From the point of view of software manufac-turers, HUDDL provides a new tool to build applications that are more data format inde-pendent. A single reader component could be developed in isolation and then these modules combined for the various data formats. If a sonar system manufacturer, or software developer, provided an HFD for their data format (which is the ideal case for a strong community), hosted either on their website or that of the project itself (Figure 9), it would significantly ease the effort involved in imple-menting the format in a data processing appli-cation. This would allow all readers to have the same understanding of the intended syntax and semantics of the data format. This will reduce some of the efforts required to maintain a set of data readers, usually one for each different format, during subsequent updates to the format, and will help to avoid problems with variant reading of data formats between different applications.

Wherever they are hosted, having the HFD web accessible has significant benefits. For example, when a new version is released, one of the commonly available mechanisms (e.g., RSS) may be used to notify interested users. This push notification allows for alerting of software maintainers as soon as a change is made, so that users do not have to search for changes when there is a sudden problem in reading a data format, or report this as a bug to software vendors. An HFD valid with respect to the HUDDL Core Schemas also allows for automated creation of standardized documentation through the use of XML style sheet technology. The HFD provides a single source for creation and documentation of code, always up to date and consistent.

These publicly available HFDs may be used as ‘trusted’ references for archived data. As long as a binary data file is paired with a HFD, the data content is described and the informa-tion can be recovered. The main benefit of this is that it is more likely that users will be able to read the data in the future, and have adequate

29


documentation, essentially for the price of a metadata link. As long as a version of HUDDLER is available, it can use these HFDs to (re-)create data format drivers to access the archive data with the same simplicity and consistency as for new data format versions. Additionally, once a valid HFD is constructed, the data in that format can be accessed on any platform regardless of the native configu-ration of the file system.

Having a separate description of the format has the potential for the description (e.g., an HFD) to become separated from its data. Future hydrographic data formats may choose to instead include the HFD as part of the binary file itself to avoid this risk. Another approach to this problem could be to use the first bytes of the file as an integer representing the unique ID reported in a future XML Hydro-graphic Formats Catalogue, or to store a URL referencing the HFD’s location in a well-known place.

At present, the development of an HFD necessarily implies the creation of an XML descriptor for the format. Although there are

many XML editors that support this, they are general tools rather than specific to HFDs, and development of the HFD for a complex format can still require significant effort. Of course, that effort is only required once, since the resulting HFD can then be shared by all members of the community so long as it is published at an appropriate URL and indexed, preferably at a clearing house such as the HUDDL community website. The structure of the HUDDL format is much more strict than a general XML file, and could be much more efficiently constructed, and checked, by a tool that reads the HUDDL Core Schemas. This provides the user with a customized interface that allows construction of XML for the HFD only within these bounds. Done graphically, this would significantly ease the burden of constructing the HFD in the first place, and their subsequent update. It is also possible to envision a graphical editing application where the HFD is rendered in diagrammatic form, and the user is able to drag-and-drop new fields and blocks, describing the structure of the data graphically before it is converted into an HFD for distribution.

Figure 9 : HUDDL framework: the online repository is used both for publicly providing format specifications (in different formats) and as a source for HUDDLER, which parses the descriptive schemas, serializes the information and creates an I/O library. Data processing applications can thus rely on this library for access the binary data.

30


5. Conclusions

Currently, the hydrographic community has to deal with a multiplicity of data formats. Each format is home grown within its specialty and in many cases is based on manufacturer tech-nology. Although generic data formats have been introduced, there has not been a suffi-ciently strong reason to rally around one par-ticular method for containing data, nor is any one format general enough to accommodate everyone’s needs. The result has been an in-vasive sea of data formats. At the same time, the temptation to convert all collected data to a selected data format does not appear to be the optimal solution, since many data formats are fundamentally incompatible with each other, and much metadata and information can be lost during this processing (or even worse, mistranslated).

The intent of HUDDL is not to describe every kind of binary data format that people have ever sent or will ever want to send from ma-chine to machine. Rather, HUDDL focuses on the most commonly used hydrographic data formats in order to simplify the problem so that the solution can be efficient and sufficiently easy to use to make it an obvious choice for most users. It can support the hydrographic community on at least at three different levels (Figure 10):

- At the descriptive level, where the user simply takes advantage of the common format repository as well as the stan-dardized templates for documentation;

- At the raw data level, where users can use the automatically created raw data parsers. That is, each parser is tailored for a given data format (with all the implicit data peculiarities) as described in the HFDs (as they are compiled by HUDDLER); and

- At the abstract data level, where an addi-tional layer of homogenization is provided with the main aim of simplifying access to hydrographic data (e.g., the same function getDepthData() for obtain-ing the collected depth from various data formats). The Hydrographic Abstraction Layer (HABLA) may also be useful for researchers coming from fields not directly related to ocean mapping. HABLA features are currently in active develop-ment.

Many types of applications could benefit from this task-oriented approach: data explorers, conversion tools, metadata archives, etc. HUDDL represents a solution for both software and hardware manufacturers to providing a strong and universal mechanism for version control of hydrographic data formats.

Figure 10 : The three expected levels of users for the HUDDL framework. The basic Descriptive level provides only description services for differ-ent file formats, including docu-mentation construction. The Raw Data level provides basic access to particular data for-mats through automatically generated data drivers. At the Abstract Data level, extra translations provide for conver-sion of the data into hydro-graphically understandable information, such as depth or backscatter, without informa-tion as to the underlying data format. The HABLA layer is currently under development.

31


Developers will have an abstraction tool for development of binary data readers and converters. In addition, the HUDDL Format Description repository is a powerful solution for propagating the publication of a format update to all interested parties (using popular electronic mechanisms such as tweets, an RSS, or a mailing list).

Based on these considerations, we believe that HUDDL represents a concrete way to reduce, in a relatively short time, existing problems related to interoperability and access to hydrographic data.

6. References

1. Calder, B., 2013, Parallel & Distributed Per-formance of a Depth Estimation Algorithm, U.S. Hydrographic Conference, (US Hydro 13), New Orleans, LA, USA, pp. 11.

2. Calder, B., and Masetti, G., 2015, HUDDLER: a multi-language compiler for automatically generated format-specific data drivers, US Hydrographic Conference (US Hydro 2015), National Harbor, Maryland, USA.

3. Eisler, M., 2006, XDR: External data repre-sentation standard.

4. Georgieva, J., Gancheva, V., and Go-ranova, M., 2009, Scientific data formats, in Mastorakis, N.E., Demiralp, M., Mladenov, V., and Bojkovic, Z., eds., 9th WSEAS Interna-tional Conference on Applied Informatics and Communications (AIC '09), Volume 9: Mos-cow, Russia, WSEAS Press, p. 19-24.

5. Hua, H., and Weiss, B., 2011, Strategies for Infusing ISO 19115 Metadata in Earth Science Data Systems, AGU Fall Meeting Abstracts, Volume 1, pp. 04.

6. ISO, 2008, ISO/TS 19139-2007 Geographic information -- Metadata -- XML schema imple-mentation, International Organization for Stan-dardization, pp. 111.

7. Kaur, G., and Fuad, M.M., 2010, An evalua-tion of Protocol Buffer, IEEE SoutheastCon

2010 (SoutheastCon), Proceedings of the, p. 459-462.

8. Kongsberg Maritime AS, 2013, EM Series Datagram formats - Instruction Manual, p. 126.

9. Masetti, G., and Calder, B., 2014, HUDDL for description and archive of hydrographic binary data, Canadian Hydrographic Conference: St. John's, NL (Canada), pp. 24.

10. Nurseitov, N., Paulson, M., Reynolds, R., and Izurieta, C., 2009, Comparison of JSON and XML Data Interchange Formats: A Case Study: Caine, v. 2009, p. 157-162.

11. Powell, A.W., Beckerle, M.J., and Hanson, S.M., 2011, Data Format Description Lan-guage (DFDL) v1. 0 Specification, Report of the Open Grid Forum. Retrieved from www. ogf. org/documents/GFD.

12. Ramachandran, R., Graves, S., Conover, H., and Moe, K., 2004, Earth Science Markup Language (ESML): a solution for scientific data-application interoperability problem: Computers & Geosciences, v. 30, p. 117-124.

13. SAIC, 2012, Generic Sensor Format Specification v.03.04, SAIC, pp. 151.

14. Triton, 2013, eXtended Triton Format (XTF) Rev. 35 Triton Imaging, Inc. , pp. 44.

15. Varda, K., 2008, Protocol Buffers: Google's Data Interchange Format.

16. Westhead, M., and Bull, M., 2003, Repre-senting Scientific Data on the Grid with BinX–Binary XML Description Language: EPCC, University of Edinburgh.

17. Widener, P., Eisenhauer, G., and Schwan, K., 2001, Open metadata formats: efficient XML-based communication for high perform-ance computing, High Performance Distrib-uted Computing, 2001. Proceedings. 10th IEEE International Symposium on, p. 371-380.

32


18. Yongguo, J., Lianying, L., and Zhongwen, G., 2009, Design of Marine Information Meta-data and Directory Service System Based on XML, Database Technology and Applications, 2009 First International Workshop on, p. 574-577.

Author Biography

Giuseppe Masetti received a MS degree in Ocean Engineering (UNH, USA) in 2012, and a Master in Marine Geomatics (2008) and a Ph.D. degree (2013) in System Monitoring and Environmental Risk Management (University of Genoa, Italy). His postdoctoral research at CCOM/JHC is focusing on signal processing and Bayesian hierarchical models f o r m a r i n e t a r g e t d e t e c t i o n . ([email protected]) Brian Calder is an Associate Research Pro-fessor and Associate Director at CCOM (UNH, USA). He has a Ph.D. in Electrical and Elec-tronic Engineering, completing his thesis on Bayesian methods in SSS processing (1997). He is currently focusing on statistically robust automated data processing approaches and tracing uncertainty in hydrographic data. ([email protected])

33


AN ENTERPRISE APPROACH TO THE NEXT GENERATION SYSTEMS ENVIRONMENT

FOR THE AUSTRALIAN HYDROGRAPHIC SERVICE (AHS) I. HALLS (Australia)

Abstract

Résumé

Resumen

The Australian Hydrographic Service (AHS) of the Royal Australian Navy (RAN) has been at the forefront of digital hydrographic information management and pro-duction for many years. In the early 2000s, the AHS had a Digital Hydrographic Data Base (DHDB) developed and implemented to enable the efficient and effec-tive management of hydrographic data, the generation of multiple hydrographic products and distribution capabilities for these products. The DHDB suffered from a number of technical shortcomings and whilst implemented, was not maintained as a contemporary capability for several years. In 2011, with the DHDB finally accepted, evolutionary sustainment commenced to re-invigorate the DHDB capa-bility to be a contemporary solution. This article provides an overview of this evolu-tionary sustainment initiative.

Le Service hydrographique australien (AHS) de la Marine royale australienne (RAN) est depuis de nombreuses années à l’avant-garde de la gestion et de la production des informations hydrographiques numériques. Au début des années 2000, le AHS a développé et implémenté une base de données hydrographiques numériques (DHDB) afin d’assurer la gestion effective et efficiente des données hydrographiques, la production de multiples produits hydrographiques et des capacités de diffusion de ces produits. La DHDB a pâti d’un certain nombre de lacunes techniques et bien qu’implémentée celle-ci n’a pas été maintenue en tant que capacité moderne pendant plusieurs années. En 2011, lorsque la DHDB a finalement été acceptée, la maintenance évolutive a commencé pour mettre à niveau la DHDB. Le présent article donne une vue d’ensemble de cette initiative de maintenance évolutive.

El Servicio Hidrográfico Australiano (SHA) de la Marina Real Australiana (RAN) ha estado a la vanguardia de la gestión y la producción de la información hidrográfica digital durante muchos años. Al principio de los años 2000, el SHA tenía una Base de Datos Hidrográficos Digitales (DHDB) desarrollada e implementada para permitir la gestión eficiente y eficaz de los datos hidrográficos, la generación de múltiples productos hidrográficos y de capacidades de distribución para estos productos. La DHDB sufrió una serie de deficiencias técnicas y, aunque imple-mentada, no se mantuvo como capacidad contemporánea durante varios años. En el 2011, con la DHDB finalmente aceptada, la autonomía evolutiva comenzó a revitalizar la capacidad de la DHDB de ser una solución contemporánea. Este artículo proporciona una visión general de esta iniciativa de autonomía evolutiva.

34


1. Background The DHDB was acquired under the auspices of Project SEA1430 Phase 1 in 1999. The DHDB was formally accepted by the Com-monwealth of Australia in October 2004. How-ever, Interim Operational Release (IOR) was deferred pending completion of contractual requirements. Acceptance into operation was finally achieved in 2011 and DHDB evolution-ary sustainment activities commenced. A new maintenance contract was signed in 2013 with BAE Systems Australia Ltd. The majority of the DHDB technical solution dated back to 2004 technology and by 2011 was obsolete, bespoke and incapable of sup-porting the exponential increase in digital ar-chive and information management required by the AHS to support contemporary require-ments. This was further compounded by up-grades in both RAN and national port data collection capabilities along with the require-ment to update a much larger number of pa-per and electronic chart products. DHDB capability shortcomings led to the de-velopment of several internal work-around so-lutions. This resulted in an operating environ-ment comprising multiple non-integrated net-works and systems, mixed software technolo-gies, bespoke solutions, inconsistent workflow operations and minimal training opportunities for existing and new staff - all leading to ineffi-ciencies in all activities. 2. AHS Responsibilities A re-engineered, effective and efficient DHDB is a fundamental technology enabler for the AHS to deliver assured information, products and services that meet Australia’s obligations under UNCLOS and SOLAS Chapter V Regulation 9, the Navigation Act 2012 and for the supply of maritime military geospatial in-formation to Defence. The RAN is the national authority responsible for the work required to meet Australia’s international commitments for the provision of hydrographic products and services within the Australian Area of Charting responsibility.

The Navigation Act 2012 confirmed the re-quirement for the AHS, as part of the RAN, to collect, compile and collate, maintain and dis-seminate hydrographic and other nautical in-formation as required under international con-ventions. The Defence White Paper 2013 reiterated the future hydrographic capability must provide maritime military geospatial information out-puts to support the following activities: Understanding and shaping Australia’s strategic environment; Conducting combined joint combat opera-tions; Conducting peace and stability operations; Providing Defence support to whole-of-government domestic security; Providing humanitarian assistance and disaster relief at home and overseas; Providing specialist support (domestic and overseas); Evacuating non-combatants; and Undertaking recovery operations. 3. Hydrographic Information and Pro-duction Initiative (HIPI)

When the DHDB came under a sustain-

ment agreement between the RAN and the Defence Materiel Organisation (DMO), a num-ber of tasks were undertaken to immediately refresh lagging core capabilities. These in-cluded upgrading the hardware environment, operating system, replacing the DHDB work-flow software and updating the production tools to support S57 Ed 3.1 sup 2. An investi-gation also commenced into the potential for CARIS HPD to replace the existing DHDB CARIS GIS and other third-party software tools used to produce ENCs and paper charts. These investigations included site visits to Land Information New Zealand (LINZ), Cana-dian Hydrographic Service (CHS) and the United Kingdom Hydrographic Office (UKHO) to understand how they were implementing contemporary solutions.

35


In 2013, the Hydrographer of Australia (CDRE Brace, RAN) appointed a technical team to lead and coordinate evolutionary sustainment activities to improve existing systems and processes. This work item was titled the Hydrographic Information and Production Initiative (HIPI). The core technical team is very small with several team members also managing daily man-agement and production operations. Other AHS staff and technical contractors are brought into the team to participate in sus-tainment tasks for various system compo-nents. This allows AHS staff, who will even-tually be using the sustained systems, to have a high level of ownership and author-ship in the way that the sustained systems will operate. This significantly improves the transition from concept to requirements, to testing and then implementation. The adopted strategy for implementing the change process is based on the Eight-Stage Change Process described by Kotter (2012). The timeframe for completing the evolution-ary sustainment of existing DHDB systems is late 2017. HIPI is assisting the DMO to achieve the CN29 sustainment program by implementing AHS Leadership Group stra-tegic objectives and business-driven activi-ties. The evolutionary sustainment activity encapsulates coordinated views of the or-ganisation using an Enterprise Architecture (EA) approach. These views encompass strategy, business and technology. As described in OMB (2012), Enterprise Architecture means a strategic information asset base which defines:

the mission;

information necessary to perform the mission;

technologies necessary to perform the mission;

transitional processes for implementing new technologies in response to changing mission needs; and

a baseline “as is” architecture, a target “to be” architecture and a sequencing plan.

The sustainment of the DHDB will be used as a benchmark EA Program (EAP) and will align and unify overall organisation strategic initia-tives, business processes, information flows, systems and services, technology infrastruc-ture and organisation structure. The EAP will strive to:

Involve stakeholders at all appropriate levels throughout the AHS enter-prise. These steps are essential to achieve maximum buy-in and ongoing support of the program;

Invest in technologies that will fully sup-port its Mission, Vision, and Strategic Ob-jectives;

To improve information sharing, AHS will invest in Commercial Off-The-Shelf (COTS) and non-proprietary products that emphasise open standards and heteroge-neity;

Allow for optimal planning whether it be a top-down or bottom-up approach by bringing together different perspectives of business and technology throughout the enterprise. Decision-making capabilities are enhanced by providing comprehen-sive views of current capabilities and re-sources while keeping in mind a number of future scenarios that may require changes in processes and resources;

Enable improved communication through-out the AHS. Open information sharing and the wealth of knowledge provided by the EA repository allows on-demand ac-cess to current and pertinent information to best support the AHS.

4. Scope of Sustainment Activities of

the EAP The key sustainment processes are:

Identify the key Lines of Business (LOB) within the AHS in terms of the current DHDB capability;

36


Model the “as is” state of the initial SEA1430 Phase 1 DHDB components and the work-around capabilities that were introduced to overcome DHDB shortcomings;

Model the “to be” state to meet current and perceived future strategic require-ments;

Describe and implement the transition stages to move from the “as is” to the “to be” states.

Sustain and maintain the EAP using the AHS Quality Management System (QMS).

4.1 Enterprise Architecture (EA) Core

Elements The HIPI approach to EA requires six core elements to exist and work together as shown in Figure 1:

Governance: Conformance of the HIPI EA process to the overall AHS planning, decision-making and oversight processes and groups in accordance with the AHS QMS;

Methodology: Specific steps to estab-lish and maintain the EA program;

Framework: Identifies the scope of the overall architecture and the type and rela-tionship of the various sub-architecture levels and threads;

Artifacts: The documentation supporting the EA process;

Standards: The business and technology standards for the AHS in each domain, segment and component. This includes all relevant standards impacting the op-erations of the AHS;

Best Practices: These are proven ways to implement parts of the overall architec-ture, in the context of the over-arching EA;

4.2 Enterprise Architecture (EA) Frame-

work

HIPI has adopted a hierarchical EA Frame-work described by Bernard (2015) shown in Figure 2:

Goals and Initiatives - high level strategic goals and initiatives of the AHS;

Business Products and Services;

Data and Information flows;

Systems and Applications – the informa-tion systems and applications used to de-liver IT capabilities;

The required Networks and Infrastructure – the connectivity grid of the architecture;

Lines of Business (LOB) with distinct business activities and resources;

Artifacts - documentation;

Threads including Security, Standards, Skills (competency) and Organisation.

Figure 1.: Enterprise Architecture core elements (Bernard, 2015)

Figure 2.: Enterprise Architecture Framework (Bernard, 2015)

37


4.3 Performance Measures For each sustainment activity, performance measures are identified. Sustainment out-comes will provide measurements of the improvements to the sustained system com-ponent. Examples of performance measures include:

Overall reduced number of enterprise software and database applications – the number of software applications, plug-ins, and extensions at the commence-ment of sustainment was 360 and has already reduced to 110;

Improved component integration;

Improved workflow operations;

Improved quality assurance of products and services;

Reduction in data and product error rates;

Increased application end-user satisfac-tion;

Improved customer response;

Improved IT investment decision-making such as utilisation of shared Defence ICT capability (e.g. Primary Data Centres) and leveraging off other Defence projects;

Improved systems’ maintenance processes;

New capabilities that weren’t present or mature when the DHDB was first imple-mented (e.g. web services, e-commerce, etc.) and external interfaces (e.g. internet gateway access for data, product and service offerings);

In conjunction with DMO, meet the contractual budget requirements.

5. AHS Lines of Business (LOB) The primary Lines of Business (LOB) to be addressed within the evolutionary sustainment activities are shown at Table 1:

Line of Business (LOB)

Description

Source Information Receipt and Registration

Managing the receipt of all incoming source data and information; registering the source material into records management system(s) and assessing it for maritime safety

Themes of Validated Information Processes and tools used to manipulate registered source information to create validated information ready for use in product. Information themes in-clude bathymetry, survey data quality, navigation aids, maritime boundaries and regulated areas, large bottom objects (e.g. wrecks), nomenclature, imagery, meteorology, oceanography etc.

Product Compilation and Maintenance Processes and tools used to manipulate validated information to create and maintain approved prod-uct ready for publication including ENC, paper charts, raster charts, publications, web service caches, etc.

Survey and Chart Planning Planning and monitoring of hydrographic surveys using risk modelling; and management of the sur-vey and chart planning schemes.

Product Distribution Processes and tools to enable the AHS to distrib-ute approved product to customers. This activity includes customer management, inventory control, product master management, data licensing, prod-uct encryption, financial management, e-commerce, chart printing, media replication and warehouse operations.

Table 1.: AHS Lines of Business (LOB)

38


Within each primary LOB, there may exist separate LOBs having their own specific requirements but sharing common require-ments of the primary LOB. For example, the Product Distribution area is a LOB. However the various operations such as data licensing, chart printing, customer management, etc. may be undertaken using different software tools, processes, workflows, etc. within the context of the specific activity and its informa-tion. 5.1 Source Information Receipt and Regis-

tration LOB The AHS currently has several separate sections managing incoming information. Most of this data is managed within each section’s file management solutions and/or the official document records management system (DRMS). Key requirements are:

Enable legal traceability of data, infor-mation and product records;

Meets the necessary key (if not all) accreditation criteria to Commonwealth of Australia records policy;

Provides a single point of information receipt within the AHS - all geospatial information coming into the AHS goes to one place for receipt, acknowledgement, registration, assessment for navigational safety impact, and distribution. The AHS may have one or many incoming conduits, but only one registration point. Multiple users can view incoming corre-spondence but cannot access it until it has been registered in a compliant records management system;

No duplication of data and information in the records management system although records can be replicated as required;

Business process workflow tools to model and manage the flow of informa-tion through defined quality assurance activities, validation rules, decision points, etc. for receipt, replication,

extraction, registration (including meta-data content, geospatial location tag-ging), storage, archive, disposal, promo-tion, alerts, etc.

The system will provide geospatial data discovery tools – search, query, display, report and output in appropriate formats;

Metadata conforming to the ISO 19115 metadata standard.

5.2 Validated Information Theme LOBs Validated Information Theme LOBs are currently managed across various AHS sections using various technologies as shown in Table 2. Key requirements are:

Reduce the number of existing software systems and tools used across the organisation where possible;

Use the best tool for managing the ap-propriate validated information theme;

The system will use only “registered” in-formation and provide the appropriate tools for staff to process the information to create a “validated” set of data that can be used for further activities (i.e. chart production, publications, etc.)

5.3 Product Compilation and Maintenance LOB The Product Compilation and Maintenance LOB produces, manages and maintains offi-cial, authorised products that are fit for pur-pose in accordance to national and interna-tional product specifications and standards.

These products are shown in Table 3: Key requirements are: Reduce the different production method-

ologies to a single solution for each prod-uct type (e.g. ENC, paper chart). This will provide consistent compilation practices, system interfaces, data encoding, QC and process workflow;

39


Topography and Imagery

ERDAS IMAGINE ERDAS IMAGINE

Tides and Geo-detic Control

Tides Information System (TIS); ESRI ArcGIS, DRMS, local file system

Tides Information System (TIS), ESRI ArcGIS, DRMS

Survey and Chart Planning / Risk Assessment

ESRI ArcGIS (multiple bespoke data-bases), MS ACCESS, DRMS

To be determined (2015/2016)

Meteorology and Oceanography

ESRI ArcGIS and other tools To be determined (2015/2016)

Navigation Aids CARIS ENC Composer, ESRI ArcGIS, DRMS

CARIS HPD, DRMS

Table 2.: Validated Information Themes

Product

Current Technology

Sustainment Technology

AusENC (S57, S63) and web services

CARIS ENC Composer, L3/SevenCs ENC Designer and ENC Analyzer, dKart Inspector

CARIS HPD, ENC Analyzer, dKart Inspector

Aus Paper Charts

CARIS GIS, L3/SevenCs ENC Carto-grapher

CARIS Paper Chart Composer

AusGeoTIFF Contractor (L3 Oceania) Contractor (L3 Oceania)

Australian Notices to Mariners

Various DHDB software tools (connected and standalone) and DRMS

Improved DHDB management tools interfaced to production tools and DRMS

Australian National Tide Tables and AusTides

Tides Information System (TIS); ESRI ArcGIS, bespoke software, DRMS, local file system

Tides Information System (TIS), ESRI ArcGIS, DRMS

Hydroscheme ESRI ArcGIS (multiple databases), MS ACCESS, DRMS

ESRI ArcGIS, DRMS

Australian Chart Index (ACI)

Bespoke software tools Sustainment activities will include revised and/or new web service capabilities

Table 3.: AHS Products

Theme Current Technology Sustainment Technology Bathymetry DHDB (dataset storage); CARIS GIS

(survey spatial extents (MBR), thinned soundings - 12m ground resolution); and ESRI ArcGIS (Marine Survey Index)

CARIS BathyDatabase (BDB) – various data resolutions to support military and national charting requirements

Survey/chart ZOC MS ACCESS and bespoke database application

CARIS BDB

Maritime Bounda-ries

ESRI ArcGIS, DRMS ESRI ArcGIS, DRMS

Bottom Objects including Wrecks

MS ACCESS, DRMS ESRI ArcGIS, DRMS

Geographic Names

ESRI ArcGIS, DRMS CARIS HPD, DRMS

Chart Notes and Navigational Views

MS ACCESS, DRMS ORACLE, DRMS

40


Improved production monitoring for pro-duction and maintenance reporting;

Remove the high reliance on individual knowledge;

Consolidate the processes to improve moving product files and metadata into the product master management system;

Simplify the current complex QMS docu-ments;

In conjunction with sustainment of validated information themes, improve efficiencies and quality in the transfer of validated information;

Reduce overheads in training and soft-ware support.

5.4 Survey and Chart Planning LOB The AHS has the responsibility to plan the survey activities for RAN survey assets. Sur-vey planning is closely associated with the planning for charting work. The key require-ments are:

Sustainment and integration into the

DHDB of the current bespoke tools to en-able the AHS to identify survey require-ments (see Figure 3) and enable input from Defence and other agencies and interested parties responsible for the safety of navigation within Australian wa-ters (national maritime safety agency, port authorities, mining companies, etc.);

Existing software tools will be investi-gated and sustained as required to im-prove the planning capabilities and to en-able interoperability with other DHDB sys-tems to assist with risk assessment of survey operations, current and future maritime activities, survey data quality (e.g. CARIS BDB will provide enhanced capability for change detection in bathym-etry and survey data quality information and new web service capabilities will pro-vide interfaces for ship movements, determining acceptable meteorological and oceanographic conditions for survey assets, reporting planned versus achieved survey measures, etc);

Enhance the current published survey and chart planning documents (currently known as Hydroscheme – a 3 year plan-ning document maintained annually and published electronically as a document). The future planning documents are yet to be finalised in terms of content and output format (e.g. web enabled);

Issue Hydrographic Instructions (HI) de-scribing the survey requirements to RAN survey assets in a format that can be easily ingested into RAN surveying sys-tems;

Improve interoperability of the overall planning capability with Defence mission planners and other external agencies.

5.5 Product Distribution LOB The Product Distribution LOB has several functions including:

Handling requests for product from Defence users, chart distributors and other users;

Inventory management;

Product Master Management;

Product protection/encryption;

Customer sales and payments;

Product master replication and packag-ing;

Print-on-Demand paper chart printing;

Licence management.

The initial DHDB implementation included a number of system and workflow processes that were not adopted. This then led to the implementation of several disconnected software solutions that have led to inefficient work practices, repetitive data entry and repli-cated data. Key requirements are:

Implementation of an external access email and internet connectivity and user interfaces (e.g. Australian Chart Index and website access tools) to improve the

41


Figure 3. Survey plan for Broome, WA (AHS, 2013)

42


interfaces (e.g. Australian Chart Index and website access tools) to improve the electronic service (e-service) delivery capabilities;

Integrating and implementing updates to current software used for data protection/encryption (e.g. S-57 to S-63), license management, chart POD printing, master product management, inventory man-agement onto the DHDB software base-line;

Use of or interfacing to a Whole of Government finance solution for accounts management.

6. Threads The architecture includes “threads” of com-mon activity that are present in all levels of the framework. These threads are: 6.1 Workflow Workflow is the definition, execution and control of business processes where tasks, information or documents are passed from one state to another for action according to a set of procedural rules. Workflows make proc-esses more efficient, compliant, repeatable, agile and visible. Activities can be explicitly defined, monitored over time, and optimised, leading to improved efficiencies, quality and decision making. The DHDB workflow in use has not been re-viewed or re-aligned to current business proc-esses. In many cases, the current processes are not thoroughly documented particularly where the DHDB workflow is not used (i.e. in a number of workaround solutions). In terms of the organisation operations as a whole, very few AHS activities, through the lack of sustainment, use any standard workflow. This results in: Poor control over routine activities involv-

ing manual and automated processes. This leads to inconsistent work practices that potentially expose the AHS to opera-tional errors and relies heavily on experi-enced human oversight to mitigate;

Poor tracking of operations progress and minimal capability to report and monitor performance;

An abundance of non-conforming bespoke solutions held together by manual processes and uncontrolled docu-mentation; and

A high reliance on individual skills rather than a shared corporate capability.

6.2 Business Intelligence Business Intelligence represents the tools that provide a key capability in the strategic planning (see Figure 4) process of the AHS enterprise. These tools allow staff to gather, store, access and analyse corporate data to aid decision making. They will assist in the areas of customer profiling, customer support, product capability and sales, operations analy-sis, statistical analysis, use in measuring performance and results.

Figure 4. Business Intelligence (ITPRO, 2014) Key requirements are:

An integrated system that provides a

consistent and single point of truth from which corporate decisions and strategy can be made based on facts and objective assessment;

Extraction of data that is usually stored deep within specific DHDB database tables into a format for ongoing analysis;

Display key data in simple outputs such as dashboards, traffic lights, graph metrics, etc.

43


6.3 Information and Communications Technology (ICT) Infrastructure ICT Infrastructure is the range of technologies to assist the AHS in running efficiently. It is essential to the everyday mechanics of the organisation and efficient and effective service delivery. These include hardware, software, networking and implementation. The AHS op-erates across multiple air-gapped networks with different operating systems. The remediation of DHDB ICT is an ongoing requirement and current activities include: DHDB software application baseline

remediation – ensuring that all key appli-cations are integrated into the DHDB baseline so that version control configura-tion and maintenance is applied consis-tently; manage 32 and 64 bit processing configurations, improve service delivery through enhanced communications infra-structure and ensure hardware and soft-ware are refreshed and maintained;

Reduce the number of operating net-works;

DHDB maintains the appropriate security accreditation in accordance with Whole of Government and Defence policies;

Implementation of a new Internet Gateway capability – this is a core capability to pro-vide improved services to external (public and Defence) and internal customers as well as creating opportunities for service providers (e.g. hydrographic survey and production agencies) to interface directly into DHDB systems;

To support enhancements to the produc-tion environment, a new prototype/training and test environment;

Upgraded connectivity – internet band-width, fibre to the desktop, etc.

Upgraded desktop configurations i.e. user ergonomics (desk and chair/stand), moni-tor configurations, desktop space require-ments;

Consider future Whole of Government and Defence Primary Data Centres and

Defence Integrated Environments (DIE) to potentially reduce internal AHS server, data storage, and backup requirements.

6.4 Standards Standards enable a consistent approach to all activities across the enterprise and the associ-ated framework. Standards relate to national, international and industry activities and promote the use of non-proprietary solutions within enterprise components. The AHS is familiar with standards compli-ance. Other Australian Defence agencies have commented that the AHS is one of the most advanced agencies in addressing geospatial standards issues, particularly in relation to the ISO, Defence and hydrographic standards implemented under the guidance of the IHO. IHO S-100 and its relevant product specifica-tions such as S-101 (ENC) are yet to be implemented within the main software systems used in the AHS. There will be a ma-jor transition activity from S-57 to S-100 some-time around 2017-2018 and the timing will be in part driven by the software vendors and ECDIS manufacturers as they update their systems. 6.5 Organisation Organisations are social entities that have a culture, a formal and informal structure, goals, activities and resources. On the whole, the AHS organisation structure has not experi-enced much change since the original imple-mentation of the DHDB in early 2000s. Evolutionary sustainment of the DHDB will address the following organisation issues:

The implementation of the DHDB im-posed a workflow process based on a 1990s organisation structure. Whilst the organisation structure has changed, the workflow and business processes were not able to be maintained due to the implementation issues;

The organisation has predominantly

44


been a “product-centric” agency and there is a need to move to an “information-centric” agency;

The majority of AHS staff involved in production and data management activities do not necessarily have formal qualifications in geospatial data management or nautical cartography. This results in the AHS having to provide “ on-the-job” training or seek external training;

Changes in the LOB activities and workflows will result in changes to the physical organisation structure support-ing such activities. This will require an active change management approach with staff.

The primary outcomes of the change manage-ment process through and EA approach are: Service Delivery – missions, programs, services, data and products are required of the AHS. The requirements for these are rapidly changing with electronic products and contemporary service delivery enablers and models for internal and external customers. Functional Integration – since the introduc-tion of the DHDB, the AHS has grown to include Meteorology and Oceanography (i.e. METOC) and the Geospatial Intelligence Library (GSIL) sections. The planned inclusion of other sections requires their systems and personnel to be integrated through transition programmes. Resource Optimisation – In accordance with the requirements of the Australian Public Governance, Performance and Accountability Act 2013, the AHS has a responsibility to optimally manage its resources (i.e. human, financial, system and infrastructure). Under a tightening resource model, the AHS is going to have to achieve its mission with:

Reduced civilian resources;

Ensure products are value-for-money, fit-for-purpose and meet end-user expec-tations;

Re-configured and streamlined ICT environment and business processes that continue to be contemporary through an enduring sustainment process;

Strategic use of the AHO building to ensure floor space is optimized and meets Defence expectations regarding effective use of the space.

7. Conclusions The evolutionary sustainment of the DHDB is a lengthy programme commencing in 2011 and planned for completion in 2017 based on the scoping to sustain what already exists. In reality, the sustainment is already stretching the scope by incorporating METOC elements and the need to address contemporary issues that didn’t even exist when the DHDB was originally scoped in the mid-late 1990s. The adoption of an Enterprise Architecture (EA) approach won’t guarantee success. It does however reduce risk by providing a rigorous approach for the HIPI team and the AHS senior leadership to adopt and monitor. It provides an independent process by which the organisation can review its business require-ments, strategy and goals and have solutions worked through and implemented. The evolutionary sustainment of the DHDB is already positively impacting the AHO’s opera-tions and over the next 12-18 months, the majority of the key components will be updated and/or replaced with contemporary solutions. In reality, the sustainment never ends, but at least this overall evolutionary activity will provide the AHS with a contempo-rary solution the meets the legislative require-ments of Defence and its customers. The problems of DHDB-past will hopefully be soon forgotten and the AHS will have a robust capability that can be incrementally sustained into the future until the next major revolution.

8. References 1. AHS, 2013. Hydroscheme 2013-2015,

Australian Hydrographic Service,

45


ht tp : / /www.hydro.gov.au/bus iness-pub l i ca t i ons /Hyd roscheme_2013 -2015_WEB_18042012.pdf

2. Bernard, S. 2015. The EA3 Cube Approach, https://eapad.dk/ea3-cube/overview/ viewed 13 March 2015.

3. Kotter, J. P. 2012. Leading Change, Har-vard Business Review Press, Boston, USA.

4. Office of Management and Budget (OMB),

2012. The Common Approach to Federal Enterprise Architecture, Executive Office of the President of the United States, 2 May 2012, pg. 45.

5. ITPRO, 2014. What is business intelli-

gence? http://www.itpro.co.uk/business-intell igence/21861/what-is-business-intelligence viewed 23 December 2014.

Author Biography Ian HALLS commenced work at the Australian Hydrographic Office (AHO) in 1979 as a trainee nautical cartographer and has been involved in the development of nautical data management and chart production systems since the mid-1980s. This period included serving several years on IHO ECDIS/S-57 technical committees. He is a past Direc-tor of HSA Systems Pty Ltd and resumed working at the AHO in 2009 after 15 years in private industry undertaking systems engi-neering, hydrographic surveying and charting activities. He is currently managing the military hydrographic data, products and services section of the AHO. Ian is also working with a small dedicated team to sustain the Digital Hydrographic Database solution developed in early 2000. This involves the software, hardware and ICT refresh of the various source data receipt, validated data, production, distribution, and workflow sub-systems using an enterprise architecture approach. [email protected]

46



47


OPERATION TIRÚA: HYDROGRAPHIC VISION N.A. Guzmán Montesinos

Naval Hydrographic Engineer Head of the Information Technology Department

Head of the Chilean Tsunami Warning Center, Chile

Abstract On 6 October 2013, a small commercial flight with 5 people on board and flying between Mocha Island and Tirúa off the coast of Chile went missing, leaving no traces. The Armed Forces established a search party with personnel and staff from the 3 forces. Considering the possible fall of the plane into the ocean, the Navy deployed several naval and maritime resources with the purpose of finding any signs of it. A number of vessels with hydrographic capacities searched carefully for over 20 days with no positive results. Some parts of the small plane were found 7 months later on the coast on Mocha Island. The hydrographic experiences from this operation provide a number of lessons that can be used in similar operations and to optimize the use of the resources. Background

A Cessna 172 plane took off from Mocha Island on 6 October 2013, heading to Tirúa, 8th Region of Biobío, with a regular flight time of less than 15 minutes and with 5 passengers on board. The small plane took off at 14:45 with the landing expected sometime around 15:00 hours on the same day. Assuming a possible tragedy, a Search and Rescue (SAR) operation was activated at 18:00 hours later that same day. The Army, Navy and Air Force were part of an operation led by the Air Force. To search the ocean area, the Navy used a series of naval and maritime capabili-ties, along with help from civilian resources to locate the small plane or its remains underwater. The ocean search used the following technologies: 1. Three platforms fitted with multi-beam systems. This operation used two

multibeam systems installed in the naval units, the BMS Merino and the PSG Ortiz and a portable device, which belonged to the Skyring Marine Enterprise, installed onboard a minor vessel. The multi-beam systems consist of small sonars designed to measure the depth of the seabed, with hundreds of beams per second to determine the topography and objects on the seabed. The resolution of the multi-beam system depends on the seafloor depth, vessel speed and the system characteristics. The BMS Merino system is 50 KHz and is designed for medium to deep waters. It also has a theoretical range of operation between 30 and 3,000 meters and sends out 126 beams per second. The PSG Ortiz and of Skyring Enterprise systems are 300 KHz, with a depth range between 0.5 and 150 meters. The 254 beams per second provide a higher discrimination capacity.

2. Three Side Scan Sonar (SSS) systems. SSS detect objects on the seabed (Figure 1) and provide good capabilities for this kind of operation. One system belonged to the Navy and the other two systems belonged to the Marine Scope

48


and Skyring Marine. SSS can be operated from a small rubber boat, so wind, sea state and daylight hours can pose operational limitations.

Figure 1. Container detected by a Side Scan Sonar, sunk in the Bay of Talcahuano.

3. Two Submarine Scan Robots. The Remotely Operated Underwater Vehicles (ROUV), commonly called Submarine Scan Robots, have the capacity of acquiring images and video of the seabed. This makes them useful for this kind of operation. Its major limitation is visibility determined by the water turbidity. For Operation Tirúa, the visibility was poor and never exceeded two meters, so the ROUVs had a very limited use in this search.

4. Divers. Divers go underwater to recover whatever is necessary once something has been detected by the multi-beam, SSS or ROUV. The limitations of the divers are visibility, depth and time of operation.

5. Naval and Maritime units provided general logistical support and visual scanning for floating objects.

The only previous local experience for this type of search operation, was in the 2011 search for the fighter plane that crashed into the ocean around Juan Fernández Island, killing 21 passengers. Unfortunately, personnel and units engaged in the current search were not part of that previous operation. The search area’s hydrographic and oceanographic conditions determines what sort of vessels are needed, the type of equipment and the skills and experience of professional staff to perform the search efficiently. All the resources were deployed gradually as new informa-tion was gathered and more hypotheses developed as to what could have happened and how to find the aircraft. Description of the Operation

The operational instructions came from the Coordination Center of the FACH (Air Force of Chile) on land and the coordination for searching the ocean was assigned to the BMS Merino. The initial information given for the search was practically zero, since there was no wreckage found or seen, nor witnesses that saw the small plane fall. No photographic or video record of the flight existed to indicate that the aircraft had fallen. There wasn’t any certainty that it had even fallen into the ocean. Hence, the first phase of the operation was a visual search with the three branches of the Armed Forces on land, sea and air, to look for any sign that could indicate a way to narrow the search.

49


Local fishermen informed the search coordinator that they had seen oil slicks. The units checked the area, took some oil samples to be analyzed, but they didn’t find any sign of the airplane. The samples were sent to the Chemical Oceanography Laboratory of the Hydrographic and Oceanographic Service of the Navy (SHOA), where an analysis of hydrocarbons was conducted using the gas chromatographer. These returned negative results for aviation oil. Later information was received about an alleged connection of a cell phone of one of the passengers, who would have had a signal on his or her cell phone for about 6 minutes of the flight. With the speed of the flight, it could be inferred the presumptive distance that the airplane should have flown, if it had fallen after 6 minutes. The problem was that the airplane’s speed could have been between 60 and 100 knots, so the distance from the Mocha Island varies significantly, and even worse, the path of the plane was completely unknown. With the purpose of carrying out a search in a well-planned manner, the search was divided in sub-areas and assigned to the different units and determined their progress. The assumption that the plane had followed a regular flight route and had fallen into the ocean was taken as a starting point after the exhaustive but unsuccessful search on land. Sounding operations were initiated to search for the plane or its wreckage. The summary of the hydrographic and oceanographic characteristics of the search area were: Hydrography Depth: Shallow waters, between 20 and 30 meters in the area of the alleged plane course. The size of the areas, combined with a shallow depth, lengthened the search. The coverage area of the seabed using multi-beam equipment is three times the depth, meaning the swath of the beam pattern was between 60 and 90 meters wide per each sounding line. The distance between Mocha Island and Tirúa is 17 nautical miles, requiring approximately 420 sounding lines given a coverage area of the multi-beam swath of 75 meters of the bottom. If a four-mile long sounding line is determined with an average speed of 7 knots for a vessel, which is the appropriate speed to be able to detect an object the size of a small plane, the result is 34 minutes per line. If this is multiplied by the 420 lines, approximately ten days sounding is obtained. This only covered the area approximating the possible track of the plane. Existing sounding lines: This data was limited and were used to compare any differences to the first sounding lines. In the areas where there were no previous sounding lines, the sound-ing lines required were between 30 and 45 meters wide. This doubled the sounding time which was originally 20 days. Oceanography: Waves: the general wave direction in the area is predominantly from the south-west, 1 to 3 meters high with a period between 12 and 20 seconds. The length grows when there are swells, which happens regularly between March and November. During the search days, tidal waves were evident practically all the time, which made it much more difficult with the sound-ing operations. Severe movement of the vessel often caused the multi-beam system to crash. This meant that during most of the time, soundings could only be recorded along lines from south to north and not the lines from north to south, further extending the sounding time to 40

50


days. To cover the total area, considering the detour of the plane and its maximum distance according to the 15 minutes of the flight, it would have taken approximately 3 months of uninterrupted sounding, that is to say 4 months of scheduled hydrographic work. Currents: the currents between Mocha Island and the continent are predominantly south-west and south-east with modeled intensities that vary between the 0.5 and 2 knots. During the survey, three experimental drifts were deployed with elements monitored by GPS. This equipment provided actual results for comparison against those that were modeled. Tides: variations of the sea level in the area were minimal with no influence on this operation. Taking into consideration these conditions, the search operation was established deploying the different units for sounding capture in the search area. The BMS Merino and the PSG Ortiz, which used multi-beam sonars and SSS, both detected what seemed to be a wrecked plane. Small objects, less than 3 meters in length, showed up on the sonar screens, different from what usually is found on the seabed. The extensive experience of the sonar operators from the Hydrographic and Oceanographic Service of the Navy (SHOA) enables determination of the seabed characteristics and its structure. However, the personnel have not had much experience in the search for sunken wrecks. Many of the new contacts found where fish shoals abundant in this fishing area. The formation of the fish looked similar to a tube and was very close to the seabed, confusing the operators and making them think they had detected something. When they repeated the process, they found that there was nothing in the same spot. After days of searching and not finding any floating wreckage, oil slicks or any other signs that a plane had crashed in the ocean, it was assumed that the plane had not broken up upon impact with the water and had sunk intact. The experience of the Skyring Marine personnel in similar searches in other countries, with similar sonars to the one used by the PSG Ortiz, indicated that based on the area’s water depth, they expected the multi-beam system to detect the plane almost in its intact form. An example of this type of detection is shown on Figure 2 where a small plane can be seen at less than 30 meters of depth using a 400 kHz multi-beam system.

Figure 2: Small plane visualized at less than 30 meters of depth, in Panama.Image prov ided by Fernando Landaeta, Skyring Marine.

51


During the search, information was collected from different sources. Several opposing theo-ries of what might have happened also existed. Search vessels with the multibeam systems kept on searching along the alleged flight route, as shown in Figure 3.

It needs be pointed out that the searched area only represented a small area, assuming that the plane had problems and could have changed course in any direction. Conclusions and Recommendations

The execution time of a search of any sunken object in the ocean will be considerably reduced if evidence that confirms the sinking location is available.

The hydrographic and oceanographic conditions are the first things to consider in planning the operation; together with the search methodology to be used, such as vessels, sonar equipment, divers, ROV and their respective capabilities and their limi-tations.

The sonars to be used include high frequency multibeam systems and Side Scan Sonars (SSS). The use of SSS equipment can be limited to the platform from which they are deployed.

It is highly recommended to follow a sequential search methodology to maximize technology and human resource efficiency and the likelihood of finding any sunken wreckage: first to search using multibeam and SSS; and after making contact with the object, to deploy the ROUV down and finally the divers, only when video or images confirm the presence of the object.

Specialized trained personnel are needed to operate the search equipment.

Figure 3: Sounding performed during the 20 days of search by the BMS Merino, PSG Ortiz and Side Scan Sonars, based in the regular flight track of the plane.

52


Author Biography

Nicolás A. Guzmán joined the Navy School as a Cadet in 2000 undertaking various Naval assignments. In 2008, he was promoted to Lieutenant and started his studies in Hydrographic Engineering at the Naval Academy. In 2010, he graduated from his “Category A” course at the Hydrographic and Oceanographic Service of the Chilean Navy (SHOA), completing his thesis on wave energy. The following two years, he was involved in activities related to improve the Chilean Tsunami Warning Center. This included periods of study and further training, specializing in Tsunami’s and attending the Early Warning Training Program in Hawaii, at the Pacific Tsunami Warning Center (PTWC). At the end of 2012, he was promoted to Senior Lieutenant. Since 2011, he has led several Hydrographic and Oceanographic campaigns, in different loca-tions in Chile, including Antarctic. In addition to these operations, he is the current Head of the Information Technology Department, the Head of the Chilean Tsunami Warning Center and has been a Hydrography and Oceanography professor at the Training Center of SHOA since 2012. [email protected]

53


OBITUARY

IN MEMORIAM Richard Michael (Mike) Eaton, C.M, B.Sc.

1928-2014

by Mathias Jonas Chair of the Hydrographic Services and Standards Committee

Hydrographer of Germany The hydrographic community learned, with sadness and respect, of the passing of Richard Michael Eaton, always known as Mike to his hydrographic colleagues, on 9 October 2014 at his home in Cole Harbour, Nova Scotia, Canada at the age of 86. Coincidentally, he passed away as the International Hydrographic Organi-zation was meeting in Monaco for its 5th Extraordinary Conference, where it agreed new conditions for the award of the Prince Albert 1st Medal for Hydrogra-phy. In future, the medal will be awarded to recognize the “heroes of hydrogra-phy”. Undoubtedly Mike Eaton was one of those “heroes of hydrography”. Mike Eaton, born in 1928 near Hull, United Kingdom, migrated to Canada and joined the Canadian Hydrographic Service in Ottawa in 1957 after serving in the UK Royal Navy as a hydrographic surveying officer. As he explained in an interview with the magazine Hydro International in 2005, there were three major overlapping phases in his Canadian hydrographic career: Arctic developments in the Canadian Hydrographic Service, positioning systems development while at the Bedford Institute and electronic chart development after his retirement. In the Arctic he found ways to acoustically measure the depth beneath the ice and he also developed a somewhat precarious technique that involved towing an echo sounder transducer from a low-flying helicopter over broken ice and open water to measure water depths. At the Bedford Insti-tute he worked on the propagation of the radio ground waves used by Decca, Loran-C etc. From 1988 he became deeply involved in the specifications for the Electronic Chart Display and Information System (ECDIS), and was keen to listen to the users’ voice which led him to guide the development of the display standards for ECDIS as the Chair of the IHO Working Group on Colours and Symbols. He began his ground-breaking work on the standardization of the display of chart information in ECDIS; at an age when others start to think about retirement. Digital charts were in their infancy at that time: personal computers were not yet powerful enough to process mass data; monitors were mostly black and white and with only low resolution and the idea of relying upon software driven devices on board ships sounded like science fiction. Inspired by the early attempts to

54


visualize digital charts based on vector data, Mike Eaton began to design and specify the core of digital charting: the digital symbolization of the information contained in the chart. Logically, he started to adapt paper chart symbology to the technical capabilities and in many ways at the time, the limitations of a computer screen. But he went way beyond this because he realized at an early stage the potential additional capabilities available through a computerized display of charts of the oceans and waterways. We must credit him with breakthrough concepts such as a range of standardised colour tables to enable the chart display to be adapted for ambient light condi-tions (day, night or dusk) and conditional procedures that enabled the presentation of features on the bridge display to be adjusted automatically according to the situation at hand. Mike Eaton ex-perimented at length and with characteristic thoroughness, for example, on how closely two shades of blue could be displayed on a screen before the eye could not distinguish between them. The development of IHO Publication S-52 - Specifications for Chart Content and Display Aspects of ECDIS, one of the two governing standards for ECDIS, benefited hugely from his per-sonal input for more than a decade. Much of his original input remains as a foundation and fun-damental principle of the standard today. His work was key in making ECDIS a reality and his legacy endures in current ECDIS units to this day. Mike Eaton was a courtly and modest man who always retained his British reserve. Humble and reluctant to acknowledge his outstanding contribution, it was most pleasing that his efforts and lasting legacy was recognized by Canada when he was awarded the Canada Marine Safety Award in 2000 and appointed to the Order of Canada in 2004. The citation for his award read:

With vision, innovation and ingenuity, Michael Eaton has made outstanding contribu-tions to the advancement of hydrography in Canada. Scientist Emeritus with the Ca-nadian Hydrographic Service, he developed techniques to accurately map frozen bod-ies of water and combined various positioning systems to more precisely survey ocean waters. Renowned nationally and internationally as the “father of the electronic chart”, he envisaged a computerized version of the traditional marine chart. This elec-tronic chart has become a common navigation tool for many shipping and recreational vessels, contributing to greater marine safety around the world.

If there will ever be a Hall of Fame for digital nautical cartography, Mike Eaton’s commemorating plaque will occupy a prominent place.

Documents

THE INTERNATIONAL HYDROGRAPHIC REVIEW - iho.int · international hydrographic review may 2015 the international hydrographic review international hydrographic organization monaco