EVALUATION OF DIFFERENT SOLUTIONS TO FILL · PDF fileEVALUATION OF DIFFERENT SOLUTIONS TO FILL VOIDS IN SRTM ELEVATION DATASETS Megan Delaney, GIS Data Technician Trina Kuuskivi, Production

EVALUATION OF DIFFERENT SOLUTIONS TO FILL VOIDS IN SRTM ELEVATION DATASETS

Megan Delaney, GIS Data Technician Trina Kuuskivi, Production Manager Xiaopeng Li, Ph.D., Mapping Scientist

Intermap Technologies Corp. 2 Gurdwara Road, Suite 200

Ottawa, Ontario, Canada K2E 1A2 Phone: (613) 226-5442 Fax: (613) 226-5529

[email protected]@intermap.com

[email protected]

ABSTRACT Due to the complex nature of deriving elevation from Interferometric Synthetic Aperture Radar (IFSAR) data, most finished SRTM data contains voids where no elevation value could be determined. This is especially the case for some water surfaces and patches of very rough terrain. In many cases, these voids must be filled, as fully populated elevation models are required for the intended applications. Several companies have attempted to fill these voids by creating their own software solutions. These solutions utilize interpolation algorithms, source datasets as patches, or a combination of both to fill the Shuttle Radar Topography Mission (SRTM) voids. Due to the large number of solutions available, and the varying nature of the intended applications of void-free data, it is necessary to explore such methods in detail and evaluate the results.

INTRODUCTION

Starting in 2004, finished SRTM data became available to the general public. Finished SRTM data contains voids, or areas where no elevation could be determined by the radar sensor. Since many users look to the SRTM dataset for large, seamless, uniform and continuous data, it is necessary to address these areas of missing values. In many cases, it is as simple as a basic Internet search to find an application claiming the ability to fill such voids. While many void fill solutions are similar in nature, they can produce drastically dissimilar results. A few representative applications include: 3DEM (Visualization software), Landserf (Landserf), VTBuilder (Visual Terrain), Voidkiller (DG ADVANCED), Blackart (Terrainmap), SRTMfill Utility (3DNature) and SURFACE (GMT). Industry standard GIS platforms also provide many tools for interpolating and patching void data. For example, ESRI’s ArcInfo software package includes the subroutine TOPOGRID.

As a follow-up to the ASPRS Spring 2005 paper submitted by Intermap Technologies, entitled “Void Fill of SRTM Elevation Data: Performance Evaluation” (Kuuskivi et al., 2005), which provided an in-depth evaluation of the Intermap-Boeing Voidfill process, the focus of this paper is to evaluate the performance of commercially available and freeware software solutions that claim to be capable of filling voids in SRTM data. This paper also explores alternative methods for filling voids using standard GIS platforms such as ArcGIS. Cells with varying terrain and void sizes were used, and a statistical/visual analysis of the performance of the void fill solution was conducted. Practicality, ease of use and flexibility of the solutions are addressed.

BACKGROUND OF VOID FILL AND EVALUATED PACKAGES While the general ideas are similar, each void fill solution package involves slight variations in the methods used to fill missing data. Listed below are explanations of the algorithms used, as described by the authors and or the documentation accompanying the void fill solution package.

The method used by 3DEM to fill in SRTM holes is a simple interpolation process. For each point at which an elevation value is missing, the program finds the elevation and position of the nearest valid points in the north

PECORA 16 “Global Priorities in Land Remote Sensing” October 23 – 27, 2005 * Sioux Falls, South Dakota

mailto:[email protected]



direction and in the south direction. A linear interpolation between these two valid points is then calculated. The program finds the elevation and position of the nearest valid points in the east and west directions, then it calculates a linear interpolation between these two valid points and two estimates of the missing value – one is an interpolation between valid points east to west, and the other one is an interpolation between valid points north to south. These two estimates are then averaged to produce the final elevation estimate at the void point. This works fairly well for small holes in the data. However, it is not suitable for filling large areas (Horne, 2005).

The gap-filling algorithm in VTBuilder is straightforward: like 3DEM, iteratively, gap pixels with valid neighbors are filled with the average value of those neighboring pixels (Discoe, 2005).

SRTMfill Utility uses a box-filtered in-fill. All pixels that are NULL that are adjacent to a sufficient number of non-NULL pixels are replaced by a distance-weighted average of their neighboring valid pixels. The process is repeated until no NULL values remain (Hanson, 2005).

Voidkiller relies on a quadratic interpolation, using pixel boundaries. If the user chooses, SRTM30 base data can also used in a poundered mean model for assuring terrain continuity on edges of data/no data (Gil, 2005).

The SURFACE model utilizes gridding with continuous curvature splines in tension. This method interpolates the data with a surface having continuous second derivatives and minimal total squared curvature. This model also employs an elastic plate flexure and tension function to find a solution for local variation (Smith and Wessel, 1990).

Blackart involves a two-level iterative computational method for solution of the Franklin Approximation Algorithm. This is essentially a nearest neighbor approximation and fills voids from the edge inward to create a “well behaved” surface (Terrainmap, 2005).

Landserf void removal works by replacing the outer 'ring' of a void with the average of its non-void neighbors. The process is repeated until no voids remain. For most small voids, this results in a pretty reliable interpolation. For larger patches, this can leave minor artifacts in the interpolated area that look like radiating 'spokes' from the centre of the void patch. Landserf can also combine the SRTM DEMs with other DEMs of the same area for which you have elevation data (SOI City, 2005).

ArcInfo TOPOGRID is based on ANUDEM software developed by the Australian National University to create hydrologically correct DEM datasets, which is modeled from established algorithms of Hutchinson (1989) to use contour data and stream point data (if available) to produce such DEMs. Data for NoData regions in the SRTM cell is then merged with the original dataset.

DESIGN AND PROCEDURES OF THE EVALUATION In order to compare the test results of each solution, three finished SRTM DTED®Level 2 cells (Figures 1, 2 and 3) were selected from California (w121n38), Florida (w082n28) and Texas (w102n34) and subjected to filling procedures. Since many of the void fill solutions are intended for unfinished SRTM data, only working with SRTM height field (.hgt) files, a certain level of pre-processing was involved in order for the packages to ingest the finished SRTM data. Files were subsequently filled and tested against higher resolution reference datasets, such as Intermap’s 5-meter STAR-3i Digital Surface Models (DSM). Larger substantial artificial voids were introduced over areas where ground control points had been obtained (NGS GPS) and compared statistically in order to examine how the void fill solutions handled large voids, and to test the output surface over known terrain as a measure of accuracy. Terrain profiles were examined and compared to the original SRTM surface and reference elevations. Difference surfaces were calculated. Test tiles were selected based on their terrain, location and presence of voids. Location was a consideration due to the availability of a higher-resolution reference dataset for statistical comparison. Input tiles, in appropriate formats, were opened in selected void filling solution packages and processed according to provided instruction and saved for post-processing analysis. Reference datasets were compiled to test the accuracy of the filled regions. GPS points were downloaded from the NGS data center by the geographic bounds of test tiles. STAR-3i data was obtained from the Intermap Store, mosaicked and resampled to match the SRTM dataset. Accuracy of the resampled STAR-3i dataset was assessed by deriving summary statistics against the NGS control GPS points. The filled datasets were then differenced with the resampled STAR-3i reference dataset. Accuracy was then determined for the resulting data. ArcGIS Spatial Analyst was used to generate regions of void data in the SRTM datasets which intersected with overlayed GPS points. Tiles were then processed in the void filling solution packages as before. Statistics were generated for filled regions by comparing the overlayed GPS points with the newly filled elevation value at that particular point of intersection. Profiles of the filled void regions were generated using PCI and compared to the


original DTED® surface to determine overall visual integrity of the solution, and to observe any overall spatial trends in the filled datasets. Difference surfaces between the filled tiles and original SRTM DTED® were derived and summary statistics were generated.

Figure 1. DTED®Level 2 Test Dataset: Cell w082n28

(Note: voids are displayed as white)






RESULTS AND ANALYSIS

False Void and GCP Analysis Artificial voids were added to the cell w121n38 in areas where GPS ground control data was available. Table 1

describes the results of the subsequent comparison.

Table 1. Statistical Analysis of Filled and Control Regions Against GPS Ground Control Datasets

Reference Data Fill Applications

DTED2 resampled STAR-3i NED (30m) ArcInfo Surface 3DEM VTBuilder Landserf Blackart SRTMfill

Standard Deviation 1.7 1.6 1.9 1.8 13.7 14.9 16.7 20.3 n/a n/a RMSE 2.7 2 2.7 9.9 15.6 16.9 19.7 22.8 n/a n/a Accuracy (LE90) 4.5 3.3 4.5 16.3 25.7 27.9 32.5 37.6 n/a n/a

It can be seen from the table above that the overall accuracy of the filled datasets varies from over 15 meters to

almost 40 meters in some cases. Of the reference datasets, in this particular study region, the original SRTM data proved to be quite accurate against the GPS points. The resampled STAR-3i data was of the higher accuracy than the SRTM data, proving the STAR-3i data to be a valid reference dataset to assess the overall accuracy of the filled void regions. The large void areas introduced and filled in w121n38 exceeded DTED®2 absolute vertical accuracy of +/-9m that was determined for North America (Rodriguez, 2005).

Table 2. Statistical Analysis of Filled Data against DTED® Values for Artificial Voids

3DEM ArcInfo Landserf VTBuilder SURFACE Standard Deviation 20.4 0.5 25.6 42 22.1 RMSE 34.3 0.6 28.7 65.4 36.1 Accuracy (LE90) 56.6 1.0 47.4 107.9 59.6


The following images compare the original SRTM surface versus surfaces that have been filled using the tested

void filling solution packages. Note the lack of results for Blackart, SRTMfill Utility and Voidkiller. These applications are intended for unfinished SRTM data only and failed to process derived raw formatted cells, despite developer’s efforts to provide a solution to be included in this paper. Blackart will process and fill DTED®1 data, as analyzed in the previous paper (Kuuskivi et al., 2005), but fails to process DTED®2, which was the desired test dataset in this publication.

Shaded Relief Images for Large Void Area 1

In the case of artificial void 1 (as seen in Figure 4), none of the void fill solutions preserved the peaks in the mountainous data as interpolation algorithms typically do not. ArcInfo was the closest to preserving this artifact. ArcInfo was the most successful at mimicking the overall nature of the terrain.

Original SRTM surface 3DEM

Landserf ArcInfo

SURFACE VTBuilder

Figure 4. Original and Filled Solutions for Artificial Void Region 1


Shaded Relief Images for Large Void Area 2 All of the solutions filled the valley in void 2 artificially high, except for VTBuilder, despite its poor aesthetic

quality and unrealistic terrain. All datasets resulted in unnatural linear artifacts. In spite of the apparent grid patterned artifact in the ArcInfo filled solution, it is still the most natural representation of the area in question. SURFACE closely mimics ArcInfo’s results.


ArcInfo Landserf

SURFACEVTBuilder



Shaded Relief Images for Large Void Area 3 As in the previous cases, ArcInfo and SURFACE produced the most natural results, most closely mimicking the

original moderate terrain. Artifacts are present in most of the filled datasets below.


ArcInfo Landserf

VTBuilder SURFACE


(Note: Red dot in image merely distinguishes which filled void region in the image being discussed and profiled.) Profiles across Artificial Voids

Overall profile trends indicate that ArcInfo filled the large voids exceptionally well, as seen in Figures 7 – 9. VTBuilder filled voids artificially low in all cases, and SURFACE and 3DEM were moderate approximations. VTBuilder did not respect the surface trends and was significantly lower than its competing datasets. For example, the resulting VTBuilder surface dipped when all other datasets (including the original surface) peaked and vice versa. SURFACE and 3DEM produced very similar surfaces. Landserf’s results were unpredictable but performed rather well over a lower relief surface (Figure 9).


Figure 7. Profiles across Large Artificial Void Region 1




Filled Genuine Void Results

The following tables represent the difference in elevation values between the filled void regions and the reference elevation dataset (Intermap STAR-3i DSM).

Table 3. Difference in Elevation across Filled Voids and STAR-3i Data

w102n38 3DEM ArcInfo Landserf VTBuilder SURFACE Min Diff (m) -19 -15 -18 -19 -21 Max Diff (m) 17 17 20 24 23 Standard Deviation 4.5 4.7 4.8 6.3 4.8 RMSE 4.8 5.5 5.2 6.5 5.3 Accuracy(LE90)+/- 7.9 9.1 8.7 10.8 8.7


w082n28 3DEM ArcInfo Landserf VTBuilder SURFACE Min Diff (m) -26 -16 -19 -23 -19 Max Diff (m) 76 51 55 83 50 Standard Deviation 5.1 4.8 5.1 5.9 4.9 RMSE 7 6.3 6.3 7.6 6.2 Accuracy (LE90) +/- 11.6 10.4 10.4 12.6 10.2



w121n38 3DEM ArcInfo Landserf VTBuilder SURFACE Min Diff (m) -2740 -216 -604 -573 -199 Max Diff (m) 148 186 369 258 156 Standard Deviation 26.5 30.9 76.5 82.8 22.9 RMSE 28.1 31 77.8 107.5 23 Accuracy (LE90) +/- 46.4 51.2 128.4 177.4 38.0

The results are consistent across the void fill solution packages for average sized voids and are slightly better in the Texas cell (w102n38). There are extreme results for the mountainous cell, w121n38. In the case of 3DEM, the results show extreme values occurred on the edge of the dataset. In terms of the relative magnitude of the error in elevation, the size of the error in the mountainous cell is not as significant as the statistics imply. For example, an error of 8 meters when the elevation range across the terrain is 200 meters, is the same as an error of 80 meters across a cell when the range of elevations across the terrain is 2000 meters. Table 6 shows the range in elevation in the input datasets and the average accuracy of the filled tiles.

Table 6. Relative Magnitude of Error

w082n28 w102n38 w121n38 Range in Elevation (m) 194 540 3120 Average Accuracy (m) 11.0 9.0 88.2 Ratio of Error 0.057 0.017 0.028

It can be seen that the relative error (as measured by the ratio of error) obtained in filling the flatter w082n28 tile is much greater than the mountainous tile w121n38, despite the obvious difference in accuracy on a pixel-by-pixel basis. All of the void fill solution packages generated elevations within the specified accuracy level for DTED®2 in tiles with moderate terrain and average void size. The overall documented accuracy for DTED®2 is +/- 16 meters (Jet Propulsion Laboratory, 2005). Usability Analysis The following table summarizes the overall usability of each of the selected void fill solution packages.

Table 7. Breakdown of Criteria to Convey Level of Usability

3DEM ArcInfo Landserf VTBuilder SURFACE Blackart SRTMfill

Utility Voidkiller Time Fast Slow Fast Fast Fast Moderate Fast Fast Intuitive (Yes/No) Yes No Yes Yes Yes No Yes Yes

Steps Few Many Few Few Single Few Single Few Batching Ability No Partial Partial Yes Yes Yes Yes

(registered) No

Processing Required Yes Yes Yes No No Yes Yes Yes

Patching Ability No Yes Yes No No Yes No Yes

Other Functionality Yes Yes Yes Yes No Yes No No

It can be seen from the above table that Landserf is the most flexible cost-free filling utility among all the tested packages, as it is meant to perform as a functional free GIS platform.


OBSERVATIONS AND DISCUSSION

While the point of the paper is not to determine, in absolute terms, the value of one application over another, statistically and visually the results obtained from contours in ArcInfo were the best amongst the solutions that filled voids solely via interpolation. However, it was also the most time consuming and required the most knowledge and understanding to execute. It is also not freely available to use without purchasing a costly license. Just as there are many applications to fill SRTM voids, there are many more intended uses for filled SRTM data. Less accurate results may be an acceptable compromise. Many of the free solutions filled average sized voids in moderate terrain fairly well with little need for pre-processing of the finished data. There were no discrepancies in this regard. One of the bonuses of algorithms that use contours to interpolate void regions (Ex. ArcInfo) is the ability to produce aesthetically pleasing terrain in areas with a great deal of slope (dense contours). None of the applications provided the ability to ignore large voids greater than a certain number of pixels to avoid poor interpolation results. Such a feature would greatly reduce the risk in filling voids inadequately.

A number of the solutions employ a mix of interpolation methods and replace null values with elevations found in alternative datasets. However, the results can only be as good as the input source data intended to fill the void areas. In many cases, good source data is difficult to obtain due to lack of availability. Input data types for most applications are also limited to a small number of sources, so in the case of a better alternative, much processing would be involved to format the data for the filling process. Besides quality of the source data for filling, is the issue of filling finished (edited to DTED® level 2 specification) SRTM data with data that has not been subjected to the same editing rules and standards. This defeats the purpose of having obtained a consistently edited dataset. Again, this risk depends on the intended use of the filled dataset. Finished DTED® products have been subjected to a number of production rules and guidelines developed by the National Geospatial-Intelligence Agency (NGA) to ensure consistent editing. Moreover, finished SRTM data includes water bodies that have been identified, delineated and fixed to a determined elevation (Slater, 2005).

In most cases it is desirable to achieve a filled dataset that is statistically accurate, topologically correct and natural looking. While the results of the free void filling solutions did not satisfy all of these requirements, they all filled small voids in moderate terrain in an adequate manner.

SUMMARY AND CONCLUSION

This paper evaluated the performance of commercially available and freeware software solutions that claim to be capable of filling voids in SRTM data. A statistical/visual analysis of the performance of each void fill solution was conducted. Practicality, ease of use and flexibility of the solutions were also addressed. From this study, it can be concluded that:

Many freeware void fill solution packages do not effectively handle finished SRTM DTED® data without significant levels of pre-processing.

Most quick “Nearest Neighbor” algorithms fill large voids in a manner that is neither statistically accurate, nor visually appealing.

Solutions that create surfaces from contours or more complex algorithms fill most voids in a manner that is more or less statistically accurate and visually appealing.

Filling voids via interpolation or using other source data (not edited to DTED® specifications) can compromise the overall accuracy and consistency of finished SRTM data.

The ideal void filling solution would directly accept finished DTED®1, DTED®2 and various other input formats such as SRTM Height and or files of varying sizes or tiles that have been mosaicked/edited in any way. The ideal solution would also effectively fill small voids to a desired level of precision and flag large void areas to be filled with other sources of elevation data where available. The ability to batch process data is important. This level of functionality would be difficult to find in a freely available solution. Depending on the intended use of filled and finished SRTM data, as well as the size of the voids in the input data, free void filling solutions are a viable alternative to expensive GIS software packages for this task. The effectiveness of such solutions greatly increases when the missing values can be obtained from another dataset of similar quality, especially in rough terrain.


REFERENCES

Hutchinson, M. (1989). A New Procedure for Gridding Elevation and Streamline Data with Automatic Removal of Spurious Pits. Journal of Hydrology 106: 211-232

Kuuskivi, T., X. Li, J. Lock (2005). Void Fill of Elevation Data: Performance Evaluations. In: Geospatial Goes Global: From your Neighborhood to the Whole Planet, ASPRS Spring Conference, March 7 – 11, 2005 Baltimore, Maryland

Slater, J. (2005). The Generation and Dissemination of “Finished SRTM Data Products by NGA. In: The Shuttle Radar Topography Mission – Data Validation and Applications Workshop. June 14-16 2005 Reston, Virginia http://edc.usgs.gov/conferences/SRTM/slater.pdf

Smith, W.H.F, P. Wessel (1990). Gridding with Continuous Curvature Splines in Tension. Geophysics 55(3): 293-205

Rodriguez, E. (2005). A Global Assessment of SRTM Accuracy. In: The Shuttle Radar Topography Mission – Data Validation and Applications Workshop. June 14-16 2005 Reston, Virginia http://edc.usgs.gov/conferences/SRTM/Rodriguez.pdf

Online Linkages ANUDEM http://cres.anu.edu.au/outputs/anudem.php (accessed on August 23, 2005) DG ADVANCED http://www.dgadv.com/dgvk (accessed on August 27, 2005) Intermap Store https://istore.intermap.com (accessed on August 19, 2005) Jet Propulsion Laboratory (JPL, 2005) http://www2.jpl.nasa.gov/srtm/statistics.html (accessed on August 27, 2005) Landserf http://www.landserf.org (accessed on August 19, 2005) National Geodetic Survey http://www.ngs.noaa.gov/GPS/GPS.html (accessed on August 19, 2005) SOI City http://www.soi.city.ac.uk/~jwo/landserf/landserf220/doc/howto/srtm.html (accessed on August 23, 2005) TERRAINMAP (2005) http://www.terrainmap.com (accessed on August 23, 2005) Visualization Software (3DEM) www.visualizationsoftware.com (accessed on August 19, 2005) Visual Terrain Project (2005) http://www.vterrain.org/Elevation/srtm.html (accessed on August 19, 2005) Personal Exchange Discoe, Ben (2005) Written communication ([email protected]) Gil, David (2005) Written communication ([email protected]) Hanson, Chris (2005) Written communication ([email protected]) Horne, Richard (2005) Written communication ([email protected])


http://edc.usgs.gov/conferences/SRTM/slater.pdf

http://edc.usgs.gov/conferences/SRTM/Rodriguez.pdf

http://cres.anu.edu.au/outputs/anudem.php

http://www.dgadv.com/dgvk

https://istore.intermap.com/

http://www2.jpl.nasa.gov/srtm/statistics.html

http://www.landserf.org/

http://www.ngs.noaa.gov/GPS/GPS.html

http://www.soi.city.ac.uk/%7Ejwo/landserf/landserf220/doc/howto/srtm.html

http://www.terrainmap.com/

http://www.visualizationsoftware.com/

http://www.vterrain.org/Elevation/srtm.html

Documents

EVALUATION OF DIFFERENT SOLUTIONS TO FILL · PDF fileEVALUATION OF DIFFERENT SOLUTIONS TO FILL VOIDS IN SRTM ELEVATION DATASETS Megan Delaney, GIS Data Technician Trina Kuuskivi, Production