Embed Size (px)
Citation preview
An Efficient Automatic Geo-registration Technique for High Resolution Spaceborne SAR Image Fu-
sion
IGARSS 2011
28/July 2011
Woo-Kyung Lee and A.R. Kim
Korea Aerospace Univer-sity
Motivation
Simple approach to the SAR image registration and fusion
- As the resolution level improves, * the unique feature of the radar imaging becomes prominent and the task of image fusion with optical image becomes complicated, * the number of pixels increases and the amount of resources for calculation such as memory and time consumption escalates exponentially.
To relieve the burden of the work and make it done in real time.
Efficient image matching in both rural and urban regions
One clickLet the machine do the rest of the job in almost real time
SAR Sensor and Geometric Characteristic
Side-looking Observation
- Image processing depends on the surface characteristics and structures - Radar images suffer from unrealistic distortions
- Non-linear distortions along range, Shortening from shadow region
System error
- Inaccurate Doppler parameter estimation leads to geocoding er-rors
- Unstability in internal system clock and orbit parameters
Image acquisition
SAR vs. optics images
Error correction methodError correction method
Effect of ErrorEffect of ErrorError SourceError Source
SAR Sensor and Geometric Characteristic
Source of SAR geocoding errors
Platform - Image Orientation Error - Squint Angle - Doppler Centroid
Platform - Image Orientation Error - Squint Angle - Doppler Centroid
Earth
- Earth Rotation
- Side-looking
- Target Height
Earth
- Earth Rotation
- Side-looking
- Target Height
SAR sensor
- Electronic Time Delay
- Slant Range Error
- Incidence Angle Estima-tion
- PRF Fluctuation
SAR sensor
- Electronic Time Delay
- Slant Range Error
- Incidence Angle Estima-tion
- PRF Fluctuation
Platform - Inclination Angle - Yaw Angel Error - Pitch Angle Error
Platform - Inclination Angle - Yaw Angel Error - Pitch Angle Error
- Geometric Calibration
- Deskew
- Ground Projection
- Image Rotation
- Terrain Correction
- Geometric Calibration
- Deskew
- Ground Projection
- Image Rotation
- Terrain Correction
Earth - Azimuth Skew - Range Non-Linearity - Foreshortening, Layover, Shadowing
Earth - Azimuth Skew - Range Non-Linearity - Foreshortening, Layover, Shadowing
Effect of Error
- Range Location
- Range Scale
- Azimuth Scale
Effect of Error
- Range Location
- Range Scale
- Azimuth Scale
SAR Sensor and Geometric Characteristic
Geometrical distortions in SAR images
- Mismatch between SAR and Optical images
Optics SAR
(a) Azimuth Distortion
(b) Non linear Range Error (c) Deskew
SAR Geo-correction with satellite internal data
Ground projection example
- Slant range function has non linear scale variations
Azi
muth
RangeSland based
Ground Based
Slant range image
Ground range image
Reference image(EO image)
- Discrepancy exist compared with the reference image- Distortion between SAR and EO are case-sensitive
GCP based geometry registration
Basic Principle
- A reference image is chosen to be used for geometric correction or fusion
- Multiple GCP(Ground Center Point)s are selected and directly ap-plied to individual position error calculation and correction i
- Based on the selected GCPs, image transforml function is charac-terized that best describes the discrepancy between the images
- Original image is re-sampled and re-arranged by the generated transform function
- In general, distinctive features such as road, building, bridges, re-flectors are chosen that are easily discriminated for convenience
- Manually? Or Automatically?? Who will chose what points??
Choice of GCP
- To perform geometrical calibration and restore distortion, the GCPs in the SAR images would be re-arranged into the true ground positions
- It becomes most essential to pick up the best candidates of GCPs
Selection of GCPs within SAR images
Difficulty of GCP choice
- Speckle noise inherence in SAR image makes it difficult to guar-antee to pinpoint precise positions that correspond to the refer-ence points.
- A human work of manual GCP selection is never reliable - The number of available points are case-sensitive and still lim-
ited by the existence of the distinctive features- The precision of the GCP location is not fully guaranteed and the
error variance may increase in coarse resolution images.
Optical image GCP SAR image GCP
Methodology
SURF algorithm
- Speeded-Up Robust Features (SURF) is known to have highly ro-bust performance
- Scale, rotation and illumination-invariant feature descriptor.- Adaptive for noisy environment and mutll-scale images- Only summing operation is involved in producing integral image to match the scale and calculation is speeded up
Selection of GCP and matching
- Selection of matching points(GCP) are performed using feature vectors described by Hessian Matrix
- The size of the constructed Hessian matix can be varied and can be increased to multiple dimensions as desired
- The number of dimensions is limited by the complexity, time con-sumption and precision of the image matching.- case sensitive
- Parameters required for the decision algorithms is set intuitively- This work is motivated to find the decision parameters automati-
cally compromising the performance and the complexity
Block diagram for GCP pair selection
Integral image generation
- For the given image, an integral set of points are summed together- The size is variable depending on the scale and complexity of the image
- Simple summations of intensity levels are performed over two di-mensional domain
: A +B + C + D
GCP candidate generation
- The Hessian matrix , corresponding to each pixel, is simplified by summation with adjacent cells
- The image scale is varied and the simplified Hessian matrix is ob-tained for each scale space
- Harr-wavelet responses are calculated and the feature descriptor is generated
- The polarity of the image intensity variation is investigated and stored
X direction Y direction X, Y direction
Harr wavelet
GCP selection
Principle
- Find a pair of feature descriptors that best fit to each other- One-by-one comparison is straightforward but time-consuming
and does not guarantee successful matching due to increased ambiguity- Construct a look-up table for the feature descriptor
- Each feature descriptor is indexed depending on their size, variation rate, orientation
- For a given GCP , a “search process” is performed within other look-up table generated from reference image and the best matching pair is selected
- Nearest neighbor search is adopted to find the correct matching pair
- Among the selected GCPs, the Euclidian distance (x, y) – (x’, y’) are estimated to find the nearest points with similar feature descriptor- The rates of intensity variations along orientations (denoted as A and B) are
considered as weighting factors- Distance multiplication is performed
- The number of orientation can be increased in order to reduce ambiguity and avoid wrong decision.
- Appropriate threshold level is required to compare with the distance multipli-cation and make a decision
- The GCP match is confirmed when the distance multiplication is less than the threshold level
- Image Projective Transform function is deduced from the matching GCPs
GCP pair selection
Define threshold level
Overall procedure diagram for image matching
지상 기준점 선정 지상 기준점 정합 기하 보정
GCP extraction demonstration
GCP extraction from SAR images
(a) Stripmap image (b) Scan image
- ScanSAR image is exposed to higher noise level and composed of extended resolution cells.
- GCP candidates are extracted from both images using the same Hessian matrix structure
- The number of GCP points appear to be close to each other de-spite the gap in the image quality
GCP number
SURF
a 1870
b 1667
Geo-registration demonstration
- Original SAR image is corrected using GCP matching and transformation - Strip mode SAR image over Vancouver, Canada is geo-registered using
the reference image in Radarsat-1 SSG format- The threshold level is set to be zero for convenience
Raw Reference
GCP # vs. Threshold level Time consumption vs Th. Level
GCP selection for raw image
GCP selection for reference image
881
557
912
544
GCP selection
GCP # vs. Threshold level Time consumption vs Th. Level
Image Matching Demonstration
- Original image is geo-corrected
Fusion image
Measure of registration errors
- RMSE(Root Mean Square Error) is calculated for the selected GCPs
Corrected Reference
Displacement Error in pixel
x y
RMSE (x, y) 0.63 0.8
RMSE
(average)1.02
The average deviation is about one point pixel size
Application to higher resolution images
- Reference image of TerraSAR in EEC format, Toronto,- The number of GCP increases consistently when the level of
correlation between the two images are high
Original Reference
GCP selection
1595
252
2680
404
1.72
0.81
3.21
1.47GCP variation rate
As the similarity of the images are high, the GCP in-creases consis-tently as the “Threshold Level” decreases
After fusion
Image Matching Demonstration
Mismatch Error Estimate
The average position error is less than one pixel
The performance of the matched GCP selection is affected by the image resolution
Mismatch error is reduced as the image resolution improves
Corrected Reference
Displacement Error in pixel
x y
RMSE (x, y) 0.7 0.32
RMSE
(average)0.77
Image fusion of SAR over EO
Number of GCPs
a 375
b 1326
(a) JERS SAR image (b) LANDSAT EO image
- This case is where SAR image constitute a small portion of the EO image
- GCPs from the two images are distinguished
- The matching GCPs are easily identified by the nearest search algorithm
Automated threshold level selection
- Threshold variance
Threshold 500, Matching image (14 points)
Threshold 350, Matching image (15 points) Threshold 200, Matching image (15 points)
There exist a turning point, where further reduction of the threshold level stops affecting the number of GCP matching points
Computer traces the variation of the available GCP matches and find the turning point
GCP matching and Image transformation
- Matching GCP selection process stops automatically and image transform function is obtained from the selected points
Feature points extraction
Source image Reference image
x y x y
139 88 74 247
119 143 359 37
459 175 94 203
254 232 137 234
350 268 365 366
509 294 514 425
192 402 231 209
503 160 137 234
236 221 94 203
362 272 390 386
266 224 173 218
280 283 177 372
349 250 365 329
440 296 87 391
383 237 467 270
Transform equation
15 points extraction
Find equation
Result of the geo-registration
Overlaid image
- RMSE is affected by the resolution discrepancy and inherent image prop-erty. - Here it is given as 1.26 pixel
Automated Geo-registration software
- Usually, the threshold level is manually set-up by user looking into the complexity of the images and resultant fusion perfor-mance
- This procedure is replaced by compute search algorithm, where the threshold level is traced to find the turning point
- Total elapsed time is within several minutes and will be further reduced by adaptive search algorithm
Original Reference Corrected
GCP selection and matching
Fusion
Performance analysis vs. Resolution
- The number of total GCP is not affected by modes(Stripmap and Scan)
(a) Stripmap image (b) Scan image
GCP number
SURF
a 1870
b 1667
- However, the RMSE is measured as 0.94 for ScanSAR mode while it is 0.34 for stripmap mode
- It appears that the performance improves as the resolution improves
Insufficient information for SAR geometry
Limited information
- Internal data within SAR instrument fails to retrieve shadow re-gion
- There is non-linear discrepancy between slant and ground ranges - Generate Errors in geometrical coordinate
- Need external references to retrieve broken information and cor-rect errors in ground range allocations
- foreshortening, layover, shadowing
Foreshortening Layover Shadowing
Impact of the ground characteristics
- Diverse ground geometry becomes a source of matching errors- Mountain areas are severely distorted from the EO case- Need to adopt separate transform functions within the image
Coast line area Mountain area After correction
Coast line fusion Mountain area fusion
After correction
Matching Performance Comparison
- Image distortion is not compensated for by the simple GCP transforma-tion
- Need to divide blocks and adopt modified transform functions separately
Coast line
Error
X Y
RMSE (x,
y) 0.33 0.26
RMSE 0.42
Mountain
Error
X Y
RMSE (x,
y) 1.46 1.73
RMSE 2.27
Modified Transform functions
- Image is divided into blocks according to the geographical properties
Mountain Area
Errors
x y
RMSE (x,
y) 1.35 1.2
RMSE 1.8Average 1.8 RMSE error
Application of separate transform function leads to the reduction of RMSE
Urban Area
Errors
x y
RMSE (x,
y) 0.6 0.3
RMSE 0.67Average 0.67 RMSE Error
Summary
- SURF provides an efficient tool to perform SAR image geo-registration- A choice of threshold level is required to perform efficient of GCP matching
and it can be automated by tracing its variation curve- The image matching algorithm works with various SAR and EO images and
the average RMSE is measured to be around 1 pixel.
- Image blocks containing mountain areas need separate GCP matching and transform function to compensate for image distortion
- Need to develop optimized transfer function for different type of ground char-acteristics
- Indexing of GCP is performed based on their intensity and variation vector - With the introduction of adaptive indexing table for selected GCP, the auto-
mated image matching is expected to be carried out in real time-
Conclusion
Further work
