EPIPOLAR RESAMPLING OF HIGH RESOLUTION SATELLITE IMAGERY

Tetsu Ono, Graduate School of Engineering, Kyoto University, Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501, JAPAN,

ono@jf.gee.kyoto-u.ac.jp

ABSTRACT

This paper presents a practical method of epipolar resampling of high-resolution satellite imagery. Satelliteimagery imaged with a linear array CCD sensor has quite different geometric characteristics from aerialphotographs, therefore the conventional method of epipolar resampling is not applicable to it. On the other hand,epipolar resampling method based on rigorous orientation model with high geometric fidelity becomes toocomplicated and then is not suitable practical use. In order to overcome this problem, the author proposes toapply the well-established 2D affine orientation model to the epipolar resampling of satellite imagery. Firstly,this paper roughly mentions the characteristics of this model. Then it is shown how the model is suitablyapplicable to epipolar resampling. Secondly, the paper proposes the improved method that does not require anyDTM or rigorous geometric parameters for reduction of vertical parallaxes. Finally an experiment validates theproposed method with SPOT imagery, where the RMS value of vertical parallaxes between a pair of stereoepipolar images was less than half a pixel.

KEYWORDS: Epipolar Resampling, High-Resolution Satellite Imagery, CCD line scanner imagery, 2D AffineOrientation Model, Affine Transformation

1. INTRODUCTION

High-resolution satellite imagery is expected to be amajor source of 3D measurement of ground in thenear future. Especially automatic stereo plotting togenerate DTMs by stereo matching technology ishighly required. However, the projection of satelliteimagery, which is imaged with a CCD line sensor, isquite different from that of conventional aerialphotographs. This leads to failure of application ofwell-known epipolar geometry. CCD line scannerimagery is not characterized by rigorous threedimentional perspective projection in a solid frame,but two-dimensional sequential perspective projectionin a line. It is already reported that strict epipolarimages cannot be generated from SPOT imagerywithout DTMs (Otto, 1988). The same applies tohigh-resolution satellite imagery with CCD linescanner. For this reason, several procedures which generatepseudo epipolar image by using DTMs have beenpresented.

1. In order to generate coarse pseudo epipolar image,each pixel of satellite image is projected on ahorizontal plane located at an average terrain

height. A pair of the pseudo epipolar images hasstill large vertical parallaxes (Haala, 1998). IfDTMs of the corresponding area exist, thesatellite images are projected onto the DTMs andreprojected onto a new image plane along epipolarline (ONeill et al, 1988). The idea of this methodis very simple and applicable to every orientationmodel. The procedures, however, are complicatedfor practical purposes.

2. The epipolar images are resampled under theassumption that all time-variant factors linearlyaffect to satellite image (Otto, 1988). Therelationship between height differences and valueof vertical parallaxes can be described by anequation. This procedure is not quite complicatedas far as non-linear effects are not considered.

Both of the procedures cannot achieve practicalaccuracy without highly precise orientationparameters. In this study, the author proposes analternate method to generate epipolar imagery, whichis very simple, highly accurate and does not requirethe rigorous orientation parameters nor DTMs.

2. GEOMETRIC CHARACTERISTICS OFHIGH-RESOLUTION SATELLITE IMAGERY

In comparison with mid resolution (5m-10m on theground) satellite imagery such as SPOT, 1m high-resolution satellite imagery has a much narrower fieldangle. This means that the projection of images isnearly approximated by parallel rather than centralone. If conventional orientation parameters are used,very high correlation between them occurs. A high-resolution satellite image covers a very smallground area in single scene, which is imaged in ashort time span on orbit. The movement of thesatellite can be approximately expressed by linearfunction and its attitude parameters are expected to bealmost constant during the short period.

3. 2D AFFINE ORIENTATION MODEL 3.1 The Basic Equations Okamoto (1999) proposed the orientation theory ofCCD line sensor imagery based on 2D affineprojection. The model named 2D affine orientationmodel can be derived from conventional collinearityequation by consideration of situation mentioned inthe previous section. Each line of an image is imaged by one-dimensionalcentral perspective, and each has different exteriororientation. Let exterior orientation parameters forline number i be expressed by coordinates of theprojection center Xoi, Yoi, Zoi and angles i, i, i.These parameters are time variant. Many studies haveindicated that the satellite sensor geometry can bemodeled by elliptical orbit and in this case its attitudeparameters can be expressed with polynomials. Theeffect of earth rotation and earth curvature must beconsidered. The collinearity equation is described as:

( )

=

oi

oi

oiT

iii

ZZYYXX

RRRc

y 0

(1)

where (X,Y,Z) is the ground coordinates of an objectpoint, is scale parameter, c is principal distance, y iscoordinate of image point and Ri, Ri, Ri are rotationmatrixes. Now that the scene is projected to the image byparallel projection, c can be set to infinity. The third

equation in Equation 1 loses the meaning and theequation can be described as follows:

0 = a11(X-Xoi) + a12(Y-Yoi) + a13(Z-Zoi) (2) y = a22(X-Xoi) + a23(Y-Yoi) + a23(Z-Zoi) (3)

where aij (i=1,2; j=1,2,3) are elements of the matrix(RiRiRi) T . We assume further that the sensormoves linearly in space and the attitude does notchange. The projection center in each line isdescribed as follows:

Xoi = Xo + X i

With Xo and X being constant value. The similarexpressions are defined likewise for Yoi and Zoi. Linenumber i is expressed by Equation 2 and these ones.

ZaYaXa

ZZaYYaXXai ooo

++++

=

131211

131211 )()()( (4)

Now, line number i can be replaced by imagecoordinate x. Assuming that the attitude does notchange, aij are regarded as constant parameters.Equation 4 arranged for the constant coefficients isdescribed by algebraic expression.

4321 AZAYAXAx +++= (5)

Equation 3 is also expressed by similar arrangement.

8765 AZAYAXAy +++= (6)

where Ai (i=1,,8 ) are independent coefficients.Equation 5 and 6 describe the collinearity relationshipbetween the coordinates (x,y) of 2D affine image andground coordinates (X,Y,Z).

3.2 Image Transformation

In reality, satellite images are taken central-perspectively in scanning direction. For rigorousanalysis, the affine image coordinate y must betransformed to the corresponding original imagecoordinate yp. The relationship between yp and y atplain field is given in the form (Okamoto et al, 1992)

)/)(tan1/( cyyy pp = (7)

Fig. 1. Transformation of a central-perspective lineimage into an affine one

However, at hilly area or mountainous area, wecannot ignore the image transformation errors due toheight differences in the terrain. Let Z indicateheight difference of a ground point from the averageheight and denote the half of the field angle of thescanner. The image transformation error y due toneglecting the height difference Z is described asfollows (Okamoto et al, 1992):

cos)tan)(tan( += Zy (8)

Fig. 2. Image Transformation Error Due toNeglecting Height Difference in the Terrain

3.3 Characteristics of 2D Affine Orientation Model

2D affine orientation model has only 8 algebraicparameters and the basic equations are linear withrespect to the object space coordinates. Therefore, itis very simple, stable and fast for mapping processing.

The model based on affine projection can expressany liner movement and distortion relating to theimages. Although the model is derived under theassumption that the attitude of sensor does not changeduring the acquisition of one scene image, smallchanges of attitude parameters can be also estimatedby the model as far as the effects are regarded asapproximately linear. Under the geometricallypeculiar condition of high-resolution satellite images,the attitude parameters highly correlate to themovement parameters. For example, small changes of are very similar to changes of Zo. Small changes of are almost same as changes of Xo and Yo. Becausethe attitude of satellite is stable, the effects can beembodied as linear movement or distortion in theaffine images. The effects of earth curvature andearth rotation are also estimated with them in smallarea. Moreover, this model is capable of geometricallypreprocessed images (e.g., SPOT Level-1B), becausethe model does not treat the geometric orientationparameters directly and then the model allowsdeformed images as far as the deformations are linear.The model is applicable for even the rotated images orflipped images. This characteristic is very importantfor the epipolar resampling method in this study.

4. THE PRIMARY THEORY OF EPIPOLARRESAMPLING FOR AFFINE IMAGES

4.1 Epipolar Geometry of Affine Imagery

The parallel projection can be expressed as thecentral perspective projection with infinity focallength. Accordingly, we can say that the affineprojection is a special case of the central perspectiveprojection. For this reason, epipolar geometry of theaffine images is considered by same approach as thatof the central perspective projection images. Thewell-known epipolar geometry is illustrated by Fig. 3.The epipolar plane is defined as a plane in which theprojection center of left image, that of right imagesand an object point lie. The projection center of eachimage is only one, thus the epipolar plane isdetermined by each object point location. On the contrary, the affine image has no projectioncenter. In the affine image the direction of projectedray is same for any point on the image. Nowconsidering a ray projected from an object point to anaffine image, the epipolar plane can be defined as aplane in which the rays of the two images lie. The

epipolar line is an intersection line between the affineimage and the epipolar plane. In order to generate theepipolar images, each affine image can be projected toa same virtual plane by parallel projection (Fig. 4, Fig.5). Let the virtual plane be a plane parallel to the rightimage. The image projected on the plane from rightimage is identical to the right image. On thecondition, projecting the left image to the virtual planeis just same as processing affine transformation fromthe left image to the right image. The relationshipbetween the left image point coordinates (xl, yl) andthe corresponding right image point coordinates (xr,yr) is simply described in the following form:

654

321

KyKxKyKyKxKx

rrl

rrl

++=

++= (9)

where Ki (i = 1,,6) are independent coefficients.Number of unknown parameters is 6 and number ofindependent equations is 2. Thus, if more than 3known points are given, the equations can be solved. Another question is how to determine the directionof the epipolar line. For this purpose, we shallconsider algebraic solutions of epipolar line of affineimages. As the author has mentioned before, the basicequations of affine images are described by equation 5and 6. These equations are written down for a stereopair of affine images in the form:

rrrrr

rrrlrr

lllll

lllll

AZAYAXAyAZAYAXAxAZAYAXAyAZAYAXAx

8765

432

8765

4321

+++=

+++=

+++=

+++=

(10)

By eliminating X and Y from these equations andrearranging them, the relationship of the left imagepoint and the right image point with change an objectheight Z is described as follows:

8765

4321

BZByBxByBZByBxBx

llr

llr

+++=

+++= (11)

By eliminating Z from these equations, the epipolarline of affine images is expressed in the form:

Fig. 3. Well-Known Epipolar Geometry

Fig. 4. Epipolar Geometry of Affine Imagery

Fig. 5. Epipolar Resampling fromAffine Imagery

4321 CxCyCxCy llrr +++= (12)

Since number of unknown coefficients is 4 inEquation 12, more than 4 known identical pointscoordinates between left image and right image arerequired for the solution of this equation. Theepipolar resampled images can be generated byrotating the virtual plane images by the anglecorresponded to C1. Finally, in order to generateepipolar images from affine images, all we need toknow is the coordinates of more than 4 identicalpoints of the left and right affine images. Theinformation such as the attitude of the images is notrequired at all.

4.2 Application to satellite imagery

Since the actual satellite images are not affine ones,the images should be approximately transformed intoaffine ones by Equation 7 and 8. Equation 8 indicatesthat DTMs are required for rigorous transformation.As we shall see later, however, we do not have tonecessarily use DTMs for the purpose of epipolarresampling. The transformation error y in scanningdirection causes vertical parallax x (Fig. 6). In orderto avoid this problem, the transformation should becarried out along the epipolar lines instead of thescanning lines. Although it is very hard to find thetrue epipolar lines on the original image, the directionof the epipolar line corresponding to the affine imagescan be determined easily. The procedures of epipolarresampling of satellite images are as follows.

1. Approximately, transform the original images tothe affine ones along scanning line

2. Determine the direction of the approximateepipolar lines on the affine image

3. Transform the original images to the affineimages along the approximate epipolar lines

4. Determine the affine transformation coefficients(Equation 9)

5. Carry out the affine transformation6. Determine the direction of the accurate epipolar

line7. Rotate the affine transformed images by angle of

the epipolar line.

Fig. 6. Vertical Parallax due to TransformationError into Affine Images

The epipolar images are generated from the affineimages by using affine transformation. The epipolarimage, that is, also can be treated as another affineimage. As mentioned in the previous section,therefore, 2D affine orientation model can be directlyapplicable to the epipolar images. The relationshipbetween ground coordinates (X, Y, Z) of an objectpoint and image coordinates (xe, ye) of thecorresponding point on epipolar image are describedby same expression as Equation 5 and 6.

8765

4321

DZDYDXDyDZDYDXDx

e

e

+++=

+++= (13)

The coefficients Di ( i = 1,,8) are determined by theleast squares method with more than 4 ground controlpoints data. Since the basic equations are very simple,this method is appropriate for real time mapping ofsatellite imagery.

5. PRACTICAL EVALUATION

5.1 Test Field and Images

Since unfortunately 1m high-resolution satelliteimages were not available, the author used a stereopair of SPOT images in order to investigate thecharacteristics of this method. Table-1 shows thecondition of the test images. The stereo scene coversHanshin area (Osaka, Kobe and the suburbs) inJAPAN. The southern area of test field is city areaand almost flat. The northern area and the westernarea are mountainous terrain. The maximum heightdifference is about 1,000m. For the purpose ofverification, 141 check points were measured bymanual. The measurement accuracy of these points is1/2 pixel to 1/4 pixel. 9 points among the checkpoints are used for determination of coefficients inbasic equations (Equation 13). Fig.7 shows the testimages and the distribution of the check points.

Table-1 Test Image DataLeft Image Right Image

Image type SPOT pan Level-1ADate 1996.11 1995.2Lat./Long. N34.7/E135.5 N32.7/E135.2Incident angle L23.0 R17.9B/H 0.75

5.2 Results and Discussion

In this study, four different approaches for epipolarresampling were evaluated.

1. Ottos approach which uses geometric orientationparameters

2. Proposed approach, but direction of epipolar lineis not considered.

3. Proposed approach, but DTMs is used attransformation into affine images.

4. Proposed approach

The results obtained by these approaches are shownin Table-2. The accuracy of Ottos approach dependson that of orientation parameters. In this study,Equation 1 was used as the collinearity equations fordetermination of the orientation parameters. Thechanges of the parameters were assumed to be linear,because non-linear model isnt appropriate for Ottosapproach. The RMSE in X,Y of the orientation was5.1m and that in Z was 6.7m. This result was very

good. But, the vertical parallax by this approach wascomparatively large. It is likely that the geometricparameters were not gotten accuracy enough.

Left Image

Right ImageFig. 7. Test Images and Check Points

Black points: check points White points: control points

On the contrary, it seems that the proposed approachworked efficiently. The different between theaccuracy of 2nd approach and that of 4th approachindicates that consideration of epipolar line direction

is effective for reduction of transformation error dueto neglecting height differences. Besides, theproposed approach without DTMs is no less accuratethan the case using DTMs. From these results, it canbe concluded that epipolar resampling of satelliteimagery in practical accuracy can achieve withoutDTMs by using the proposed method. For the reference of discussion, the results of theorientation with 2D affine orientation model areshown in Table-3. We can see that application of the2D affine orientation model is quite adequate for theepipolar images.

Table-2 Results of Each Approach (pixel)1 2 3 4

RMS ofv-parallax 0.615 0.562 0.460 0.469

Table-3 Results of Orientation (m)Original Images Epipolar Images

o (x 10 8 ) 5.92 4.92RMSE in X,Y 5.814 5.996RMSE in Z 7.951 7.671

6. CONCLUSIONS

The epipolar resampling approach presented in thispaper is based on affine projection imagery. Thismethod is, therefore, appropriate for small areamapping with almost parallelly projected imagerysuch as high-resolution satellite imagery. Theproposed method does not require DTMs or rigorousgeometric orientation parameters. In the practicalexperiments, it was shown that accuracy better thanhalf a pixel was achieved by the proposed approach.Furthermore, since the resampling process isindependent from the orientation process and the basicequations of the orientation are very simple, thismethod is appropriate for real time mapping.

ACKNOWLEDGEMENTS

The author would like to dedicate this paper to thelate Dr. A. Okamoto, who had built the fundation ofthis method and had given many advices. The authoralso wishes to thank to Dr. S.Hattori and Mr. H.Hasegawa for their assistance with the preparation ofthis paper.

REFERENCES

1. G. P. Otto, 1988. Rectification of SPOT Data forStereo Image Matching, International Archives ofPhotogrammetry and Remote Sensing, Vol.27,B3, pp.635-645.

2. M. A. ONeill and I. J. Dowman, 1988. TheGeneration of Epipolar Synthetic Stereo Mates forSPOT Images Using a DEM, InternationalArchives of Photogrammetry and RemoteSensing, Vol.27, B3, pp.587-598.

3. Norbert Haala, Dirk Stallmann and ChristianSttter, 1998. On The Use of Multispectral andStereo Data from Airborne Scanning Systems forDTM Generation and Landuse Classification,Internationl Archives of Photogrammetry andRemote Sensing, Vol. 32, B4, pp.204-210

4. T. Ono, A. Okamoto, S. Hattori, H. Hasegawa,1996. Fundamental Analytic of Satellite CCDCamera Imagery Using Affine Transformation,International Archives of Photogrammetry andRemote Sensing, Vol.31, Commision III, pp.611-615.

5. A. Okamoto, C. Fraser, S. Hattori et al, 1998. AnAlternative Approach to the Triangulation ofSPOT Imagery, International Archives ofPhotogrammetry and Remote Sensing, Vol.32,B4, pp.457-462.

6. A. Okamoto, T. Ono, S. Akamatsu et al, 1999.Geometric Characteristics of AlternativeTriangulaion Models for Satellite Imagery,Proceedings of ASPRS 1999 Annual Conference.

7. A. Okamoto, S. Akamatsu, H. Hasegawa, 1992.Orientation Theory for Satellite CCD Line-Scanner Imagery of Hilly Terrains, InternationalArchives of Photogrammetry and RemoteSensing, Vol.29, Commission II, pp.217-222.

8. O. Hofmann, 1986. Dynamische Photo-grammetrie. BuL, Vol.54(5), 105-121.

9. V. Kratky, 1989. On-Line Aspects of Stereo-photogrammetric Processing of SPOT Images,Photogrammetric Engineering & Remote Sensing,Vol,55(3), 311-316.

ABSTRACTTable-2 Results of Each Approach (pixel)Table-3 Results of Orientation (m)6. CONCLUSIONS