Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Digital Fictitious Datafor Automatic Mapping Research *
]. D. NEWTON and H. F. DODGEt
Graphic Data Redudion Dept.IBM, Kingston, New York
(Abstract on page 169)
INTRODUCTION
T HE purpose of this paper is to describe anIBM 7094 Data Processing System pro
gram designed to prod uce fictitious photographic data. Such data are now being used asan important tool in carrying out research indigital automatic mapping at IBM.
In this research, an independent effort isbeing made to produce and verify techniquesfor automatically generating contoured ortho-
known. The fictitious terrain data are composed of both the gray-scale val ue and theexact geographic position (X, Y, Z) forground points as close together horizontallyas 0.1 foot. From such terrain data and fromknowledge of exact position, attitude, andprincipal distance of the camera, values ofwhich are arbitrarily chosen, fictitious photographic data with points as close to eachother as one micron can be obtained. Allphoto coordinate data are accurate within the
A stereoscopic pair of "photographs" of a hypothetical terrainmodel can be simulated, stored on magnetic tape, and printed forvisual inspection.
photographs from control-annotated stereodiapositives, ground control data, cameracalibration information, exposure stationposition estimates, and output specifications.Current activity is largely concerned withfinding a solution to the problem of automatically associating corresponding images onthe several photos of the strip or block onwhich the images appear. Another objectiveis to establish specifications for practicalspecial purpose equipment which will carryout orthophoto compilation automaticallyat high speed. One approach to an automaticmapping system is described in [1].
PURPOSE OF FICTITIOUS DATA
The function of the fictitious data is toestablish strong technical control over thecorrelation research. Digital means have beendevised for generating fictitious terrain andphotographic data about which everything is
limits of round-off error. No presently available electronic scanning equipment can extract such error-free photo data. Moreover, itis impractical to attempt to obtain accurategeodetic positions for such a multitude ofground points in a real photo model.
The primary use of such error-free data is tofacilitate the evaluation of the correlationprogram and to establish various decisionmaking criteria in the correlation process.
The main function of the fictitious photogeneration process is to introduce stereoscopicor binocular parallax into two photographs.I n human stereoscopic vision, binocularparallax is the principal depth cue [2]. All otherdepth cues, binocular or monocular, such aslinear perspective, monocular movementparallax, shadow patterns, interposition, etc..are of much less importance and merely serve
* Presented at 30th Annual Meeting of the Society, Hotel Shoreham, Washington, D. c., March17-20, 1964.
t Mr. Dodge is now Principal Engineer, Recon. & Mapping Dept., Aeronutronics Div., Ford MotorCo., Newport Beach, Calif.
166
GENERATING DIGITAL FICTITIOUS STEREOGRAMS
PHOTO [Md °2
PLANES ~p-ff-~_
FICTITIOUSMODEL
FIG. 1. Geometry of projection of model element to photo-planes. (Text on page 168.)
167
to help stereoscopic vision. The use of arandom gray-scale pattern in a pair of fictitious stereo photographs demonstrates thatstereo vision can be accomplished with onlybinocular parallax; a random gray-scalepattern is devoid of all other depth cues.
Therefore, any correlation process performed on the fictitious photos will have toutilize the property of binocular parallax inorder to derive height information. No otherdepth cues need to be present in the photos.In fact, it does not appear possible or practicalto devise a correlation process which will takeadvantage of depth cues other than binocularparallax, except in very restricted cases (e.g.,in determining the height of moon cratersfrom shadow cues on a single photograph).
If binocular parallax is the only depthindicator required in fictitious photographs,it is clear that the gray-scale pattern need notcorrespond to the elevation data in the groundterrain model in order to establish stereoscopic vision. This correspondence is likewisenot required for curren t photo data correlation processes which make use of only thebinocular parallax property.
Realistic errors can be inserted into thedata. These errors fall into two categories:(1) gray-scale; (2) geometric. Gray-scale
errors can be used to simulate noise, simulateuneven transmission of light through theatmosphere or lens, and simulate differencesin reflectivity from the ground. The use ofsuch errors tests the ability of the correlationprogram to adapt itself to normal photographic variations in film densities. Geometric errors which can be easily determined ordescribed, such as residual aerotriangulationerror and errors inherent in an electronicscanner, can also be introduced.
]. D. NEWTON
168 PHOTOGRAMMETRIC ENGINEERING
\Vhen a satisfactory correlation program isdeveloped, it is planned to introduce thesearbitrary errors into the data. For design ofan optimum mapping system, a series of testscan be run to determine the allowable errorsin the electronic gear, the required spot size,and the gray scale quantization needed forany desired final accuracy. Trade-offs of thesediA'erent factors can be investigated.
I n all these tests, the fictitious data provide the framework within which controlledexperiments can be conducted.
EVALUATION OF THE CORRELATION PROCESS
As previously mentioned, the fictitiousdata consist of ground terrain elevation data,two stereo photographs, the principal distanceof the camera, and "aerotriangulation" data(the positions and attitudes of the two cameras at the time of exposure). In the automatic mapping program, only the two fictitious stereo photographs and the principaldistance of the camera are provided as inputto the correlation process in its function ofassociating corresponding images. The quality of this process of association or correlation is checked by an evaluation routine.Evaluation proceeds as follows for each pairof correlated points:
1. The object rays are projected in accordance with the aerotriangulation data,with the camera principal-distance, andwith the photo coordinates found by thecorrelation process; in this way, theposition of the virtual ground intersection is found (i.e., horizontal and ver,tical position).
2. The ground model data are interrogatedfor the true terrain elevation at thehorizontal ground position.
3. A comparison of the true elevation withthat computed from the correlationprocess provides the basis for determining if \'ertical accuracy requirementsha ve been met.
4. The original horizontal position of thatground point corresponding to one of theobject rays, as selected by the correlation process, is obtained from theficti tious data.
5. A comparison of the original horizontalposition with that found as a result ofthe correlation process forms the basisfor determining if horizontal accuracyrequirements have been met.
6. Horizontal and vertical errors are sum,marized by the evaluation routine.
To verify the fideli ty of the test loop priorto the testing of a highly complex correlationprogram, it is expected that the loop will beclosed first by using an "a-priori correlationprogram." This program will select, from thefictitious photo data, matched pairs of pointsas they are generated. \Vhen the entire testloop has been proved, the "a-priori" correlation program will be replaced by the actualcorrelation programs to be evaluated.
MECHANICS OF DIGITAL FICTITIOUS
DATA GENERATION
The geometry involved in producing afictitious stereogram, such as those given inFigures 5, 6, 10, 11, and 13, is illustrated inFigure 1 (shown on page 167). It will beshown how, over each photo plane, a discretegray-scale picture pattern corresponding to aperspective photograph of the model can begenerated. Each photo coordinate system isgiven any desired spatial position Randattitude M. (M is an orientation matrixwhose values are direction cosines of thephotographic coordinate system taken relative to the master ground X, Y, Z coordinateframe.)
The grou nd model is an arbi trary discretesurface defined on a square grid on the X- Yplane. Elevation information is given at eachX, Y grid intersection. Figure 1 shows suchelevations for a typical square element in themodel. Such terrain data may be computedfrom a mathematical surface or interpolatedfrom contour map data for a realistic effect.Any grid interval can be chosen for the model.
The gray-scale val ues for the model may beany arbitrary density pattern; in general, thegray values are obtained by electronic scanni ng of an aerial photograph. The gray val uesare assigned on the basis of one to a groundgrid square. In the computer program, capability exists for carrying along 1 to 6 grayscale values per ground square; thus, 1 to 6different gray-scale density picture patternsmay be assigned at one time to the reliefmodel. \\"hen carrying out the stereo-projection process (see below), the 1 to 6 gray-scalevalues per ground grid square are carriedalong as a single entity. Hence, after the entire photo-plane projection process has beencompleted, 1 to 6 pairs of stereograms areobtained. These are all derived from the sameground terrain but have different picturepatterns.
GENERATING DIGIT.\!. FICTITIOUS STEREOGR.\MS 169
To generate a fictitious photograph, squareelemen ts along each row of the model areprojected in sequence by the program to thephoto plane (on which is superimposed asquare photo grid). The projection equationsused are those originated by von Gruber(1932) :
computer program for setting the photo-gridspacing (resolution) to any desired value. Ingeneral, a 1-1 ratio between the number ofphoto-grid intersections and the corresponding number of ground-grid intersections ismaintained.
:t =
y=
_ jMII(XO - Rx) + M 21 (Yo - Rl·) + M,,(Zo - R z)
M 13 (Xo - Rx) + M 23(YO - RJ·) + M 33(ZO - R z )
_ jMI2 (XO - Rx) + M 22(YO - R y) + M 32(ZO - Rz)
M 13 (XO - Rx) + M 23(YO - Ry) + M 33(Zo - Rz)
in which
x, yare the photo-coordi nates of the projected point
f is the principal distance of the camera
Because the photographic axes are notnecessarily parallel to the master coordinatesystem and because of the usual presence ofrelief in the model, a sort-and-merge routine
ABSTRACT: Techniques for generating digital fictitious stereograms on an IE ill7094 Data Processing System are described. The stereograms are obtained by aperspective projection of a ground model with known elevation and gray shadedetail upon two "photo" plcmes with known spatial position and orientation.Realistic photographic and geometrical errors can be introduced into the dataif desired. Such fictitious data are very useful for evaluation of digital automatic mapping programs now being developed and perfected at JEll!.
M is the orientation matrix of the photoplane
(Xa, Ye, Ze) is the position of the groundpoint
(Rx , Ry , Rz , components of R) is the location of the camera exposure station
The four rays from the corner elevations ofthe ground-square element form the cornersof a quadrilateral on the photo-plane as indicated in Figures 1 and 2. Next, the vertical(or horizontal) grid line intercepts (see intercept points, Figure 2) on the sides of thequadrila teral are obtained. The intersectionsof the square photo-grid within the projected quadrilateral boundary then consist ofthe points lying in between each corresponding pair of intercepts. These enclosed gridintersections (photo-points) are assigned thegray-scale density value (or values) of thecorresponding ground-square element.
The photo-grid intersections with theirassociated gray-scale densities (spots) simulate the scan-by-scan and spot-by-spot discrete sampling of a photograph by an electronic-optical scanner. The photo-grid spacing corresponds to the spot interval (or spotsize) of the scanner. Capability exists in the
is required to order the gray-scale densityvalues of the generated photo by spots in xand by scans in y. This routine takes up thebulk of computer lime in generating a fictitious stereogram.
An a-priori correlation routine is includedin the program to save the x and y coordinatesof a large number of points projected up toeach photo-plane, as corners of quadrilaterals.Matched pairs of points on the two photos arethus retained for later usc in checking outother programs in the closed simulation loop.
The computer program provides for sequential numbering of the ground gridsquares. These numbers are carried to onephoto-plane along wi th the gray-scale valuesassociated with the same ground-grid squaresthroughout the entire simulation processstereo-projection, correlation, and evaluation.In this way, a horizontal check can be madeon the correlation process. As a resul t of thecorrelation progra m, matched pairs of photopoints are obtained; each pair of the~e isassociated with a gray-scale value and with aground-grid square number corresponding toone of the photo points. The horizontal error
170 PHOTOGRAMMETRIC ENGINEERING
PHOTO GRID COORDINATE SYSTEM
FIG. 2. Assignment of photo-grid pointsto a quadrilateral.
due to correlation is then given by the difference in position between the original groundgrid square (denoted by the grid square number) and the ground-grid square containingthe position computed from the photo-toground projection of the correlated pair ofphoto points.
The fictitious stereograms are idealized inthat gray-scale density variations do not existfor corresponding regions in the two pictures-if both pictures of a pair are derived from areiief model with the same density pattern.Inreal photography, such gray-scale densitydifferences do exist because of differences inlight reflectivity on the ground and differentamounts of atmospheric haze for the twoexposure stations; in addition, the corners of aphotograph are made darker than the rest ofthe photograph because less light passesthrough the camera lens for these regions. Tosimulate real'photography, the computer program incorporates a routine to interpolate fineground-grid gray-scale density changes(± val ues) from gross grid data determined
manually. The ground grid gray-scale densitypattern may then be modified by these grayscale changes prior to the generation of afictitious photograph.
Because of electronic, optical, and mechanical errors in a scanner, linear and nonlinear x, y errors in positioning the light beamresul t. This type of error can be measured andeither entered in tabular form or approximated by 1-variable or 2-variable polynomials.
Rotation and translation errors occur whenthe photographic axes (fiducial marks) are notlocated where they are thought to be whensetti ng up the photographic plates for scanning.
In addition to these gross changes made togray-scale density, noise may also be introduced into the fictitious photo data. For thispurpose, a random-number generator routinewas devised. The muliplicative congruentialscheme in use at MIT was adopted. It produces normally distributed noise (± values)with any desired mean and variance whichbecome the changes to be made to the grayscale density pattern of a photograph.
The following geometric errors, present inany digital system, can also be introducedinto the fictitious data to determine how theyaffect output accuracy:
1. Rotation error2. Translation error3. Linear and non-linear x, y position
errors.
Aerial photographs often have featureswhich appear in one photograph but are notvisible in a second overlapping photograph.These "hidden" features occur wheneverground terrain slope is sufficient to obscurethe terrain from a camera exposure station.
~ PROJECTEDCORNERS
JlI INTERCEPT
* ENCLOSED GRIDINTERSECTION
.-/1-
~v
\ ...-'~1--'
-
.~~- - - ~
y
I( »1REGION 2
PHOTO 2-7"'---+--"2
A'
A"
NADIR
-y
+I
~-t-
+----t+---------t-+---------t-+---------t-
x
y REGION OFMODEL
FIG. 3. Hidden features. FIG. 4. Projection sequence of model elements.
GENERATING DIGITAL FICTITIOUS STEREOGRAMS 171
FIG. 5. Fictitious stereogram derived from a photo pattern superimposed on mathematical model.
FIG. 6. Fictitious stereogram (with enlarged section) derived from arandom number pattern superimposed on mathematical model.
In Figure 3, for example, region 1 is hiddenfrom photo 1 and region 2 from photo 2. Inthe region of a hidden feature, three groundterrain points (two at points of tangency)project along a single object ray to a singlepoint on the photo (e.g., points A, A', A"project to point a, Figure 3), whereas onlyone point, point A, is visible. To insure thatonly the visible points are projected onto thephoto, the computer program always projects ground-square elements in a sequencemoving inward toward the ground nadir
point (for the given photo-plane) from theoutermost corner of the model, inward inspots along a scan, then inward to the nextscan, etc. (Figure 4). In the computer program, each photo grid intersection receivesonly the last gray-scale density value that isassigned to it. I n the two dimensional ill ustrations of Figure 3, point a on photo 1 isassigned first the gray-scale value from pointA", then from A', and finally from A, whenthe projection sequence is taken from right toleft toward the nadir point N. The projection
TOP VIEW
(4) s::J['Z~--
SIDE VIEW
FIG. 7. Elevation model with mathematical surfaces used for stereogral1ls in Figures 5 and 6.
172 PHOTOGRAMMETRIC ENGINEERING
Real photography has certain propertieswhich are not simulated by the fictitious datagenerator. These are:
1. Multi-valued elevation data. In realphotography, many situations exist inwhich three elevations are required todescribe an object at a given horizontalposition. For example, detail under atree or under an overhanging roof of abuilding can be seen. The fictitious datagenerator, as designed, can only repro-
FIG. 8. Ground random number pattern forstereograms in Figures 10 and 11.
sequence therefore assures that only thevisible gray-scale density values are transferred to the photo-plane.
Surface
EllipsoidX cylinderInclined planeElliptical paraboloid
TABLE 1
Surface Numbers
1,3,5216All remaining surfaces
100' JAGGED PEAKS
90'
(14)
60'
35':>
(5)c
401
("'~--:'(:::-2):-----..:..------t...-.~~::.I..---------_4
0 1
FIG. 9. Terrain model for stereograms in Figures 10, 11, and 13.
FIG. 10. Fictitious stereogram generated from the terrain model of Figure 9.
GENERATING DIGITAL FICTITIOUS STEREOGRAMS 173
F[G. 11. Stereogram of Figure 10 modified by noise and gross gray scale changes.
duce ground terrain with:slopes of lessthan ± 90 degrees.
2. Shadow effects. With additional programming, shadows probably can beintroduced into fictitious photo data, ifdesired.
3. Individual local gray shade variation dueto differences in reflectivity. This property is simulated only crudely by the useof gross gray-scale changes and normallydistri bu ted noise.
4. Those photographic errors which cannotbe adequately defined.
A most realistic fictitious model, in whichthe photo patterns correspond to the terrai ndata, can be generated. I t would make use of adigital photo-data correlation program asfollows:
1. The correlation program is used togenerate two outputs upon correlatinga given stereo pair of real photographs;these are:(a) terrain elevation data(b) an orthophoto based on orthocorrec
tion of one of the two photographs.2. Using a terrain model made up from
these two outputs of the correlationprogram, two stereo fictitious photos aregenerated.
I t does not matter if the correlation program produces elevation data which is inaccurate with respect to the actual terrain because these data are now being defined as being"true" terrai n da ta for the ficti tious model.
A second way to obtain a realistic modelrequires a large amount of manual work. Thesteps required are:
1. Contour lines from a contour map aretransferred to one of the two photographs while viewing the diapositives instereo.
2. Fine grid elevation data are interpolatedfrom gross grid elevations extracted fromthe warped contour map overlay on thegiven photograph.
3. Using a terrain model consisting of thefine grid elevation data and the givenphotograph gray pattern, a ficti tiousstereogram is genera ted.
EXAMPLES OF FICTITIOUS STEREOGRA 'IS
Figures 5, 6, 10, II, and 13 are fictitiousstereo photographs prepared on an I B i\l7094-II computer. All are vertical photographs. All gray-scale patterns have a rangeof eight shades of gray (0 to 7).
Figures 5 and 6 are fictitious stereogramsprepared with the same terrain elevation databut with the different gray-scale patternsshown. The elevation data is composed of thesix mathematical surfaces indicated below andappearing in Figure 7, from left to right:
1. Side plane2. Elliptical paraboloid3. Ellipsoid4. Elliptical paraboloid
FIG. 12. Ground photo pattern for stereogramin Figure 13.
174 PHOTOGRAMMETRIC ENGINEERING
FIG. 13. Stereogram with photo pattern modified by noise and gross gray scale changes.
5. Elliptical cylinder rUllning up and down6. Inclined plane in upper right corner
The use of a random number pattern inFigure 6 shows that stereoscopic parallax provides the only depth cue in the stereogram.
Figure 8 shows the basic ground randomnumber pattern used for the stereograms inFigures 10 and 11. The terrain model forFigures 10, 11, and 13 is shown in Figure 9.In this figure are depicted the rough outlinesof surfaces which can be seen in the stereograms. The elevation model consists of anupper and lower section. The upper section isfine grid elevation data which was interpolated from gross grid elevations man uallyassigned and is generally self-explanatory.The lower part consists of mathematicalshapes given in Table 1.
Figures 10 and 11 are the same stereogramexcept that normally distributed noise (with
mean 0 and standard deviation 1.2) and grossgray-scale changes (ranging from +2 to -3)have been introduced into the left and rightphotos of Figure 11, respectively.
For the stereogram of Figure 13, the basicground pattern used was that of Figure 12.The same terrain data (Figure 9) used forFigures 10 and 11, is used in Figure 13. As inFigure 11, noise and gross gray-scale changesare included in Figure 13.
While all features in Figures 5 and 6 in boththe left and right photos are clearly visible tothe eye, there are ground features which arehidden from both photos in Figures 10, 11,and 13. This may be seen by comparing thelatter with Figures 8 and 12, the originalground gray-scale patterns. Close examination of Figure 10 will disclose features visiblein the left photo but not in the right photo.
TABLE 2
Generate unsorted photo data 10.5 16.0Merge and sort photo data 36.0 36.0Introduce gross gray scale
changes 3.6Introduce gray scale noise 3.0Prepare data for recording on
film 13.6 13.6
Total for stereo photo(minutes) 63.1 69.2
Time (Minutes)
Generate ground model elevationdata (640 X 480 spots)
LeftPhoto
6.3
RightPhoto
Because (1) the elevation changes in theterrain model (Figure 9) are quite large and(2) the model is located in the upper leftquadrant relative to the ground nadir pointsof both camera stations, large binocularparallaxes result. Since it is quite difficult toview stereograms 10, 11, and 13 in stereo andbecause they overlap, they are not mountedfor easy stereo viewing. evertheless, thesestereograms are interesting when viewedmonocularly. The terrain features are clearlyidentifiable. (This is particularly true in thecase of Figure 10.) It is apparent that whenhidden features are present, the monoculardepth cue of interposition (the superimposingof near objects upon far objects) comes intoplay [2J.
Total running times on an IBM 7094-II forthe stereograms in Figures 10, 11, and 13 areshown in Table 2.
GENERATING DIGITAL FICTITIOUS STEREOGRAMS 175
SUMMARY
This paper has shown how ficti tious stereophotography can be generated on a generalpurpose digital computer. A fictitious terrainmodel must first be constructed from anarbitrary gray-scale pattern and an elevationmodel. The elevation data can be interpolatedfrom a contour map or from manuallyassigned elevations, or it can be computedfrom mathematical shapes. A fictitious stereogram is then easily obtained by the mathematical projection of the terrain model upontwo tilted photo-planes in space.
Such fictitious data can be utilized as apowerful tool in research on digital correlationtechniques. Since most correlation techniquesunder investigation today for use in automaticmap compilation employ only the binocularparallax property as a clue to depth, the lackof correspondence between gray-scale patternand elevation data, the absence of shadows,
and the inability to reproduce the widevariety of tonal differences between the twophotographs (which are present in real photography) are not detrimental to correlationresearch. Neither is the inability to generatemulti-valued terrain data considered a significant disadvantage.
ACKNOWLEDGMENTS
The authors are grateful to Mr. C. I.Homan for his major contribution in programming and in the generation of fictitiousdata and to Mr. A. L. Anthony for his work ingeneration of mathematical terrain models.
REFERENCES
1. Dodge, Hugh F. "Automatic Mapping SystemDesign," paper presented at the 1963 Semiannual Meeting of the American Society ofPhotogrammetry, Wellesley Island, New York.
2. Julesz, Bela, "Binocular Depth Perception ofComputer-Generated Patterns," Bell SystemTechnical Journal, September 1962, vol.XXXIX, No.5, pp. 1125-1162.
BOARD OF DIRECTION FOR 1964
WILLIAM A. FISCHER .
GOMER T. McNEIL .
CAPT. LAWRENCE W. SWANSON .C. EARL PALMER .
G. C. TEWINKEL... . .
THEODORE W. NORCROSS ....•..
MRS. BARBARA N. MUIR .....
. .President. First Vice President
· .Second Vice President· Secretary- Treasurer
. Editor. Editor Emeritus· .. Advertising Manager
Term expires in 1964CLARICE L. NORTON
MARVIN B. SCHER
HELMUT H. SCHMID
CHARLES THEURER
GLENN C. WELDEN
DIRECTORS
Term expires in 1965HUBERT C. CAIN
JOHN O. EICHLER
HEINZ GRUNER
EUGENE]. SCHLATTER
PAGE TRUESDELL
PAST PRESIDENTS
JAMES P. WEBB AND R. S. QUACKENBUSH, JR.
Term expires in 1966PAUL H. ALBAN
FREDERICK J. DOYLE
WILLIAM C. MAHONEY
ARTHUR R. SHOPE, JR.
JOHN H. WICKHAM, JR.
EXECUTIVE COMMITTEE, 1964
PRESIDENT-William A. FischerVICE PRESIDENTs-Gomer T. McNeil and Lawrence W. Swanson
DIRECTORs-Faul H. Alban, F. J. Doyle, Charles Theurer, Glenn C. Welden
COUNCIL OF THE SOCIETY, 1964
CHAIRMAN-Gomer T. McNeilDEPUTY CHAIRMAN-R. S. Quackenbush
SECRETARy-Glenn A. HarperCOUNCIL REPRESENTATIVES-Clarice L. Norton, Marvin B. Scher, Helmut H. Schmid
AND PRESIDENTS OF ALL LOCAL REGIONS