1
Real Time Building Identification System Development Using Aerial Imagery
Presented by:Mr. Duong Van HIEU RSA028033
Examination Committee:Dr. Kiyoshi Honda ChairmanDr. Xiaoyong Chen MemberDr. Frederic Borne MemberDr. Manukid Parnichkun Member
1. Background
2D GIS databases are available, especially building
geometric information on the buildings contains polygonal shape and the number of floors (height)
Constructing a 3D building model is possible
Availability of small and low-power consumer devices (PDA, GPS, Tilt sensor and Digital Camera) and network access capabilitiesA real-time system development is feasible
2
1. Background (cont.)Access building information based on landscape image will give very intuitive interface Help people to know and grasp what is happening in the real world and help their decision making.Tourist support systemUrban disaster Mitigation; Fire, Earthquakes and etc.AircraftBig market in recent time
3. Objectives
To develop a core algorithm for real time building identification system by linking images from high places ( airplane or high buildings) to existing 2D GIS building database
(1). Design building identification system.> Images are captured from high places> A link between images and 2D GIS building data
are automatically generated
(2). Develop matching algorithm with short processing time.
3
Reference Image CreationData Acquisition
4. Methodology - System Overview
Rectification byLine Matching
Coordinate Conversion
Real Time Data Acquisition
Digital Camera
Digital Image
Approximate
Camera Position
(X,Y,Z), GPS
Camera Attitude (ω,ϕ,κ), Electronic Compass
2D Building data with ID
numbers
3D Computer Graphicswith (X,Y,Z) and (ω,ϕ,κ)
4. Methodology - System Overview
Rectification byLine Matching
Geo-rectified imagewith (X,Y Z)and (ω,ϕ,κ)
Link Creation & Display
2319
Bld_id: 2319Name: …Owner: ……
GIS Building Database
4
4. Study AreaViewing Point
Phathum Wan District
BaiYoke Sky2X= 1,520,785.552Y= 666,868.817Z = 275m
4. Study Area – GIS data
Phathum Wan District
BaiYoke Sky2X= 1,520,785.552Y= 666,868.817Z = 275m
Viewing Point
Spatial data (1:4000 map scale)
Provided by MapPointAsia Company
5
4. Study Area – GIS dataAttribute data
Number of people working insideNresident13Bl_unit_f12Bl_nunit11
Number of floors of buildingBl_nstorey10Building depthBl_depth9Building heightBl_height8
Bl_frontag7Act_name 6Act_code 5
Building typeBl_type4Building id numberBldg_id3PerimeterPerimeter2AreaArea1
DescriptionField NameField No.
Survey_dat35Bldg_code34Index_link33Bl_remark32Bl_address31Postcode30Changwat29District028District27Bl_road26Bl_village25Bl_villnum24Housenum23Bl_name22Bl_matl21Bl_date20Bl_use19Bl_tax_id18Area_flag17Bl_area16Bl_owner15Bl_employ14
Number of people working insideNresident13Bl_unit_f12Bl_nunit11
Number of floors of buildingBl_nstorey10Building depthBl_depth9Building heightBl_height8
Bl_frontag7Act_name 6Act_code 5
Building typeBl_type4Building id numberBldg_id3PerimeterPerimeter2AreaArea1
DescriptionField NameField No.
4. Study Area – Landscape Photos
Landscape Photo 2D GIS dataN
6
1. Edge Detection2. Thresholding3. Vertical Edge Detection
by Morphology
3D Computer Graphicswith (X,Y,Z) and (ω,ϕ,κ)
(Reference Image)
Flow chart of System
Digital Camera Image(Original Image)
4. Mask Image Creation
1. Edge Detection2. Thresholding
Building Selection
7s
10s
2s2s2s
2s2s
5s
Reference Edge Image(Binary)
Extract True Building Edges By Hough Transform
Original Edge Image with mask(Binary)
5. Methodology (cont.)
5. Masking Original Edge Image
Line Matching by Image Correlation
Rectified Original Image
Building ID Display
Morphology to reduce texture
6s
2s
10s
20s
3s
1s
7
Computer Digital Camera
Electronic Compass(ω,ϕ,κ)
GPS (X,Y,Z)
Data Acquisition System
Digital Landscape ImagePhathum Wan BuildingsImage size (1856x1392) Focus length = 18.1mmX= 1,520,785.552Y= 666,868.817Z= 275mω= -7.590, ϕ=154.6, κ=0
8
X= 1,520,785.552Y= 666,868.817Z = 275mω = -7.590ϕ = 154.6κ = 0
Phathum Wan Buildings
Same image size (1856x1392)
& Viewing angle (FOV)
Reference Image-3D CG with (X,Y,Z) & (ω,ϕ,κ)
Criteria for Building SelectionVisibility=B/A > 0.7
A
A
B
ID:2319
No. of Floor = 23
Height = 4*23 =92
Number of Floor >20
1
2
Number of obstructive objects = 0
9
Subset 3D CG image
X= 1,520,785.552Y= 666,868.817Z = 275mω = -7.590ϕ = 154.6κ = 0
Subset image with the same size (550x500) & the same upper left location (59x893)
Digital Camera Image (Original Image)Phathum Wan Buildings
Same image size (1856x1392)
& Viewing angle (FOV)X= 1,520,785.552Y= 666,868.817Z = 275mω = -7.590ϕ = 154.6κ = 0
10
Subset 3D Digital Camera Image image
X= 1,520,785.552Y= 666,868.817Z = 275mω = -7.590ϕ = 154.6κ = 0
Subset image with the same size (550x500) & the same upper left location (59x893)
Edge Detection & Thresholding
Edge Detection by Robert filter Thresholding
Binary Edge Image
11
5. Result (cont.) 1. Edge Detection2. Thresholding3. Vertical Edge Detection
by Morphology
Thresholding & Vertical Edge Detection (Openning)
Edge Detection by Robert filter
Mask Image Creation
Some assumption of accuracyAzimuth angle is measured by electronic compass (1 degree accuracy).Altimeter is used for height measurement with 10m accuracy.Elevation angle ω is estimated by manually, around -5 degree (downward).
12
Mask Image CreationBased on heading information accuracy of tilt sensor and distance from viewing point to target buildingα = ±0.5, d=1100m
d=1100m
α: ±0.5
Target Building
Viewing Point
Delta_Xs
Delta_Xs = α * d = 1100*0.5*π/180 = 10 m
Mask Image Creation
GIS Building Edge
Actual building in the landscape image
The difference between GIS and actual building
Delta_Xg = 10m
10 m10 m
13
Mask Image Creation
Focus Point
Viewing AngleVAh,VAv
Focus Length
Image Sizew=1856h=1392
f=18.1mm
CCD
VA = (tan-1 (a / f)) x 2VA: Viewing anglea: ½ of CCD Sizef: Focus length
4.8 mm
6.4 mm
d=1100Dx(m) = (w/2)*(d/f) = 194m1m = 1856/2/Dx(m) ~ 5 pixels
Delta_Xs = Delta_Xg = 50 pixelsDx
Mask Image Creation
d=1100m
α
Target Building
Viewing Point
Delta_Xs
Delta_Xg
Mask Area
Mask Area Width = = 2*(Delta_Xs + Delta_Xg)= 200 pixels
14
Hough Transform - Right Mask Area
Convert back into image SpaceMax Value = 93r = 490, theta = 1
r = x*cosθ + y*sinθ
θ
Hough Space
r
Image Space
y
x
Line Matching by Image Correlation
Shift X1 = -40Rotation1 = 0.8 degree
Red : Left Edge of Reference ImageGreen : Left Edge of Original Image
Shift X Range = -100 to +100Rotation Range -3 to +3, step 0.1 degree
15
Line Matching by Image Correlation
Shift X2 = -22Rotation2 = 0.7 degree
Red : Right Edge of Reference ImageGreen : Right edge of Original Image
Shift X Range = -100 to +100Rotation Range -3 to +3, step 0.1 degree
Matching parameters
Shift X = -31Rotation = 0.75 degree
Shift X Range = -100 to +100Rotation Range -3 to +3, step 0.1 degree
Image Not Rectified Yet
16
Rectified Original ImageShift X = -31Rotation = 0.75 degree
Original Landscape Image
Rectified Image
Rectified Original ImageShift X = -31Rotation = 0.75 degree
Rectified Image& Reference Image
Original Landscape Image& Reference Image
17
6. Accuracy Assessment – Area Method
Extraction of true shape of real building as reference
Accuracy Assessment – Area MethodOverlay true shape of real building with 3DCG building shape
Landscape building iswithin completely 3D CG building.Accuracy is 100%
18
Rectified Whole Landscape ImageShift X = -31Rotation = 0.75 degree
6. Accuracy Assessment – Center PointExtract shape of 16 buildings by manually
20
6. Accuracy Assessment - CenterPointCorrectIncorrect Accuracy = 93.75%
Correct: 15 Incorrect: 1
6. Accuracy Assessment – Center PointOverlaying Original Landscape Image and Center Point Image
21
6. Accuracy Assessment - CenterPoint
7. Conclusion
(1). Core algorithm for matching digital camera image and 3D from 2D GIS was developed. Digital Image from BaiYoke Sky to Pathumwan was georectified
(2). The accuracy for identification was acceptable at 93.75% which was measured at the center of the gravity