Automated Pavement Distress Survey

Wang, Gong, submitted to 2002 Pavement Evaluation Conference, 21-25, 2002, Roanoke, Virginia

2

Automated Pavement Distress Survey: A Review and A New Direction

KELVIN C.P. WANG AND WEIGUO GONG

4190 Bell Engineering Civil Engineering

University of Arkansas, Fayetteville, AR 72701 Email: [email protected]

ABSTRACT

Pavement condition survey normally includes surface distresses, such as cracking, rutting, and other surface defects. Broadly, pavement roughness is also included as a condition survey item in some literature. This paper reviews past research efforts in this area conducted at several institutions, including the automated survey system of pavement surface cracking at the University of Arkansas. The paper also proposes a new direction of technology development through the use of stereovision technology for the comprehensive survey of pavement condition in its broad definition. The goal is to develop a working system that is able to establish three-dimensional (3D) surface model of pavements for the entire pavement lane-width at 1 to 2-millimeter resolution so that comprehensive condition information can be extracted from the 3D model.

INTRODUCTION

The current system developed at the University of Arkansas with 2D imaging for automated cracking survey is fully automated and real-time based. However, surveys of other surface defects of pavements, such as rutting, roughness, pothole, and others, require using different equipment, or cannot be automated at all at this time.

Stereovision with a pair of simultaneous image sources is not new. Highway agencies have been using the principle in photogrammetry in highway design and planning. In the recent decade, the failure of newer technology to provide solutions to comprehensive pavement condition survey prompted us to examine the possibility of using stereovision for that purpose. With rapid advances in digital camera and computing power of microcomputers, we believe that the time is right to integrate such a system with commonly available devices and develop efficient and accurate algorithms in software sub-systems to achieve the following goals:

1. High-resolution. The 3D surface model will provide 1 to 2-millimeter resolution in all three coordinate axis directions, X, Y, and Z.

2. Comprehensive survey. The information obtained from the 3D surface model includes cracking, roughness, rutting, potholes, and other surface defects associated with both flexible and rigid pavements.

3. Real-time at highway speed for both image acquisition and processing. Based on the new real-time technology developed at the University of Arkansas for


3

pavement cracking with 2D images, the realization of this goal will provide all condition results while images are being acquired at real-time.

Current pavement condition data collection uses a mixture of automated devices and manual methods. For pavement roughness and rutting, the automated method is using static laser sensors to determine longitudinal and transverse profiles of pavements. This technology is mature, but, costs over $100,000 in materials (not including the vehicle and R&D cost) to make such a system with five lasers for the transverse profiling.

For surveying all other pavement surface defects, the predominant method is to use manual labor to walk pavements or examine visual information collected in the field to determine the conditional parameters. The speed for such data analysis is slow, normally 1 to 2 miles per hour per person if complete pavement surface is covered with the visual examination of collected images. It usually costs over $20 per mile for manual surface condition survey, not including the cost to collect the images.

BACKGROUND AND LITERATURE SEARCH

Results from pavement condition survey are critical for pavement engineers to determine the needs of maintenance and rehabilitation for both network and project level study. Pavement condition survey includes identification and classification of various types of surface cracks, identification of patching and potholes, and quantification of rutting and shoving. Determination of other types of distresses, such as bleeding, polished aggregate, and raveling, are also made during certain pavement condition surveys. For pavements with jointed Portland cement concrete surfaces and continuously reinforced concrete surfaces, in addition to cracks, spalling, corner breaks, and other types of distresses may be also determined during condition survey (LTPP Distress Manual). In this paper, pavement roughness is also broadly defined as a part of condition survey; even though, it is highly correlated with serviceability.

In the past two decades, various agencies tried to develop automated methods to identify, classify, and quantify pavement surface distresses (Wang 2000). It is generally agreed that rutting and roughness measurements can be automatically collected at highway speed with sufficient number of laser sensors. Collection of many other types of distresses, particularly cracking, still cannot be fully automated. Most federal and state highway agencies are still using manual labor to collect distress data by either patrolling pavements, or rating pavements via looking at pavement surface images. A limited number of highway agencies are using semi-automated survey systems that are post-processing based, require substantial human interaction, and its processing speed is about a fraction of normal driving speed.

In the past few years, researchers at the University of Arkansas made substantial progress in automatically identifying and classifying pavement surface cracks at highway speed using a data collection system with one high-resolution digital frame camera and parallel processing. Currently, the developed system can collect two-dimensional pavement surface images, identify and classify four types of cracks at over 60 MPH. The four types of cracks are longitudinal, transverse, alligator and block. The size of cracks that can be identified and classified is about 2-millimeters.


4

The goal of the 3D-modeling research for comprehensive pavement survey has two folds. One goal is to refine the current system to have an acquisition system to have the capability of 1 to 2-millimeter resolution and utilize multiple camera configuration of the proposed research to improve the accuracy of automated cracking detection. The primary goal is to study the feasibility to apply stereovision technology to develop a three-dimensional (3D) surface model of pavement surface. When the resolution of the 3D surface model is sufficiently high, such as 1-millimeter, the vast majority of surface distresses for both flexible and rigid pavements can then be identified and quantified through geometric modeling. Roughness can be also determined by using the 3D surface model, as longitudinal profiles of pavement surface can be conveniently established through the use of the 3D surface model.

It should be noted that there have been numerous publications on image processing algorithms for pavement crack survey. The literature search concentrates on system design and implementation, which are more related with the proposed research. In the past two decades, there have been a number of efforts at automating pavement condition survey. One direction of the research is on establishing 3D surface models of pavement surface by using laser technologies and shadow Moire optical interference method. The other direction is using imaging or laser approaches to detecting pavement surface cracks based on 2D characteristics of pavement surface.

3D Approaches

Phoenix Scientific Inc. in California (http://phnx-sci.com) developed a phase-measurement Laser Radar (LADAR) to measure the 3D properties of pavements. The Laser Radar uses scanning laser and reflector to measure the reflecting times across pavement surface, therefore establishing a 3D pavement surface after the Laser Radar moves longitudinally along the traveling direction. Its system, as claimed, is able to produce roughness and rutting data. Another company, GIE Technologies Inc. in Canada (http://www.gietech.com/), has the LaserVISION system, which also models the 3D surface of pavements. The lasers are stationary and four of them are used to cover full lane-width. The service it provides is primarily for roughness and rutting survey. At this time, there is no independent evidence that laser based technologies are able to provide usable data for pavement cracking survey and other condition survey.

In addition, researchers at Illinois Institute of Technology developed a 3D technology for pavement distress survey through a grant provided by NCHRP IDEA in the mid-1990s (Guralnick 1995). That technology uses the shadow Moire optical interference method and can only identify off-the-plane crack, that is it can only find cracks, if both sides of which are not on the same plane. The limitation of this technology is apparent, as relative to the size of cracks, sides of most cracks are on the same plane.

2D Approaches

The following are abbreviated discussions on some significant developments based on 2D approaches in chronological order.

In the late 1980s, the Japanese consortium Komatsu built an automated-pavement-distress-survey system (Fukuhara, et al. 1990), comprising a survey vehicle and data-


5

processing system on board to simultaneously measure cracking, rutting, and longitudinal profile. Maximum resolution of 2048 x 2048 is obtained at the speed of 10 km/h. The Komatsu system worked only at night to control lighting conditions. The system represents an implementation of the most sophisticated hardware technologies at that time. However, it does not output the types of cracking and only works during the night. There are no known operating units at this time.

From late 1980s to early 1990s, Earth Technology Corporation created a research unit called Pavement Condition Evaluation Services (PCES). The automated system created by PCES was the first to use line-scan cameras at 512-pixel resolution to collect pavement data. One important factor for discontinuing the effort is that the necessary technologies associated with the image capturing and processing were not mature enough. PCES designed, produced their own hardware and made their own system level software, which were not only costly, but also limited the research team from obtaining higher performance equipment from third parties at a later time.

In the early 1990s, NCHRP also funded a research called Video Image Processing for Evaluating Pavement Surface Distress, project 1-27 (Fundakowski 1991). The research approach in that project used traditional imaging algorithms with 2D images of pavements. The research was experimental and the final report was not formally published.

The Swedish PAVUE acquisition equipment includes four video cameras, a proprietary lighting system, and four S-VHS videocassette recorders. The image collection subsystem is integrated into a Laser RST van. The off-line workstation is based on a set of custom designed processor boards in a cabinet to analyze continuous pavement data from the recorded video images. Surface images are stored on S-VHS tapes in analog format.

Dr. Max Monti of the Swiss Federal Institute of Technology (EPFL) completed his Ph.D. with the goal to design and implement a new pavement imaging system (Monti, 1995). With the developed system, Crack Recognition Holographic System (CREHOS), the pavement surface is scanned with a focused laser beam along a straight line in the lateral direction, while the longitudinal scan is conducted with the movement of the vehicle. The hardware system was dismantled due to performance limitations.

From 1995, the US Federal Highway Administration awarded contracts to LORAL Defense Systems in Arizona, now a unit of Lockheed-Martin, to provide an Automated Distress Analysis for Pavement, or ADAPT for short. The researchers used techniques developed for military purposes. The data source is digitized images from PASCO’s 35-mm film. AADPT was written in MATLAB and is not being used in any operating products. The delivered system after project completion could not be used and Arizona DOT currently is funding Lockheed-Martin to continue the research.

Since late 1996, RoadWare Corporation has been actively using a new product, WiseCrax, for automated survey of pavement surface. The data collection uses two analog cameras synchronized with a strobe illumination system, with each camera covering about half-width of a pavement lane. The image processing is done in the off-line office environment relying on the host CPUs to conduct image processing at the speed of two or three miles per hour with substantial operator assistance.


6

DEVELOPMENTS AT THE UNIVERSITY OF ARKANSAS

A lesson learned from past endeavors is that developing proprietary hardware not only requires high capital, but also limits the research from using better and cheaper hardware in the future. The fundamental approach taken by us in the past few years and will be used in the proposed research is to develop algorithms and their implementations in software with general-purpose computing devices, and to use digital systems only. Data acquisition hardware components, such as digital cameras, acquisition boards, sensors, and others, have been supplied by third parties who engage in constant performance improvements.

The general technical approach based on the vehicle technology at the University of Arkansas is illustrated in Figure 1. In order to achieve the necessary resolution, two digital cameras at the resolution of 2048-pixel/cameras or 1300-pixle/camera are used to cover half of a lane-width, approximately two meters. The first step is to analyze 2D images from each of the two cameras to detect and classify any cracks. The results from analyzing two image sources of the same pavement are then combined so that cracks missed by one analysis are still counted, therefore potentially achieving higher accuracy.

The pair of images on the same pavement surface is also used to establish 3D surface model, which is then used to detect other deficiencies of the pavement through straightforward geometric modeling. The geometric modeling including establishing the longitudinal and transverse profiles of the pavement, and detect abnormalities on pavement surface based on vertical variations on pavement surface. Rutting information is obtained by scanning transverse profiles with a virtual straight line. With the established longitudinal profile values, roughness is obtained through computing International Roughness Index (IRI) and Ride Number (RN) based on common accepted procedures developed at the University of Michigan. Computer code has been already realized by the authors of the proposed research on calculating IRI and RN based on known profile values.

Potholes are detected based on the characteristics of shape and depth of defects. Other deficiencies, particularly those associated with rigid pavements, can also be found based on the 3D characteristics of the deficiencies. Furthermore, cracking information obtained based on 2D imaging can be refined based on the 3D information by either eliminating noises that were classified as cracks, or adding new cracks that exhibit depths and were missed in 2D image analysis. To cover a complete lane surface plus a portion of shoulder area, a four-camera setup is necessary with two systems, each of which is similar to Figure 1 in design.

Application of stereovision has long been used in remote sensing, particularly with airborne and deep space based imaging devices. There were several obstacles for using it for pavements. First, image resolution had to be sufficiently high for pavement surveys, for instance, 1-millimeter. Digital cameras capable of this resolution at reasonable cost became only available in the past few years. Second, the required input bandwidth of the computing device is more than sustained 100 megabytes per second. The limitation of I/O sub-systems and microprocessor power prohibited a desktop computer from being used for acquiring and processing images from two camera sources. The new research is based on the fact that today’s digital cameras are capable of providing needed resolution at reasonable cost, for instance about $5,000 per camera, and I/O of most recent microcomputers is able to provide over 100 megabytes per second per channel.


7

Figure 1 General Procedures for Automated Condition Survey with Stereovision

Furthermore, it is evident that by using the most advanced microcomputer in the summer of 2001 at the cost of $6,000, we achieved 60 MPH real-time processing of 2D pavement images. Future technology advances in the next two years will provide computing hardware platforms for a real-time system at highway speed to establish a 3D terrain model of pavement surface at 1 to 2-millimeter resolution.

Automated System for Pavement Surface Distress Survey with 2D Images

As a part of a larger research effort to develop a digital highway data vehicle started in the mid 1990s, the researchers at the University of Arkansas focused on the development of a real-time automated system for distress survey. After examining the feasibilities of using neural net and fuzzy sets, traditional imaging algorithms and customized approaches have been developed to achieve the goal of real-time processing at highway speed. Figure 2 shows the digital highway vehicle at the university. Its basic features are:

(1) The vehicle is based on a full-digital design.

(2) The vehicle includes two sub-systems for pavement surface image collection and automated distress analyzer to identify and classify pavement cracks at real-time.

(3) The vehicle collects right-of-way video or still frame images and save them in digital format in real-time.

(4) The vehicle acquires the location through the use of a GPS device and a distance-measuring instrument (DMI).

(5) Real-time relational database engine, inter-computer communication techniques, and multi-computer and multi-CPU based parallel computing.

Pavement images are stored in JPEG format in RAID disks when the images are being acquired. The resolution of the images is currently at 1300-pixel per lane. The imaging system is able to cover the complete surface of several thousand miles of pavement without changing storage device or downloading. The data collection can be conducted at

2-Meter Wide

Two Cameras, Resolution 2048-Pixel/Camera

2D Crack Detection

2D Crack Detection

Overlapping Crack Recognition

Establish 3D Surface Model

Condition Survey with Geometric Modeling

Pavement Surface Condition Database


8

highway speed. Vehicle exterior are shown in Figure 2. The firing of the strobes and the shutter opening of the camera are synchronized to obtain images at high shutter speed.

Figure 2 Exterior of the Data Vehicle

SPEED, ACCURACY, AND CRACK GEOMETRY

Two key issues in automated survey include improving processing speed, and developing sufficiently accurate algorithms. Highway pavement surface contains numerous foreign objects, such as oil residue, dirt, lane markings, vehicle’s tire mark, tree limbs, and other non-distress related items. The task of the research was to develop algorithms to correctly distinguish cracks from these non-distress noises.

The implementation of the developed algorithms in a parallel computing environment produced a real-time system that can automatically identify and classify pavement surface cracks while high-resolution digital images are being acquired and archived into a multimedia database at highway speed. This image processing system for cracks is called distress analyzer.

The first step in the image processing process is to distinguish any cracks from other non-distress noises. The primary method in this step relies on analytical descriptions of distresses’ characteristics. The second step is to connect and vectorize the detected cracks, and establish a distress database related to location, orientation, and other geometry information of each crack. Based on the geometric information obtained in the second step, cracks can be classified using any pre-defined distress categorization protocols. Several distress protocols are incorporated into the system, including AASHTO Interim Distress Protocol, Texas distress manual, and Universal Cracking Indicator (CI) from World Bank. These indices are immediately computed and available for analysis when the vehicle completes a data collection field trip.

Figure 3 illustrates the framework of the data acquisition and processing in a parallel environment. A dual-CPU computer is used for data acquisition of GPS data, DMI data,


9

and images. These data sets are moved to a multi-CPU computer at real-time for the distress analyzer. The distress analyzer has a project manager for parallel processing, which coordinates the processing of images among the n processors. The current implementation is using two CPUs for the distress analyzer. Figure 4 shows a screen shot of the distress analyzer working at real-time with two Pentium-III processors at 733-MHz per CPU. At the bottom of each process are the analysis results for crack geometry and basic classification. Each process shows the original image on the left and processed binary image on the right. Each binary image shows each identified crack in a bounding box with a unique integer number. The right-most window illustrates the processing status of the analyzers. When a computer with two Pentium 4 processors at 1.7-GHz is used, the processing speed for the distress analyzer is well over 60 MPH on consistent basis.

Figure 3 The Framework of the Data Acquisition and Processing with One Image Source

Figure 4 Dual-Processing for Cracks in a Parallel Environment with the Crack Analyzer

SEGMENTATION AND NOISE REMOVAL

The most prominent imaging characteristics of pavement surface distresses include (1) their gray levels are lower than the surrounding background; (2) they are normally composed of curves or lines of irregular shapes, which may or may not be connective; (3)

GPS DMI Camera

Dual-CPU Acquisition

Multi-CPU,Distress Analyzer

CPU1

CPU N

Project Manager for Parallel Processing

Expanded View of the Distress Analyzer


10

every distress belongs to a classification depending on definitions. Distresses can be classified based on their geometric orientation, location, and density. Distresses such as longitudinal crack and transverse crack are in the orientations of longitudinal profile and transverse profile. Alligator cracks consist a number of intertwining cracks and are normally located in traffic’s wheel-paths. Block cracks’ geometric characteristics are similar to alligator cracks, but their density is less than alligator cracks.

Pavement noises are darker than the background and their shapes are abnormal. Other non-distress objects such as lane markings are brighter than the background. The positions and orientations of lane markings need to be detected before the locations of cracks respective to pavement lane are determined.

A classical method in image processing, thresholding, is employed at the initial stage of processing. The method of adaptive thresholding is used to segment the image, which converts gray-scale image into black-white images. In order to preserve the continuity of cracks in the process of removing noises, a known imaging technique called Morphological Closing is used to pre-process the image to fill small and thin holes in objects, connect nearby objects without changing their area significantly, and smooth the boundaries of objects.

For noises whose size is only a limited number of pixels, and whose areas range from one pixel to nine pixels, we developed an Elimination with Scanning Distribution (ESD) method to discriminate small size noises from actual cracks for each sub-image with the size of 5 pixels by 5 pixels. Figure 7 illustrates the effect of applying the ESD method. Following the use of ESD, for noises whose shapes look like unconnected grains with more than several pixels for each grain, they are removed by using a grid filter method (GFM). Figure 8 illustrates the effect of applying the GFM method. It should be noted that even large number of noises are removed after the process, certain noises still exist, which will be identified and removed at later stages.

DETECTING LANE-MARKING AND ANALYSIS OF OVERALL CHARACTERISTICS OF IMAGES

Detecting lane markings becomes critical to determine the geometric positions of cracks relative to the traffic lane. A projection technique is applied to the image in the longitudinal direction, so that the maximum accumulation of white pixels exists for an area where the marking is located. In addition, for images without any lane marking, adjacent lane markings are used as basis to estimate lane markings for the images.

In preparation for analyzing cracks, a set of statistics is obtained at this stage for entire images of full-lane width. An entire image is classified into three categories: punctuated image with only black dots, lined image with only line cracks, and blocked image with intertwining cracks. This processing has the capabilities of identifying an entire image primarily containing black dots, or noises. In addition, it is able to categorize the image to be primarily with line cracks or of block type of cracks. Even though the classification of the image at this stage is still preliminary, the processing provides important information for the final classification of distresses. Figure 10 illustrates the geometric categories of the distresses that are identified at this stage for a larger area of pavement.

After initial noise removal, a black-white image covering a full lane is divided into a number of blocks. As most of the blocks do not contain any black pixels, the processing


11

for crack classification is focused on blocks containing black pixels, resulting in substantial improvement of processing speed.

CONNECTING AND LINKING OBJECTS

Actual cracks are frequently presented with broken dark objects. This step merges the objects into a larger one if they are determined to be a part of a larger object. It is likely that small objects identified in the previous step are actually parts of a larger crack crossing several blocks. The adjacent blocks are scanned to determine if the candidate object actually connects to an object in another block. If the determination is positive, the image is updated so that the two objects are identified as belonging to one crack. Area size, perimeter, and boundary of identified cracks can then be determined. Hough transformation is used to determine directions of merged objects.

When objects are linked together to form a merged larger object, the direction of the merged object is determined through a weighting process. Length, li, of object, i, is the diagonal distance of the bounding box. αi is the direction of object i. The length, direction, perimeter, and area size of the merged object, l, α, p, and s are:

illll +++= ...21 ........................................................................ (1)

llll ii αααα ×++×+×

=...2211 .................................................. (2)

∑=i

ipp1

.................................................................................. (3)

∑=i

iss1

.................................................................................... (4)

At this stage, all dark objects, some of which have been merged, are presented in line forms with basic geometric properties associated with them in any lane-wide images. The above method is also used to determine if a particular object is a man-made joint, such as concrete joint, if the direction is uniform, the length is long, and the width varies little.

CONNECTING CRACKS, CRACK GEOMETRY AND CLASSIFICATION

For a punctuated image and a lined image determined at an earlier stage, further analysis is made based on the characteristics of long objects in an image:


12

Figure 7 Before and After the Use of ESD, Sub-Image Size, 100-pixel x 100-pixel

Figure 8 Before and After the use of GFM, (Same Source Image as in Figure 7)

Figure 10 Dot Noises, Line Cracks, and Block Cracks


13

(1) Determining long objects for potential classification as cracks,

(2) Connecting the long objects with other objects if the other objects are in the path of the long objects,

(3) Naming the connected object as longitudinal, transverse, block cracks, including alligator cracks and actual block cracks.

DATA ANALYSIS WITH DISTRESS PROTOCOL

Universal Cracking Indicator (CI) gives only one number for each sub-section. Test runs were conducted four times for same 2.8-mile road section to verify the repeatability of the system. The Figure 11 shows different test runs have the same pattern in the chart, indicating the same trouble spots on the pavement. To quantify the repeatability of the crack recognition system, the standard deviation of the Universal Cracking Indicators (CI) for each sub-sections are calculated. The average of the standard deviation is 0.0083 for the average UI value of 0.0556 for the entire 112 data points. In addition, the system is able to produce result based on the new AASHTO Cracking Protocol (AASHTO 2000).

FIGURE 11 FOUR TEST-RUNS WITH UNIVERSAL CRACK INDICATOR (CI)

METHODS TO ESTABLISH THREE-DIMENSIONAL (3D) PAVEMENT SURFACE

Highway agencies have used photogrammetry for planning and design for quit some time. 3D surface model of pavements built with similar technology may be established at sufficient resolution with the latest technology, so that comprehensive analysis of pavement condition can be automatically conducted at highway speed, including roughness, rutting, cracking, and other distresses exhibiting physical distortions.

The principle of acquiring two images on the same pavement surface is illustrated in Figure 12. The projected points for the two cameras on the level plane are p1 and p2 with camera heights of h1 and h2 respectively. The interested point in question is P. p3 and p4 are points representing point P in the two images captured with cameras 1 and 2 respectively. d1 and d2 are projected distances from p1 and p2 to p3 and p4 respectively. d is the distance between p3 and p4. The key in establihsing any 3D model based on a pair of

Repeatability Test

0

0.05

0.1

0.15

0.2

0.25

0.00

000 -

0.10

000

0.10

000 -

0.20

000

0.20

000 -

0.30

000

0.30

000 -

0.40

000

0.40

000 -

0.50

000

0.50

000 -

0.60

000

0.60

000 -

0.70

000

0.70

000 -

0.80

000

0.80

000 -

0.90

000

0.90

000 -

1.00

000

1.00

000 -

1.10

000

1.10

000 -

1.20

000

1.20

000 -

1.30

000

1.30

000 -

1.40

000

1.40

000 -

1.50

000

1.50

000 -

1.60

000

1.60

000 -

1.70

000

1.70

000 -

1.80

000

1.80

000 -

1.90

000

1.90

000 -

2.00

000

2.00

000 -

2.10

000

2.10

000 -

2.20

000

2.20

000 -

2.30

000

2.30

000 -

2.40

000

2.40

000 -

2.50

000

2.50

000 -

2.60

000

2.60

000 -

2.70

000

2.70

000 -

2.80

000

Sub-section

Uni

vers

al C

rack

ing

Indi

cato

r

1st Run2nd Run3rd Run4th Run


14

images is to correctly find the height of any particular point, H. As coordinates for points, p1, p2, p3, and p4, are known, therefore the relevant angles, α1, α2, and β, are also known. When H is calculated for every point in an image, the 3D surface model of the object, in our case, the pavement surface, is then established.

Digital Image Matching and Height Calculation

The first step in finding H is to correctly identify the same point on the pavement surface in each of the pair images. The positioning difference, d, for the same point on the pavement surface in the two images is therefore used as one input to determine H in Figure 12. The method to recognize similar image characteristics in small regions in both images is called digital image matching (Wolf 2000). The technique of area-based matching is used in our test.

Area-based methods perform the image match by a numerical comparison of digital numbers, commonly called gray-scale, in small arrays from each image. The mathematical technique chosen for area-based matching is Normalized Cross-Correlation. In this approach, a statistical comparison is computed from digital numbers taken from same-size sub-arrays in both images. A correlation coefficient is computed by the following equation, using digital numbers from sub-arrays A and B (Wolf and Dewitt 2000):

( )[ ]

−

−

−−=

∑∑∑∑

∑∑

= == =

= =

m

i

n

jij

m

i

n

jij

m

i

n

jijij

BBAA

BBAAc

1 1

2

1 1

2

1 1

)()(

)(................................ (5)

Where:

c: the correlation coefficient,

m and n: the numbers of rows and columns in the sub-arrays,

Aij and Bij: digital number (gray value) from sub-arrays A and B at row i, column j,

A and B : the average of all digital numbers in sub-arrays A and B,

The correlation coefficient, c, can range from –1 to +1, with 1 indicating a perfect match, –1 indicating a negative correlation, and zero indicating a non-match. Negative-correlation means identical images from a photographic negative and positive are compared. The image matching process is conducted by using a candidate sub-array from one image, and a search is performed for its corresponding sub-array in the second image. A search array is selected from the second image with dimensions much larger that those of the candidate sub-array. A moving window approach is then used to compare the candidate sub-array from the first image with all possible window locations within the search array from the second image. At each window location in the search array, c is computed, resulting in a correlation matrix C after the search computation is completed. The largest correlation value in C is tested against a threshold. If the test is positive, the corresponding location in the search array is considered a match.


15

Figure 12 Using Two Cameras to Establish Height, H

(a) and (b) in Figure 13 show the concept of a DEM model and the illustration of the positions of the sub-arrays for the pair images when the digital image matching is completed with a pair of images. Note the irregular positioning of sub-arrays in the second image (right image) after the processing. The differences in the positioning of sub-images from the pair images are directly used to produce height, H in the second step.

From Figure 12, the height of any given point is determined as follows:

2

22 )(

)(dSin

SinhdH×

××=βα

.......................................................... (6)

Where:

H: height of a given point on the pavement surface,

h2: the height of the second camera from a reference point on the pavement surface,

Camera 2

P p4

p3

p2

p1

d1

Camera 1

d2

H

d

h2

h1

α2 α1

β

d

y x

d

x y

H


16

d2: the projected distance from camera 2 to the point of interest based on image 2,

2α and β : known angles determined by p1, p2, p3, and p4.

Figure 13 DEM Digital Image Matching Principles (based on Wolf and Dewitt 2000)

With the principles described earlier, Figure 14 demonstrates a test to establish a 3D surface model of a pavement surface with the size about 6-ft by 11-inch, using the principles of digital image matching and height calculation presented in the previous pages. In Figure 14, a and b are a pair of images for the same pavement section taken from two positions. Images c, d, and e are taken from the same 3D model based on the pair original images. Image c is looking at the model from top. Image d is looking at the model from an isometric angle. Image e is looking at the transverse profile of the pavement. Several cracks are shown in all images. Also rutting distress is apparent in images d and e.

Establishing accurate reference points on pavement images is critical to produce longitudinal and transverse profiles for pavement surface. We plan to use a high precision gyro system in the vehicle, so that vehicle’s body movement in the vertical and lateral directions can be determined and compensated. In addition, grading in longitudinal and transverse directions of pavement is also monitored with the gyro system.

CONCLUSION

A four-camera system to cover 5,000 kilometers of lane-mile at full lane-width would require two computers and one-terabyte storage. It is expected that 500 GB single drives will be available in less than two years. At the cost of a high-end laser profiler vehicle, the new system, including four cameras, two high-end microcomputers, and all necessary peripherals and circuits, will be able to collect the vast majority of pavement condition and serviceability data, produce comprehensive survey results, including roughness, cracking, rutting, and other surface deficiencies, at the spatial resolution of 1 to 2-millimeter.

(a) (b)


17

(a)

(b)

(c)

(d)

(e)

FIGURE 14 ILLUSTRATION OF 3D SURFACE MODELING FOR A PAVEMENT SECTION


18

REFERENCES AASHTO (2000), Standard Practice for Quantifying Cracks in Asphalt Pavement Surface,

AASHTO Designation: PP44-00 Fukuhara, T., et al. (1990). "Automatic Pavement-Distress-Survey System," ASCE Journal

of Transportation Engineering, Vol. 116, No. 3, May/June, pp.280-286. Fundakowski, R. A., et al. (1991). Video Image Processing for Evaluating Pavement

Surface Distress, Final Report, National Cooperative Highway Research Program 1-27, September, Washington, D.C.

Guralnick S. A. and Suen, E.S. (1995). Advanced Testing of An Automated NDE System for Highway Pavement Surface Condition Assessment. Project Report of the Idea Program, Transportation Research Board, Washington, D.C.

Monti, Max. (1995), "Large-Area Laser Scanner with Holographic Detector Optics for Real-Time Recognition of Cracks in Road Surfaces" Optical Engineering, July, Vol. 34, No. 7, pp.2017-2023.

Strategic Highway Research Program National Research Council (1993), Distress Identification Manual for the Long-Term Pavement Performance Project

Wang, K.C.P. (2000) “Design and Implementation of Automated Systems for Pavement Surface Distress Survey”, ASCE Journal of Infrastructure Systems, Vol.6, No1, March, pp. 24-32

Wolf and Dewitt (2000). Elements of Photogrammetry with Applications in GIS, 3rd Edition, McGraw Hill

Documents

Automated Pavement Distress Survey