Counting windowsCounting windows

Project participants (in alphabetical order):

• Akif Durdu Middle East Technical University, Turkiye• Viktor Jónás Budapest Polytechnic, Hungary• Csaba Pintér University of Szeged, Hungary • Péter Rieger Budapest Polytechnic, Hungary• Umut Tilki Middle East Technical University, Turkiye

1 2 3 4

765 8

Project OverProject OvervvieiewwMain task was to count windows on the photo of a building

Input:- Image that the user wants to count windows in- A length of the diagonal of an average sized window

(user selects it on the input picture)

Output:- Number of found windows- Output picture containing the original picture and the contours of the found windows

Overview:Overview:

PPreprocessingreprocessing

DDetecting lines with Hough transformationetecting lines with Hough transformation

IIt will define boxes (creates separate quads)t will define boxes (creates separate quads)

unifying “similar” boxesunifying “similar” boxes

CClassify remaining boxes (windows <-> non-lassify remaining boxes (windows <-> non-windows)windows)

Preprocessing:Preprocessing:

CConverting image to grayscaleonverting image to grayscale NNormalizingormalizing MMedian filter (blurring)edian filter (blurring) Sobel operator (finding edges)Sobel operator (finding edges) OOpeningpening

Preprocessing 1 – Preprocessing 1 – Converting to grayscale:Converting to grayscale:

CColor information is not so olor information is not so important during preprocessingimportant during preprocessing

FFasteraster EEasier to handleasier to handle

Preprocessing 2 – Preprocessing 2 – NormalizingNormalizing

GGreater contrastreater contrast MMore determining edgesore determining edges

Preprocessing 3 – Preprocessing 3 – Median filterMedian filter

BBlurs the unimportant edges (e.g. lurs the unimportant edges (e.g. gutters, window frames gutters, window frames (inner+outer=2))(inner+outer=2))

SStill preserves the other edgestill preserves the other edges DDisadvantage: rounds corners (Hough isadvantage: rounds corners (Hough

can compensate)can compensate) PParameters:arameters:

repetitions:repetitions: 33 window size:window size: square root of the user square root of the user

input window diagonalinput window diagonal

Preprocessing 3 – Preprocessing 3 – Median filterMedian filter

Preprocessing 4 – Preprocessing 4 – Sobel operatorSobel operator

FFinds the edgesinds the edges PPrepares the picture for repares the picture for

Hough transformHough transform

Preprocessing 5 – Preprocessing 5 – OpeningOpening

DDilation + erosionilation + erosion RReduces remaining noise educes remaining noise

(unifies broken edges)(unifies broken edges) MMakes possible double akes possible double

frame edges disappearframe edges disappear

Preprocessing 5 – Preprocessing 5 – OpeningOpening

Detecting lines with Detecting lines with Hough transformation:Hough transformation:

AApplying Hough transformationpplying Hough transformation FFinding local maxima (finding inding local maxima (finding

the important lines)the important lines) AApplying inverse Hough pplying inverse Hough

transformationtransformation(projecting found lines back)(projecting found lines back)

Hough 1 – ApplyingHough 1 – Applying

IImage space -mage space ->> Hough space Hough space transformationtransformation

SStrong lines with higher trong lines with higher intensityintensity

Hough 2 – Hough 2 – Finding local maximaFinding local maxima SSeparating dominant lineseparating dominant lines WWindow size: indow size:

3*sqrt(UI diagonal)3*sqrt(UI diagonal)

Hough 3 – Inverse Hough 3 – Inverse Hough transformationHough transformation

Hough space -> image spaceHough space -> image space[inverse Hough formulas][inverse Hough formulas]

PProjecting dominant lines backrojecting dominant lines back IIt defines boxes (general quads)t defines boxes (general quads)

ideal actual

Unifying “similar” Unifying “similar” boxesboxes

CClassify image to boxes lassify image to boxes (floodfill)(floodfill)

computing statisticscomputing statistics unifying “similar” boxesunifying “similar” boxes

(similar <-> their statistics (similar <-> their statistics have little difference)have little difference)

Unifying 1 – Flood fillUnifying 1 – Flood fill IIt classifies image to boxest classifies image to boxes EEach class is a general quadach class is a general quad it is done with a simple floodfill it is done with a simple floodfill

until there are any black spots until there are any black spots remainingremaining

the color values of the areas the color values of the areas represent the classesrepresent the classes

Unifying 2 – StatisticsUnifying 2 – Statistics TThey are used to represent a kind hey are used to represent a kind

of similarity measureof similarity measure SStatistics: mean and variance of tatistics: mean and variance of

each color channelseach color channelssome more statistics: center some more statistics: center coordinates, area (for an easier coordinates, area (for an easier determination of neighbourhood)determination of neighbourhood)

The sample variance Eqn.The sample mean Eqn.

Unifying 3 – Unifying 3 – The unifying itselfThe unifying itself

we must decrease the number we must decrease the number of classes radicallyof classes radically

the reamaining classes may bethe reamaining classes may be whole windows (we hope whole windows (we hope ) ) larger homogeneous areas (roof, larger homogeneous areas (roof,

grass, sky …)grass, sky …)

Unifying 3 – Unifying 3 – The unifying itselfThe unifying itself

TThe unifying algorithmhe unifying algorithmput each class into a queueput each class into a queuewhile (there is a class in the queue) {while (there is a class in the queue) {

while (there is a similar neighbor) {while (there is a similar neighbor) {unify themunify themcompute new statisticscompute new statistics

}}remove from queueremove from queue

}}

Classifying Classifying remaining boxesremaining boxes

we must decide if an area we must decide if an area represents a window or notrepresents a window or not

we do this according to the user we do this according to the user input “ideal” windowinput “ideal” window

the idea:the idea:if the ratio of edges or area is if the ratio of edges or area is reasonable, let’s consider it to be reasonable, let’s consider it to be a window, else it possibly belong a window, else it possibly belong to the class of non-windowsto the class of non-windows

The resultThe result

THE ENDTHE END

Thank YOU

Have a nice day