View
2
Download
0
Category
Preview:
Citation preview
Computer Vision
Pictures as Functions
Image Function (I) maps the values at pixels
I (x, y) -> Returns value for pixel at x,y position in the image
Color Images: I(x, y,) -> Returns array [r, g, b] for pixel
BW Images I(x,y) -> Returns single value for pixel
Pictures as Functions: Python
Pictures as Functions and numpyarrays: Python Color Images• Mathematical
Expression:
I(100, 50) = {61, 150, 184}
• Python Expression
img[50][100]
# Returns:
array([ 61, 150, 184], dtype=uint8)
NOTE! In Mathematical expressions we write I(x, y)
In programming (most languages) we write image arrays as:img_array[row][column]
-That becomes img_array[y][x]
Pictures as Functions and numpyarrays: Grayscale images• Mathematical
Expression:
I(100, 50) = 150
• Python Expression
img_g[50][100]
# Returns value of 150
NOTE! In Mathematical expressions we write I(x, y)
In programming (most languages) we write image arrays as:img_array[row][column]
-That becomes img_array[y][x]
Numpy Arrays in Python
• Numpy Arrays can be written two ways:
• Nested:• img[50][100][3]
• Means Row Index 50, Column Index 100, Blue Channel
• Vectorised• img[50, 100, 3]
Why Vectors? Can call range of Values without For Loops• Slow Way: Iteration
# Nested For Loop
part_img = [[]]
fill = []
for r in range (50, 100):
fill = []
for c in range(60, 80):
fill.append(img_g[r][c])
part_img.append(fill)
• Fast Way: Vectors and Numpy
part_img = img_g[50:100, 60:80]
# Yes – this one line is equal to
# the nested for loop on the left!
# And, this is much faster to
# to process – needed for real time
# Movie processing
Some Python Examples:
Vectorised Array: Set Green and Blue to 0
Vectorised Array: Divide entire picture by 2
Vectorised Array: Get the Horizontal Derivative
Picture Values
• Usually two choices:• 0 to 255: 0 is dark, 255 is completely bright
• 0 to 1: 0 is dark. 1 is Bright. This is sometimes called “Normalized Value” Usefull in statistical analysis.
• Data Types• Be CAREFUL! Images by default are unsigned 8 bit
numbers (0 to 255 with no decimals)
• When doing processing and calculations – convert the image array to a float 32 bit number before the math. Or, the values will be lost in calculations!
Python Language Guide
• Interpreted: Does not need to be compiled
• Sequential – Define functions before using
• Object Orientated (Like Java)• Both Java and Python are built on C and C++
• Can code short ideas in real time in Interpretter
• Write Longer Text Files and Packages (Called Modules)
Python Language Guide
# Single Line comment
“”” Multi line
Comment “””
# No semicolons in Python!
(But they are OK to use . . .)
Python Variables
• Same datatypes as Java (Boolean, int, float, double, String . . .)
• Do not need to declare type of variable before creating
• Java:int age = 14;
• Pythonage = 14
Python (and Numpy) Lists / Arrays
• Pure Python Arrays are mutable (Like ArrayList or List in Java)
students = [“John”, “Frank”, “George”]
# Note use of brackets for array.
students.append(“Mr. Michaud”)
# Adds “Mr. Michaud” to List
Python Lists(Arrays)
• Python Indexing of Lists/Arrays is like java.
• Zero based Indexing
students = [“John”, “Frank”, “George”]
# Note use of brackets for array.
students[0] # Returns “John”
Python List Functions
• len(array) will return length of array
numbers = [3, 8, 7]
len(numbers) # returns 3
# Numpy: min, max, sum
numbers.min() # Returns 3
numbers.max() # Returns 8
sum(numbers) # Returns 18
Numpy Array Operations
import numpy as np #Import numpy library as np
img = cv2.imread(path, 0)
# Grayscale image as numpy array
len(img) # returns number of rows in np array
len(img[0]) # returns number of columns in np array
np.where(array == value)
# Returns Array object with point location of value
Control Structures in Python
• Iteration:
• Java
for (int i = 0; i < 10: i++) {
System.out.println(i);
}
• Python
for i in range(10):
print (i)
If Statements in Python# Standard If statement
if (x > 200):
pin9.write(120)
# Else Statements
if (x > 200):
pin9.write(190)
else if(x < 150):
pin9.write(60)
else:
pin9.write(90)
While Statements
# While Statement: 100 Frames shown
while(x < 100):
ret, frame = cap.read()
if (ret == true):
cv2.imshow(“Picture”, frame)
x = x + 1
Defining Functions in Python
• Do not need to specify return types
• Keyword is “def”
def addTwoNumbers(x, y):
return x + y
Longer Function Example
# Function to return dx
def getDX(frame):
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
diff = np.float32(gray[:, 2:len(gray[0])]) - np.float32(gray[:, 1:len(gray[0])-1])
out = diff + -1*diff.min()
out = out / out.max()
return out
Python and Tabs
• Python uses Tabs to encapsulate code. (No curly brackets in Python)
• For loops, conditionals, while loops, and functions all use :’s and tabs to encase code.
• Be very careful with tabs in Python. If you get a parse error – your tabs are not consistent.
Open Computer Vision 2
• Library to handle image Processing
• Uses Numpy and Scipy for graphics and computer vision.
• Provides access to cameras
• Reading and writing image files
• Creating Image arrays
• Filters, Harr Cascades, and many other features used in Computer Vision.
• Standard Library for Research at University and Industry applications.
Selected Open CV and numpy API commandsimport cv2 as cv2
import numpy as np
#Imports CV2 Library as object cv2
cv2.imread()
cv2.cvtColor()
cv2.matchTemplate()
cv2.imshow
cv2.rectangle
cv2.VideoCapture()
cv2.waitKey()
cv2.destroyAllWindows()
np.copy()
Open CV File Operationscv2.imread(pathToFile, channels)
pathToFile: Address in file system to image
channels: Number of colors
3: R, G, B Full Color
0: Grayscale (One Channel)
Returns an Image array
Example:
#Path to image
path = "C:\Users\michaudc\OneDrive\BCT 440 Computational Perception\images\lisa.png"
#Open Image and assign to np array. Color Image
img = cv2.imread(path, 3)
Open CV Image Operations
cv2.cvtColor(image, ConversionType)
Converts between color and grayscale images.
Returns an Image array
Example:
frame = cv2.imread(path, 3) # color image
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
Open CV Image Operations
cv2.rectangle(image, (x1,y1), (x2, y2), color, thickness)
Draws a rectangle on an image file. Color is (B, G, R) tuple. x1 and y1 are upper left corner. x2 and y2 are lower right corner.
Example:cv2.rectangle(image, (x, y), (x+50, y+50), (0, 0, 255), 2)
Open CV Image Output
cv2.imshow(‘WindowName’, image)
‘WindowName’: Name of Window
image: numpy array object
Example
cv2.imshow(‘My Picture’, image)
Writing images to files
Example:
• Convert image array to 0-255 np.uint8 datatypeimg = img * 255
img = np.uint8(img)
• Use the cv2.imwrite(path, image) to write to filecv2.imwrite(‘output\name.png’, img)
cv2.imwrite(pathToFile, imageArray)
Template Matching (More on this later . . .)cv2.matchTemplate(img_g, patch, type)
img_g: Grayscale image to be searched
patch: Image patch to search for
type: Type of of search (Normalized correlation, Sum of Squared Differences . . .)
Returns a map of values from the filter
Example:f = cv2.matchTemplate(gray, patch, cv2.TM_CCORR_NORMED)
Using the Video Camera
cv2.VideoCapture(camera)
Returns a Camera object for video image capture.
camera: Number assigned to camera in system. Usually ‘0’ is the internal camera and ‘1’ is an external USB camera.
Example:
cap = cv2.VideoCapture(0)
Using the Video Camera
cameraObject.read()
Returns two objects to program:
Boolean value if the camera is working
Numpy Image array of captured image
Example
ret, frame = cap.read()
Create Numpy Array
np.array(arrayData)
Creates a Numpy array
Example: Creates a 2 x 4 array of integers
img = np.array([[4, 3, 5, 6],
[3, 5, 7, 8]])
Convert Datatype in Numpy Array to 32 bit floating pointnp.float32(arrayData)
Converts incoming data to floating point 32 bit numbers. Returns the an array with the converted and copied data
Example:
patch = np.float32(img)
Copy Numpy Arrays (image data)
np.copy(numpyArray)
Takes input of numpy array and creates a new numpyarray and copies the data.
Example:
# img is an existing numpy image array
img_copy = np.copy(img)
2D Filter Function (Convolution)
cv2.filter2D(img, -1, kernel)
Takes input of grayscale img data filters with kernel.
The kernel is an m x m array used to filter the data. Usually a Gaussian, Laplacian, Sorbel, or box filter.
Example:
# img is an existing numpy image array
kernel = np.ones((5, 5), np.float32)/25
box_blur = cv2.filter2D(img, -1, kernel)
CV2 Canny Edge Function
• Built in Edge detection function in Open CV that uses a more involved algorithm:• Filters image with derivative of Gaussian
• Uses magnitude and orientation of gradients
• Non-Maximum suppression
• Linking and thresholding
• Result is a more refined edge image.
img_edge_canny = cv2.Canny(img_gauss, 50, 100)
Gaussian Blur: Role of Sigma
• We will blur images with the cv2.GaussianBlur() function.
cv2.GaussianBlur(img, (m x m), sigma)
Where:
img = Image Array to Blur
(m x m): size of ‘window’ of Gaussian (odd number)
sigma: The spread of the blur (High for more depth)
2D Gaussian Filter Image
Example:
Why Blur with Gaussian?
• Remove Noise from the Picture.
Code Sample: Blur with Gaussian
Code Sample: Canny EdgeNote: You will have to experiment with the parameters for GaussianBlur() and Canny() to get a good edge image.
Exit Sequences
• Open CV will “lock” your python when showing images. You will need “Exit Sequences” to cleanly exit running programs to preserve data and not crash your python apps.
• Two Kinds: Non-Looping and Looping• Non-Looping: For Still Images
• Looping: when using the Camera Object
Non-Looping Exit Sequence
Looping Exit Sequence
Thank You!
• Many thanks to Charlie Fligg (Class of ’20) for researching Python and Computer Vision and helping gather software, materials, and tinkering with open CV2 for the Perception Class!
Recommended