Scientific Computing with MATLAB3. Manipulating Data Sets

Dra. K.-Y. Daisy FanDepartment of Computer ScienceCornell UniversityCornell UniversityIthaca, NY, USA

UNAM IIM 2012

Working with data and files

Data that you have created or manipulated using MATLAB can be saved to a data file for future use. MATLAB can be saved to a data file for future use. Or you may use data provided by others. The MATLAB internal format mat is convenient The MATLAB internal format, mat, is convenient

if you and the future user uses MATLABTh ASCII i i l b f i The ASCII type is universal, but often requires more low-level work

Spreadsheet format is commonly used Image data comes in many form, with JPEG as one g y J

of the most commonly used.3

By the end of this lecture, you will be able to …

Save Workspace variable values to MATLAB’s .mat formatted file; import .mat data to .mat formatted file; import .mat data to Workspace

Save Workspace matrices to ASCII files; import Save Workspace matrices to ASCII files; import ASCII files as matricesI d f d h Import data from a spreadsheet

Import an image file, process the data, and write back an image file

Work with irregularly formatted ASCII filesg y

4

Saving Workspace data to file

save filename

Simple way to save data with basic formats Simple way to save data with basic formats Options include file type and saving select

i bl S lvariables. Some examples:% myFile.mat stores all variable values:sa e m Filesave myFile

% myFile.mat stores values of variables x,u:save myFile x, uy ,

% myFile.txt stores values of matrices ma and% mi in ASCII (plain text) format:

Fil t t i ii

5

save myFile.txt ma mi -ascii% matrices must have same no. of columns!

Importing .mat data into Workspace

load filenameOptions include loading only specified variables Examples:Options include loading only specified variables. Examples:% Load all variable values from myFile.mat:load myFileoad y e

% Load values from myFile.txt as a matrix:load myFiley% Any file with an extension that is not% .mat is treated as ASCII file.% Matrix name now in workspace is myFile% Matrix name now in workspace is myFile.

% Load values from myFile.dat as a matrix:load myDat dat

6

load myDat.dat% Matrix name now in workspace is myDat

Importing data from a spreadsheet (xls)

[ar, txt, raw] = xlsread(filename)

Options include specifying the range sheet Options include specifying the range, sheet Filename does not need to include extension .xls Which versions of Excel spreadsheets?

If your machine has Excel for windows—all versions supported by that version of Excel, e.g., xls, xlsx, …

If your machine does not have Excel for windows—basic functionality only (no range, no sheet, i.e., Excel 97-2003)

7

Importing data from a spreadsheet (xls)

[ar, txt, raw] = xlsread(filename)

ar – a numeric array storing only the numeric l i d t i t d N Nvalues; non-numeric data is stored as NaN

txt – a cell array storing only the text (non-numeric data); non-text is stored as the empty string, ’’

raw – a cell array storing all the data; each cell is numeric or text depending on the datap g

8

Single quotes enclose strings in Matlab

Anything enclosed in single quotes is a string (even f l k l k h l )if it looks like something else)

‘100’is a character array (string) of length 3 100 is a numeric value ‘pi’ is a character array of length 2 pi is the built-in constant 3.1416… ‘x’ is a character (vector of length 1) x may be a variable name in your program

9

Strings are vectorsVectors Strings

Assignmentv= [7 0 5];

Indexing

g Assignment

s= ‘hello’;

Indexing Indexingx= v(3); % x is 5v(1)= 1; % v is [1 0 5]

(2 3) % i [0 5]

Indexingc= s(2); % c is ‘e’s(1)= ‘J’; % s is ‘Jello’t (2 4) % t i ‘ ll’w= v(2:3); % w is [0 5]

: notationv= 2:5; % v is [2 3 4 5]

t= s(2:4); % t is ‘ell’

: notations= ‘a’:‘g’; % s is ‘abcdefg’

Appendingv= [7 0 5];v(4)= 2; % v is [7 0 5 2]

Appendings= ‘duck’;s(5)= ‘s’; % s is ‘ducks’( ) ; [ ]

Concatenationv= [v [4 6]];

% v is [7 0 5 2 4 6]

( ) ;

Concatenations= [s ‘ quack’];

% s is ‘ducks quack’

10

% v is [7 0 5 2 4 6] % s is ducks quack

Some useful string functionsstr= ‘Cs 1112’;str= Cs 1112 ;

length(str) % 7isletter(str) % [1 1 0 0 0 0 0]isspace(str) % [0 0 1 0 0 0 0]lower(str) % ‘cs 1112’lower(str) % ‘cs 1112’upper(str) % ‘CS 1112’

ischar(str)% Is str a char array? True (1)

t ( t (1 2) ‘ ’)strcmp(str(1:2),‘cs’)% Compare strings str(1:2) & ‘cs’. False (0)

strcmp(str(1:3),‘CS’)

11

strcmp(str(1:3), CS )% False (0)

Characters arrays

We have used strings already: n= input(‘Next number: ’) sprintf(‘Answer is %d’, ans)

A string is made up of individual characters, so a string is g p , ga 1-d array of characters

‘UNAM iim2012!’ is a character array of length 13; it has 7 letters, 4 digits, 1 space, and 1 symbol.

‘U’‘N’‘A’ ‘ ’‘M’ ‘i’ ‘m’‘2’‘0’‘1’‘2’‘!’‘i’

Can have 2-d array of characters as well‘u’‘n’‘a’‘m’

‘2’‘0’‘1’‘2’ 2×4 matrix

Matrix vs. Cell Array A cell array is a special array

Vectors and matrices store values of the same type in

whose individual components may contain different types of datayp

all componentsdata

-4 -1 ‘ ’‘ ’ ‘ ’3.1

2‘c’ ‘o’ ‘m’ ‘ ’ ‘s’

‘1’ ‘1’ ‘1’ ‘2’ ‘ ’

-4 -1 .

‘m’‘c’ ‘o’

-1

9

1 1‘ ’ ‘ ’ ‘L’ ‘A’ ‘B’

‘M’ ‘a’ ‘t’ ‘ ’ ‘ ’

.91

5 7

1.1

.4‘M’ -1 74 x 5matrix5 x 1

matrix

13

3 × 2 cell array

Cell Arrays of Strings

C= { ‘Alabama’,’New York’,’Utah’}

C ‘Alabama’ ‘New York’ ‘Utah’

C= { ‘Alabama’;’New York’;’Utah’}

C i h ‘Alabama’

‘New York’

C

M= [‘Alabama ’; ...

Contrast with 2-d array of characters

‘Utah’1-d cell array of

‘New York’; ...‘Utah ’]

‘A’ ‘l’ ‘a’ ‘b’ ‘a’ ‘m’ ‘a’ ‘ ‘My

strings ‘N’ ‘e’ ‘w’ ‘ ‘ ‘Y’

‘U’ ‘t’ ‘h’

‘o’ ‘r’ ‘k’

‘ ‘‘a’ ‘ ‘‘ ‘‘ ‘

M

Use braces { } for creating and addressing cell arrays

Matrix

Create

Cell Array

Create Create

m= [ 5, 4 ; …1 2

Create

C= { ones(2,2), 4 ; …‘ b ’ (3 1) 1, 2 ; …

0, 8 ]‘abc’ , ones(3,1) ; …9 , ‘a cell’ }

Addressing

m(2 1)= pi

Addressing

C{2 1}= ‘ABC’m(2,1) pi C{2,1} ABCC{3,2}= pidisp(C{3,2})

15

A picture as a matrix

1458-by-2084

150 149 152 153 152 155150 149 152 153 152 155151 150 153 154 153 156153 151 155 156 155 158154 153 156 157 156 1591 6 1 1 8 1 9 1 8 161

16

156 154 158 159 158 161157 156 159 160 159 162

Images can be encoded in different ways

Common formats include JPEG: Joint Photographic Experts GroupJ J g p p p GIF: Graphics Interchange Format

Data are compressed p We will work with jpeg files:

imread: read a .jpg file and convert it to a “normal imread: read a .jpg file and convert it to a normal numeric” array that we can work with

imwrite: write an array into a .jpg file (compressed data)

17

Grayness: a value in [0..255]

0 = black255 = white

These are integer valuesType: uint8Type: uint8

150 149 152 153 152 155150 149 152 153 152 155151 150 153 154 153 156153 151 155 156 155 158154 153 156 157 156 1591 6 1 1 8 1 9 1 8 161

18

156 154 158 159 158 161157 156 159 160 159 162

Reading a jpeg file and displaying the image

% Read jpg image and convert to % an array P

P = imread(‘bwduck.jpg');jpg

% Show the data in 3-d array P as% Show the data in 3-d array P as % an image

imshow(P)

19

A color picture is made up of RGB matrices 3-d array114 114 112 112 114 111 114 115 112 113114 113 111 109 113 111 113 115 112 113115 114 112 111 111 112 112 111 112 112116 117 116 114 112 115 113 112 115 114113 112 112 112 112 110 111 113 116 115115 115 115 115 113 111 111 113 116 114112 113 116 117 113 112 112 113 114 113115 116 118 118 113 112 112 113 114 114115 116 118 118 113 112 112 113 114 114116 116 117 117 114 114 112 112 114 115

153 153 150 150 154 151 152 153 150 151153 152 149 147 153 151 151 153 150 151154 153 151 150 151 152 150 149 150 150155 156 155 152 152 155 151 150 153 153151 150 150 150 150 148 149 151 152 151153 153 153 153 151 149 149 151 152 150150 151 152 153 151 150 150 151 152 151153 154 154 154 151 150 150 151 152 152154 154 153 153 149 149 150 150 152 153

212 212 212 212 216 213 215 216 213 213212 211 211 209 215 213 214 216 213 213213 212 210 209 212 214 213 212 213 212213 212 210 209 212 214 213 212 213 212214 215 214 214 213 216 214 213 215 212213 212 212 212 212 210 211 213 214 211215 215 216 216 213 211 211 213 212 210212 213 214 215 213 212 212 213 214 213215 216 216 216 213 212 212 213 214 214216 216 215 215 213 213 213 213 214 215

0 ≤ A(i,j,1) ≤ 255

0 ≤ A(i,j,2) ≤ 255E.g., color image data is stored in a 3-d array A:

0 ≤ A(i,j,3) ≤ 255

20

A color picture is made up of RGB matrices 3-d array

Color image 3-d Array

0 ≤ A(i,j,1) ≤ 255

0 ≤ A(i,j,3) ≤ 255

0 ≤ A(i,j,2) ≤ 255

O ti i t t ti

( ,j, )

Operations on images amount to operations on matrices!

21

Problem: produce a negative

22

Problem: produce a negative

“Negative” is what we say, but all color values are positive numbers!p

Think in terms of the extremes, 0 and 255. Then the “negative” just means the opposite side.g j pp

So 0 is the opposite of 255;1 … 254;1 … 254;5 … 250;30 225;30 … 225;x … 255-x

23

function newIm = toNegative(im)% newIm is the negative of image im% im, newIm are 3-d arrays; each component is uint8

[nr,nc,np]= size(im); % dimensions of imnewIm= zeros(nr,nc,np,’uint8’); % initialize newIm

for r= 1:nrfor r= 1:nrfor c= 1:nc

for p= 1:npnewIm(r,c,p)= ___________________;

endend

end

24

function newIm = toNegative(im)% newIm is the negative of image im% im, newIm are 3-d arrays; each component is uint8

[nr,nc,np]= size(im); % dimensions of imnewIm= zeros(nr,nc,np,’uint8’); % initialize newIm

for r= 1:nrfor r= 1:nrfor c= 1:nc

for p= 1:np2newIm(r,c,p)= 255 - im(r,c,p);

endend

end function newIm = toNegative(im)% newIm is the negative of image im% im, newIm are 3-d uint8 arrays

25

newIm = 255 – im; % vectorized code!

Finding Edges

26

What is an edge?

Near an edge, grayness values h b lchange abruptly

200 200 200 200 200 200200 200 200 200 200 100200 200 200 200 100 100200 200 200 200 100 100200 200 200 100 100 100200 200 100 100 100 100200 100 100 100 100 100

27

General plan for showing the edges in in image

Identify the “edge pixels” Highlight the edge pixels Highlight the edge pixels

make edge pixels white; make everything else black

200 200 200 200 200 200200 200 200 200 200 100200 200 200 200 100 100200 200 200 200 100 100200 200 200 100 100 100200 200 100 100 100 100200 100 100 100 100 100

28

General plan for showing the edges in in image

Identify the “edge pixels” Highlight the edge pixels Highlight the edge pixels

make edge pixels white; make everything else black

200 200 200 200 200 200200 200 200 200 200 100200 200 200 200 100 100BLACK

200 200 200 200 100 100200 200 200 100 100 100200 200 100 100 100 100BLACK200 100 100 100 100 100BLACK

29

Finding Edges

30

The Rate-of-Change-Array

Suppose A is an image array with integer values between 0 and 255.Let B(i,j) be the maximum difference between jand its eight neighbors.So B(i,j) is the maximum value inSo B(i,j) is the maximum value in

A( (1 i 1) i ( i+1)A(max(1,i-1):min(m,i+1),...

max(1,j-1):min(n,j+1)) - A(i,j)

31

Neighborhood of A(i,j)

Rate-of-change example

90 6581Rate-of-change at

iddl i l i 3059

58

60

56

62

57

middle pixel is 30

5856 57

Be careful! In “uint8 arithmetic”57 – 60 is 0

32

The Pixel (i,j) Neighborhood

iMin = i-riMax = i+riMax = i+rjMin = j-rjMax = j+rC = A(iMin:iMax,jMin:jMax)( ,j j )

r = 1 r = 2

Am

33

r = 1 r = 2n

The Pixel (i,j) Neighborhood

iMin = max(1,i-r)iMax = min(m i+r)iMax = min(m,i+r)jMin = max(1,j-r)jMax = min(n,j+r)C = A(iMin:iMax,jMin:jMax)( ,j j )

r = 1 r = 2

Am

34

r = 1 r = 2n

Edge finding: Rate-of-change matrix

A: grayness values B: rate-of-change values

Aij

i

j

Bij

Ν|| AAB

j

Let N be the set of subscripts (p,q) that is the neighborhood of (i,j)

Ν),(||max qpijpqij AAB

Edge finding: Rate-of-change matrix

A: grayness values B: rate-of-change values E: edge values

Aij

i

j

Bij Eij

Ν|| AAB

j

Let N be the set of subscripts (p,q) that is the neighborhood of (i,j)

Ν),(||max qpijpqij AAB

High rate of change Abrupt change in grayness An edgeg g p g g y g

ij

ij BblackBwhite

Eifif

ijBblack if

function Edges(jpgIn,jpgOut,tau)% jpgOut is the “edge diagram” of image jpgIn.jpg g g g jpg% At each pixel, if rate-of-change > tau% then the pixel is considered to be on an edge.A i d(j I ) fi i h (A)A = imread(jpgIn); figure; imshow(A)A = rgb2gray(A);[m,n] = size(A);

Built-in function to convert to grayscale [ , ] ( );

B = zeros(m,n,’uint8’);for i = 1:m

convert to grayscale. Returns 2-d array.

for j = 1:n

B(i j) = ?????B(i,j) = ?????

end

38

end

Recipe for rate-of-change B(i,j)

% The 3-by-3 subarray that includes A(i,j) % and its 8 neighbors (for an interior pixel)Neighbors = A(i-1:i+1,j-1:j+1);

% Subtract A(i,j) from each entryDiff= abs( double(Neighbors) – double(A(i,j)) );

% Compute largest value in each columncolMax = max(Diff);co a a ( );% Compute the max of the column max’sB(i,j) = max(colMax);

39

function Edges(jpgIn,jpgOut,tau)% jpgOut is the “edge diagram” of image jpgIn.% At each pixel, if rate-of-change > tau% At each pixel, if rate of change > tau% then the pixel is considered to be on an edge.A = imread(jpgIn); figure; imshow(A)A = rgb2gray(A);[m,n] = size(A);B = zeros(m,n,’uint8’);for i = 1:m

for j = 1:nNeighbors = A(max(1,i-1):min(i+1,m), ...

max(1,j-1):min(j+1,n));

endend

41

end

function Edges(jpgIn,jpgOut,tau)% jpgOut is the “edge diagram” of image jpgIn.% At each pixel, if rate-of-change > tau% At each pixel, if rate of change > tau% then the pixel is considered to be on an edge.A = imread(jpgIn); figure; imshow(A)A = rgb2gray(A);[m,n] = size(A);B = zeros(m,n,’uint8’);for i = 1:m

for j = 1:nNeighbors = A(max(1,i-1):min(i+1,m), ...

max(1,j-1):min(j+1,n));B(i j) ( ( b (d bl (N i hb )B(i,j)=max(max(abs(double(Neighbors)– ...

double(A(i,j)))));

endend

42

end

“Edge pixels” are now identified; display them with maximum brightness (255)

A

1 1 1 1 1 11 1 1 1 1 1

A

if B(i j) > tau

threshold1 1 1 1 1 11 1 1 1 90 901 1 1 90 90 901 1 90 90 90 90

if B(i,j) > tauB(i,j) = 255;

end 1 1 90 90 90 901 1 90 90 90 90

B(i j)0 0 0 0 0 00 0 0 89 89 89

B(i,j)0 0 0 0 0 00 0 0 255 255 255

0 0 89 89 0 00 89 89 0 0 00 89 0 0 0 0

0 0 255 255 0 00 255 255 0 0 00 255 0 0 0 0

43

0 89 0 0 0 00 89 0 0 0 0

0 255 0 0 0 00 255 0 0 0 0

function Edges(jpgIn,jpgOut,tau)% jpgOut is the “edge diagram” of image jpgIn.% At each pixel if rate of change > tau% At each pixel, if rate-of-change > tau% then the pixel is considered to be on an edge.A = imread(jpgIn); figure; imshow(A)A = rgb2gray(A);A = rgb2gray(A);[m,n] = size(A);B = zeros(m,n,’uint8’);for i = 1:mfor i = 1:m

for j = 1:nNeighbors = A(max(1,i-1):min(i+1,m), ...

max(1 j 1):min(j+1 n));max(1,j-1):min(j+1,n));B(i,j)=max(max(abs(double(Neighbors)– ...

double(A(i,j)))));if B(i j) > tif B(i,j) > tau

B(i,j) = 255;end

44

endend

function Edges(jpgIn,jpgOut,tau)% jpgOut is the “edge diagram” of image jpgIn.% At each pixel if rate of change > tau% At each pixel, if rate-of-change > tau% then the pixel is considered to be on an edge.A = imread(jpgIn); figure; imshow(A)A = rgb2gray(A);A = rgb2gray(A);[m,n] = size(A);B = zeros(m,n,’uint8’);for i = 1:mfor i = 1:m

for j = 1:nNeighbors = A(max(1,i-1):min(i+1,m), ...

max(1 j 1):min(j+1 n));max(1,j-1):min(j+1,n));B(i,j)=max(max(abs(double(Neighbors)– ...

double(A(i,j)))));if B(i j) > tif B(i,j) > tau

B(i,j) = 255;end

45

endendimwrite(B,jpgOut,’jpg’)

tau = 30

46

Edge finding: Effect of edge threshold,

tau = 20