Amazon My eye finding program, given an image with a human's eyes in it, is supposed to tell you where eyes of people are located. As you can see finding and locating eyes reliably is extremely easy for people to do but difficult for machines to do.

I used Matlab to develop my eye detection algorithms because Matlab has a very useful image toolbox that I can use to my advantage. My eye detection program is broken down into two stages: find the face, and then find the eyes. To find the face, I designed two face finding algorithms: one utilizing the high symmetry of a human face and the other using average face matching.

In the first algorithm I scan the image for the region that has the best symmetry, and then perform some optimization to get better results. Also I apply a skin filter (2 types the user can choose from) to the face detection window to further shrink down the search space.

In the second algorithm I scan the image for the region that has the best correlation with the given average face template. To find eyes I also implemented two ways: rule based one and average eye matching.

The rule-based method simply means the program tries to find two clusters of dark pixels (eyes are dark...) with some constraints applied to them. It takes too long and is highly error prone.

The average eye matching, however, is extremely fast and gives accurate results, but user must supply the approximate ratio of the face size to the image size. This requirement renders this method less practical because ideally, user shouldn't tell the program anything other than the image itself.

I made it so that the user can choose any combination of algorithms and methods via command-line options. I tested my system with thirty images and averaged eighty percent accuracy. The following is a snapshot of running my eyes locating program (symmetry for face finding and average-eye matching for eye finding):

The following is a snippet of the source code for symeye.m, the main function for the eye detection program. function [coordinates]=symeye(imfile,display,exh_search,method,eye_method);%imfile: file name of the given image, which must be 3-colored%display: 1 if you want to see the process of reaching the eye location;% other values if you do not (1 makes it run more slowly)%exh_search: 1 if you want exhaustive search in the image for face; other% values if you want only horizontal search, assuming face appears on the% top of the image (1 makes it run more slowly)%method: skin filter method; 0 for skin texture map (slower but better results) % and others for thresholding in YCbCr color space (much faster but worse% results)%eye_method: eye finding method; 0 for average eye matching method and% others for rule-based method

fprintf('Your arguments:\n');

fprintf('File to process: %s\n',imfile);if display==1 fprintf('Display the finding process.\n');else fprintf('Do not display the finding process.\n');end

if exh_search==0 fprintf('Do only horizontal face window search.\n');else fprintf('Do exhaustive horizontal and vertical face window search.\n');end

if method==0 fprintf('Use skin texture map to filter out non-skin regions.\n');else fprintf('Use YCbCr color space to filter out non-skin regions.\n');end

if eye_method==0 fprintf('Use eye template matching to locate eyes.\n');else fprintf('Use rule-based method to locate eyes.\n');end

fprintf('\n');

%dark weights we generateddark_weights=[.05+.09i .053+.085i .06+.08i .07+.075i .08+.07i .025+.19i .08+.22i .09+.39i .11+.38i .18+.36i;.18+.4i .21+.4i .14+.47i .12+.53i .11+.54i .11+.55i .12+.56i .14+.52i .16+.52i .155+.54i;.155+.55i .155+.56i .21+.49i .19+.5i .21+.5i .2+.51i .2+.52i .19+.54i .2+.54i .19+.55i;.26+.42i .25+.44i .25+.455i .25+.49i .24+.5i .255+.51i .22+.51i .23+.53i .225+.53i .26+.46i;.26+.47i .27+.48i .275+.445i .28+.46i .29+.47i .31+.495i .33+.505i .325+.51i .320+.515i .315+.52i;.95+.09i .947+.085i .94+.08i .93+.075i .92+.07i .975+.19i .92+.22i .91+.39i .89+.38i .82+.36i;.82+.4i .79+0.4i .86+.47i .88+.53i .89+.54i .89+.55i .88+.56i .86+.52i .84+.52i .845+.54i;.845+.55i .8450+.56i .79+.49i .81+.5i .79+.5i .8+.51i .8+.52i .81+.54i .8+.54i .81+.55i;.74+.42i .75+.44i .75+.455i .75+.49i .76+.5i .745+.51i .78+.51i .77+.53i .775+.53i .74+.46i;.74+.47i .73+.48i .725+.445i .72+.46i .71+.47i .69+.495i .67+.505i .675+.51i .68+.515i .685+.52i];

%bright weights we generatedbright_weights=[.06+.75i .09+.805i .11+.84i .12+.82i .12+.87i .14+.86i .16+.84i .15+.89i .17+.87i .175+.89i;.21+.835i .21+.9i .22+.87i .345+.84i .475+.875i .49+.86i .16+.62i .155+.56i .17+.54i .2+.555i;.2+.54i .205+.51i .22+.575i .22+.55i .23+.535i .255+.575i .275+.55i .475+.475i .49+.44i .485+.39i;.465+.37i .43+.37i .42+.325i .39+.325i .325+.34i .35+.31i .375+.3i .36+.29i .39+.27i .4+.25i;.3+.32i .32+.29i .335+.275i .35+.25i .355+.255i .36+.22i .37+.24i .375+.2i .345+.275i .325+.29i;.94+.75i .91+.805i .89+.84i .88+.82i .88+.87i .86+.86i .84+.84i .85+.89i .83+.87i .825+.89i;.79+.835i .79+.9i .78+.87i .655+.84i .525+.875i .51+.86i .84+.62i .845+.56i .83+.54i .8+.555i;.8+.54i .795+.51i .78+.575i .78+.55i .77+.535i .745+.575i .725+.55i .525+.475i .51+.44i .515+.39i;.535+.37i .57+.37i .58+.325i .61+.325i .675+.34i .65+.31i .625+.3i .64+.29i .61+.27i .6+.25i;.7+.32i .68+.29i .665+.275i .65+.25i .645+.255i .64+.22i .63+.24i .625+.2i .655+.275i .675+.29i;];

gray255map=gray(255);

[coordinates,bw_map]=recur(imfile, bright_weights, dark_weights, gray255map, ... 0, display, exh_search, method, eye_method);

function [coordinates,bw_map]=recur(imfile, bright_weights, dark_weights, ... gray255map, recur_factor, display, exh_search, ... method, eye_method);

%clock starts time~tic;

%store pixel values of different format in arraysimrgb=im2double(imread(imfile));imrgb_255=double(imread(imfile));[imh, imw, imnc]=size(imrgb);

%report error if image is not 3-color channeledif imnc~=3 fprintf('%s is not a 3-color image; cannot use skin filter, forced exit\n',... imfile); coordinates=[]; bw_map=[]; return;end

max_face=0.7; %max percentage size of face in input imagemin_face=0.2; %min percentage size of face in input imagebig=min(imh,imw); %get the smaller valuemax2=round(big*max_face);%max pixel width/height of face in input imagemax2=max2+mod(max2,2); %make even for facevalmin2=round(big*min_face); %min pixel width/height of face in input imagejump= round(imw*0.015); jump= jump+mod(jump,2); %make even for faceval

%use image() instead of imshow() because image() displays calibrationfigure, image(imrgb),title('Orignal Image');axis([0 imw 0 imh]); axis ij; axis equal; axis tight;

y=0; x=0;%if user wants to see the process, get graphics window's handlerif display==1 rh=rectangle('Position',[x,y,max2,max2]); set(rh,'EraseMode','xor'); %set window's color to black set(rh,'EdgeColor','k'); end

detection=-10; %highest value for faceness that have obtaineddetection2=-10;faceness=-10; %value returned by facevalwinsize=max2; %biggest winsize (max)winsize=uint32(0.7*min(imh,imw));winsize=winsize+mod(winsize,2);stop_threshold=100;

%begin searching for the window with best symmetry valuey=1;x=1;if display==1 title('Searching for best symmetry...');end

%either exhaustive search or just top horizontal searchif exh_search==1 while x+winsizedetection2 & detection>faceness detection2=faceness; coordinates2=[detection2,x,y,winsize,winsize]; end y=y+jump; end x=x+jump; end else while x+winsizedetection detection=faceness; coordinates=[detection,x,y,winsize,winsize]; end

%try to get second best window if faceness>detection2 & detection>faceness detection2=faceness; coordinates2=[detection2,x,y,winsize,winsize]; end

x=x+jump; end

end

...

AssumptionsI wrote two functions that use two different eye locating algorithms. In the first approach, symeye.m, you only need to specify some control parameters (type help symeye for help). You can only input 3-channel color image in this function due to skin filtering method. In the second function, avgeye.m, you need to specify the approximate ratio of face size to image size (type help avgeye for help). You can use color or gray-scale images in this function.

I assume that each image contains one face which covers at least 20% of the image and at most 70% of the total image size so that it is not too big or small (although face violating these size constraints may nonetheless be detected successfully).

I assume frontal, upright view of the face. Image size should be around 300 by 300 pixels to allow efficiency.

In the first function, the face should be located higher than the center horizontal line of the image to allow maximum efficiency. If the face is very low, use option exh_search = 1 to enable exhaustive search vertically and horizontally. Directions

Add path of the source code directory via addpath command. For example if our source code is in C:\eye_detection\, then do addpath C:\eye_detection\images\, addpath C:\eye_detection\code\, addpath C:\eye_detection\code\m1, and addpath C:\eye_detection\code\m2.

The picture files should reside in C:\eye_detection\images\ and function 1 should reside in C:\eye_detection\code\m1\ and function 2 should reside in C:\eye_detection\code\m2\.

Do symeye('1.jpg',1,0,0,0) to run the first function and avgeye('1.jpg',0.4) to run the second function on '1.jpg'.

You can run the script I wrote as following to run a range of pictures: Run function 1: eyescript(0,1,30,[0,0,0,0],'.jpg'); This means you run the function 1 on 1.jpg, 2.jpg, ..., 30.jpg with args [0,0,0,0].

Run function 2: eyescript(1,1,30,[.5,0],'.jpg'); This means you run the function 2 on 1.jpg, 2.jpg, ..., 30.jpg with args [.5,0].

For further help please type "help eyescript" in matlab. Caveats

In function 2 if your image is too large you will get a warning saying the image is too large and is displayed at a certain size. In that case you may not see the eye detection window clearly; you will need to make the matlab window larger to see it.

Occasionally you may get an error due to the image you select. If it happens simply use another image.

Here's the entire source code for my eye finding algorithms. Use at your own risks.