1

OpenCV

Computer Vision API for everyone

Vadim PisarevskyIntel Corporation

2

What is OpenCV? OpenCV stands for Open Source Computer Vision

Library It is a collection of computer vision, image

processing and general-purpose numeric algorithms.

Very efficient (implemented in C/C++) Has BSD-like license (that is, absolutely free for

academic and commercial use) Available at http://

sourceforge.net/projects/opencvlibrary

3

Why OpenCV? Provide de-facto standard API as a base for

image processing and computer vision applications. Create new usage models for PC’s, mobile platforms.

Make IA attractive for such applications by using Intel Performance Libraries to improve OpenCV performance

4

The Project History“CVL” project was started; the main goals/features:

• Human-Computer interface is a main target• Real-Time Computer Vision Library for using by UI Developers, Videoconferences, Games• Highly optimized for IA.

1999/01

OpenCV alpha 3 released and presented at CVPR’00

2000/06 2000/12

Continuous development and various research projects

OpenCV beta 1 withstereo support released and presented at CVPR’01

2004/08

OpenCV beta 4 released.

5

Community There is a large community including people

from major companies (IBM, Microsoft, Intel, SONY, Siemens,…) and research centers (Stanford, MIT, CMU, Cambridge, INRIA etc.)

>9500 members of the forum OpenCV@yahoogroups.com, with average daily traffic ~10-20 messages.

Contributions: bug reports, patches, new features (video acquisition, 3d tracking, textures, Python interface)

6

Statistics Code Volume (beta 4):

the code: 500+ functions, ~170K lines, ~6MB documentation (HTML): 13K lines, ~650K tests: ~51K lines, ~1.7MB

Downloads: ~46000 beta3.1 downloads (36000 – Win32,

10000 – Linux), 20000+ beta 4 downloads (15000+ – Win32, 5000 – Linux)

7

Supported Platforms, Tools The processing (algorithmic) part:

Windows: Microsoft Visual C++ (6.0, .NET 2003), Intel Compiler,

Borland C++, Mingw (GCC 3.x): via project files and command-line Makefile’s.

Linux: GCC (2.9x, 3.x), Intel Compiler: “./configure-make-make install”, RPM (spec file is provided)

C and “light” C++ are used. A few and well localized #if and #pragma’s… (i.e. very portable)

GUI, Video/image acquisition: Windows: DirectShow, VFW, MIL, CMU1394 Linux: V4L2, DC1394, FFMPEG

Documentation: plain HTML

8

OpenCV and Performance Libraries Performance libraries at Intel have a long

history (since mid-90’s) Currently include IPP (low-level API for signal

& image processing, media codecs etc.) and MKL (native LAPACK and FFTPack)

Non-commercial versions for Linux and trial versions for Windows and Linux are freely available from Intel site.

Optimized for IA32, IA64, XScale (IPP), SMP (the best version for particular configuration is automatically selected).

Along with Intel compilers and performance tuning tools provide a solid base for development of high-performance applications on IA.

OpenCV can automatically detect IPP and MKL at runtime and use them to speedup processing.

CV

CXCore

IPP

MKL

9

OpenCV Architecture

Low level C-functions

Switcher

OpenCV(few C++ classes, High-level C functions)

Open source Open source

Open sourceOpen source

IPP, MKL

Demo Applications

IPP & MKL are not required to build and use OpenCV!

10

The Functionality in Words Core (cxcore, partly used by Intel Open Source Probabilistic Network Library as well):

Simple operations on dense arrays Matrix algebra, math functions, RNG DFT, DCT Serialization to XML/YAML (data persistence) Drawing (2D graphics) Complex data structures: sparse matrices, growing sequences, graphs

Vision (cv): Basic image processing (filters, geometrical transformations, color space transforms) Image analysis (feature selection, morphology, contour retrieval, histograms) Structural analysis (shape descriptors, planar subdivisions) Motion analysis and object tracking Object/face detection Camera calibration and elements of 3D reconstruction

IO/GUI (highgui) Image/video acquisition. Simple GUI facilities (all OpenCV visual samples use HighGUI)

Experimental/obsolete functions (cvaux): 3D vision: stereo calibration, trifocal tensor, bundle adjustment Stereo correspondence, graph cliques Face details detection and tracking Shape matching, skeletons … HMMs Textures

11

The Functionality in Pictures

12

The future OpenCV directions/plans The strategy shifts and expands from “a-man-in-front-

of-camera” paradigm towards generic “visual information processing” (cameraphones, digital home, security, medical applications, media retrieval etc.)

Next major release is scheduled for 2005’Q4 Will remain free and open source Even better stability (to call it 1.0?) and performance Better use of IPP (=> yet higher performance) More open development model Some new functionality to be transferred from

past/present research projects

13

The all-time OpenCV Crew

14

Useful Links http://www.intel.com/research/mrl/research/opencv/ (OpenCV at Intel) http://sourceforge.net/projects/opencvlibrary/ (the project page at SF. Take the latest

release or CVS snapshots here) http://groups.yahoo.com/group/OpenCV/ (The forum. Subscribe and ask your questions

here) http://www.opencv.org/ (OpenCV Wiki pages, still being formed) http://www.cs.ru.ac.za/research/sharpercv/ (C# interface for OpenCV) http://www.ient.rwth-aachen.de/team/asbach/opencv-python.html (more wrappers for

OpenCV. To be included into the official distribution) http://lush.sourceforge.net/ (Lisp-like language by Yann LeCun with interface to

OpenCV) http://www.intel.com/software/products/ (Intel Software page. Take IPP, Intel Compiler

and other tools here) http://sourceforge.net/projects/openpnl (OpenPNL - Bayesian Nets library from Intel;

shares some code with OpenCV) http://www-2.cs.cmu.edu/~cil/v-source.html (a lot of links to open source computer

vision and image processing software) http://www.google.com/search?q=opencv (find more OpenCV resources with Google)

15

Backup:Sample OpenCV code

16

The simplest OpenCV program#include <cv.h>#include <highgui.h>

int main( int argc, char** argv ){ IplImage* image; if( argc != 2 ) return -1; image = cvLoadImage( argv[1] ); if( !image ) return -1;

cvNamedWindow( “Sample”, 1 ); cvShowImage( “Sample”, image ); cvWaitKey(); return 0; }

$ g++ –o sample `pkg-config --cflags opencv` sample.cpp `pkg-config –-libs opencv`…$ ./sample lena.jpg

sample.cpp:

Build it and run:(the screenshot is from Windows version)

17

Using contours#include <cv.h>#include <highgui.h>

IplImage* image = 0;int thresh = 100;

void on_trackbar(int){ IplImage* gray = cvCreateImage( cvGetSize(image), 8, 1 ); CvMemStorage* storage = cvCreateMemStorage(0); CvSeq* contours = 0; cvCvtColor( image, gray, CV_BGR2GRAY ); cvThreshold( gray, gray, thresh, 255, CV_THRESH_BINARY ); cvFindContours( gray, storage, &contours ); cvZero( gray ); if( contours ) cvDrawContours( gray, contours, cvScalarAll(255), cvScalarAll(255), 100 ); cvShowImage( "Contours", gray ); cvReleaseImage( &gray ); cvReleaseMemStorage( &storage );}

int main( int argc, char** argv ){ IplImage* image; if( argc != 2 || !(image = cvLoadImage(argv[1])) ) return -1; cvNamedWindow( "Contours", 1 ); cvCreateTrackbar( "Threshold", "Contours", &thresh, 255, on_trackbar ); on_trackbar(0); cvWaitKey(); return 0; }

18

Detecting faces in Video#include "cv.h"#include "highgui.h"int main( int argc, char** argv ){ static CvMemStorage* storage = 0; static CvHaarClassifierCascade* cascade = 0; CvCapture* capture = 0; int optlen = strlen("--cascade="); if( argc != 3 || strncmp( argv[1], "--cascade=", optlen )) return -1; cascade = (CvHaarClassifierCascade*)cvLoad( argv[1] + optlen ); capture = cvCaptureFromAVI( argv[2] ); if( !cascade || !capture ) return -1; storage = cvCreateMemStorage(0); cvNamedWindow( "Video", 1 ); for(;;) { IplImage* frame = cvQueryFrame( capture ), *img; CvSeq* faces; if( !frame ) break; img = cvCloneImage(frame); img->origin = 0; if( frame->origin ) cvFlip(img,img); cvClearMemStorage( &storage ); faces = cvHaarDetectObjects( img, cascade, storage, 1.1, 2, CV_HAAR_DO_CANNY_PRUNING, cvSize(20, 20) ); for( int i = 0; i < (faces ? faces->total : 0); i++ ) { CvRect* r = (CvRect*)cvGetSeqElem( faces, i ); cvRectangle( img, cvPoint(r->x,r->y), cvPoint(r->x+r->width,r->y+r->height), CV_RGB(255,0,0), 3 ); } cvShowImage( "Video", img ); cvReleaseImage( &img ); if( cvWaitKey(10) >= 0 ) break; } cvReleaseCapture( &capture ); return 0;}

$ ./facedetect –-cascade=opencv/data/haarcascades/haarcascade_frontalface_alt2.xml screetcar.avi