Install OpenCV and Python on your Raspberry Pi 2 and B+by Adrian Rosebrock on February 23, 2015 in Raspberry Pi, Tutorials

6124

My Raspberry Pi 2 just arrived in the mail yesterday, and man is this berry sweet.

This tiny little PC packs a real punch with a 900mhz quadcore processor and 1gb of

RAM. To give some perspective, the Raspberry Pi 2 is faster than the majority of the

desktops in my high school computer lab.

Anyway, since the announcement of the Raspberry Pi 2 I’ve been getting a lot of

requests to provide detailed installation instructions for OpenCV and Python.

So if you’re looking to get OpenCV and Python up-and-running on your Raspberry Pi,

look no further!

In the rest of this blog post I provide detailed installation instructions for both

the Raspberry Pi 2 and the Raspberry Pi B+.

I’ve also provided install timings for each step. Some of these steps require a lot of

processing time. For example, compiling the OpenCV library on a Raspberry Pi 2

takes approximately 2.8 hours versus the 9.5 hours on the Raspberry Pi B+, so

please plan your install accordingly.

Finally, it’s worth mentioning that we’ll be utilizing the Raspberry Pi inside

thePyImageSearch Gurus computer vision course. Our projects will include home

surveillance applications such as detecting motion and tracking people in a room.

Here’s a quick example of detecting motion and tracking myself as I walk around my

apartment on the phone:

Install OpenCV and Python on your Raspberry Pi 2 and B+

I’m going to assume that you have either your Raspberry Pi 2 or Raspberry Pi B+

unboxed and setup. If you don’t have a Raspberry Pi yet, I definitely suggest picking

one up. They are super cheap and a lot of fun to play with.

Personally, I prefer to spend a little extra money and purchase from Canakit — their

shipping is fast and reliable, plus their complete ready-to-go bundles are really nice.

Anyway, let’s get into the OpenCV and Python install instructions.

Step 0:

Again, I’m going to assume that you have just unboxed your Raspberry Pi 2/B+. Open

up a terminal and we’ll start by updating and upgrading installed packages, followed by

updating the Raspberry Pi firmware:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

123

$ sudo apt-get update$ sudo apt-get upgrade$ sudo rpi-update

Step 1:

Install the required developer tools and packages:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

1 $ sudo apt-get install build-essential cmake pkg-config

Both build-essential  and pkg-config  are likely already installed, but just in case

they are not, be sure to include them in your apt-get  command.

Timings:

Raspberry Pi B+: < 2 minutes

Raspberry Pi 2: < 40 seconds

Step 2:

Install the necessary image I/O packages. These packages allow you to load various

image file formats such as JPEG, PNG, TIFF, etc.

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

1 $ sudo apt-get install libjpeg8-dev libtiff4-dev libjasper-dev libpng12-dev

Timings:

Raspberry Pi B+: < 5 minutes

Raspberry Pi 2: < 30 seconds

Step 3:

Install the GTK development library. This library is used to build Graphical User

Interfaces (GUIs) and is required for the highgui  library of OpenCV which allows you

to view images on your screen:Install OpenCV and Python your Raspberry Pi 2 and B+Shell

1 $ sudo apt-get install libgtk2.0-dev

Timings:

Raspberry Pi B+: < 10 minutes

Raspberry Pi 2: < 3 minutes

Step 4:

Install the necessary video I/O packages. These packages are used to load video files

using OpenCV:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

1 $ sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev

Timings:

Raspberry Pi B+: < 5 minutes

Raspberry Pi 2: < 30 seconds

Step 5:

Install libraries that are used to optimize various operations within OpenCV:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

1 $ sudo apt-get install libatlas-base-dev gfortran

Timings:

Raspberry Pi B+: < 2 minutes

Raspberry Pi 2: < 30 seconds

Step 6:Install pip :Install OpenCV and Python your Raspberry Pi 2 and B+Shell

12

$ wget https://bootstrap.pypa.io/get-pip.py$ sudo python get-pip.py

Timings:

Raspberry Pi B+: < 2 minutes

Raspberry Pi 2: < 30 seconds

Step 7:Install  virtualenv  and virtualenvwrapper :Install OpenCV and Python your Raspberry Pi 2 and B+Shell

1 $ sudo pip install virtualenv virtualenvwrapper

Then, update your ~/.profile  file to include the following lines:Install OpenCV and Python your Raspberry Pi 2 and B+Shell

123

# virtualenv and virtualenvwrapperexport WORKON_HOME=$HOME/.virtualenvssource /usr/local/bin/virtualenvwrapper.sh

Reload your .profile  file:Install OpenCV and Python your Raspberry Pi 2 and B+Shell

1 $ source ~/.profile

Create your computer vision virtual environment:

$ mkvirtualenv cvShell

1 $ mkvirtualenv cv

Timings:

Raspberry Pi B+: < 2 minutes

Raspberry Pi 2: < 2 minutes

Step 8:

Now we can install the Python 2.7 development tools:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

1 $ sudo apt-get install python2.7-dev

Note: Yes, we are going to use Python 2.7. OpenCV 2.4.X does not yet support

Python 3 and OpenCV 3.0 is still in beta. It’s also unclear when the Python bindings

for OpenCV 3.0 will be complete so I advise to stick with OpenCV 2.4.X for the time

being.

We also need to install NumPy since the OpenCV Python bindings represent images

as multi-dimensional NumPy arrays:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

1 $ pip install numpy

Timings:

Raspberry Pi B+: < 45 minutes

Raspberry Pi 2: < 15 minutes

Step 9:

Download OpenCV and unpack it:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

123

$ wget -O opencv-2.4.10.zip http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.10/opencv-2.4.10.zip/download$ unzip opencv-2.4.10.zip$ cd opencv-2.4.10

Setup the build:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

123

$ mkdir build$ cd build$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D BUILD_NEW_PYTHON_SUPPORT=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON  -D BUILD_EXAMPLES=ON ..

Timings:

Raspberry Pi B+: < 3 minutes

Raspberry Pi 2: < 1.5 minutes

Compile OpenCV:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

1 $ make

Important: Make sure you’re in the  cv   virtual environment so OpenCV is compiled

against the virtual environment Python and NumPy. Otherwise, OpenCV will be

compiled against the system Python and NumPy which can lead to problems down the

line.

Timings:

Raspberry Pi B+: < 9.5 hours

Raspberry Pi 2: < 2.8 hours

Finally, we can install OpenCV:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

12

$ sudo make install$ sudo ldconfig

Timings:

Raspberry Pi B+: < 3 minutes

Raspberry Pi 2: < 1 minute

Step 10:

If you’ve gotten this far in the guide, OpenCV should now be installed

in /usr/local/lib/python2.7/site-packages

But in order to utilize OpenCV within our cv  virtual environment, we first need to sym-

link OpenCV into our site-packages  directory:Install OpenCV and Python your Raspberry Pi 2 and B+Shell

123

$ cd ~/.virtualenvs/cv/lib/python2.7/site-packages/$ ln -s /usr/local/lib/python2.7/site-packages/cv2.so cv2.so$ ln -s /usr/local/lib/python2.7/site-packages/cv.py cv.py

Step 11:

Finally, we can give our OpenCV and Python installation a test drive:

Install OpenCV and Python your Raspberry Pi 2 and B+Shell

12345

$ workon cv$ python>>> import cv2>>> cv2.__version__'2.4.10'

OpenCV and Python is now successfully installed on your Raspberry Pi!

Here is an example of me ssh’ing (with X11 forwarding) into my Raspberry Pi, followed

by loading and displaying an image:

SummaryIn this blog post I detailed how to install OpenCV and Python on your Raspberry Pi 2

or Raspberry Pi B+. Timings for each installation step were also provided so you could

plan out the install accordingly.

As the Raspberry Pi (along with Raspbian/NOOBS) evolves the installation instructions

will likely change. If you run across any edge cases or variations in the install

instructions, please feel free to let me know. While I can’t promise that I can reply to

every email, but I think it would be good to curate a list of methods to setup OpenCV

and Python on Raspberry Pi systems.

And in future blog posts we’ll explore how to utilize the camera add-on for the

Raspberry Pi.

Until then, take a look at the PyImageSearch Gurus computer vision course. We’ll be

utilizing the Raspberry Pi inside the course for a few projects, including building a

home surveillance application that can detect motion and people in rooms.

RASPBERRY PI 1 MODEL B+

The Model B+ is the final revision of the original Raspberry Pi. It replaced

the Model B in July 2014 and was superseded by the Raspberry Pi 2 Model B in

February 2015. Compared to the Model B it has:

More GPIO. The GPIO header has grown to 40 pins, while retaining the same

pinout for the first 26 pins as the Model A and B.

More USB. We now have 4 USB 2.0 ports, compared to 2 on the Model B, and

better hotplug and overcurrent behaviour.

Micro SD. The old friction-fit SD card socket has been replaced with a much nicer

push-push micro SD version.

Lower power consumption. By replacing linear regulators with switching ones

we’ve reduced power consumption by between 0.5W and 1W.

Better audio. The audio circuit incorporates a dedicated low-noise power supply.

Neater form factor. We’ve aligned the USB connectors with the board edge,

moved composite video onto the 3.5mm jack, and added four squarely-placed

mounting holes.

The Model B+ is perfectly suitable for use in schools: it offers more flexibility for

learners than the leaner Model A or A+, which are more useful for embedded

projects and projects which require very low power, and has more USB ports than

the Model B.

See the documentation for technical details.

The hardware in the Raspberry Pi

Schematics

o Schematics for the Raspberry Pi Model A, B and B+

BCM2835

o The Broadcom processor used in Raspberry Pi Model A, B and B+

Mechanical Drawings

o Mechanical drawings of the Raspberry Pi Model B+

Power

o Powering the Raspberry Pi

USB

o USB on the Raspberry Pi

GPIO

o General Purpose Input/Output pins on the Raspberry Pi

SPI

o SPI on the Raspberry Pi

High Definition 1080p Embedded Multimedia Applications ProcessorThe BCM2835 is a cost-optimized, full HD, multimedia applications processor for advanced mobile and

embedded applications that require the highest levels of multimedia performance. Designed and optimized for

power efficiency, BCM2835 uses Broadcom's VideoCore® IV technology to enable applications in media

playback, imaging, camcorder, streaming media, graphics and 3D gaming.

Request Product Info to learn more about Broadcom products or contact a Manufacturer Representative in

your area.

Features

Low Power ARM1176JZ-F Applications Processor

Dual Core VideoCore IV® Multimedia Co-Processor

1080p30 Full HD HP H.264 Video Encode/Decode

Advanced Image Sensor Pipeline (ISP) for up to 20-megapixel cameras operating at up to 220 megapixels per

second

Low power, high performance OpenGL-ES® 1.1/2.0 VideoCore GPU. 1 Gigapixel per second fill rate.

High performance display outputs. Simultaneous high resolution LCD and HDMI with HDCP at 1080p60

ARM1176 Processor

(View Larger ARM1176 Processor Image)

The ARM1176™ applications processors deployed broadly in devices ranging from smart

phones to digital TV's to eReaders, delivering media and browser performance, a secure

computing environment, and performance up to 1GHz in low cost designs. The

ARM1176JZ-S processor features ARM TrustZone® technology for secure applications

and ARM Jazelle® technology for efficient embedded Java execution. Optional tightly

coupled memories simplify ARM9™ processor migration and real-time design,

while AMBA ®   3 AXI TM interfaces improve memory bus performance. DVFS support enables

power optimization below the best-in-class nominal static and dynamic power of

the ARM11TMprocessor architecture.

 

ARM1176

Architecture ARMv6

Dhrystone Performance 1.25 DMPS/MHz

Multicore No - Single core only

ISA SupportARM

Thumb ®

Jazelle ® DBX

DSP extension

ARM1176

Floating Point Unit (Optional)

Memory Management Memory management unit

Debug & Trace CoreSight™ Design Kit for ARM11 (available separately)

ARM1176 Key Features

Performance to enable excellent end-user experience

750MHz (TSMC 65GP) with conservative design 1GHz+ with design optimizations

Low latency mode for interrupt responsiveness

TCM for ARM9™ processor migration, real-time demands

Physically addressed caches for multi-tasking performance

Broad OS support, multiple Linux distributions, amazing ARM ecosystem

Full Internet experience

Product maturity enables rapid time to market and low risk

Well proven technology in wide range of applications

Available as soft core or hard macro from ARM (TSMC (90G) or from 3rd parties

(Socle/GLOBALFOUNDRIES - 65nm, TSMC - 65nm)

AMBA AXI  supported by wide range of fabric

CoreSight  debug offering unrivalled system visibility

Comprehensive range of development tools from ARM and from third parties

Range of Reference Methodologies supplied

Low Power Leadership

Shutdown modes, Clock Gating, and DVFS capability

93% of flops are clock gated

Separate Main TLB and Micro-TLBs --> main TLB is not clocked unless micro-TLB misses

Avoids unnecessary Tag-RAM and Data-RAM activity for sequential accesses

Predictive use of Cache or TCM

Mode Power

Run .208 mW/MHz in 65G

Standby Leakage Only

Dormant Memory Retention current

Shutdown Zero power draw

DVFS - supported by the ARM1176

Physical IP Power Management Kit  Clamps allow core to shut down while TCM still

enabled in Dormant mode

Asynchronous interfaces allow core to run at non-integer multiple of bus frequency

L-Shift clamps allow core voltage to differ from rest of SoC

POWER SUPPLY

The device is powered by a 5V micro USB supply. Exactly how much current (mA)

the Raspberry Pi requires is dependent on what you connect to it. We have found

that purchasing a 1.2A (1200mA) power supply from a reputable retailer will

provide you with ample power to run your Raspberry Pi.

Typically, the model B uses between 700-1000mA depending on what peripherals

are connected; the model A can use as little as 500mA with no peripherals

attached. The maximum power the Raspberry Pi can use is 1 Amp. If you need to

connect a USB device that will take the power requirements above 1 Amp, then

you must connect it to an externally-powered USB hub.

The power requirements of the Raspberry Pi increase as you make use of the

various interfaces on the Raspberry Pi. The GPIO pins can draw 50mA safely,

distributed across all the pins; an individual GPIO pin can only safely draw 16mA.

The HDMI port uses 50mA, the camera module requires 250mA, and keyboards

and mice can take as little as 100mA or over 1000mA! Check the power rating of

the devices you plan to connect to the Pi and purchase a power supply

accordingly.

BACKPOWERING

Backpowering occurs when USB hubs do not provide a diode to stop the hub from

powering against the host computer. Other hubs will provide as much power as

you want out each port. Please also be aware that some hubs will backfeed the

Raspberry Pi. This means that the hubs will power the Raspberry Pi through its

USB cable input cable, without the need for a separate micro-USB power cable,

and bypass the voltage protection. If you are using a hub that backfeeds to the

Raspberry Pi and the hub experiences a power surge, your Raspberry Pi could

potentially be damaged.

USB

PAGE CONTENTS

Overview

o Supported Devices

o General Limitations

o Port Power Limits

Known Issues

Troubleshooting

OVERVIEW

The Raspberry Pi Model B is equipped with two USB2.0 ports. These are

connected to the LAN9512 combo hub/Ethernet chip IC3, which is itself a USB

device connected to the single upstream USB port on BCM2835.

On the Model A, the single USB2.0 port is directly wired to BCM2835.

The USB ports enable the attachment of peripherals such as keyboards, mice,

webcams that provide the Pi with additional functionality.

There are some differences between the USB hardware on the Raspberry Pi and

the USB hardware on desktop computers or laptop/tablet devices.

The USB host port inside the Pi is an On-The-Go (OTG) host as the application

processor powering the Pi, BCM2835, was originally intended to be used in the

mobile market: i.e. as the single USB port on a phone for connection to a PC, or to

a single device. In essence, the OTG hardware is simpler than the equivalent

hardware on a PC.

OTG in general supports communication to all types of USB device, but to provide

an adequate level of functionality for most of the USB devices that one might plug

into a Pi, the system software has to do more work.

SUPPORTED DEVICES

In general, every device supported by Linux is possible to use with the Pi, subject

to a few caveats detailed further down. Linux has probably the most

comprehensive driver database for legacy hardware of any operating system (it

can lag behind for modern device support as it requires open-source drivers for

Linux to recognise the device by default).

If you have a device and wish to use it with a Pi, then plug it in. Chances are that

it'll "just work". If you are running in a graphical interface (such as the LXDE

desktop environment in Raspbian), then it's likely that an icon or similar will pop up

announcing the new device.

If the device doesn't appear to work, then refer to the Troubleshooting section.

GENERAL LIMITATIONS

The OTG hardware on Raspberry Pi has a simpler level of support for certain

devices, which may present a higher software processing overhead. The

Raspberry Pi also has only one root USB port: all traffic from all connected devices

is funnelled down this bus, which operates at a maximum speed of 480mbps.

The USB specification defines three device speeds - Low, Full and High. Most

mice and keyboards are Low-speed, most USB sound devices are Full-speed and

most video devices (webcams or video capture) are High-speed.

Generally, there are no issues with connecting multiple High-speed USB devices to

a Pi.

The software overhead incurred when talking to Low- and Full-speed devices

means that there are soft limitations on the number of simultaneously active Low-

and Full-speed devices. Small numbers of these types of devices connected to a Pi

will cause no issues.

PORT POWER LIMITS

USB devices have defined power requirements, in units of 100mA from 100mA to

500mA. The device advertises its own power requirements to the USB host when it

is first connected. In theory, the actual power consumed by the device should not

exceed its stated requirement.

The USB ports on a Raspberry Pi have a design loading of 100mA each - sufficient

to drive "low-power" devices such as mice and keyboards. Devices such as WiFi

adapters, USB hard drives, USB pen drives all consume much more current and

should be powered from an external hub with its own power supply. While it is

possible to plug a 500mA device into a Pi and have it work with a sufficiently

powerful supply, reliable operation is not guaranteed.

In addition, hotplugging high-power devices into the Pi's USB ports may cause a

brownout which can cause the Pi to reset.

See Power for more information.

DEVICES WITH KNOWN ISSUES

1. Interoperability between the Raspberry Pi and USB3.0 hubs

There is an issue with USB3.0 hubs in conjunction with the use of Full- or Low-

speed devices (most mice, most keyboards) and the Raspberry Pi. A bug in most

USB3.0 hub hardware means that the Raspberry Pi cannot talk to Full- or Low-

speed devices connected to a USB3.0 hub.

USB2.0 high-speed devices, including USB2.0 hubs, operate correctly when

connected via a USB3.0 hub.

Avoid connecting Low- or Full-speed devices into a USB3.0 hub. As a workaround,

plug a USB2.0 hub into the downstream port of the USB3.0 hub and connect the

low-speed device, or use a USB2.0 hub between the Pi and the USB3.0 hub, then

plug low-speed devices into the USB2.0 hub.

2. USB1.1 webcams

Old webcams may be Full-speed devices. Because these devices transfer a lot of

data and incur additional software overhead, reliable operation is not guaranteed.

As a workaround, try to use the camera at a lower resolution.

3. Esoteric USB sound cards

Expensive "audiophile" sound cards typically use far more bandwidth than is

necessary to stream audio playback. Reliable operation with 96kHz/192kHz DACs

is not guaranteed.

As a workaround, forcing the output stream to be CD quality (44.1kHz/48kHz 16-

bit) will reduce the stream bandwidth to reliable levels.

4. Single-TT USB hubs

USB2.0 and 3.0 hubs have a mechanism for talking to Full- or Low-speed devices

connected to their downstream ports called a Transaction Translator. This device

buffers high-speed requests from the host (i.e. the Pi) and transmits them at Full-

or Low-speed to the downstream device. Two configurations of hub are allowed by

the USB specification: Single-TT (one TT for all ports) and Multi-TT (one TT per

port).

Because of the OTG hardware limitations, if too many Full- or Low-speed devices

are plugged into a single-TT hub, unreliable operation of the devices may occur. It

is recommended to use a Multi-TT hub to interface with multiple lower-speed

devices.

As a workaround, spread lower-speed devices out between the Pi's own USB port

and the single-TT hub.

TROUBLESHOOTING

IF YOUR DEVICE DOESN'T WORK AT ALL

The first step is to see if it is detected at all. There are two commands that can be entered into a terminal for this:  lsusb  and  dmesg . The first will print out all

devices attached to USB, whether they are actually recognised by a device driver

or not, and the second will print out the kernel message buffer (which can be quite big after booting - try doing  sudo dmesg -C  then plug in your device and

retype  dmesg  to see new messages).

As an example with a USB pendrive:

pi@raspberrypi ~ $ lsusb

Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp.

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp.

Bus 001 Device 005: ID 05dc:a781 Lexar Media, Inc.

pi@raspberrypi ~ $ dmesg

... Stuff that happened before ...

[ 8904.228539] usb 1-1.3: new high-speed USB device number 5 using

dwc_otg

[ 8904.332308] usb 1-1.3: New USB device found, idVendor=05dc,

idProduct=a781

[ 8904.332347] usb 1-1.3: New USB device strings: Mfr=1, Product=2,

SerialNumber=3

[ 8904.332368] usb 1-1.3: Product: JD Firefly

[ 8904.332386] usb 1-1.3: Manufacturer: Lexar

[ 8904.332403] usb 1-1.3: SerialNumber: AACU6B4JZVH31337

[ 8904.336583] usb-storage 1-1.3:1.0: USB Mass Storage device detected

[ 8904.337483] scsi1 : usb-storage 1-1.3:1.0

[ 8908.114261] scsi 1:0:0:0: Direct-Access Lexar JD Firefly

0100 PQ: 0 ANSI: 0 CCS

[ 8908.185048] sd 1:0:0:0: [sda] 4048896 512-byte logical blocks:

(2.07 GB/1.93 GiB)

[ 8908.186152] sd 1:0:0:0: [sda] Write Protect is off

[ 8908.186194] sd 1:0:0:0: [sda] Mode Sense: 43 00 00 00

[ 8908.187274] sd 1:0:0:0: [sda] No Caching mode page present

[ 8908.187312] sd 1:0:0:0: [sda] Assuming drive cache: write through

[ 8908.205534] sd 1:0:0:0: [sda] No Caching mode page present

[ 8908.205577] sd 1:0:0:0: [sda] Assuming drive cache: write through

[ 8908.207226] sda: sda1

[ 8908.213652] sd 1:0:0:0: [sda] No Caching mode page present

[ 8908.213697] sd 1:0:0:0: [sda] Assuming drive cache: write through

[ 8908.213724] sd 1:0:0:0: [sda] Attached SCSI removable disk

In this case, there are no error messages in  dmesg  and the pendrive is detected

by the usb-storage driver. If your device did not have a driver available, then

typically only the first 6 new lines will appear in the dmesg printout.

If a device enumerates without any errors, but doesn't appear to do anything, then

it is likely there are no drivers installed for it. Search around, based on the

manufacturer's name for the device or the USB IDs that are displayed in lsusb (e.g.

05dc:a781). The device may not be supported with default Linux drivers - and you

may need to download or compile your own third-party software.

IF YOUR DEVICE HAS INTERMITTENT BEHAVIOUR

Poor quality power is the most common cause of devices not working,

disconnecting or generally being unreliable.

If you are using an external powered hub, try swapping the power adapter supplied

with the hub for another compatible power supply with the same voltage rating and

polarity.

Check to see if the problem resolves itself if you remove other devices from the

hub's downstream ports.

Temporarily plug the device directly into the Pi and see if the behaviour improves.

Powerful Vector Floating Point Unit

Fully IEEE 754-compliant mode

Rare cases bounced to software support code for full IEEE 754 support

Suited to safety-critical application like Automotive and Avionics

RunFast mode for fast, near-compliant hardware execution of all instructions

No software support needed, performance increased as no software involved

Suited to consumer applications like 3D graphics

Compiler automatically targets VFP

Functions supported in hardware:

Multiplication, addition, and multiply-accumulate ( various variants)

Division and square root operation (multi-cycle, not pipelined)

Comparisons and format conversions

Operations can be performed on short vectors (From assembler only)

Separate pipelines allow load/store and MAC operations to occur simultaneously with

divide/square root unit operation

VFP may be powered-down when not in use

Clock gated and/or power completely removed

VFP has it's own additional register set

32 single precision

16 double-precision

Sufficient to double-buffer algorithms

o Linpack, Filters, Graphics transforms, etc.

Secure Computing Environment with TrustZone® technology

Secures against software-only "hack attacks" and most low budget hardware "shack

attacks"

Delivers two "virtual" cores with deep separating of context & data.

o S/W and JTAG attacks can not enter secure domain

TrustZone : two virtualized CPUs in one

o CPU MHz/resources are dynamically shared between according to demands

o The two isolated domains are implemented in the same machine with no duplication of HW

o Simpler and more flexible platform designs, lower costs and high power/performance efficiency

GPIO

General Purpose Input/Output pins on the Raspberry Pi

OVERVIEW

This page expands on the technical features of the GPIO pins available on

BCM2835 in general. For usage examples, see the GPIO Usage section. When

reading this page, reference should be made to the BCM2835 ARM

PeripheralsDatasheet, section 6.

GPIO pins can be configured as either general-purpose input, general-purpose

output or as one of up to 6 special alternate settings, the functions of which are pin-

dependant.

There are 3 GPIO banks on BCM2835.

Each of the 3 banks has its own VDD input pin. On Raspberry Pi, all GPIO banks

are supplied from 3.3V. Connection of a GPIO to a voltage higher than 3.3V will

likely destroy the GPIO block within the SoC.

A selection of pins from Bank 0 is available on the P1 header on Raspberry Pi.

GPIO PADS

The GPIO connections on the BCM2835 package are sometimes referred to in the

peripherals datasheet as "pads" - a semiconductor design term meaning "chip

connection to outside world".

The pads are configurable CMOS push-pull output drivers/input buffers. Register-

based control settings are available for

Internal pull-up / pull-down enable/disable

Output drive strength

Input Schmitt-trigger filtering

POWER-ON STATES

All GPIOs revert to general-purpose inputs on power-on reset. The default pull

states are also applied, which are detailed in the alternate function table in the

ARM peripherals datasheet. Most GPIOs have a default pull applied.

INTERRUPTS

Each GPIO pin, when configured as a general-purpose input, can be configured as

an interrupt source to the ARM. Several interrupt generation sources are

configurable:

Level-sensitive (high/low)

Rising/falling edge

Asynchronous rising/falling edge

Level interrupts maintain the interrupt status until the level has been cleared by

system software (e.g. by servicing the attached peripheral generating the interrupt).

The normal rising/falling edge detection has a small amount of synchronisation

built into the detection. At the system clock frequency, the pin is sampled with the

criteria for generation of an interrupt being a stable transition within a 3-cycle

window, i.e. a record of "1 0 0" or "0 1 1". Asynchronous detection bypasses this

synchronisation to enable the detection of very narrow events.

ALTERNATIVE FUNCTIONS

Almost all of the GPIO pins have alternative functions. Peripheral blocks internal to

BCM2835 can be selected to appear on one or more of a set of GPIO pins, for

example the I2C busses can be configured to at least 3 separate locations. Pad

control, such as drive strength or Schmitt filtering, still applies when the pin is

configured as an alternate function.

For more detailed information see the Low level peripherals page on the elinux wiki

SPI

PAGE CONTENTS

Overview

Software

Hardware

Linux driver

Troubleshooting

OVERVIEW

The Raspberry Pi is equipped with one SPI bus that has 2 chip selects.

The SPI master driver is disabled by default on Raspian. To enable it, remove the blacklisting for  spi-bcm2708  in  /etc/modprobe.d/raspi-blacklist.conf , or

use raspi-config. Reboot or load the driver manually with:

$ sudo modprobe spi-bcm2708

The SPI bus is available on the P1 Header:

MOSI P1-19

MISO P1-21

SCLK P1-23 P1-24 CE0

GND P1-25 P1-26 CE1

SOFTWARE

WIRINGPI

WiringPi includes a library which can make it easier to use the Raspberry Pi's on-

board SPI interface. Accesses the hardware registers directly.

http://wiringpi.com/

BCM2835 LIBRARY

This is a C library for Raspberry Pi (RPi). It provides access to GPIO and other IO

functions on the Broadcom BCM 2835 chip. Accesses the hardware registers

directly.

http://www.airspayce.com/mikem/bcm2835/

USE SPIDEV FROM C

There's a loopback test program in the Linux documentation that can be used as a starting point. See the Troubleshooting section. Uses the Linux  spidev  driver to

access the bus.

SHELL

# Write binary 1, 2 and 3

echo -ne "\x01\x02\x03" > /dev/spidev0.0

HARDWARE

The BCM2835 on the Raspberry Pi has 3 SPI Controllers. Only the SPI0 controller

is available on the header. Chapter 10 in the BCM2835 ARM

Peripherals datasheet describes this controller.

MASTER MODES

Signal name abbreviations

SCLK - Serial CLocK

CE - Chip Enable (often called Chip Select)

MOSI - Master Out Slave In

MISO - Master In Slave Out

MOMI - Master Out Master In

MIMO - Master In Master Out

STANDARD MODE

In Standard SPI master mode the peripheral implements the standard 3 wire serial

protocol (SCLK, MOSI and MISO).

BIDIRECTIONAL MODE

In bidirectional SPI master mode the same SPI standard is implemented except

that a single wire is used for data (MIMO) instead of two as in standard mode

(MISO and MOSI).

LOSSI MODE (LOW SPEED SERIAL INTERFACE)

The LoSSI standard allows issuing of commands to peripherals (LCD) and to

transfer data to and from them. LoSSI commands and parameters are 8 bits long,

but an extra bit is used to indicate whether the byte is a command or

parameter/data. This extra bit is set high for a data and low for a command. The

resulting 9-bit value is serialized to the output. LoSSI is commonly used with MIPI

DBI type C compatible LCD controllers.

Note:

Some commands trigger an automatic read by the SPI controller, so this mode

can't be used as a multipurpose 9-bit SPI.

TRANSFER MODES

Polled

Interrupt

DMA

SPEED

The CDIV (Clock Divider) field of the CLK register sets the SPI clock speed:

SCLK = Core Clock / CDIV

If CDIV is set to 0, the divisor is 65536. The divisor must be a power

of 2. Odd numbers rounded down. The maximum SPI clock rate is of the

APB clock.

Errata: "must be a power of 2" probably should be "must be a multiple of 2"

See the Linux driver section for more info.

CHIP SELECT

Setup and Hold times related to the automatic assertion and de-assertion of the CS

lines when operating in DMA mode are as follows:

The CS line will be asserted at least 3 core clock cycles before the msb of the first

byte of the transfer.

The CS line will be de-asserted no earlier than 1 core clock cycle after the trailing

edge of the final clock pulse.

LINUX DRIVER

The default Linux driver is spi-bcm2708.

The following information was valid 2014-07-05.

SPEED

The driver supports the following speeds:

cdiv speed

2 125.0 MHz

4 62.5 MHz

8 31.2 MHz

16 15.6 MHz

32 7.8 MHz

64 3.9 MHz

128 1953 kHz

256 976 kHz

512 488 kHz

1024 244 kHz

2048 122 kHz

4096 61 kHz

8192 30.5 kHz

16384 15.2 kHz

32768 7629 Hz

When asking for say 24 MHz, the actual speed will be 15.6 MHz.

Forum post: SPI has more speeds

SUPPORTED MODE BITS

SPI_CPOL - Clock polarity

SPI_CPHA - Clock phase

SPI_CS_HIGH - Chip Select active high

SPI_NO_CS - 1 device per bus, no Chip Select

Bidirectional mode is currently not supported.

SUPPORTED BITS PER WORD

8 - Normal

9 - This is supported using LoSSI mode.

TRANSFER MODES

Only interrupt mode is supported.

DEPRECATED WARNING

The following appears in the kernel log:

bcm2708_spi bcm2708_spi.0: master is unqueued, this is deprecated

SPI DRIVER LATENCY

This thread discusses latency problems.

DMA CAPABLE DRIVER

This is a fork of spi-bcm2708 which enables DMA support for SPI client drivers that

support DMA.

https://github.com/notro/spi-bcm2708 (wiki)

TROUBLESHOOTING

LOOPBACK TEST

This can be used to test SPI send and receive. Put a wire between MOSI and

MISO. It does not test CE0 and CE1.

wget https://raw.githubusercontent.com/raspberrypi/linux/rpi-

3.10.y/Documentation/spi/spidev_test.c

gcc -o spidev_test spidev_test.c

./spidev_test -D /dev/spidev0.0

spi mode: 0

bits per word: 8

max speed: 500000 Hz (500 KHz)

FF FF FF FF FF FF

40 00 00 00 00 95

FF FF FF FF FF FF

FF FF FF FF FF FF

FF FF FF FF FF FF

DE AD BE EF BA AD

F0 0D

If you get compilation errors, try the latest version instead:

wget

http

s://raw.github.com/torvalds/linux/master/Documentation/spi/spidev_test

.c

How to Use Background Subtraction Methods¶

Background subtraction (BS) is a common and widely used technique for generating a foreground mask (namely, a binary image containing the pixels belonging to moving objects in the scene) by using static cameras.

As the name suggests, BS calculates the foreground mask performing a subtraction between the current frame and a background model, containing the static part of the scene or, more in general, everything that can be considered as background given the characteristics of the observed scene.

Background modeling consists of two main steps:

1. Background Initialization;

2. Background Update.

In the first step, an initial model of the background is computed, while in the second step that model is updated in order to adapt to possible changes in the scene.

In this tutorial we will learn how to perform BS by using OpenCV. As input, we will use data coming from the publicly available data set Background Models Challenge (BMC) .

Goals

In this tutorial you will learn how to:

1. Read data from videos by using VideoCapture or image sequences by using imread;

2. Create and update the background model by using BackgroundSubtractor class;

3. Get and show the foreground mask by using imshow;

4. Save the output by using imwrite to quantitatively evaluate the results.

Code

In the following you can find the source code. We will let the user chose to process either a video file or a sequence of images.

Two different methods are used to generate two foreground masks:

1. MOG

2. MOG2

The results as well as the input data are shown on the screen.

//opencv#include <opencv2/highgui/highgui.hpp>#include <opencv2/video/background_segm.hpp>//C#include <stdio.h>//C++#include <iostream>#include <sstream>

using namespace cv;using namespace std;

//global variablesMat frame; //current frameMat fgMaskMOG; //fg mask generated by MOG methodMat fgMaskMOG2; //fg mask fg mask generated by MOG2 methodPtr<BackgroundSubtractor> pMOG; //MOG Background subtractorPtr<BackgroundSubtractor> pMOG2; //MOG2 Background subtractorint keyboard;

//function declarationsvoid help();void processVideo(char* videoFilename);void processImages(char* firstFrameFilename);

void help(){ cout << "--------------------------------------------------------------------------" << endl << "This program shows how to use background subtraction methods provided by " << endl << " OpenCV. You can process both videos (-vid) and images (-img)." << endl << endl << "Usage:" << endl << "./bs {-vid <video filename>|-img <image filename>}" << endl << "for example: ./bs -vid video.avi" << endl << "or: ./bs -img /data/images/1.png" << endl

<< "--------------------------------------------------------------------------" << endl << endl;}

int main(int argc, char* argv[]){ //print help information help();

//check for the input parameter correctness if(argc != 3) { cerr <<"Incorret input list" << endl; cerr <<"exiting..." << endl; return EXIT_FAILURE; }

//create GUI windows namedWindow("Frame"); namedWindow("FG Mask MOG"); namedWindow("FG Mask MOG 2");

//create Background Subtractor objects pMOG = createBackgroundSubtractorMOG(); //MOG approach pMOG2 = createBackgroundSubtractorMOG2(); //MOG2 approach

if(strcmp(argv[1], "-vid") == 0) { //input data coming from a video processVideo(argv[2]); } else if(strcmp(argv[1], "-img") == 0) { //input data coming from a sequence of images processImages(argv[2]); } else { //error in reading input parameters cerr <<"Please, check the input parameters." << endl; cerr <<"Exiting..." << endl; return EXIT_FAILURE; } //destroy GUI windows destroyAllWindows(); return EXIT_SUCCESS;}

void processVideo(char* videoFilename) { //create the capture object VideoCapture capture(videoFilename); if(!capture.isOpened()){ //error in opening the video input cerr << "Unable to open video file: " << videoFilename << endl; exit(EXIT_FAILURE); }

//read input data. ESC or 'q' for quitting while( (char)keyboard != 'q' && (char)keyboard != 27 ){ //read the current frame if(!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); } //update the background model pMOG->apply(frame, fgMaskMOG); pMOG2->apply(frame, fgMaskMOG2); //get the frame number and write it on the current frame stringstream ss; rectangle(frame, cv::Point(10, 2), cv::Point(100,20), cv::Scalar(255,255,255), -1); ss << capture.get(CAP_PROP_POS_FRAMES); string frameNumberString = ss.str(); putText(frame, frameNumberString.c_str(), cv::Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5 , cv::Scalar(0,0,0)); //show the current frame and the fg masks imshow("Frame", frame); imshow("FG Mask MOG", fgMaskMOG); imshow("FG Mask MOG 2", fgMaskMOG2); //get the input from the keyboard keyboard = waitKey( 30 ); } //delete capture object capture.release();}

void processImages(char* fistFrameFilename) { //read the first file of the sequence frame = imread(fistFrameFilename); if(!frame.data){ //error in opening the first image cerr << "Unable to open first image frame: " << fistFrameFilename << endl; exit(EXIT_FAILURE); } //current image filename string fn(fistFrameFilename); //read input data. ESC or 'q' for quitting while( (char)keyboard != 'q' && (char)keyboard != 27 ){ //update the background model pMOG->apply(frame, fgMaskMOG); pMOG2->apply(frame, fgMaskMOG2); //get the frame number and write it on the current frame size_t index = fn.find_last_of("/"); if(index == string::npos) { index = fn.find_last_of("\\"); } size_t index2 = fn.find_last_of("."); string prefix = fn.substr(0,index+1); string suffix = fn.substr(index2);

string frameNumberString = fn.substr(index+1, index2-index-1); istringstream iss(frameNumberString); int frameNumber = 0; iss >> frameNumber; rectangle(frame, cv::Point(10, 2), cv::Point(100,20), cv::Scalar(255,255,255), -1); putText(frame, frameNumberString.c_str(), cv::Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5 , cv::Scalar(0,0,0)); //show the current frame and the fg masks imshow("Frame", frame); imshow("FG Mask MOG", fgMaskMOG); imshow("FG Mask MOG 2", fgMaskMOG2); //get the input from the keyboard keyboard = waitKey( 30 ); //search for the next image in the sequence ostringstream oss; oss << (frameNumber + 1); string nextFrameNumberString = oss.str(); string nextFrameFilename = prefix + nextFrameNumberString + suffix; //read the next frame frame = imread(nextFrameFilename); if(!frame.data){ //error in opening the next image in the sequence cerr << "Unable to open image frame: " << nextFrameFilename << endl; exit(EXIT_FAILURE); } //update the path of the current frame fn.assign(nextFrameFilename); }}

The source file can be downloaded here.

Explanation

We discuss the main parts of the above code:

1. First, three Mat objects are allocated to store the current frame and two foreground masks, obtained by using two different BS algorithms.

2. Mat frame; //current frame3. Mat fgMaskMOG; //fg mask generated by MOG method4. Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method

5. Two BackgroundSubtractor objects will be used to generate the foreground masks. In this example, default parameters are used, but it is also possible to declare specific parameters in the create function.

6. Ptr<BackgroundSubtractor> pMOG; //MOG Background subtractor7. Ptr<BackgroundSubtractor> pMOG2; //MOG2 Background subtractor8. ...

9. //create Background Subtractor objects10. pMOG = createBackgroundSubtractorMOG(); //MOG approach11. pMOG2 = createBackgroundSubtractorMOG2(); //MOG2 approach

12.The command line arguments are analysed. The user can chose between two options:

o video files (by choosing the option -vid);o image sequences (by choosing the option -img).

13. if(strcmp(argv[1], "-vid") == 0) {14. //input data coming from a video15. processVideo(argv[2]);16. }17. else if(strcmp(argv[1], "-img") == 0) {18. //input data coming from a sequence of images19. processImages(argv[2]);20. }

21.Suppose you want to process a video file. The video is read until the end is reached or the user presses the button ‘q’ or the button ‘ESC’.

22. while( (char)keyboard != 'q' && (char)keyboard != 27 ){23. //read the current frame24. if(!capture.read(frame)) {25. cerr << "Unable to read next frame." << endl;26. cerr << "Exiting..." << endl;27. exit(EXIT_FAILURE);28. }

29.Every frame is used both for calculating the foreground mask and for updating the background. If you want to change the learning rate used for updating the background model, it is possible to set a specific learning rate by passing a third parameter to the ‘apply’ method.

30. //update the background model31. pMOG->apply(frame, fgMaskMOG);32. pMOG2->apply(frame, fgMaskMOG2);

33.The current frame number can be extracted from the VideoCapture object and stamped in the top left corner of the current frame. A white rectangle is used to highlight the black colored frame number.

34. //get the frame number and write it on the current frame35. stringstream ss;36. rectangle(frame, cv::Point(10, 2), cv::Point(100,20),37. cv::Scalar(255,255,255), -1);38. ss << capture.get(CAP_PROP_POS_FRAMES);39. string frameNumberString = ss.str();40. putText(frame, frameNumberString.c_str(), cv::Point(15, 15),41. FONT_HERSHEY_SIMPLEX, 0.5 , cv::Scalar(0,0,0));

42.We are ready to show the current input frame and the results.

43. //show the current frame and the fg masks44. imshow("Frame", frame);45. imshow("FG Mask MOG", fgMaskMOG);46. imshow("FG Mask MOG 2", fgMaskMOG2);

47.The same operations listed above can be performed using a sequence of images as input. The processImage function is called and, instead of using a VideoCapture object, the images are read by using imread, after individuating the correct path for the next frame to read.

48. //read the first file of the sequence49. frame = imread(fistFrameFilename);50. if(!frame.data){51. //error in opening the first image52. cerr << "Unable to open first image frame: " <<

fistFrameFilename << endl;53. exit(EXIT_FAILURE);54. }55. ...56. //search for the next image in the sequence57. ostringstream oss;58. oss << (frameNumber + 1);59. string nextFrameNumberString = oss.str();60. string nextFrameFilename = prefix + nextFrameNumberString +

suffix;61. //read the next frame62. frame = imread(nextFrameFilename);63. if(!frame.data){64. //error in opening the next image in the sequence65. cerr << "Unable to open image frame: " << nextFrameFilename

<< endl;66. exit(EXIT_FAILURE);67. }68. //update the path of the current frame69. fn.assign(nextFrameFilename);

Note that this example works only on image sequences in which the filename format is <n>.png, where n is the frame number (e.g., 7.png).

Results

Given the following input parameters:

-vid Video_001.avi

The output of the program will look as the following:

The video file Video_001.avi is part of the Background Models Challenge (BMC) data set and it can be downloaded from the following link Video_001 (about 32 MB).

If you want to process a sequence of images, then the ‘-img’ option has to be chosen:

-img 111_png/input/1.png

The output of the program will look as the following:

The sequence of images used in this example is part of the Background Models Challenge (BMC) dataset and it can be downloaded from the following link sequence 111 (about 708 MB). Please, note that this example works only on sequences in which the filename format is <n>.png, where n is the frame number (e.g., 7.png).

Evaluation

To quantitatively evaluate the results obtained, we need to:

Save the output images; Have the ground truth images for the chosen sequence.

In order to save the output images, we can use imwrite. Adding the following code allows for saving the foreground masks.

string imageToSave = "output_MOG_" + frameNumberString + ".png";bool saved = imwrite(imageToSave, fgMaskMOG);if(!saved) { cerr << "Unable to save " << imageToSave << endl;}

Once we have collected the result images, we can compare them with the ground truth data. There exist several publicly available sequences for background subtraction that come with ground truth data. If you decide to use the Background Models Challenge (BMC), then the result images can be used as input for the BMC Wizard. The wizard can compute different measures about the accuracy of the results.

References

Background Models Challenge (BMC) website, http://bmc.univ-bpclermont.fr/ Antoine Vacavant, Thierry Chateau, Alexis Wilhelm and Laurent Lequievre. A

Benchmark Dataset for Foreground/Background Extraction. In ACCV 2012, Workshop: Background Models Challenge, LNCS 7728, 291-300. November 2012, Daejeon, Korea.