The CLAM framework

Developing Cross-Platform
Audio and Music Applications with the
CLAM Framework

ACM Multimedia '06

Xavier Amatriain (UCSB)

Pau Arum (UPF)

David Garcia (UPF)

Index

Introduction

Applications

Infrastructure

Repositories

Tools

Conclusions

Demo!

Started in October 2000

There are more than 250 C++ classes (50.000 loc), compiled under GNU/ Linux, Windows and OSX.

CLAM has been used for various audio projects: time-stretching, real-time sax synthesis, content analysis and Mpeg7 description, real-time audio effects, audio annotation...

It has also been used for educational purposes.

Factsheet

Highlights

Although it currently specializes in audio and music, it presents a metamodel that has proven valid for general multimedia.

Deals well with spectral-domain processing

Can process streams with different kinds of data types, even user-defined

Two different working modes: application framework and rapid-prototyping.

It is really Object-oriented and has been documented through a Pattern Language

It is efficient and can be used for real-time applications

Applications. Annotator

Applications. SMSTools

Applications. Network Editor

Applications. Others

CLAM Components

Infrastructure. CLAM Network model

The CLAM network is a graphical model of computation based on Dataflow Process Networks

Scheduling can be performed both statically and dynamically, depending on the particular application.

Infrastructure. Ports, Controls, and Configurations

Two kinds of connection mechanisms between Processing Objects:

Ports

Data is transmitted synchronously through FIFO queues

Controls

Events are transmitted asynchronously

Configurations, unlike Controls, hold parameters that produce expensive or structural changes in a Processing Object

They can only be set when the Processing Object is not running

Infrastructure. Processing

Infrastructure. Dynamic vs. Static Compositions

Processing Objects can be grouped using two different mechanisms:

Networks

Created dynamically at run-time

Strive for flexibility and understandability

Offer automatic flow and data management

Processing Composites

Created statically at compile-time

Strive for efficiency

The programmer is in charge of most flow management issues.

Repositories (black-box)

Ready-to-use processing classes (almost 150):

Analysis (FFT, spectral analysis, SMS analysis, Tonal Analysis, descriptor extraction...), Arithmetic Operators, Input/Output Processing Objects (Audio, AudioFile, MIDI, SDIF), Generators, Transformation, Synthesis

Ready-to-use data classes:

Audio, Spectrum, SpectralPeakArray, Fundamental, Frame, Segment, Descriptors...

Tools. Platform Abstraction

Audio I/O

MIDI I/O

Audio File I/O: wav, aiff, mp3, ogg...

ID3 tags.

SDIF File support

Support for OSC, JACK, LADSPA, SDIF, VST, ASIO...

Tools

XML

Any Processing Data or Configuration has automatic XML persistency.

GUI

Visualization module based on the Qt toolkit

Plus many ready-to-use graphical components (widgets)

Including debugging tools

External Open Source libraries

FFTW (FFT)

Xercesc & libxml (XML using DOM API)

FLTK and QT GUI toolkits

RtAudio, PortAudio or DirectX (for Windows audio)

Libsndfile, Ogg-Vorbis, libmad (mp3), id3lib, for handling audio files.

oscpack

libjack

CppUnit (testing framework, only used for development)

pthreads (multithreading on Windows)

Conclusions

CLAM is currently in release 0.91 and we plan on getting to 1.0 by the end of 2006.

But it already offers:

A powerful framework that can yield efficient and robust applications.

A prototyping tool for rapid development

A number of ready-to-use applications.

Demo

Visit us at....

www.clam.upf.es

Xavier Amatriain, Pau Arumi, David Garcia ...

Click to edit the title text format

Click to edit the outline text format

Second Outline Level

Third Outline Level

Fourth Outline Level

Fifth Outline Level

Sixth Outline Level

Seventh Outline Level

Eighth Outline Level

Ninth Outline Level