space.light.sound

introduction

The reason to choose the topic of the interaction between sound, light and space

is my already longer existing interest in this kind of art and technology. I am

fascinated which creative possibilities have arisen in the last couple of years with

the development of new software for musicians and visual artists. A new field of

artists and culture has emerged; videojockeys manipulate videodata real-time-

based like a dj mixes his records. In the first part I want to introduce the artist

Carsten Nicolai who makes the attempt to transfer his video-experiments and his

electronic music into the architectural space.

In the second part I go further into this idea and present an own attempt of real-

time-manipulated video on the basis of a patch in the software max/msp and

jitter.

biography

The artist and musician Carsten Nicolai is born

1965 in Chemnitz and studied from 1985 till 1990

landscape design. He founded 1994 “noton.archiv

für ton und nichtton” (archive for sound and non-

sound) and 1999 (merged with rastermusic) the

label “raster-noton”, a platform for conceptional

and experimental related projects in electronic

music, art and science.

Nicolai explores issues of binaries and opposed polarities: pure black and

white, protons and electrons, noise and silence, order and chaos. He is using the

methodology of a scientist and plays with the rules of physicality. His audiovisual

works explore the generative processes of nature and our experience of light,

sound, space and time.

He arranges his music without sequencers; instead he uses mathematically edits

to give his compositions a precise rhythmic structure. He wants to break down

the walls between categories like fine arts and music, between the visual and

acoustic experiences.

telefunken

Nicolai’s “telefunken”, 2000 instructs the listener to

run an audiosignal through the S-VHS input of a

television set. It generates an array of white

horizontal bars that rise and fall, merge and part,

expand and contract on the TV-screen. The TV is

transformed from an information overloaded

medium into a viewer of abstract minimalist art. The

CD-player is transformed as well. The device for

playing music triggers now an audio-visual

assemblage.

insen

Carsten Nicolai and the pianist Ryuichi Sakamoto created in the project “insen” a

synergy between puristic piano-melodies, electronic sounds and their dynamic real-

time visualisation on stage. Nicolai worked directly with the sounds of Sakamoto.

He cut them into small parts (micro-editing), looped

them and put these compressed rhythms underneath

his piano tracks. Sakamoto´s piano playing and

Nicolai´s electronic processing triggered the visuals

of strips of light and grid-like patterns on the screen.

syn chron

In 2005 Nicolai did for an exhibition in the “Neue

Nationalgalerie”, Berlin the biggest project in his

effort to connect architecture, sound and light. He

placed a crystalline object in the main hall of the

museum. The visitors already could see this shining

sculpture from the outside of the museum, but it was

also allowed to enter the inside. The steel beams of

the construction were placed outside that the visitor

has inside the feeling of a neutral space. The

object’s outer and inner skins were the same and

acted as a loudspeaker.

The architectural body was a resonating acoustic space which was not only

emitting the sound, it was generating it. The piezo-ceramic loudspeakers on the

construction material projected the sound material into the inside. His standard

pitch is the frequency of voltage around 50Hz, a low bass frequency. Normally

the standard pitch A is around 440 Hz. He likes also the ultra-high frequencies

around 1100Hz like if you switch on the TV. The bass-speakers were installed in

the floor plates and started vibrating them continuously. The visitors felt the

sounds in their feet at the same time as they heard them.

Lasers projected light-modulations on the translucent

skin which were generated by Nicolai´s electronic

music. The composition was arranged for the whole

duration of the exhibition with different programs

for day and night situations or special presentations

by live musicians. The budget of this project was

around 500.000 Euros.

max/msp + jitter

Carsten Nicolai used the software max/msp and jitter for his projects. This

program is a graphical programming environment in which you are able to

create your own software. You are using a toolkit of some building blocks and

connecting them together with patch chords. Max is responsible for the control of

midi-based equipment, e.g. electronic music instruments like keyboards or

synthesizers. Msp is a set of audio processing objects to sample or generate

sounds. Jitter is a plug-in for max/msp to manipulate graphics (also 3D) or

videodata (also real time-based). I think the powerful on max/msp and jitter is

not only the freedom to build your own software but also the possibilities to

connect the different programs, in such a way you are able to use video tracking

to generate sounds, control visuals with your midi-keyboard or generate visual

effects with audio material.

jitter- patch flashbulbs

I tried to get to know the software and how I can use it for generating real-time-

based visuals like in the insen-project of Nicolai and Sakamoto. I want to explain

on the basis of a modified patch how it works to track the amplitude of an

audiotrack and use it to trigger the motion of a visual.

In the upper right corner of the patch there are

some building blocks which are responsible for the

audio input. You can open there another audiofile,

start the file from the beginning with the 0 and 1

buttons, pause and start the file again with stop

and startwindow or change the output volume.

Interesting is the input rate where you can pitch the

speed of the audiotrack. The left outlet of the left

selector object is connected to the part of the patch

in the upper left corner which is responsible for the

amplitude tracking.

The peakamp~ is used to track the sound’s

amplitude and reports the peak amplitude

automatically at regular intervals, by sending a

changeable interval in milliseconds in its right inlet,

in my example every 20 milliseconds.

The p AtodB subpatch converts the peak amplitude reported by peakamp~ into

decibels. That’s better because more than half of the sound pressure we’re

capable of hearing resides in the bottom one percent of its linear amplitude. The

patch deals now with the sound level on a logarithmic decibel scale and not with

a straight amplitude value.

The noise floor threshold is used to filter out a level of unwanted sounds and

focus only on a certain portion of the decibel range.

The patch dBexpander expands the levels to fill the full range of decibel scale

from 0dB to -120dB. In this example we have a noise floor threshold of -20dB.

The amplitude of the MSP signal at this moment is 0.204 which is a level of

-13.80dB. The subpatch expands that level, originally in the range from 0 to -20,

now it gets a position in the range from 0 to -120. The expanded level is now

around 82.78dB which is sent out the right outlet of the dBexpander and into the

next part of the patch.

The patch detectevent provides three parameters for

its subpatch:

-The Note-on Threshold is the level above which the

sound must rise to be an event or a note. In my

case an event will be reported when the level

exceeds -30dB.

-The Min. Off Time is the amount of time that the

level must remain below the threshold for the note

to be considered ended. The note will in my

example only be considered off when the level

stays below -30dB for at least 25 milliseconds.

-The Min. Note Duration is the time to wait before

looking for a level that goes back below the

threshold. The patch waits at least 50 ms before it

begins looking for a note-off level.

The result triggers the patcher flashbulbs that

generates a 16 x 12 matrix of random dots. Those

dots are upsampled with interpolation in the

window and they look now like flashes of light.

conclusion

Artists like Carsten Nicolai are experimenting with the media of sound, light and

space as one medium and link them to one experience. New technologies help

them to transfer their ideas in the real space.

For me was the work with max/msp and jitter helpful to get an idea of the

potentials of this software and hopefully a starting point to experiment further

with audiovisuals in architectural space.

references

Carsten Nicolai´s label: http://www.raster-noton.de/

his electronic music-pseudonym: http://www.alvanoto.com/

art- gallery with his works: http://www.eigen-art.com/

texts and interviews

Jansen, M.: 2005, Gendertronics, Suhrkamp Verlag, Frankfurt am Main

Kleiner, M.: 2003, Soundcultures, Suhrkamp Verlag, Frankfurt am Main

Nicolai, C.: 2002, autopilot, Die Gestalten-Verlag GmbH, Berlin

http://www.de-bug.de/texte/3646.html

http://www.artnet.de/magazine/features/wackernagel/wackernagel03-04-

05.asp

http://www.medienkunstnetz.de/source-text/122/

http://www.pingmag.jp/2005/05/01/carsten-nicolai-creating-space-between-

things/

picture- sources:

http://www.kultureflash.net/archive/141/priview.html

http://www.eigen-art.com/

youtube-videolinks

Insen-live: http://www.youtube.com/watch?v=gS4jE0SMXxs

Berlin-visual: http://www.youtube.com/watch?v=gjMvQHgAgjg

Alva Noto + Ryuichi Sakamoto: http://www.youtube.com/watch?v=PSYuu74FiqI

synchron-installation: http://www.youtube.com/watch?v=f7bgkpTJITE

software max/msp and jitter:

http://www.cycling74.com/