Embed Size (px)
Citation preview
Music VisualizerWith JavaFXTM,JMFTM + JavaTM SoundLucas JordanEffectiveUIDenver - Rochester - Vancouver
2
IntroductionJavaFX Launch> Sun came to us
“You tell us what you should make”> Music Visualizer - Synethesiator
Fancy UI Complex computation Music is hip (right?) Perfect for JavaFX
What is a music Visualizer and is JavaFX a reasonable platform for this?
What is an Audio Visualizer> Digital Audio
Sound is air pressure on your eardrum The pressure is proportional to the movement of a speaker The movement is proportional to the voltage in a wire The voltage is proportional to a byte in your application A byte of sound
> Digital Graphics Take that byte of sound Display a graphic proportional to the byte
> Now you are seeing something proportional to what you are hearing!
Examples
5
How do we get JavaFX to do this?> Playing audio with java
javafx.scene.media.* java.applet.AudioClip sun.audio.* java.sound.sampled.* javax.media.* Quicktime?
But which give us access to the audio data as it is played?
6
How do we get JavaFX to do this?> JavaFX can’t do this on its own
JMF or Java Sound required> But it can when there is a J2SE stack
Desktop Web Probably the fancier phones soon
7
How do we get JavaFX to do this?> JMF - Java Media Framework
Kind of ancient Really flexible Poor codec support / we are cheap Insane deployment story
Originally designed as an extension to the JVM> Java Sound
Poor codec support JavaFX can play MP3s, but Java out of the box can’t? Find third party jars
Built into the JVM
8
Getting at the audio data (JMF)> JMF requires a lot of code to set up> Here are the highlights on setting it up
To simply play audio Player player = Manager.createPlayer(url); We can’t use a Player
Implement Effect Register your Effect with the Manager Create a Processor from the Manager
A Processor extends Player Plug-in your Effect
9
Getting at the audio data (JMF)> The Effect interface
Extends Codec Gets inserted after the audio is decoded by a codec Has lots of other methods, process is what is interesting to us
public int process(Buffer inputBuffer, Buffer outputBuffer){//Copy inputBuffer into outputBuffer//Look at inputBuffer, do something interesting return BUFFER_PROCESSED_OK;}
10
Getting at the audio data (Java Sound)> Steps to get at the byte of sound
Create an AudioInputStream of a known AudioFormat Create a SourceDataLine Create a Thread
Reads from AudioInputStream Does something interesting Write to SourceDataLine
> We want a known format because we don’t want to write a spectrum analyzer for each possible encoding, sample rate, etc
11
Getting at the audio data (Java Sound)> The code to set up the audio input and outputAudioInputStream originalInputStream = AudioSystem.getAudioInputStream(inputStream);
//Format of data read inAudioFormat originalFormat = originalInputStream.getFormat();
//Normalize the data to something we knowdecodedFormat = new AudioFormat(AudioFormat.Encoding.PCM_SIGNED,
originalFormat.getSampleRate(), … );
//Input stream of the normalized format.AudioInputStream decodedInputStream = AudioSystem.getAudioInputStream(decodedFormat,
originalInputStream);
SourceDataLine line = (SourceDataLine)AudioSystem.getLine(info);
line.open(decodedFormat);line.start();
12
Getting at the audio data (Java Sound)> Thread to play the audio
Runnable.run() dis is a decodedInputStream
byte[] data = new byte[4096]int nBytesRead;while (nBytesRead = dis.read(data, 0, data.length) != -1) {
//Do Something Interesting! line.write(data, 0, nBytesRead);}
13
Doing Something Interesting Considerations> Sample Rate
Audio processing thread updates too fast for JavaFX A Spectrum Analysis or other FFT code might want more data then just one
read-write loop We want to accumulate audio bytes in the audio processing thread Create another thread which produces a final value at a more reasonable
rate We want to accumulate audio bytes in the audio processing thread
> Interacting With JavaFX Presenting our processed value in a JavaFX way We want to bind to the value JavaFX wrapper class Observable/Observer
14
Java and JavaFX classes> Create a Java class SoundDetailspublic class SoundDetails extends Observable implements Runnable{
private byte[] accumulatedData; private float currentLoudness;
public void run() { while(true){ try { Thread.sleep(1000 / 33); currentLoudness = calculateLoudness(accumulatedData); setChanged(); notifyObservers(); } catch (InterruptedException ex) { //log or something } } }}
15
Java and JavaFX classes> Create a class SoundDetailsFX
public class SoundDetailsFX extends Observer{ public-init var soundDetails:SoundDetails; public-read var currentLoudness:Number; init{ soundDetails.addObserver(this); } override function update(observable: Observable, arg: Object) { currentLoudness = soundDetails.getCurrentLoudness(); }}
16
Java and JavaFX classes> Finally, doing something interesting
Where sdfx is a SoundDetailsFX object
var rect = Rectangle{ width: 10 height: bind sdfx.currentLoudness fill: Color.ANTIQUEWHITE }
17
Recap> Exposed bytes of sound as they are being played
JMF or Java Sound> Accumulated and processed those bytes
Processing Thread> Wrapped the result of that process in a JavaFX object
Observable/Observer> Used a JavaFX programming paradigm to display a graphic
bind
18
Resources> Example Code
http://lucasjordan.com/audiovisualizer> JMF
http://java.sun.com/javase/technologies/desktop/media/jmf/> Java Sound
http://java.sun.com/products/java-media/sound/> Tritonus: Open Source Java Sound
http://www.tritonus.org/> EffectiveUI
http://effectiveui.com
