Music Visualizer With JavaFX, JMF + Java Sound What is an Audio Visualizer > Digital Audio Sound is

  • View
    1

  • Download
    0

Embed Size (px)

Text of Music Visualizer With JavaFX, JMF + Java Sound What is an Audio Visualizer > Digital Audio Sound...

  • Music Visualizer With JavaFXTM, JMFTM + JavaTM Sound Lucas Jordan EffectiveUI Denver - Rochester - Vancouver

  • 2

    Introduction JavaFX Launch > Sun came to us

     “You tell us what you should make” > Music Visualizer - Synethesiator

     Fancy UI  Complex computation  Music is hip (right?)  Perfect for JavaFX

    What is a music Visualizer and is JavaFX a reasonable platform for this?

  • What is an Audio Visualizer > Digital Audio

     Sound is air pressure on your eardrum  The pressure is proportional to the movement of a speaker  The movement is proportional to the voltage in a wire  The voltage is proportional to a byte in your application  A byte of sound

    > Digital Graphics  Take that byte of sound  Display a graphic proportional to the byte

    > Now you are seeing something proportional to what you are hearing!

  • Examples

  • 5

    How do we get JavaFX to do this? > Playing audio with java

     javafx.scene.media.*  java.applet.AudioClip  sun.audio.*  java.sound.sampled.*  javax.media.*  Quicktime?

    But which give us access to the audio data as it is played?

  • 6

    How do we get JavaFX to do this? > JavaFX can’t do this on its own

     JMF or Java Sound required > But it can when there is a J2SE stack

     Desktop  Web  Probably the fancier phones soon

  • 7

    How do we get JavaFX to do this? > JMF - Java Media Framework

     Kind of ancient  Really flexible  Poor codec support / we are cheap  Insane deployment story

     Originally designed as an extension to the JVM > Java Sound

     Poor codec support  JavaFX can play MP3s, but Java out of the box can’t?  Find third party jars

     Built into the JVM

  • 8

    Getting at the audio data (JMF) > JMF requires a lot of code to set up > Here are the highlights on setting it up

     To simply play audio  Player player = Manager.createPlayer(url);  We can’t use a Player

     Implement Effect  Register your Effect with the Manager  Create a Processor from the Manager

     A Processor extends Player  Plug-in your Effect

  • 9

    Getting at the audio data (JMF) > The Effect interface

     Extends Codec  Gets inserted after the audio is decoded by a codec  Has lots of other methods, process is what is interesting to us

    public int process(Buffer inputBuffer, Buffer outputBuffer){ //Copy inputBuffer into outputBuffer //Look at inputBuffer, do something interesting return BUFFER_PROCESSED_OK; }

  • 10

    Getting at the audio data (Java Sound) > Steps to get at the byte of sound

     Create an AudioInputStream of a known AudioFormat  Create a SourceDataLine  Create a Thread

     Reads from AudioInputStream  Does something interesting  Write to SourceDataLine

    > We want a known format because we don’t want to write a spectrum analyzer for each possible encoding, sample rate, etc

  • 11

    Getting at the audio data (Java Sound) > The code to set up the audio input and output AudioInputStream originalInputStream = AudioSystem.getAudioInputStream(inputStream);

    //Format of data read in AudioFormat originalFormat = originalInputStream.getFormat();

    //Normalize the data to something we know decodedFormat = new AudioFormat(AudioFormat.Encoding.PCM_SIGNED,

    originalFormat.getSampleRate(), … );

    //Input stream of the normalized format. AudioInputStream decodedInputStream = AudioSystem.getAudioInputStream(decodedFormat,

    originalInputStream);

    SourceDataLine line = (SourceDataLine)AudioSystem.getLine(info);

    line.open(decodedFormat); line.start();

  • 12

    Getting at the audio data (Java Sound) > Thread to play the audio

     Runnable.run()  dis is a decodedInputStream

    byte[] data = new byte[4096] int nBytesRead; while (nBytesRead = dis.read(data, 0, data.length) != -1) {

    //Do Something Interesting! line.write(data, 0, nBytesRead); }

  • 13

    Doing Something Interesting Considerations > Sample Rate

     Audio processing thread updates too fast for JavaFX  A Spectrum Analysis or other FFT code might want more data then just one

    read-write loop  We want to accumulate audio bytes in the audio processing thread  Create another thread which produces a final value at a more reasonable

    rate  We want to accumulate audio bytes in the audio processing thread

    > Interacting With JavaFX  Presenting our processed value in a JavaFX way  We want to bind to the value  JavaFX wrapper class  Observable/Observer

  • 14

    Java and JavaFX classes > Create a Java class SoundDetails public class SoundDetails extends Observable implements Runnable{

    private byte[] accumulatedData; private float currentLoudness;

    public void run() { while(true){ try { Thread.sleep(1000 / 33); currentLoudness = calculateLoudness(accumulatedData); setChanged(); notifyObservers(); } catch (InterruptedException ex) { //log or something } } } }

  • 15

    Java and JavaFX classes > Create a class SoundDetailsFX

    public class SoundDetailsFX extends Observer{ public-init var soundDetails:SoundDetails; public-read var currentLoudness:Number; init{ soundDetails.addObserver(this); } override function update(observable: Observable, arg: Object) { currentLoudness = soundDetails.getCurrentLoudness(); } }

  • 16

    Java and JavaFX classes > Finally, doing something interesting

     Where sdfx is a SoundDetailsFX object

    var rect = Rectangle{ width: 10 height: bind sdfx.currentLoudness fill: Color.ANTIQUEWHITE }

  • 17

    Recap > Exposed bytes of sound as they are being played

     JMF or Java Sound > Accumulated and processed those bytes

     Processing Thread > Wrapped the result of that process in a JavaFX object

     Observable/Observer > Used a JavaFX programming paradigm to display a graphic

     bind

  • 18

    Resources > Example Code

     http://lucasjordan.com/audiovisualizer > JMF

     http://java.sun.com/javase/technologies/desktop/media/jmf/ > Java Sound

     http://java.sun.com/products/java-media/sound/ > Tritonus: Open Source Java Sound

     http://www.tritonus.org/ > EffectiveUI

     http://effectiveui.com

  • 19

    Lucas Jordan lucasjordan@gmail.com lucas.jordan@effectiveui.com

    effectiveui.com lucasjordan.com

    Music Visualizer With JavaFXTM, JMFTM + JavaTM Sound Introduction JavaFX Launch What is an Audio Visualizer Examples How do we get JavaFX to do this? Slide 6 Slide 7 Getting at the audio data (JMF) Slide 9 Getting at the audio data (Java Sound) Slide 11 Slide 12 Doing Something Interesting Considerations Java and JavaFX classes Slide 15 Slide 16 Recap Resources Slide 19