Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan

  • View
    214

  • Download
    0

Embed Size (px)

Text of Rafi Khan, Mike Levine, Jacob Metrick, and Hari Ganesan

Music Visualizer 2.0

Soundulous

A better music visualizerRafi Khan, Mike Levine, Jacob Metrick, and Hari GanesanConceptStreaming music and visualizer to mobile deviceInteractive visualizer with touchesSync to multiple phonesTwo user functions:New: a user (master) creates a sessionJoin: a user joins an existing session

What Is It Good For?Audience: teens, young adults, other bored peopleFun group activityParties! Bar Mitzvahs! Weddings!

Technical ChallengesStreaming images to each phoneCombining touch events from various phonesIntegrating touches into visualizer quicklyNetwork OptionsTCP/HTTPUDPSome combination of both

Technical ChallengesThe Visualizer!Need to do audio parsing for volume, frequency detectionFFT, amplitude, etc.Map touch events to changes in volume, or frequency, or something elseMake it fastMake it cool

The Client SideListening to touch eventsMixing eventsDifferentiating between touches and swipes

Rendering the bitmap images successfully

The Server SideGetting packets and transferring to input of visualizerSo far: one threadGetting data via UDP from client

The Server SideUsing Processing (processing.org)Making visualizer based on inputsSo we need three more threads:One for doing the actual rendering One for receiving video data from processingOne for sending video data to client(s)Total threads: 4! Wow!Workflow (creating session)Phone posts to /newServer responds with a port number (eg. 10001)Server starts sending visualization data (compressed as JPG) on 10001Phone opens an HTTP connection on 10001, and renders the incoming JPG dataWhenever the phone detects a touch event, it sends the event via UDP

Workflow (joining session)Phone posts to /join with the port numberServer responds to confirm.Exact same workflow as with creating!

Demo

Future ConsiderationsMaking the full-scale appLimiting number of people/sessionViewers/Enablers?Private/Public sessions?More optionsMultiple visualizersDifferent touch functionalities

Thanks!