Real-World Game Audio Development with HTML5 Craig Robinson, Absolute Hero Martin Reurings, Spil...

Real-World Game Audio Development with HTML5

Craig Robinson, Absolute HeroMartin Reurings, Spil Games

About us

• Craig Robinson craig@absoluteherogames.com

– Absolute Hero– Mobile games– Multimedia software

2

• Martin Reurings martin.reurings@spilgames.com

– Spil Games– Webbased UI’s– Ajax before it had a

name

3

What's out there?

Let's get a feeling for the landscape of HTML5 audio shall we?

It can't be all bad after all…

HTML, audio after the silent years

• <bgsound>, remember the midi-tracks?

• Flash, good enough while it lasted

4

5

HTML5 Audio vs Web Audio API

• <audio />, a ray of light, tainted by fear– Fragmented codec support– Autoplay got disabled– Only 1 sound at a time

• Web Audio API, the promised land– We’ll get back to this

6

7

HTML5 Audio in Desktop Browsers

• Fast response• Multiple streams• Fragmented codec support• Solid codec detection

• Supported in: Chrome, Safari, IE9+, Opera• Chrome and Safari support Web Audio API, with

support for FireFox just around the corner (FF 23)!

8

Playing a sound on Desktop

9

What about Mobile?

• There seems to be a difference of opinion• iOS5, audio-tag, but only on user-interaction, only 1

stream• iOS6+, same <audio /> support but Web Audio API• Android 4, audio-tag, only on user-interaction, only

1 stream• Windows Phone, not enough market-share yet.• BB10, audio-tag• Latency, let’s get around that

10

Playing low-latency audio on Mobile

11

Codecs

12

Source: http://html5doctor.com/html5-audio-the-state-of-play/

Mobile browser codec support

13

Source: http://html5doctor.com/html5-audio-the-state-of-play/

Codecs

• Commonly supported audio codecs– ac3/aac– mp3/mpeg-1– ogg/vorbis– m4a/h.264

• Ogg FTW?• Just in case, there’s people creating JS decoders for

unsupported Codecs!

14

Handling different Codecs

15

Right, audio-tag, it's been around

• Pros– Widely supported– Built in capabilities for streaming/buffering

• Cons– User-interaction only, or the sound takes a long time

to load, no snappy sound-effects here

• Solutions– Audio-sprites, yay!! Easy to make with NPM and (

https://github.com/tonistiigi/audiosprite)• In Chrome, server must support byte range requests• FF doesn't support seeking in .ac• Chrome doesn't support seeking in .ogg• Ok, never tested in IE, no noticeable market-share…

16

Using Sound-Sprites

17

This too shall pass.~ancient

proverb~

18

Web Audio API – What is it?

• JavaScript interface to access advanced native audio capabilities

• Modular, graph-based to support complex mixing and effects

• Allows playback of sampled audio and audio synthesis

• Integration with <audio> media element• Spatialized audio and advanced filter effects• Ability to synthesize and process audio stream in JS

code• Provides sample accurate scheduling and low

latency• Can play multiple, simultaneous sounds!

19

Simple routing

20

Source: https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html

Advanced routing

21

Source: https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html

Testing for availability

22

Loading and playing a sound

23

Loading and playing a sound

24

Use with HTML5 audio tag to support streaming

25

Use with HTML5 audio tag to support streaming

26

Demo

27

Three things to remember

• A bit of trouble, but it’s not that bad!

• HTML5 <audio> tag --> Web Audio API

• Desktop audio != mobile audio

28

Resources

• https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html

• http://www.html5audio.org/

• http://uglyhack.appspot.com/webaudiotoy/

29