Rickroll To Go With PHP, WURFL, and Other Open Source Tools

Brian DeShongSeptember 16, 2008

Rickroll To Go With PHP, WURFL, and Other Open Source Tools

Intro

2

Intro

Brian DeShong

2

Intro

Brian DeShongSenior Manager, Technology for Schematic

2

Intro

Brian DeShongSenior Manager, Technology for SchematicApplication and systems architect

2

Intro

Brian DeShongSenior Manager, Technology for SchematicApplication and systems architectDeveloper

2

Who is Schematic?

3

Who is Schematic?

Full-service interactive agency

3

Who is Schematic?

Full-service interactive agencyOffices in Atlanta; New York; LA; San Francisco; Austin, TX; Minneapolis, MN; San Jose, Costa Rica

3

Who is Schematic?

Full-service interactive agencyOffices in Atlanta; New York; LA; San Francisco; Austin, TX; Minneapolis, MN; San Jose, Costa RicaSome clients include...

DellTargetDisneyNokiaComcastMTV

3

What are we talking about?

4


PC and mobile development are very different

4


PC and mobile development are very differentIdentifying devices with WURFL

4


PC and mobile development are very differentIdentifying devices with WURFLGenerating content for mobile consumption

Images, video, audioPHP, open source tools

4

Computers vs. Mobile Devices

5

6

6

6

6

6

Dial-up Broadband T1+, etc.6

Dial-up Broadband T1+, etc.6

7

7

7

7

EDGE 3G EVDO

7

EDGE 3G EVDO Proxy, no proxy

7

WURFL

8

What is it?

9

What is it?

Wireless Universal Resource File

9

What is it?

Wireless Universal Resource FileXML file containing characteristics of many mobile devices

9

What is it?

Wireless Universal Resource FileXML file containing characteristics of many mobile devicesOpen source

9

What is it?

Wireless Universal Resource FileXML file containing characteristics of many mobile devicesOpen sourcehttp://wurfl.sourceforge.net/

9

10

WURFL Examples

10

WURFL Examples

10

WURFL Examples

10

WURFL Examples

10

WURFL Examples

How can we use WURFL?

11


Lookup device by user agent

11


Lookup device by user agentDevice data indicates its capabilities

iPhone320x4803GP, MP4 video @ 320x240, 30 fps

11

Data offered

12

Data offered

Display size

12

Data offered

Display sizeUser agent strings

12

Data offered

Display sizeUser agent stringsSupported audio, video, image formats

12

Data offered

Display sizeUser agent stringsSupported audio, video, image formatsStreaming capabilities

12

Data offered

Display sizeUser agent stringsSupported audio, video, image formatsStreaming capabilitiesHas QWERTY keyboard?

12

Data offered

Display sizeUser agent stringsSupported audio, video, image formatsStreaming capabilitiesHas QWERTY keyboard?XHTML, Ajax capabilities 12

Gotchas

13

Gotchas

wurfl.xml is ~8 MB in size!

13

Gotchas

wurfl.xml is ~8 MB in size!Not a great idea to parse it from front end

13

Gotchas

wurfl.xml is ~8 MB in size!Not a great idea to parse it from front endExtract data you need, place in database

Youʼll need a plan for updating from your source data

13

Use what you need

14

Use what you need

Display sizes

14

Use what you need

Display sizesRelevant video, audio formats supported

14

Use what you need

Display sizesRelevant video, audio formats supportedMIDP, CLDC versions supported

Handy in determining compatibility for applications

14

Use what you need

Display sizesRelevant video, audio formats supportedMIDP, CLDC versions supported

Handy in determining compatibility for applications

Bitmasks values are handy

14

Applying bitmask and DB

15

<?phpdefine(‘VIDEO_MP4’, 1);define(‘VIDEO_3GP’, 2);define(‘VIDEO_WMV’, 4);

// $videoFormats = 7;$videoFormats = VIDEO_MP4 | VIDEO_3GP | VIDEO_WMV;?>

id | manuf | model | video_formats----------------------------------1 | Nokia | N95 | 7

Using it from PHP

16

Using it from PHP

Itʼs just XML!

16

Using it from PHP

Itʼs just XML!SimpleXML, DOM, etc.

16

Using it from PHP

Itʼs just XML!SimpleXML, DOM, etc.Caching your lookups

MemcachedSQLite, other DBs

16

Using it from PHP

Itʼs just XML!SimpleXML, DOM, etc.Caching your lookups

MemcachedSQLite, other DBs

Lookup by user agent$_SERVER[‘HTTP_USER_AGENT’]Opera: X-OperaMini-Phone-UA

16

Summary and other options

17


Use WURFL responsibly and efficiently!PHP scripts on its websiteEmploy some caching techniques

17


Use WURFL responsibly and efficiently!PHP scripts on its websiteEmploy some caching techniques

WURFL and other options are key to a great mobile experience

DeviceAtlas

17

Mobile Content Challenges

18

Some challenges

19

Some challenges

Varying amounts of bandwidth available

19

Some challenges

Varying amounts of bandwidth availableCostly data plans

Unlimited vs. per-MB/GB

19

Some challenges



Different capabilities across devices

19

Some challenges



Different capabilities across devicesLimited memory in devices

19

How to workaround them

20


Serve content tailored to userʼs deviceEfficient use of resources and bandwidthDetermine if a given piece of content will work on the device before download

20


Serve content tailored to userʼs deviceEfficient use of resources and bandwidthDetermine if a given piece of content will work on the device before download

Hope for the best!User will sort it outIt will work or it wonʼt

20

To be fair...

21

To be fair...

None of this may be relevant in a few years

21

To be fair...

None of this may be relevant in a few yearsMobile browsers and devices are evolving

21

To be fair...

None of this may be relevant in a few yearsMobile browsers and devices are evolvingLow-end devices are prevalent in many parts of the world

Who is your audience, and where are they located?

21

Tailoring Content for Devices

22

My Personal Approach

23


23

I pre-generate files and write them to disk


23

I pre-generate files and write them to diskI can rely on them being in place when making decision


23

I pre-generate files and write them to diskI can rely on them being in place when making decisionImages:foo_240x320.jpgfoo_320x240.jpg


23


Video:foo_176x144.3gpfoo_176x144.mp4


23


Video:foo_176x144.3gpfoo_176x144.mp4

Audio:foo.aac

Images

24

Optimizing image delivery

25


25

User visits site


25

User visits site

Device identified?


25

User visits site

Device identified?

No


25

User visits site

Device identified?

YesNo

Optimizing images with PHP

26


Graphics extensions availableGDimagick (ImageMagick)

26


Graphics extensions availableGDimagick (ImageMagick)

Command-line ImageMagick binaries

26

Using imagick

$imagick = new Imagick();$imagick->readImage(‘/astley/rick.jpg’);

27

Example

28

Example

Source Image: 507x398

28

Example

Thumbnailed image: 240x188

28

Example

Thumbnailed image: 240x188

$imagick->thumbnailImage(240, 0);

28

Example

Cropped thumbnail image: 240x320

28

Example

Cropped thumbnail image: 240x320

$imagick->cropThumbnailImage(240, 320);

28

Gotchas

29

Gotchas

Generating images on-demand is expensive

29

Gotchas

Generating images on-demand is expensiveKeep it cheap and/or infrequent

Write the different sizes to disk; routine cleanupMemcachedPre-generate them

29

Summary

30

Summary

Serving image tailored for deviceUses least amount of bandwidth possibleUses device resources efficiently

30

Summary


Be smart with the server-side implementation!

30

Summary


Be smart with the server-side implementation!Less complex than dealing with video, audio

30

Video

31

Video on mobile devices

32


Most devices canʼt play the formats weʼre accustomed to on PCs

32


Most devices canʼt play the formats weʼre accustomed to on PCs3GP is widely supported

32


Most devices canʼt play the formats weʼre accustomed to on PCs3GP is widely supportedMPEG-4 on newer devices

32

Optimizing video delivery

3321


3321

User visits site


3321

User visits site

Device supports MPEG-4 video?


3321

User visits site


No

Serve3GP


3321

User visits site


YesNo

Serve3GP

ServeMPEG-4


3321

User visits site


YesNo

Serve3GP

ServeMPEG-4

Max video resolution?

Mobile Video Sites

34

Mobile Video Sites

34

Mobile Video Sites

34

Mobile Video Sites

34

Mobile Video Sites

34

Optimizing for mobile

35


Lower resolution176x144, 320x240

35



Reduce video frameratePlayback looks jerkyBut it requires less bandwidth!

35



Reduce video frameratePlayback looks jerkyBut it requires less bandwidth!

Reduce bitrates35

Tools

36

Tools

FFmpeg, Mencoder

36

Tools

FFmpeg, MencoderOn2 Flix Engine for Flash (commercial)

36

Tools

FFmpeg, MencoderOn2 Flix Engine for Flash (commercial)Anystream Agility

36

Tools

FFmpeg, MencoderOn2 Flix Engine for Flash (commercial)Anystream AgilityCommercial tools to read Windows Media, other DRMʼd formats

36

Expense

37

Expense

Generating videos is not cheap

37

Expense

Generating videos is not cheapShould not be done in real time

End user will have to wait for it!

37

Expense



Pre-generate with an offline processStore them on diskPurge when and if needed

37

Expense



Pre-generate with an offline processStore them on diskPurge when and if needed

Weʼll look at FFmpeg shortly!

37

Audio

38

Audio on mobile devices

39


Devices have been playing MP3s for years now

39


Devices have been playing MP3s for years nowWider support than video

39


Devices have been playing MP3s for years nowWider support than videoMP3, AAC formats are well supported

39

Optimizing audio delivery

402921


402921

User visits site


402921

User visits site

AAC audio

supported?


402921

User visits site

AAC audio

supported?

No

ServeMP3


402921

User visits site

AAC audio

supported?

YesNo

ServeMP3

ServeAAC

Tools

41

Tools

FFmpeg

41

Tools

FFmpegTiMidity++ (MIDI to PCM)

41

Tools

FFmpegTiMidity++ (MIDI to PCM)Note:

Iʼm not dealing with MP3 licensing issues here!

41

Expense

42

Expense

Again, generating audio isnʼt cheap

42

Expense

Again, generating audio isnʼt cheapShouldnʼt transcode in real time

42

Expense

Again, generating audio isnʼt cheapShouldnʼt transcode in real timePre-generate in an offline process

42

Expense


42

Expense


What can we use?Surprise! FFmpeg! 42

What’s this FFmpeg stuff?

43

Basics

44

Basics

Decodes and encodes audio and video streams

44

Basics

Decodes and encodes audio and video streamsMany A/V formats supported

44

Basics

Decodes and encodes audio and video streamsMany A/V formats supportedSupports use of external libraries to handle additional formats

44

Basics

Decodes and encodes audio and video streamsMany A/V formats supportedSupports use of external libraries to handle additional formatsRuns on many platforms (Windows, OS X, Linux)

44

Usage

ffmpeg [[infile options] -i infile]... {[outfile options] outfile}...

45

Usage


45ffmpeg -h: 504 lines!Extensive amount of options for both audio and video

Usage


45ffmpeg -h: 504 lines!Extensive amount of options for both audio and video

FFmpeg: Video

46

47

Notable video encoding options

47


-acodec: audio codec to use

47



-vcodec: video codec to use

47




-s: frame size

47




-s: frame size

-r: video framerate

47




-s: frame size

-r: video framerate

-b: bitrate

47




-s: frame size

-r: video framerate

-b: bitrateffmpeg -i rr.mp4 \ -acodec libfaac \ -vcodec h263 \ -s 176x144 \ -r 12 \ foo.3gp

Examples

48

Examples

48

Examples

320x240MPEG-4, AAC

30 fps244kbps6.2 MB

48

Examples

320x240MPEG-4, AAC

30 fps244kbps6.2 MB

176x1443GP, AAC

30 fps90kbps2.3 MB

48

Examples

320x240MPEG-4, AAC

30 fps244kbps6.2 MB

176x1443GP, AAC

30 fps90kbps2.3 MB

176x1443GP, AAC

8 fps63kbps1.6 MB

48

Grab screenshots!

$ ffmpeg \ -i rr.mp4 \ -f image2 \ frame%2d.jpg 49

50

50

FFmpeg: Audio

51

52

Notable audio encoding options

52



52



-ar: audio sampling rate

52




-ab: audio bitrate

52




-ab: audio bitrate

-ac: number of channels

52




-ab: audio bitrate

-ac: number of channels

ffmpeg -i iib.mp3 \ -ab 64000 \ -ac 1 \ foo.mp3

Examples

53

Examples

53

Original:128kbps4.1MB

Examples

53


Second Clip:64kbps

2.05MB (1/2)

Examples

53


Second Clip:64kbps

2.05MB (1/2)

Third Clip:32kbps

1.025MB (1/4)

Examples

53


Second Clip:64kbps

2.05MB (1/2)

Third Clip:32kbps

1.025MB (1/4)

Final Clip:8kbps

1.025MB (???)

FFmpeg Summary

54

FFmpeg Summary

54

Powerful tool!

FFmpeg Summary

54

Powerful tool!Takes a lot to master it

Probably makes more sense to A/V experts

FFmpeg Summary

54



But itʼs easy enough to get the job done right

FFmpeg Summary

54



But itʼs easy enough to get the job done rightIts manpage is your friend!

In Closing...

55

Mobile is tricky!

56

Mobile is tricky!

All of those combinationsDevicesProvidersBandwidth / data plansCapabilities

56

Mobile is tricky!


Use resources efficiently56

Mobile is tricky!


Use resources efficientlyOptimize the experience for the end user 56

Resources

57

Resources

WURFL: http://wurfl.sourceforge.net/

57

Resources

WURFL: http://wurfl.sourceforge.net/FFmpeg: http://ffmpeg.mplayerhq.hu/

57

Resources

WURFL: http://wurfl.sourceforge.net/FFmpeg: http://ffmpeg.mplayerhq.hu/ImageMagick: http://www.imagemagick.org/

PECL extension: http://pecl.php.net/package/imagick/

57

For more information, please see me or visithttp://www.schematic.com/#/JoinUs/

It goes without saying: Schematic is only as good as the people who work here. That’s why we’re so particular about recruiting, training, nurturing, and retaining the very best people in our field.

If you have digital expertise (technical, creative, managerial–or something else entirely), enthusiasm, curiosity, and the ability to collaborate with others, we’d love to hear from you.

We’re Hiring!

58

Thanks!

http://www.deshong.net/http://www.schematic.com/

[email protected]@schematic.com

59

Thanks!

http://www.deshong.net/http://www.schematic.com/

[email protected]@schematic.com

59