Brian DeShongSeptember 16, 2008
Rickroll To Go With PHP, WURFL, and Other Open Source Tools
Intro
2
Intro
Brian DeShong
2
Intro
Brian DeShongSenior Manager, Technology for Schematic
2
Intro
Brian DeShongSenior Manager, Technology for SchematicApplication and systems architect
2
Intro
Brian DeShongSenior Manager, Technology for SchematicApplication and systems architectDeveloper
2
Who is Schematic?
3
Who is Schematic?
Full-service interactive agency
3
Who is Schematic?
Full-service interactive agencyOffices in Atlanta; New York; LA; San Francisco; Austin, TX; Minneapolis, MN; San Jose, Costa Rica
3
Who is Schematic?
Full-service interactive agencyOffices in Atlanta; New York; LA; San Francisco; Austin, TX; Minneapolis, MN; San Jose, Costa RicaSome clients include...
DellTargetDisneyNokiaComcastMTV
3
What are we talking about?
4
What are we talking about?
PC and mobile development are very different
4
What are we talking about?
PC and mobile development are very differentIdentifying devices with WURFL
4
What are we talking about?
PC and mobile development are very differentIdentifying devices with WURFLGenerating content for mobile consumption
Images, video, audioPHP, open source tools
4
Computers vs. Mobile Devices
5
6
6
6
6
6
Dial-up Broadband T1+, etc.6
Dial-up Broadband T1+, etc.6
7
7
7
7
EDGE 3G EVDO
7
EDGE 3G EVDO Proxy, no proxy
7
WURFL
8
What is it?
9
What is it?
Wireless Universal Resource File
9
What is it?
Wireless Universal Resource FileXML file containing characteristics of many mobile devices
9
What is it?
Wireless Universal Resource FileXML file containing characteristics of many mobile devicesOpen source
9
What is it?
Wireless Universal Resource FileXML file containing characteristics of many mobile devicesOpen sourcehttp://wurfl.sourceforge.net/
9
10
WURFL Examples
10
WURFL Examples
10
WURFL Examples
10
WURFL Examples
10
WURFL Examples
How can we use WURFL?
11
How can we use WURFL?
Lookup device by user agent
11
How can we use WURFL?
Lookup device by user agentDevice data indicates its capabilities
iPhone320x4803GP, MP4 video @ 320x240, 30 fps
11
Data offered
12
Data offered
Display size
12
Data offered
Display sizeUser agent strings
12
Data offered
Display sizeUser agent stringsSupported audio, video, image formats
12
Data offered
Display sizeUser agent stringsSupported audio, video, image formatsStreaming capabilities
12
Data offered
Display sizeUser agent stringsSupported audio, video, image formatsStreaming capabilitiesHas QWERTY keyboard?
12
Data offered
Display sizeUser agent stringsSupported audio, video, image formatsStreaming capabilitiesHas QWERTY keyboard?XHTML, Ajax capabilities 12
Gotchas
13
Gotchas
wurfl.xml is ~8 MB in size!
13
Gotchas
wurfl.xml is ~8 MB in size!Not a great idea to parse it from front end
13
Gotchas
wurfl.xml is ~8 MB in size!Not a great idea to parse it from front endExtract data you need, place in database
Youʼll need a plan for updating from your source data
13
Use what you need
14
Use what you need
Display sizes
14
Use what you need
Display sizesRelevant video, audio formats supported
14
Use what you need
Display sizesRelevant video, audio formats supportedMIDP, CLDC versions supported
Handy in determining compatibility for applications
14
Use what you need
Display sizesRelevant video, audio formats supportedMIDP, CLDC versions supported
Handy in determining compatibility for applications
Bitmasks values are handy
14
Applying bitmask and DB
15
<?phpdefine(‘VIDEO_MP4’, 1);define(‘VIDEO_3GP’, 2);define(‘VIDEO_WMV’, 4);
// $videoFormats = 7;$videoFormats = VIDEO_MP4 | VIDEO_3GP | VIDEO_WMV;?>
id | manuf | model | video_formats----------------------------------1 | Nokia | N95 | 7
Using it from PHP
16
Using it from PHP
Itʼs just XML!
16
Using it from PHP
Itʼs just XML!SimpleXML, DOM, etc.
16
Using it from PHP
Itʼs just XML!SimpleXML, DOM, etc.Caching your lookups
MemcachedSQLite, other DBs
16
Using it from PHP
Itʼs just XML!SimpleXML, DOM, etc.Caching your lookups
MemcachedSQLite, other DBs
Lookup by user agent$_SERVER[‘HTTP_USER_AGENT’]Opera: X-OperaMini-Phone-UA
16
Summary and other options
17
Summary and other options
Use WURFL responsibly and efficiently!PHP scripts on its websiteEmploy some caching techniques
17
Summary and other options
Use WURFL responsibly and efficiently!PHP scripts on its websiteEmploy some caching techniques
WURFL and other options are key to a great mobile experience
DeviceAtlas
17
Mobile Content Challenges
18
Some challenges
19
Some challenges
Varying amounts of bandwidth available
19
Some challenges
Varying amounts of bandwidth availableCostly data plans
Unlimited vs. per-MB/GB
19
Some challenges
Varying amounts of bandwidth availableCostly data plans
Unlimited vs. per-MB/GB
Different capabilities across devices
19
Some challenges
Varying amounts of bandwidth availableCostly data plans
Unlimited vs. per-MB/GB
Different capabilities across devicesLimited memory in devices
19
How to workaround them
20
How to workaround them
Serve content tailored to userʼs deviceEfficient use of resources and bandwidthDetermine if a given piece of content will work on the device before download
20
How to workaround them
Serve content tailored to userʼs deviceEfficient use of resources and bandwidthDetermine if a given piece of content will work on the device before download
Hope for the best!User will sort it outIt will work or it wonʼt
20
To be fair...
21
To be fair...
None of this may be relevant in a few years
21
To be fair...
None of this may be relevant in a few yearsMobile browsers and devices are evolving
21
To be fair...
None of this may be relevant in a few yearsMobile browsers and devices are evolvingLow-end devices are prevalent in many parts of the world
Who is your audience, and where are they located?
21
Tailoring Content for Devices
22
My Personal Approach
23
My Personal Approach
23
I pre-generate files and write them to disk
My Personal Approach
23
I pre-generate files and write them to diskI can rely on them being in place when making decision
My Personal Approach
23
I pre-generate files and write them to diskI can rely on them being in place when making decisionImages:foo_240x320.jpgfoo_320x240.jpg
My Personal Approach
23
I pre-generate files and write them to diskI can rely on them being in place when making decisionImages:foo_240x320.jpgfoo_320x240.jpg
Video:foo_176x144.3gpfoo_176x144.mp4
My Personal Approach
23
I pre-generate files and write them to diskI can rely on them being in place when making decisionImages:foo_240x320.jpgfoo_320x240.jpg
Video:foo_176x144.3gpfoo_176x144.mp4
Audio:foo.aac
Images
24
Optimizing image delivery
25
Optimizing image delivery
25
User visits site
Optimizing image delivery
25
User visits site
Device identified?
Optimizing image delivery
25
User visits site
Device identified?
No
Optimizing image delivery
25
User visits site
Device identified?
YesNo
Optimizing images with PHP
26
Optimizing images with PHP
Graphics extensions availableGDimagick (ImageMagick)
26
Optimizing images with PHP
Graphics extensions availableGDimagick (ImageMagick)
Command-line ImageMagick binaries
26
Using imagick
$imagick = new Imagick();$imagick->readImage(‘/astley/rick.jpg’);
27
Example
28
Example
Source Image: 507x398
28
Example
Thumbnailed image: 240x188
28
Example
Thumbnailed image: 240x188
$imagick->thumbnailImage(240, 0);
28
Example
Cropped thumbnail image: 240x320
28
Example
Cropped thumbnail image: 240x320
$imagick->cropThumbnailImage(240, 320);
28
Gotchas
29
Gotchas
Generating images on-demand is expensive
29
Gotchas
Generating images on-demand is expensiveKeep it cheap and/or infrequent
Write the different sizes to disk; routine cleanupMemcachedPre-generate them
29
Summary
30
Summary
Serving image tailored for deviceUses least amount of bandwidth possibleUses device resources efficiently
30
Summary
Serving image tailored for deviceUses least amount of bandwidth possibleUses device resources efficiently
Be smart with the server-side implementation!
30
Summary
Serving image tailored for deviceUses least amount of bandwidth possibleUses device resources efficiently
Be smart with the server-side implementation!Less complex than dealing with video, audio
30
Video
31
Video on mobile devices
32
Video on mobile devices
Most devices canʼt play the formats weʼre accustomed to on PCs
32
Video on mobile devices
Most devices canʼt play the formats weʼre accustomed to on PCs3GP is widely supported
32
Video on mobile devices
Most devices canʼt play the formats weʼre accustomed to on PCs3GP is widely supportedMPEG-4 on newer devices
32
Optimizing video delivery
3321
Optimizing video delivery
3321
User visits site
Optimizing video delivery
3321
User visits site
Device supports MPEG-4 video?
Optimizing video delivery
3321
User visits site
Device supports MPEG-4 video?
No
Serve3GP
Optimizing video delivery
3321
User visits site
Device supports MPEG-4 video?
YesNo
Serve3GP
ServeMPEG-4
Optimizing video delivery
3321
User visits site
Device supports MPEG-4 video?
YesNo
Serve3GP
ServeMPEG-4
Max video resolution?
Mobile Video Sites
34
Mobile Video Sites
34
Mobile Video Sites
34
Mobile Video Sites
34
Mobile Video Sites
34
Optimizing for mobile
35
Optimizing for mobile
Lower resolution176x144, 320x240
35
Optimizing for mobile
Lower resolution176x144, 320x240
Reduce video frameratePlayback looks jerkyBut it requires less bandwidth!
35
Optimizing for mobile
Lower resolution176x144, 320x240
Reduce video frameratePlayback looks jerkyBut it requires less bandwidth!
Reduce bitrates35
Tools
36
Tools
FFmpeg, Mencoder
36
Tools
FFmpeg, MencoderOn2 Flix Engine for Flash (commercial)
36
Tools
FFmpeg, MencoderOn2 Flix Engine for Flash (commercial)Anystream Agility
36
Tools
FFmpeg, MencoderOn2 Flix Engine for Flash (commercial)Anystream AgilityCommercial tools to read Windows Media, other DRMʼd formats
36
Expense
37
Expense
Generating videos is not cheap
37
Expense
Generating videos is not cheapShould not be done in real time
End user will have to wait for it!
37
Expense
Generating videos is not cheapShould not be done in real time
End user will have to wait for it!
Pre-generate with an offline processStore them on diskPurge when and if needed
37
Expense
Generating videos is not cheapShould not be done in real time
End user will have to wait for it!
Pre-generate with an offline processStore them on diskPurge when and if needed
Weʼll look at FFmpeg shortly!
37
Audio
38
Audio on mobile devices
39
Audio on mobile devices
Devices have been playing MP3s for years now
39
Audio on mobile devices
Devices have been playing MP3s for years nowWider support than video
39
Audio on mobile devices
Devices have been playing MP3s for years nowWider support than videoMP3, AAC formats are well supported
39
Optimizing audio delivery
402921
Optimizing audio delivery
402921
User visits site
Optimizing audio delivery
402921
User visits site
AAC audio
supported?
Optimizing audio delivery
402921
User visits site
AAC audio
supported?
No
ServeMP3
Optimizing audio delivery
402921
User visits site
AAC audio
supported?
YesNo
ServeMP3
ServeAAC
Tools
41
Tools
FFmpeg
41
Tools
FFmpegTiMidity++ (MIDI to PCM)
41
Tools
FFmpegTiMidity++ (MIDI to PCM)Note:
Iʼm not dealing with MP3 licensing issues here!
41
Expense
42
Expense
Again, generating audio isnʼt cheap
42
Expense
Again, generating audio isnʼt cheapShouldnʼt transcode in real time
42
Expense
Again, generating audio isnʼt cheapShouldnʼt transcode in real timePre-generate in an offline process
42
Expense
Again, generating audio isnʼt cheapShouldnʼt transcode in real timePre-generate in an offline process
42
Expense
Again, generating audio isnʼt cheapShouldnʼt transcode in real timePre-generate in an offline process
What can we use?Surprise! FFmpeg! 42
What’s this FFmpeg stuff?
43
Basics
44
Basics
Decodes and encodes audio and video streams
44
Basics
Decodes and encodes audio and video streamsMany A/V formats supported
44
Basics
Decodes and encodes audio and video streamsMany A/V formats supportedSupports use of external libraries to handle additional formats
44
Basics
Decodes and encodes audio and video streamsMany A/V formats supportedSupports use of external libraries to handle additional formatsRuns on many platforms (Windows, OS X, Linux)
44
Usage
ffmpeg [[infile options] -i infile]... {[outfile options] outfile}...
45
Usage
ffmpeg [[infile options] -i infile]... {[outfile options] outfile}...
45ffmpeg -h: 504 lines!Extensive amount of options for both audio and video
Usage
ffmpeg [[infile options] -i infile]... {[outfile options] outfile}...
45ffmpeg -h: 504 lines!Extensive amount of options for both audio and video
FFmpeg: Video
46
47
Notable video encoding options
47
Notable video encoding options
-acodec: audio codec to use
47
Notable video encoding options
-acodec: audio codec to use
-vcodec: video codec to use
47
Notable video encoding options
-acodec: audio codec to use
-vcodec: video codec to use
-s: frame size
47
Notable video encoding options
-acodec: audio codec to use
-vcodec: video codec to use
-s: frame size
-r: video framerate
47
Notable video encoding options
-acodec: audio codec to use
-vcodec: video codec to use
-s: frame size
-r: video framerate
-b: bitrate
47
Notable video encoding options
-acodec: audio codec to use
-vcodec: video codec to use
-s: frame size
-r: video framerate
-b: bitrateffmpeg -i rr.mp4 \ -acodec libfaac \ -vcodec h263 \ -s 176x144 \ -r 12 \ foo.3gp
Examples
48
Examples
48
Examples
320x240MPEG-4, AAC
30 fps244kbps6.2 MB
48
Examples
320x240MPEG-4, AAC
30 fps244kbps6.2 MB
176x1443GP, AAC
30 fps90kbps2.3 MB
48
Examples
320x240MPEG-4, AAC
30 fps244kbps6.2 MB
176x1443GP, AAC
30 fps90kbps2.3 MB
176x1443GP, AAC
8 fps63kbps1.6 MB
48
Grab screenshots!
$ ffmpeg \ -i rr.mp4 \ -f image2 \ frame%2d.jpg 49
50
50
FFmpeg: Audio
51
52
Notable audio encoding options
52
Notable audio encoding options
-acodec: audio codec to use
52
Notable audio encoding options
-acodec: audio codec to use
-ar: audio sampling rate
52
Notable audio encoding options
-acodec: audio codec to use
-ar: audio sampling rate
-ab: audio bitrate
52
Notable audio encoding options
-acodec: audio codec to use
-ar: audio sampling rate
-ab: audio bitrate
-ac: number of channels
52
Notable audio encoding options
-acodec: audio codec to use
-ar: audio sampling rate
-ab: audio bitrate
-ac: number of channels
ffmpeg -i iib.mp3 \ -ab 64000 \ -ac 1 \ foo.mp3
Examples
53
Examples
53
Original:128kbps4.1MB
Examples
53
Original:128kbps4.1MB
Second Clip:64kbps
2.05MB (1/2)
Examples
53
Original:128kbps4.1MB
Second Clip:64kbps
2.05MB (1/2)
Third Clip:32kbps
1.025MB (1/4)
Examples
53
Original:128kbps4.1MB
Second Clip:64kbps
2.05MB (1/2)
Third Clip:32kbps
1.025MB (1/4)
Final Clip:8kbps
1.025MB (???)
FFmpeg Summary
54
FFmpeg Summary
54
Powerful tool!
FFmpeg Summary
54
Powerful tool!Takes a lot to master it
Probably makes more sense to A/V experts
FFmpeg Summary
54
Powerful tool!Takes a lot to master it
Probably makes more sense to A/V experts
But itʼs easy enough to get the job done right
FFmpeg Summary
54
Powerful tool!Takes a lot to master it
Probably makes more sense to A/V experts
But itʼs easy enough to get the job done rightIts manpage is your friend!
In Closing...
55
Mobile is tricky!
56
Mobile is tricky!
All of those combinationsDevicesProvidersBandwidth / data plansCapabilities
56
Mobile is tricky!
All of those combinationsDevicesProvidersBandwidth / data plansCapabilities
Use resources efficiently56
Mobile is tricky!
All of those combinationsDevicesProvidersBandwidth / data plansCapabilities
Use resources efficientlyOptimize the experience for the end user 56
Resources
57
Resources
WURFL: http://wurfl.sourceforge.net/
57
Resources
WURFL: http://wurfl.sourceforge.net/FFmpeg: http://ffmpeg.mplayerhq.hu/
57
Resources
WURFL: http://wurfl.sourceforge.net/FFmpeg: http://ffmpeg.mplayerhq.hu/ImageMagick: http://www.imagemagick.org/
PECL extension: http://pecl.php.net/package/imagick/
57
For more information, please see me or visithttp://www.schematic.com/#/JoinUs/
It goes without saying: Schematic is only as good as the people who work here. That’s why we’re so particular about recruiting, training, nurturing, and retaining the very best people in our field.
If you have digital expertise (technical, creative, managerial–or something else entirely), enthusiasm, curiosity, and the ability to collaborate with others, we’d love to hear from you.
We’re Hiring!
58