Creating Responsive HTML5 Touch InterfacesStephen Woods

Sunday, March 11, 12

Stephen WoodsFront End Engineer

Flickr

Sunday, March 11, 12

Sunday, March 11, 12

On the desktop we worry about browsers-moz-transform:rotate(-270deg); -moz-transform-origin: bottom left;-webkit-transform: rotate(-270deg);-webkit-transform-origin: bottom left;-o-transform: rotate(-270deg);-o-transform-origin: bottom left;filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=1);

Sunday, March 11, 12

On mobile we worry about devices.

Sunday, March 11, 12

Wait!Did you say they all

run webkit?

Sunday, March 11, 12

Wait!Did you say they all

run webkit?

Sunday, March 11, 12

Wait!Did you say they all

run webkit?

XSunday, March 11, 12

On mobile we worry about devices.

Sunday, March 11, 12

Media Queries, Break points, liquid layouts

Screen Sizes

http://www.alistapart.com/articles/responsive-web-design/Sunday, March 11, 12

Sunday, March 11, 12

iPhone 3GS256mb RAM

Geekbench: 271

Sunday, March 11, 12

iPhone 3GS256mb RAM

Geekbench: 271

==

Sunday, March 11, 12

Modern mobile devices are crappy

computers with decent video cards.

Sunday, March 11, 12

Sunday, March 11, 12

Perceived Performance

Sunday, March 11, 12

On the desktop it’s easy...

Throw up a spinner.Sunday, March 11, 12

Touch interfaces are tactile.

Sunday, March 11, 12

Touch interfaces are tactile.

Feedback must be continuous.

Sunday, March 11, 12

When the interface stops moving

during a gesture it feels like it died

Sunday, March 11, 12

Respect Convention

Sunday, March 11, 12

Mobile has conventions too

Sunday, March 11, 12

Mobile has conventions too

Sunday, March 11, 12

TouchEvent

• touchstart - fires once

• touchmove - fires continuously

• touchend - fires once

Sunday, March 11, 12

The touches Array

• You only get one on Android

• You get up to 11 on iOS

• Each touch gives you position information, and sometimes scale

Sunday, March 11, 12

iOS Gesture Events

• gesturestart

• gesturechange

• gestureend

Sunday, March 11, 12

iOS Developer Library

http://bit.ly/iOS-guide

Sunday, March 11, 12

Making Gestures Work

• Prioritize user feedback

•Use hardware acceleration

•Manage your memory

Sunday, March 11, 12

Prioritize User-feedback

•Don’t do any loading during gestures

• Treat the DOM as write-only (do your own math)

•When at all possible, use css transitions

Sunday, March 11, 12

Write-Only DOM

•DOM touches are really expensive

• You know where everything is

•Use matrix transforms to queue up positions

Sunday, March 11, 12

Swipe Basics

distance = e.touches[0].pageX - startX;

'translate3d('+distance+'px,0px,0px)'

Sunday, March 11, 12

Snap back/snap forward

• Keep track of last position, use transitions with easing to snap back

• Pick a swipe distance threshold, use that to snap forward (ontouchend)

• If the user is gesturing, the element must be moving

Sunday, March 11, 12

A Word about scrolling

•Use native if at all possible:

•-webkit-overflow-scrolling: touch;

• If not, use a library to simulate momentum scroll (iScroll 4, Scrollability)

Sunday, March 11, 12

Avoid Event Abstraction

Sunday, March 11, 12

Pinch to Zoom(there will be math)

Image © Brian Lalor Used with permission

Sunday, March 11, 12

Why you can’t use native Pinch to

Zoom

Sunday, March 11, 12

First:Use Matrix TransformsMinimize DOM touches, make your transforms simpler in the long run

Sunday, March 11, 12

http://xkcd.com/184/

Sunday, March 11, 12

It’s Not That Hard!

transform:matrix(1, 0, 0, 1, 10, 10);

Translate

Scale

Sunday, March 11, 12

[ [1,0,0,0], [0,1,0,0], [0,0,1,0], [tx,ty,tz,1]]

With Hardware Acceleration... (matrix3d)

Sunday, March 11, 12

Transforms keep complex state

without DOM reads

Sunday, March 11, 12

What is happening?

•Determine Center of the touch points

•Determine the scale factor (touch.scale)

• Scale the element by the scale factor, with the center of the touch points as the scale center

Sunday, March 11, 12

The Naive Example

Sunday, March 11, 12

The Naive Example

Sunday, March 11, 12

The Naive Example

Sunday, March 11, 12

The Right Example

Sunday, March 11, 12

The Right Example

Sunday, March 11, 12

The Right Example

Sunday, March 11, 12

Breakdown

Sunday, March 11, 12

Breakdown

Sunday, March 11, 12

Breakdown

Sunday, March 11, 12

Breakdown

Sunday, March 11, 12

translateX = scalePointX * (newWidth - oldWidth)

newWidth;

Sunday, March 11, 12

Pro Tips

• Beware the virtual pixels

•Moving the transform-origin won’t really work

•Remember to snap back

Sunday, March 11, 12

Dealing with browsers

Sunday, March 11, 12

Dealing with browsers

Sunday, March 11, 12

Remember Progressive Enhancement?

Sunday, March 11, 12

• Feature Detect

• Add transitions, don’t depend on them

•Gesture interaction is an enhancement, clicks should still work

• Be able to disable features per user-agent, if necessary

Progressive Enhancement

Sunday, March 11, 12

The Tool Chain

Sunday, March 11, 12

The dumbest thing that works

Webkit Browser with UA Spoofing

Sunday, March 11, 12

Weinre

Remote webkit debugger.

http://phonegap.github.com/weinre/

Sunday, March 11, 12

Internet Sharing + Charles Proxy

•Watch HTTP traffic

• Add breakpoints in ajax requests

• Serve web pages to your phone from your computer

Sunday, March 11, 12

Adobe Shadow

•Wrapper for Wienre

•Neat

•Has some limitations

Sunday, March 11, 12

Pile of Devices

Sunday, March 11, 12

Pile of Devices• iPad 1

• iPhone 3G

• iPhone 4

• Samsung Galaxy S

•HTC Desire

•Galaxy Tab

•Motorola Xoom

•Kindle Fire

•HTC Titan

Sunday, March 11, 12

Device Simulators & Emulators:

Basically Useless.

Sunday, March 11, 12

The Flickr Touch Light Box

Sunday, March 11, 12

UntitledBy protohiro

Sunday, March 11, 12

UntitledBy protohiro

Sunday, March 11, 12

UntitledBy protohiro

Sunday, March 11, 12

UntitledBy protohiro

Sunday, March 11, 12

Swiping Process

• Event Listener on top level for touch events

•Only visible nodes move via translate3d

•Rebuild next/previous happens when movement stops.

Sunday, March 11, 12

Performance Tricks

• Aggressive Pruning

•Clean off css transforms/transitions

•Write-only DOM.

•Do as little as possible during swipes

Sunday, March 11, 12

Frustrating Limitations

•Retina screen is huge, device memory is small

•Hardware acceleration is a crash festival

• Fighting automatic optimization

http://bit.ly/apple-image-size-restrictionsSunday, March 11, 12

http://www.flickr.com/photos/wafer/5533140316/http://www.flickr.com/photos/latca/2265637876/http://www.flickr.com/photos/spine/1471217194/http://www.flickr.com/photos/williamhook/3656233025/http://www.flickr.com/photos/isriya/4656385586/http://www.flickr.com/photos/yandle/3076451873/http://www.flickr.com/photos/uberculture/6632437677/http://www.flickr.com/photos/blalor/4934146981/http://www.flickr.com/photos/torek/3280152297/http://www.flickr.com/photos/nilsrinaldi/5157809941/

Stephen Woods @ysaw

Image Credits (http://flic.kr/y/kQ5cLh)

http://www.slideshare.net/ysaw/creating-responsive-html5-touch-interfaces

Sunday, March 11, 12