Upload
lynn-carroll
View
228
Download
4
Tags:
Embed Size (px)
Citation preview
Agenda
Microsoft Surface and the future of computing
How you can be part of this revolution How the Surface SDK enables
rapid innovation Microsoft’s multi-touch roadmap Questions and Answer
Microsoft Surface computing uses sensing and display technology to
infuse everyday surfaces with digital content and is comprised of the
following unique attributes:
Microsoft Surface And NUI
Direct interaction Object recognitionMulti-touch Multi-user
Direct interaction Object recognitionMulti-touch Multi-user
How Surface Is Being Used Today
5 Verticals • Automotive
• Financial Services• Healthcare• Leisure and Entertainment• Retail
Developing ForMicrosoft Surface
Robert LevyProgram ManagerSurface SDK
Doug KramerLead DeveloperSurface SDK
Microsoft Surface Hardware & Drivers
Surface v1 Architecture
Windows Vista SP1
Vision System Windows Integration
Core API Shell UI & APIs
Surface Applications
WPF APIs
Surface And WPF
Expose Surface capabilities in a way consistent with WPF Infrastructure integration for consistent behavior
with mouse Automatic hit testing to find the control being touched Sets attached properties on controls
(IsContactOver, etc.) Raises routed events on controls (ContactDown, etc.)
Custom versions of most built-in WPF controls Custom controls for Surface-specific UX Base classes for building custom Surface controls
XAML <s:SurfaceWindow> <Canvas
s:Contacts.ContactDown=“OnContactDown“> <s:SurfaceButton Click=“OnButtonClick”/> </Canvas> </s:SurfaceWindow>
Code void OnContactDown(object sender,
ContactEventArgs e) { // ... } void OnButtonClick(object sender,
RoutedEventArgs e) { // ... }
Surface API Usage With WPF
XAML <Window> <Canvas Mouse.MouseDown=“OnMouseDown“> <Button Click=“OnButtonClick”/> </Canvas> </Window>
Code void OnMouseDown(object sender,
MouseEventArgs e) { // ... } void OnButtonClick(object sender,
RoutedEventArgs e) { // ... }
Surface Classes For WPF Input
MouseStylus
MouseDeviceStylusDevice
MouseEventArgsStylusEventArgs
Tablet
TabletDevice
Surface Versions Of Common Controls
SurfaceWindowFull screen on the SurfaceOriented towards the user
SurfaceMenuSurfaceContextMenu
SurfaceMenuItemMultiple menus can be used at
the same time
SurfaceButtonSurfaceRadioButton
SurfaceCheckBoxSurfaceToggleButton
Only “click” when all contacts are removed
Surface Versions Of Common ControlsSurfaceListBox
SurfaceScrollViewer• Determines scrolling vs. select
• Allows flicking• Provides “elastic” effect
SurfaceSliderSurfaceScrollBar• Allows flicking
• Track input results in smooth animation and capture
• Subtle UX when not in use
SurfaceTextBoxSurfacePasswordBox
• Invokes virtual keyboard oriented towards user when
touched
Neat, But Does It Justify Buying A Surface?
We’ve made it easy to port WPF and XNA apps to Surface
Great for developers Re-use code, skills, and tools Get started quickly
…but little value to users
Is this really just a big PC?
Reading upside down makes my
head hurt
Constantly reaching across the table is
not fun
I don’t like having to share and wait
my turn
Making Surface Shine
Great apps leverage all of what makes Microsoft Surface unique Multi-touch Multi-user 360° UI Gestures and manipulations Interaction with physical objects
WPF and the Surface SDK make this easy
Introducing The ScatterView Control
Quickly enables 360°, multi-user apps Gives users direct, shared control of layout Standardizes manipulations for Resize,
Move, and Rotate Same familiar programming model
as ItemsControls, ListBox, TreeView, Menu, etc.
Recognizing Tagged Objects
Contact.Tag.Byte256 unique values
Contact.Tag.Identity>340,282,366,920,938,000,000,000,000,000,000,000,000
unique values
Byte Tags
Identity Tags
Introducing The TagVisualizer Control
Simplifies common scenarios involving tagged objects Displays interactive UI under, around, or next to objects Keeps UI in sync with movements Handles temporary lifting
TagVisualizer Control
Tag values to look for
UI for each object
Positioning of UI relative to tag
Max # of each object allowed
How to handle tag removal
Is this really just a big PC?
Reading upside down makes my
head hurt
Constantly reaching across the table is
not fun
I don’t like having to share and wait
my turn
Controls For Making Surface Shine
ScatterView Multi-touch Multi-user 360° UI Natural interactions
with virtual objects
TagVisualizer Object recognition Natural
interactions with physical objects
Surface Hardware
Windows 7
Surface Hardware
Windows VistaWindows 7
NativeWin32
Application
Touch Development Roadmap
WPF 3.5
Surface SDK 1.0
Managed Wrapper and Interop
WinForms Application
Windows 7 ReleaseNET 4.0 / Surface 2.0 Release
WPF 3.5 x WPF 4.0
Surface SDK 2.0
WPF Application
Multi-Touch API
Surface Application
Multi-Touch Controls
Multi-Touch API
Surface Multi-Touch
Controls & API
Multi-Touch API
Multi-Touch API and Controls
NET 3.5 / Surface 1.0 Release
Summary
Microsoft Surface is shaping the future Multi-touch, multi-user, 360° UI, object recognition, …
UI design requires creativity and innovation Coding builds upon what you already know Microsoft is creating a unified API for touch
Call To Action
Help us change the world Create apps using Surface’s unique capabilities
Get Microsoft Surface developer hardware Order forms on http://surface.com $1,500 discount until November 15th
Swipe your badge to receive an e-mail invitation to join http://community.surface.com Download the Microsoft Surface SDK & Simulator Participate in online forums and training
Related Content
Hands on Labs PCHOL19: Using the Microsoft Surface Simulator PCHOL20: A Lap Around the Microsoft Surface SDK Surface HOL: Exploring Vision Based Object
Recognition with Microsoft Surface Demo applications from partners Scavenger Hunt game PC03: Developing multi-touch applications
with Windows 7 Wednesday, 1:15
Evals & Recordings
Please fill
out your
evaluation for
this session at:
This session will be available as a recording at:
www.microsoftpdc.com
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market
conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Simulator for Off-Table Dev & Test
Replicates input With multiple mice With one mouse
1. Click to place multiple contacts2. Click and drag to group them together3. Move, rotate, and stretch the group
Supports all types of contacts Finger, resizable blobs, and tags
Supports input automation for testing Recording and playback UI
Simulator APIs 37
Manipulation & Inertia Processor APIs Low level APIs for building controls with consistent
behavior Reduces the cost of building new controls Increases consistency for end users
Interpreting Surface input as 2D transformations Input: list of contacts to track Output: scale, rotation, movement, velocity data (~60 FPS) Used by: ScatterView, ScrollViewer, Slider, Concierge map
Two modes of inertia Initial velocity + deceleration rate
Realistic Used by ScatterView and Concierge map
Initial velocity + final position/orientation Goal-based Used by ScrollViewer and Slider
Up & Down Event Comparison
Similarities: Naming convention & meaning Routing strategy
Tunneling (“preview”) events & bubbling events Events stop once marked as handled Hit testing & capture logic
Differences: Need to find out which contact the event is for New Contact object is created with each Down
event
40
Movements
41
Mouse.MouseMove … Mouse.MouseMove …
Contacts.ContactChanged … Contacts.ContactChanged …
MouseMoved == ContactChanged?
ContactChanged includes: Position Orientation Recognized type Size Shape Area
42
Wait a sec… he said this was supposed to be the same as mouse. Why are these events named differently?
Movement Event Comparison
Similarities: Routing strategy
Tunneling (“preview”) events & bubbling events Events stop once marked as handled Hit testing & capture logic
Event for changes in position Differences:
ContactChanged is not just for movements Change events only happen between Down & Up Need to find out which contact the event is for
43
Position
Contact.GetPosition(UIElement)
Relative to specified UI element GetPosition(window) = {50, 100} GetPosition(button) = {10, 10}
To track position changes across multiplecontrols, get position relative to a common parent
44
Orientation
Contact.GetOrientation(UIElement)
Degrees clockwise from relative east
Range and usefulness varies Tagged objects: 360° Generic blobs: 180°
Fingers: 360° but…unreliable when straight down
45
0°
90°
180°
270°
Ellipse & Bounds
Rect Contact.BoundingRect
Ellipse Contact.GetEllipse(UIElement)
void Contact.UpdateEllipse(Ellipse, UIElement)
Ellipse is returned with transforms that counteract transforms on the specified element Adding the shape as a child element results in UI
matching the physical object Bounding rectangle is always screen-aligned UpdateEllipse method can improve performance
46