PuReWidgets toolkit

The PuReWidgets toolkit for interac6ve public display

applica6ons

Jorge C. S. Cardoso [email protected]

Ubicomp@UMinho

May 30, 2012

1

Content

•  Mo6va6on •  PuReWidgets features •  Architecture •  Applica6on models •  Programming library overview •  HelloWorld

2

Mo6va6on Complexity of incorpora6ng interac6ve features in public display applica6ons. Too many things to answer when designing the interac6on for a public display applica6on: •  What controls can we use to offer interac6ve features to users?

–  No standard interac6ve controls for public displays •  What interac6on mechanisms can be used to provide those features?

–  Various possible interac6on mechanisms (SMS, BT naming, OBEX, Email, IM, mobile app, desktop, QR codes, touch, etc.)

–  Different loca6ons may have different interac6on resources •  How will those features work?

–  Desktop interac6on paradigm does not generally apply –  Applica6ons may not always be visible

•  How can the applica6on react to different users? –  The interac6on environment is mul6-‐user –  We need to iden6fy/differen6ate users

•  How can users tell what they can do with the applica6on? –  Where are the features presented?

3

Mo6va6on We can solve all those ques6ons for ad hoc applica6ons, but

•  This means wasted effort in developing applica6ons –  Programmers must deal with issues extraneous to the main applica6on logic

•  Will certainly lead to inconsistencies in the interac6on model –  Each solu6on will behave slightly differently –  Will confuse users

We need a programming toolkit for interac6ve public display applica6ons!

4

Toolkit

•  The toolkit should provide a consistent model for user interac6on and applica6on development – But s6ll allow some diversity in solu6ons

5

PuReWidgets

•  PuReWidgets tries to solve these problems. •  To make sure it covers a wide range of applica6ons, we looked at various interac6ve public display systems to learn – General requirements – Types of controls – Types of input mechanisms

6

PuReWidgets Widget-‐based toolkit.

•  A widget represents an interac6ve feature. •  Is represented by class in an object-‐oriented programming model. •  Applica6ons instan6ate widgets, define a callback, and receive

interac6on event through the callback •  Main features

–  Mul6ple, extensible, controls –  Independence of input mechanisms and modali6es –  Automa6cally generated graphical interfaces –  Asynchronous interac6on –  Concurrent interac6on –  Graphical affordances –  Server & client libraries

7

Features Mul6ple, extensible, controls. •  Allows developers to create rich interac6ve applica6ons. Controls can be extended with custom

func6onality. •  We studied various interac6ve display systems to define a set of basic interac6ve controls, common to the

majority of applica6ons:

•  Ac6on bugon –  Trigger an ac6on in the applica6on, e.g., play a video

•  Op6on selec6on –  Selects among a set of op6ons, e.g., to vote

•  Text entry –  Sends text to the applica6on, e.g., a comment, tag, search keyword

•  Download –  Receives a media file from the applica6on, e.g., download poster

•  Upload –  Uploads a media file to the applica6on, e.g., a photo to be displayed

•  Check-‐in –  Says “I’m here”

8

Features

Independence of input mechanisms and modali6es. •  Allows developers to focus on the core applica6on func6onality and not on the low-‐level interac6on details.

•  Allows applica6ons to work in heterogeneous loca6ons, with different mechanism

•  Interac6on can be accomplished with various mechanisms (SMS, Bluetooth naming, QR codes, and automa6cally generated graphical interfaces).

9

Features

Automa6cally generated graphical interfaces. •  Provide a richer interac6on for desktop and mobile devices.

•  PuReWidgets generates web-‐based interfaces for each applica6on.

•  It also generates QR codes for each applica6on's feature

10

Features

Asynchronous interac6on. •  Allows applica6ons to receive input events that were generated when the applica6on was not listening.

•  PuReWidgets provides a persistent input queue for each applica6on

•  Applica6ons can request past input at any 6me

11

Features

Concurrent interac6on. •  Allows various users to interact at the same 6me, while providing applica6ons with iden6ty informa6on that allows them to differen6ate users.

•  Every input event carries a user id (if available) – Where possible, anonymous interac6ons are s6ll “iden6fied” (more on this later)

12

Features Graphical affordances. •  Allows users to immediately recognize interac6ve features and input feedback.

•  Controls have an op6onal graphical representa6on that can be used on the public display interface. – Applica6ons may not show the features on the public display, but they will s6ll be available

•  Input feedback is also (op6onally) displayed on the public display.

13

Features

Server & client libraries. •  Programmers can choose to use just the server, just the client, or both libraries together.

•  Allows developers to choose the best applica6on model.

14

PuReWidgets Architecture

15

Interac6on services •  PuReWidgets service –  Keeps informa6on about every widget –  Generates graphical interfaces for desktop, mobile, touch plaporms •  These allow anonymous interac6ons, but they generate persistent cookie-‐based random ids.

–  Generates QR codes for every widget –  Generates unique textural reference codes for data-‐based interac6ons (SMS, BT naming, OBEX, etc.)

•  IO infrastructure –  (from Instant places) –  Provides low-‐level data-‐based interac6on

16

PuReWidgets library

•  Provides an object-‐oriented library of widgets •  Hides the communica6on details with the PuReWidgets service

•  Provides other features, unrelated to interac6on, such as – Applica6on “place-‐aware” storage

•  PuReWidgets provides a name-‐value server datastore for easy storage of place-‐specific seqngs/state

–  Skeleton admin interface for place-‐owners •  To configure place-‐specific seqngs

17

Local applica6on model

18

Local applica6on model

•  Applica6on logic on the public display (player) – Subject to the schedule 6mings

•  May not be able to react (or update data structures) immediately when user interacts – But will eventually (when it is displayed again)

•  When displayed, the toolkit periodically asks the service for input events – And triggers the appropriate widget event

19

Remote applica6on model

20

Remote applica6on model

•  Applica6on logic on the server •  Can react (or update data structures) immediately to user input – Even if not on the public display, the reac6on can be visible on other channels

•  The toolkit periodically asks the service for input events – And triggers the appropriate widget event

21

PuReWidgets Implementa6on

•  Google Appengine (server) •  Google Web Toolkit – GWT (client)

•  Overview of the library

22

Using PuReWidgets

•  Hello World applica6on (local model) •  Video

23

Ini6al Evalua6on/Development process

•  Two applica6ons developed so far (ignore the “graphic design”, please) –  Youtube player, media intensive, complex graphical components (several panels that change over 6me), local applica6on model

–  Everybody votes, based on display owner created content, simpler graphical component, combines remote and local applica6on models

24

Ini6al Evalua6on/Development process

•  Con6nuous refinement – Develop interac6ve public display applica6ons – Gain more insight about the difficul6es – Refine the toolkit to address the iden6fied problems

– Refactor the applica6ons to include the toolkit changes

25

Summary

•  PuReWidgets facilitates applica6on development – Provides ready-‐to-‐use interac6ve controls –  Integrates various input mechanisms via an I/O infrastructure

– Generates applica6on interfaces for desktop, mobile, and QR codes

– Provides client and server applica6on development models

26

The end

27

Technology

PuReWidgets toolkit