1

SHAPE WRITING TECHNOLOGY

2

CONTENT Introduction Basic Concept of Shape Writing Information and Constraints User Interface Multi channel Architecture Installation and Configuration Features of Shape Writer Tips for using Shape Writer Conclusion Reference

3

INTRODUCTION

Shape Writing is a new form of writing that use pen/finger stroke on graphical keyboard to write text.

Invented by Shumin Zhai and Per Ola Kristensson.

Previously called Shorthand Aided Rapid Keyboarding (SHARK).

4

BASIC CONCEPT In a shape writer to enter a word user slides the pen over the

entire keys of the word on the graphical keyboard . Ideal Trace is called sokgraph (Shorthand On Keyboard as a

Graph).

•When pen strokes terminate intended word will display.

5

INFORMATION AND CONSTRAINTS The shape writing recognizer decodes the message from

the code + noise and output the message. Lexicon is the Critical aspect of shape writing

For enter a word “writing” pen crosses over the letters w-r-t-o-s-i-s-e-t-e-s-i-n-g.

6

USER INTERFACE

Good user interface must

a) remove complexity

b) prevent errors

c) minimize user’s effort to correct the errors.

7

KEYBOARD DESIGN

QWERTY keyboard is designed to prevent mechanical jamming.

ATOMIK (Alphabetically tuned and Optimized Mobile Interface Keyboard) used in shape writer

3 features of ATOMIK Keyboard

1) Higher movement efficiency.

2) Alphabetically tuned

3) The letter connectivity of most common word

8

Cont..

Shape Writer on ATOMIC

Shape Writer on QWERTY

9

ERROR CORRECTION

The place where entered words displayed is called edit buffer.

Efficient error correction can be improved when words rather than characters should be corrected.

Type of error correction

* Correcting a confusion error.

* Correcting an out-of-vocabulary error.

10

Cont..CORRECTING A CONFUSION ERROR

•Word are written on word by word.•Correcting can be done using pen gesture function •Two methods to replace a word

1) Select from a list of alternative word. 2) Holding the pen down on the word.

11

Cont..CORRECTING A OUT-OF VOCABULARY ERROR

User can explicitly add the word into the lexicon.

A tapped word not in the lexicon is marked with a box.

and the user can add new word to the lexicon by clicking on it

12

FEEDBACKDISPLAY OF RECOGNIZED WORD AND REINFORCEMENT OF THE IDEAL SHAPE

Recognized word is displayed at the point of pen up. Recognized word and trace will display for 600ms

13

Cont.. MINIMIZING THE PEN TRACE

Clutter displaying the ink of the pen as the user articulates a pen gesture have to advantages

1) It gives the user information that the pen motion is still recorded.

2) It provides the sense of orientation to the user on where the pen has traveled on the keyboard.

14

Cont..

MORPHING VISUALIZATIONMorphing algorithm

Used to find part of the input gesture that matches sokgraph of the recognized wordWORKING 1) Sample the traces. 2) Sample points at the same position of both trace are joined with imaginary lines

15

Cont..

3) Area between two imaginary lines are painted with translucent blue color.

4) User’s trace transformed into ideal shape

16

Users pen trace for the word “computer” is gradually morphing to ideal shape

17

Two core channels are 1) Shape channel 2) Location channel

MULTICHANNEL ARCHITECURE

18

TEMPLATE PRUNING

It filters out a large number of sokgraph templates entering into the later stage.

It compute the start to start and end to end distance between a sokgraph and a normalized input gesture.

If the two distance are greater than threshold then template will discarded.

19

SHAPE RECOGNITION CHANNEL Classifies a pen gesture according to the normalized shape of the pen

gesture

In early shape writer’s elastic matching also considered .

Proportional shape matcher: The Proportional shape matching distance between an unknown pattern and template pattern is defined as

u-unknown patternt-template patternN-no:of sample point

20

Cont..

Pattern must be resample and normalized Scale factor is

L-predetermined lengthW-widthH-height

• Output is the distance measure of the similarity between the pattern

21

LOCATION RECOGNITION CHANNEL

Classifiers the pen gesture according to the location of the gesture on the keyboard.

Reason to use location channel

1) Increase sokgraph recognition power for reducing shape confusion pairs.

2) Location is a part of users memory of sokgraph and therefore will be reproduced during gesture production.

22

Cont..

Analysis of the confusion pairs in 20K lexicon on QWERTY and ATOMIC keyboard

Location Memory• User’s can easily remember the gestures with shape particularly the beginning and the end position.

Shape Confusion :By sokgraph shape alone some words may be near or in complete confusion with each other.

23

Cont..

Location Channel Algorithm It computes the distance of the unknown input trace u to

the ideal trace t of the word w on keyboard. Location Channel Distance

N-number of equidistant points

24

Cont.. δ can defined as

r-the radius of an alphabetical key

25

Cont..

Score is the sum of the spatial point to point distance Score will be zero if the given input trace is within the

tunnel of t. α (i) , i € (1,N) are weights for different point to point

distances . Lowest weight at middle point. Weight increase linearly towards the end.

26

CHANNEL INTEGRATION

It select exact template from the templates produced by the two channels.

Distance follows Gaussian distribution. x-distance between input gesture and template y Probability of y be the exact word can be calculated

μ = 0σ – key radius

Template with x>2σ have zero probability.

27

Cont..

Probability of word w € W with distance x be the word is

• Integrate probabilities from two channel and obtain confidence score

p’S(w) & p’L(w) –probability score from shape and location channelWS & WL word sets passed pruning stage

28

N-BEST LIST OUTPUT

It contain the list of multiple candidates for each user input, ranked by confidence.

The architecture cannot completely remove the ambiguity in recognition system

User can access next best choice from the list. Alternative words displayed in popup menu.

29

INSTALLATION

Install Sun Java Runtime version 5 or above.

Set the environment variable properly

Install Shape Writer using shark.exe

30

CONFIGURATION

General Tab Keyboard Tab Animated Morphs Phantom Keyboard Lexicon Tab Recognition Tab

31

FEATURES

Efficiency Human sensitivity to geometric pattern Intelligent pattern recognition Ease of learning Ease of error correction

32

TIPS TO USE A SHAPE WRITER

Shape writing a word. Changing Capitalization

iPhone, iphone, IPHONE,Iphone Writing word that contain double letter

for “look” trace”l-o-k” CMD-Key

CMD replay

33

CONCLUSION

Shape writer recognizes continues shorthand gesture and features a smooth learning curve for users

After some usage users can quickly enter the text,wihtout lookin much into the keyboard.

This technology is used in new IPHONE’s

34

REFERENCE ZHAI, S. AND KRISTENSSON, P.O. 2007. Introduction to

shape writing. KRISTENSSON, P.O. 2004. Large Vocabulary Shorthand

Writing on Stylus Keyboard. Per Ola Kristensson: Discrete and Continuous Shape Writing,

for Text entry and control KRISTENSSON, P.O. AND ZHAI, S. 2004. SHARK2: a large

vocabulary shorthand writing system for pen based computers ZHAI, S. AND KRISTENSSON, P.O. 2003. Shorthand

writing on stylus keyboard. In Proceedings of the ACM Conference on Human Factors in Computing Systems

35

THANK YOU