Example of Historical Research Palm v. Xerox Patent …csis.pace.edu/~ctappert/dps/d891a-09/palm-xerox.pdf · Example of Historical Research Palm v. Xerox Patent Infringement Lawsuit

1

Example of Historical ResearchPalm v. Xerox

Patent Infringement Lawsuit

Charles C. TappertSeidenberg School of CSIS, Pace University

2

Goldberg Patent (Xerox), filed 1993 (“unistroke symbols”)

Goldberg (Xerox) Patent 5596656http://patft.uspto.gov/Claims: system, method, processOriginally filed 1993Re-filed 1995Patent granted 1997

Palm v. Xerox Lawsuit 1997-2006Patent Infringement – Graffiti alphabet

3

Goldberg Alphabet, filed 1993(“unistroke symbols”)

4

Graffiti Alphabet (Palm), 1995

5

Goldberg versus Graffiti

(MacKenzie, “The Immediate Usage of Graffiti,” 1997)

6

Palm’s Defense Approach

Invalidity based on prior artWhere claims interpreted to cover any alphabet of single, unbroken strokesOr where claims interpreted to cover an alphabet of the symbols shown in the patent

Non-infringementAfter the invalidity arguments limit the interpretation of the claims, the Graffiti alphabet does not infringe

7

Goldberg Patent Claims

1. A system for interpreting handwritten text User interface … pointer for writing

mutually independent unistroke symbolssome being linear and others arcuateeach representing a predefined textual componentdelimiting operation distinguishing symbols from each other independent of their spatial relationship

Sensor mechanism coupled to user interfaceRecognition unitDisplay & character generator to display output

8

Simplified Claim’s Chartfor references against 656 patent

NYYNunistrokes are natural language alphanumeric symbols

12. Process:

*Y*Ysome symbols differ only in stroke direction

YYYYstroke direction parameter

*YYYalphabet of mutually independent unistroke symbols

10. Method:

9. 2-6 + unistrokes well separated in sloppiness space

2. 1 + planar writing surface, etc.

*YYNeach unistroke symbol is a predefined textual component

YNYYsome unistroke symbols linear and others arcuate

*YYYsymbol independent delimiting operation

*YYYmutually independent unistroke symbols

YNYY1. System: UI, tablet, pointer, x-y coord, reco unit, display

IBMAllenBurr656Claim

9

Background: Online (Pen-Centric)Handwriting Recognition

Written Languages and Handwriting PropertiesThe Fundamental Property of WritingHandwriting Recognition DifficultiesOnline (Pen-Centric) Handwriting RecognitionOnline more accurate than Offline RecognitionOnline Info Can Complicate Recognition ProcessDesign Tradeoffs / Design Decisions

10

Written Language and Handwriting Properties

AlphabetLetters, digits, punctuation, special symbols

Writing is a time sequence of strokesStroke – writing from pen down to pen upUsually complete one character before beginning the nextSpatial order – e.g., in English left to right

11

Fundamental Property of Writing

Differences between different characters are more significant than differences between different drawings of the same characterThis makes handwritten communication possibleCan there be exceptions – say, different characters written identically?

12

Fundamental Property of Writingin English

Property holds within subalphabets of uppercase, lowercase, and digits, but not across them

“I”, “l”, and “1” written with single vertical stroke“O” and “0” written similarly with an oval

13

Handwriting Recognition Difficulties

Shape, size, and slant variationSimilarly shaped characters – U and VCareless writing

in the extreme, almost illegible writing

Resolving difficult ambiguities requires sophisticated recognition algorithms, syntax/semantics

14

Handwriting Recognition Difficulties

Segmentation ambiguitiescharacter-within-character problem lowercase d might be recognized as a cl if drawn with two strokes that are somewhat separated from one another

15

Electronic tablets invented in late 1950sDigitizer and display in separate surfaces

Pen Computers arrived in 1980sCombined digitizer and displayBrought input and output into one surfaceImmediate feedback via electronic inkCreated paper-like interface

Online (Pen-Centric) Handwriting Recognition

16

Online (Pen-Centric) Handwriting Recognition

Machine recognizes the writing as the user writes Digitizer equipment captures the dynamic information of the writing

Stroke number, order, direction, speedA stroke is the writing from pen down to pen up

17

Design Tradeoffs/Decisions

No constraints on the userMachine recognizes user's normal writing

User severely constrainedMust write in particular style such as handprint Must write strokes in particular order, direction, and graphical specification

18

Tappert 1990 Journal ArticleSurveyed 44 Systems

11 experimental systems for handprint4 experimental systems for cursive script16 commercial systems for opaque tablets5 commercial systems for pen computers8 experimental applications systems

Spreadsheets, flowcharts, etc.

19

Tappert 1990 Survey English Handprint Systems

Early systems handled only specific variationsE.g., Groner experimental system

Later systems handled all common variationsE.g., CIC and Pencept commercial products

Recent systems (e.g., IBM, Linus, etc.)Trained to userBuilt-in coverage of common variations

20

Tappert 1990 SurveySymbol Segmentation Methods

Pen liftMove to new tablet area

Or new box when filling out forms

Pause – Time outUser hits finish button

21

Tappert 1990 SurveyThree Categories of Systems

1. University project systems – least robust2. Industrial project systems – more robust3. Commercial products – most robust

Fred Brooks’ Mythical Man MonthProgram – programming system –programming systems product

22

Tappert 1990 Survey Lawsuit Relevant Systems

Rand Experimental System 1966 – GronerPencept Commercial Product 1980sATT Experimental System 1983 – BurrIBM Runon System 1984 – TappertLinus Commercial Product 1987 – Sklarew

23

Rand Experimental System 1966

Different surfaces for input and outputSystem created about 1959Handwriting system article published 1966Used author’s small number of alphanumeric symbol variations

24

Pencept Commercial Product 1980s(all common symbol variations, one per box)

25

Burr Article 1983Proposed a curve matching classification techniqueTo avoid segmentation problems, users must draw each letter with one stroke

E.g., draw i,j without dots and t,x without crossings

26

Tappert Handprint System 1984Runon Patent filed 1986

Part of ThinkWrite software in IBM’s pen-enabled ThinkPad product in early 1990s

27

Sklarew Patent filed 1987Linus Commercial Product 1987

28


NNYNunistrokes are natural language alphanumeric symbols

12. Process:

***Ysome symbols differ only in stroke direction

YYYYstroke direction parameter

**YYalphabet of mutually independent unistroke symbols

10. Method:



**YNeach unistroke symbol is a predefined textual component

YYYYsome unistroke symbols linear and others arcuate

**YYsymbol independent delimiting operation

**YYmutually independent unistroke symbols

YYYY1. System: UI, tablet, pointer, x-y coord, reco unit, display

LinusIBMBurr656Claim

29

Shorthand Alphabets

Famous Uses of ShorthandHistorical Shorthand AlphabetsPen-Centric Shorthand AlphabetsPen-Centric Word/Phrase ShorthandAllegro/Chatroom Shorthand System

M.S. thesis that could be extended into a DPS dissertation

30

Famous Historical Shorthand Uses

Cicero’s orationsMartin Luther’s sermonsShakespeare’s & George Bernard Shaw’s playsSamuel Pepys’ diarySir Isaac Newton’s notebooks

31

Historical Shorthand Alphabets

We first review the history of shorthand systems prior to pen computing Shorthand is “a method of writing rapidly by substituting characters, abbreviations, or symbols for letters, words, or phrases”Shorthand can be traced back to the Greeks in 400 B.C.

32


We focus on shorthand alphabets that might be appropriate for PDAs and relevant to the lawsuitWe review two types of shorthand

Geometric shorthandSmall number of basic shapesShapes reused in multiple orientations

Non-geometric shorthand shorthand

33


Ancient Greeks – 400 BCTironian Alphabet – 63 BCJohn Willis’s Stenography – 1602Gabelsberger Alphabet – 1834Moon Alphabet – 1845

34

Tironian Alphabet, 63 B.C.Non-Geometric

35

Stenography Alphabet, 1602

36

Stenography Alphabet, 1602

Basic Shapes and Orientations

37

Gabelsberger Cursive-Style, 1834Non-Geometric Alphabet

38

Moon Geometric Alphabet, 1845

39

Other Historical Shorthand Systems

Phonetic alphabetsPitman (1837), was popular in UKGregg (1888), was popular in USA

Systems for the blindBraille (1821)

40

Pen-Centric Shorthand Alphabets

Some of the earliest were for CAD/CAMsymbols represent graphical items and commands

Others developed for text input on small consumer devices like PDAs that have limited computing powerWe review geometric and non-geometric shorthands appropriate for small devices

41


Historical alphabets presented above could be used for machine recognition

symbols drawn with a single stroke (except “K” in Tironian and “+” in Stenography)

In addition to shape and orientation, online systems can use stroke direction to differentiate among symbols

42


Geometric Pen-Centric ShorthandsOrganek – 1991Allen – filed 1991, patent 1993Goldberg (Xerox) – filed 1993, patent 1997

Non-Geometric Pen-Centric ShorthandsGraffiti (Palm Computing) – 1995Allegro (Papyrus) – 1995

43

Organek Alphabet, 1991

44

Organic Alphabet, 1991Basic Shapes and Orientations

One shape in 4 orientations.

This gives 8 directions that together with 3 lengths provide 24 symbols.

A second wheel provides additional symbols.

45

Allen patent, filed 1991

46

Allen patent, filed 1991Basic Shapes and Orientations

only linear shapes

47

Goldberg patent, filed 1993(“unistroke symbols”)

48

Goldberg patent, filed 1993Basic Shapes and Orientations

49

Goldberg patent, filed 1993

5 Basic shapes4 Orientations2 Stroked Directions40 Possible SymbolsDesigned for Speed of Input and Maximum Symbol Separation

50


NYYNunistrokes are natural language alphanumeric symbols

12. Process:

*Y*Ysome symbols differ only in stroke direction

YY*Ystroke direction parameter

*Y*Yalphabet of mutually independent unistroke symbols

10. Method:



*Y*Neach unistroke symbol is a predefined textual component

YNYYsome unistroke symbols linear and others arcuate

YY*Ysymbol independent delimiting operation

*Y*Ymutually independent unistroke symbols

YNNY1. System: UI, tablet, pointer, x-y coord, reco unit, display

Graffiti

AllenTiro656Claim

51

Design of Graffiti Alphabetfor the Palm Pilot

Small alphabetUppercase, digits, special symbols

One stroke per symbol to avoid segmentation difficultySeparate writing areas for letters and digits to avoid same-shape confusions

52

Graffiti Alphabet, 1995

53

Graffiti Mimics Keyboard Input

Character by character inputMode shifts for

UppercaseSpecial characters

Eyes can focus on application’s insertion point rather than on input area

54

Graffiti Alphabet Designedfor Ease of Learning

21 letters match the Roman alphabet5 remaining ones match partially


55

Graffiti Alphabet Designedfor Ease of Learning


frequency of use

56

Graffiti Alphabet: 11 of 26 characters have alternate variations

57

Frequently Confused Characters(MacKenzie, “The Immediate Usage of Graffiti,” 1997)

58

Other Low Performance Characters(MacKenzie, “The Immediate Usage of Graffiti,” 1997)

59

Symbol Overlap Comparison

60

Goldberg-Graffiti Differences

YNAlternate variationsNYHighly separable symbols*YAll single stroke symbolsYNLike Roman alphabetYNEasy to learnNYDesigned for fast inputNYGeometric alphabet

GraffitiGoldbergDesign Criterion

61

Palm-XeroxPatent Infringement Lawsuit

The nine-year old battle between Palm and Xerox over handwriting recognition ends in 2006, http://www.psionplace.com/articles/2006/6/2006-6-28-Palm-Xerox-Ink.htmlPalm pays Xerox $22.5 million for a fully paid-up license for Xerox patents covering its text input UnistrokestechnologyXerox first sued Palm predecessor Palm Computing back in April 1997, claiming that the Graffiti text-entry system used in its PDAs infringed on patents for Unistrokes, which allows users to input letters and numbers into personal data units with basic, one stroke movements.

62

ConclusionsPalm-Xerox Patent Infringement Lawsuit

Invalidity Historical research showed that Goldberg alphabet not so uniqueEven though the patent was accepted as valid, these arguments narrowed the scope of the patent

InfringementAnalyses and comparisons of the Goldberg and Graffiti alphabets showed major differences

Result was favorable settlement for Palm

63

References

C.C. Tappert, C.Y. Suen, and T. Wakahara, "The state-of-the-art in on-line handwriting recognition," IEEE Trans. Pattern Analysis Machine Intelligence, Vol. PAMI-12, pp. 787-808, August 1990.C.C. Tappert and J.R. Ward, "Pen-Centric Shorthand Handwriting Recognition Interfaces," Proc. 1st Int. Workshop on Pen-Based Learning Technologies, Catania, Italy, May 2007.C.C. Tappert and S. Cha, "Handwriting Recognition Interfaces," Chapter 6, pp. 123-137, in Text Entry Systems, Scott MacKenzie and Kumiko Tanaka-Ishii (Eds.), Morgan Kaufmann, 2007.

Documents

Example of Historical Research Palm v. Xerox Patent …csis.pace.edu/~ctappert/dps/d891a-09/palm-xerox.pdf · Example of Historical Research Palm v. Xerox Patent Infringement Lawsuit