Human computer interaction research at ibm t

Human-Computer Interaction Human-Computer Interaction Research at IBM T. J. Watson Research at IBM T. J. Watson

Research CenterResearch Center

John C. ThomasJohn C. Thomas

Lehman CollegeLehman College

October 20, 2004October 20, 2004

OutlineOutline

• Whirlwind Tour of the PastWhirlwind Tour of the Past• Whirlwind Tour of the PresentWhirlwind Tour of the Present• In-depth Look at One ProjectIn-depth Look at One Project• Challenge: A Science of HCIChallenge: A Science of HCI• Potential Approaches: Potential Approaches:

– Pattern Languages in HCIPattern Languages in HCI– Fifteen Properties of Great DesignFifteen Properties of Great Design– Practical HacksPractical Hacks

Whirlwind Tour of the PastWhirlwind Tour of the Past

• Query By Example (1973-1975)Query By Example (1973-1975)– Pencil and Paper studiesPencil and Paper studies– Deeper Dive on QuantifiersDeeper Dive on Quantifiers

• Natural Language Processing (1975-1976)Natural Language Processing (1975-1976)– ““Wizard of Oz” studiesWizard of Oz” studies

• Psychology of Design (1976-1979)Psychology of Design (1976-1979)– Spectrum of TechniquesSpectrum of Techniques

• Corporate Headquarters (1980-1982)Corporate Headquarters (1980-1982)• Speech Synthesis (1983-1985)Speech Synthesis (1983-1985)• Motivation (1986)Motivation (1986)• NYNEX AI Lab (1986-1998)NYNEX AI Lab (1986-1998)

– Expert Systems, Machine Vision, Speech Recognition, Expert Systems, Machine Vision, Speech Recognition, Robotics, HCI, Intelligent Tutoring SystemsRobotics, HCI, Intelligent Tutoring Systems

• IBM Research – Stories and Storytelling (1999-2001)IBM Research – Stories and Storytelling (1999-2001)• Dynamic Assembly of Learning Objects (2001-2003)Dynamic Assembly of Learning Objects (2001-2003)• Tools for Consultants (2004--)Tools for Consultants (2004--)

Whirlwind Tour of the Whirlwind Tour of the PresentPresent

• Multi-mediaMulti-media– Everywhere DisplayEverywhere Display– Video ParsingVideo Parsing– Scene ParsingScene Parsing

• User Interface TechnologyUser Interface Technology– Speech RecognitionSpeech Recognition– Handwriting RecognitionHandwriting Recognition– Natural Language ResearchNatural Language Research

• Human Computer InteractionHuman Computer Interaction– Social ComputingSocial Computing– New Learning & Teaching SystemsNew Learning & Teaching Systems– Better Tools for SW development, consultantsBetter Tools for SW development, consultants

Learning Objects: Vision and Learning Objects: Vision and ChallengesChallenges

• Learning Materials Modularized with added Learning Materials Modularized with added MetadataMetadata

• ““Best” Materials become available to allBest” Materials become available to all• Re-purposing and re-use (e.g., Tiger Re-purposing and re-use (e.g., Tiger

Woods video and physics professor)Woods video and physics professor)• ----------------------------------------------------------------------------------------------• Copyright IssuesCopyright Issues• How does metadata get added ?How does metadata get added ?• Selection, ordering, coherence issuesSelection, ordering, coherence issues

ContextContext

• Dynamic Assembly allows on-demand material Dynamic Assembly allows on-demand material based on background and needs.based on background and needs.

• System allows a person to use a Query Engine and System allows a person to use a Query Engine and then select from a list of relevance-ordered results.then select from a list of relevance-ordered results.

• System uses metadata associated with each System uses metadata associated with each selected result to construct a pedagogically selected result to construct a pedagogically coherent course from learning objects.coherent course from learning objects.

• Here, domain is WebSphere and Web ServicesHere, domain is WebSphere and Web Services• Materials come from IBM Redbooks.Materials come from IBM Redbooks.• Hypothesis: People using Dynamic Assembly will Hypothesis: People using Dynamic Assembly will

perform significantly better in an on-demand perform significantly better in an on-demand situation than those using the Query Engine alone.situation than those using the Query Engine alone.

• Experiment designed to inform this question and Experiment designed to inform this question and also to provide on-going feedback to the design also to provide on-going feedback to the design team.team.

Custom Course Delivery




Iterative DevelopmentIterative Development

• Began with a series of PowerPoint “simulations”Began with a series of PowerPoint “simulations”

• After several iterations, programming began on After several iterations, programming began on working systemworking system

• Pilot subjects with “real system”Pilot subjects with “real system”

• Followed by extensive IGS Field TrialFollowed by extensive IGS Field Trial

• Users had very positive reaction to the system Users had very positive reaction to the system based on questionnaire results; HOWEVER, it based on questionnaire results; HOWEVER, it seemed that most were either: trying it seemed that most were either: trying it because they were told to, kicking the tires, or because they were told to, kicking the tires, or seeing what it “would be like.” Only a few (and seeing what it “would be like.” Only a few (and there were some) seemed to be engaged in there were some) seemed to be engaged in serious learning. serious learning.

Experimental Design Experimental Design RationaleRationale• Trade-offs between “real people” doing Trade-offs between “real people” doing

“real work” in their “real setting” at their “real work” in their “real setting” at their convenience and controlconvenience and control

• Used IBM volunteers willing to spend two Used IBM volunteers willing to spend two hours from their own office connected via hours from their own office connected via phone, shadowing, and/or sametimephone, shadowing, and/or sametime

• Gave scenario to motivate learning and Gave scenario to motivate learning and performanceperformance

• Used very difficult task to avoid ceiling Used very difficult task to avoid ceiling effecteffect

• Used multiple measures of background, Used multiple measures of background, process, and product of learningprocess, and product of learning

• Group I used Dynamic Assembly; Group II Group I used Dynamic Assembly; Group II only used (the same) Query Engineonly used (the same) Query Engine

ProcedureProcedure

• Volunteers recruited via e-mailVolunteers recruited via e-mail• Volunteers asked for two hour time Volunteers asked for two hour time

slot; given instructions to print out slot; given instructions to print out ahead of time; most stayed on ahead of time; most stayed on phone and spoke aloud during all phone and spoke aloud during all tasks; some preferred to work tasks; some preferred to work silently or send same-time silently or send same-time messages.messages.

• Not all subjects kept appointments; Not all subjects kept appointments; not all subjects turned in all not all subjects turned in all materials – but most (>90%) did materials – but most (>90%) did turn in all materials. turn in all materials.

TasksTasks

• Subjects read overview of experiment and Subjects read overview of experiment and were asked if they had any questionswere asked if they had any questions

• Subjects read a scenario to motivate them to Subjects read a scenario to motivate them to design a solution to a customer’s problemdesign a solution to a customer’s problem

• Subjects had 55 minutes to learnSubjects had 55 minutes to learn• Subjects had 35 minutes to complete a high Subjects had 35 minutes to complete a high

level design level design • Subjects were then quizzed on objective Subjects were then quizzed on objective

knowledge of WebSphere and Web Services, knowledge of WebSphere and Web Services, answered subjective questions about their answered subjective questions about their experience with the software and asked experience with the software and asked about their relevant background knowledge. about their relevant background knowledge.

ScenarioScenario

• IBM has the chance for a moderately large IBM has the chance for a moderately large software deal with Genysis although … is a software deal with Genysis although … is a strong competitor.strong competitor. You’ve been called in at the You’ve been called in at the last minute because the person who was last minute because the person who was supposed to give a client presentation was in a supposed to give a client presentation was in a serious automobile accident.serious automobile accident. The client exec wanted to The client exec wanted to reschedule …Genysis … needs to make a decision now. You will reschedule …Genysis … needs to make a decision now. You will meet with the client in about an hour …provide preliminary high meet with the client in about an hour …provide preliminary high level “design”…. If you can pull this off, your boss would be level “design”…. If you can pull this off, your boss would be extremely impressed. Basically, their situation is as follows:extremely impressed. Basically, their situation is as follows:

• Genysis … an international provider of herbs, vitamins, and related Genysis … an international provider of herbs, vitamins, and related health care products; they have a working client-server application health care products; they have a working client-server application that allows registered pharmacists to order their products on-line. that allows registered pharmacists to order their products on-line. …software called “Doser” allows the pharmacist to enter a body …software called “Doser” allows the pharmacist to enter a body weight and get a recommended dosage for particular drugs. A weight and get a recommended dosage for particular drugs. A series of recent events, …have led them to want to publish series of recent events, …have led them to want to publish “Doser” as a Web Service available to the general public. “Doser” as a Web Service available to the general public. Moreover…they would like to offer interactions in at least six major Moreover…they would like to offer interactions in at least six major languages… as well as English. Luckily, although the current languages… as well as English. Luckily, although the current application is written for an American English, the Model View application is written for an American English, the Model View Controller Pattern was used and formats, help files and messages Controller Pattern was used and formats, help files and messages …separate from …control flow. Genysis… considering possible use …separate from …control flow. Genysis… considering possible use of another Web Service PolyGlot to do translation….of another Web Service PolyGlot to do translation….

SubjectsSubjects

• Subjects were all IBMersSubjects were all IBMers

• Subjects varied considerably both in IT Subjects varied considerably both in IT experience generally and knowledge of experience generally and knowledge of Web Services and WebSphere from Web Services and WebSphere from “complete novice” to “levels 3 to 5.”“complete novice” to “levels 3 to 5.”

• Subjects also varied widely in “the Subjects also varied widely in “the number of languages I’ve used to write number of languages I’ve used to write programs of >1000 lines” from 0 to 7. programs of >1000 lines” from 0 to 7.

Evaluating the DesignsEvaluating the Designs

• Design Behavior difficult to quantify; hence, Design Behavior difficult to quantify; hence, use multiple measures:use multiple measures:

• Quantitative Measures (often correlated with Quantitative Measures (often correlated with quality)quality)– Total Number of Pages in Design/PresentationTotal Number of Pages in Design/Presentation– Number of Words, Boxes, Arrows (not counting what Number of Words, Boxes, Arrows (not counting what

is copied)is copied)– Number of features present from an expert’s designNumber of features present from an expert’s design

• Qualitative MeasuresQualitative Measures– Three experts rating the designs Three experts rating the designs

Total Pages of OutputTotal Pages of Output

• Mean for Custom Course Group = 6.5Mean for Custom Course Group = 6.5

• Mean for Query Only Group = 1.83Mean for Query Only Group = 1.83

• t(24)=2.10, p<.05t(24)=2.10, p<.05

• Also, a significant correlation between self-Also, a significant correlation between self-reported experience and number of pages; reported experience and number of pages; r=.46, t(24)=2.535, p<.01 (one-tailed), p<.02 r=.46, t(24)=2.535, p<.01 (one-tailed), p<.02 (two-tailed)(two-tailed)

• When experience is taken into account, the When experience is taken into account, the effect of group increases: t(24)=2.66, p<.014effect of group increases: t(24)=2.66, p<.014

Registry includes herb, vitamins, etc.

WSDL document Existing Java beansImport JAR file into AAT

WAR file

Web Sphere build

Deploy

RunManage

Client

Service Registry

Sample Design from one of the Query Only subjects

Architectural concept Architectural concept

Genysis Doser WebService Genysis Doser WebService enablingenabling

Sample Design from Custom Course Assembly Subject (p.1)

Existing ArchitectureExisting Architecture

DoserBusiness

Logic

GenysisDrug

Database(DB2)

DoserController

DoserView

DoserGUI

Doser Java Client Doser Java Server


RequirementsRequirements

• WebService enabling of existing WebService enabling of existing Business LogicBusiness Logic

• Multilanguage (Mandarin, French, Multilanguage (Mandarin, French, German, Italian, Spanish and English)German, Italian, Spanish and English)


AssumptionsAssumptions

• Business Logic of existing solutions Business Logic of existing solutions runs on serverruns on server

• New solution runs exactly the same New solution runs exactly the same Business LogicBusiness Logic

• Content of the database is language Content of the database is language independent (product names and independent (product names and numbers)numbers)


New New ArchitectureArchitecture

DoserBusiness

Logic

GenysisDrug

Database(DB2)

DoserController

DoserView

DoserGUI

Doser Java Client Doser Server

RPCServlet

(WebService)

HTMLController

HTMLView

NLSProperties

BrowserWebService

Client

SOAP/HTTP HTTP HTTP

UDDI

register


Custom Course Group had Custom Course Group had more total design features that more total design features that matched expert design matched expert design suggestionsuggestion

• Custom Course Group Mean = 10.8Custom Course Group Mean = 10.8• Query Only Group Mean = 8.6Query Only Group Mean = 8.6• However, previous reported experience However, previous reported experience

dwarfed this effect, r=.463, t dwarfed this effect, r=.463, t (22)=2.35, p<.05(22)=2.35, p<.05

• When experience is taken into account, When experience is taken into account, the superiority of the Custom Course the superiority of the Custom Course Group is significantGroup is significant

• t(22) = 2.35, p<.03t(22) = 2.35, p<.03

Experience also correlated with Experience also correlated with overall “quantity” of designoverall “quantity” of design

• Correlation of Experience with words+boxes+arrowsCorrelation of Experience with words+boxes+arrows• r=.409, t(22)=2.1, p<.05r=.409, t(22)=2.1, p<.05• Custom Course group had more in their designs; Custom Course group had more in their designs;

when experience taken into account, when experience taken into account, • Custom Course outperformed the Query Only GroupCustom Course outperformed the Query Only Group• t(22)=2.59, p<.02t(22)=2.59, p<.02• Experience also correlated separately with boxes Experience also correlated separately with boxes

used (r=.418, p<.05), arrows,r=.346, p<.1) and used (r=.418, p<.05), arrows,r=.346, p<.1) and words (r=.367,p<.08)words (r=.367,p<.08)

• With experience accounted for, Custom Course Group With experience accounted for, Custom Course Group also had significantly more words t(22)=2.46, p<.03also had significantly more words t(22)=2.46, p<.03

Designs were graded Designs were graded qualitatively by three expertsqualitatively by three experts

• Designs were graded on presentation, level Designs were graded on presentation, level of detail, accuracy, completeness, depth of of detail, accuracy, completeness, depth of understanding, and overallunderstanding, and overall

• Graders tended to agree overall: r (1,2) Graders tended to agree overall: r (1,2) = .728, r (1,3) = .72, r (2,3)= .734, p<.0001 = .728, r (1,3) = .72, r (2,3)= .734, p<.0001 for allfor all

• Dynamic Assembly Group produced better Dynamic Assembly Group produced better designs for all three raters, but not designs for all three raters, but not significantly so.significantly so.

• Adjusted for experience, more “good Adjusted for experience, more “good designs” in Dynamic Assemblydesigns” in Dynamic Assembly

ConclusionsConclusions

• It appears that the subjects from the It appears that the subjects from the Dynamic Assembly Group produced Dynamic Assembly Group produced more in terms of design and, with more in terms of design and, with experience accounted for, experience accounted for, qualitatively better designs as wellqualitatively better designs as well

• Individual differences, both in terms Individual differences, both in terms of background knowledge and design of background knowledge and design quality were very largequality were very large

Why might the Custom Course Why might the Custom Course Group have done better?Group have done better?

• The extra work involved in making a course may have The extra work involved in making a course may have encouraged them to stay more focused and less encouraged them to stay more focused and less scattered.scattered.

• The additional metadata presented in the query results The additional metadata presented in the query results page may have allowed them to go to material better page may have allowed them to go to material better suited to their background and goals.suited to their background and goals.

• The expectation of building and using a “course” may The expectation of building and using a “course” may have induced a more reflective cognitive set.have induced a more reflective cognitive set.

• The learning objectives and organization may have The learning objectives and organization may have served as advanced organizers thus improving served as advanced organizers thus improving learning.learning.

• The ordering of learning objects within a course may The ordering of learning objects within a course may have allowed a more coherent learning experience.have allowed a more coherent learning experience.

ConclusionsConclusions• Behavior Behavior duringduring the experiment was the experiment was

qualitatively and quantitatively different qualitatively and quantitatively different for the two groups:for the two groups:

• The Query Only group spent a short The Query Only group spent a short amount of time on a larger number of amount of time on a larger number of Learning Objects, focusing on search and Learning Objects, focusing on search and navigationnavigation

• The Custom Course Group spent longer The Custom Course Group spent longer amounts of time on fewer (and somewhat amounts of time on fewer (and somewhat different) Learning Objects focusing on different) Learning Objects focusing on reading and understanding reading and understanding

• Subjects varied considerably in their Subjects varied considerably in their queries, Learning Object selections and queries, Learning Object selections and visits, probably due to differences in their visits, probably due to differences in their backgroundbackground

““Good” HCI Depends on…Good” HCI Depends on…

• Users (People differ in training, ability, Users (People differ in training, ability, etc.etc.

• Tasks (Speed/Accuracy, Complexity, Tasks (Speed/Accuracy, Complexity, etc.etc.

• Context (Work, Play, School, War, Context (Work, Play, School, War, Retail…Retail…

• Technology (Typing, Speech Technology (Typing, Speech Recognition, Gesture Recognition, Recognition, Gesture Recognition, Virtual Reality, Affect Recognition, etc.Virtual Reality, Affect Recognition, etc.

““Good” HCI Depends on…Good” HCI Depends on…

• Users Users • TasksTasks• ContextContext• Technology Technology

•BUT….BUT….• All of these are changing rapidly ALL All of these are changing rapidly ALL

the TIME !! the TIME !!

How then do we “Cumulate” How then do we “Cumulate” Knowledge and Develop a True Knowledge and Develop a True Science ?Science ?

• Scientific Truths are Supposedly Scientific Truths are Supposedly independent of time and placeindependent of time and place

• One Approach: Pattern LanguageOne Approach: Pattern Language• Second Approach: Fifteen PropertiesSecond Approach: Fifteen Properties

Potential Forms of Potential Forms of KnowledgeKnowledgeKnown, Predictable, Unchanging, Simple

Unknown, Unpredictable, Changing, Complex

Algorithms, Formulae, Programs, Machines

Patterns

Guidelines

Heuristics, Principles, Properties

Case Studies

Stories

Ethical values and fluid intelligence

How can we help the How can we help the designer DESIGN? designer DESIGN?

The Importance of the The Importance of the SocialSocial

• Robert Putnam: Making Democracy Robert Putnam: Making Democracy Work (Italy) Bowling Alone (America)Work (Italy) Bowling Alone (America)

• Impacts health of individual more than Impacts health of individual more than smokingsmoking

• Impacts on whether we have a Impacts on whether we have a sustainable approach to the world’s sustainable approach to the world’s resourcesresources

• Impact on war and other miseriesImpact on war and other miseries• Corporations now supporting Corporations now supporting

collaboration and communities of collaboration and communities of practicepractice

• Socially defined intelligence: Evan’s Socially defined intelligence: Evan’s Thesis on figures analogiesThesis on figures analogies

E.g. Washing DishesE.g. Washing Dishes

• Hand Washing DuoHand Washing Duo• Rhythm requiredRhythm required• Side by side Side by side

“confessional”“confessional”• Conversation OKConversation OK• Team accomplishes Team accomplishes

the workthe work• High shared High shared

stimulus contextstimulus context

• Using DishwasherUsing Dishwasher• Rhythm not requiredRhythm not required• Unitary betterUnitary better• Conversation ?Conversation ?• Team or One Team or One

prepares machine to prepares machine to accomplish the workaccomplish the work

• Moderate shared Moderate shared stimulus contextstimulus context

Fixing DinnerFixing Dinner

• Traditional cookingTraditional cooking

• Negotiation Negotiation RequiredRequired

• High shared High shared stimulus context stimulus context (same meal)(same meal)

• Synchronous Synchronous activityactivity

• Conversation likelyConversation likely

• MicrowaveMicrowave• No negotiation No negotiation

required (separate required (separate meals)meals)

• Asynchronous Asynchronous activityactivity

• Conversation less Conversation less likely (person who likely (person who is ready first starts is ready first starts some other activity)some other activity)

Traditional QueueTraditional Queue

• Some shared context; however…Some shared context; however…

• Perceived as competition for limited Perceived as competition for limited resource (tickets may run out)resource (tickets may run out)

• People in front are costing you timePeople in front are costing you time

• Face to Back of Head orientationFace to Back of Head orientation

• Asynchronous movement reinforces Asynchronous movement reinforces individual identity (cf. rowing)individual identity (cf. rowing)

Vibrating Pager QueueVibrating Pager Queue

• The obviousness of the The obviousness of the competition has been greatly competition has been greatly reducedreduced

• No requirement to “face the No requirement to “face the same direction”same direction”

• Face to face interaction possibleFace to face interaction possible

• Conversation is much more Conversation is much more likelylikely

Enhanced Telephone Help Enhanced Telephone Help Desk QueueDesk Queue• Many more people need help solving technical Many more people need help solving technical

problem than servers availableproblem than servers available• People describe problemPeople describe problem• ASR used to group similar problemsASR used to group similar problems• People are bridged onto a conference callPeople are bridged onto a conference call• Synthesis announces to group their areas of Synthesis announces to group their areas of

overlapping interestoverlapping interest• Group may be able to solve the individual Group may be able to solve the individual

problemsproblems• When available, help first gives generic advice When available, help first gives generic advice

A Pattern LanguageA Pattern Language

• Christopher AlexanderChristopher Alexander• Architectural “Patterns” that capture Architectural “Patterns” that capture

recurring problems and solutionsrecurring problems and solutions• Organized into a “Pattern Language” Organized into a “Pattern Language”

– a lattice of inter-related Patterns.– a lattice of inter-related Patterns.• Examples: Examples:

– Eccentric Town Center encourages Eccentric Town Center encourages commuter traffic to stop at Town Centercommuter traffic to stop at Town Center

– European PubEuropean Pub– Gradient of Privacy in homes: porch, Gradient of Privacy in homes: porch,

entry, living room, dinning room, kitchen, entry, living room, dinning room, kitchen, bedroombedroom

Some Socio-Technical Some Socio-Technical PatternsPatterns• Community of CommunitiesCommunity of Communities• Reality CheckReality Check• Radical Co-locationRadical Co-location• Small Successes EarlySmall Successes Early• Who Speaks for Wolf?Who Speaks for Wolf?• Support Conversation at BoundariesSupport Conversation at Boundaries• Social ProxySocial Proxy• Context-setting EntryContext-setting Entry• Answer GardenAnswer Garden• Registered AnonymityRegistered Anonymity• Anonymized Stories for Organizational LearningAnonymized Stories for Organizational Learning• Mentoring CircleMentoring Circle• Levels of AuthorityLevels of Authority• Rites of PassageRites of Passage

Reality CheckReality Check

Who Speaks for Wolf?Who Speaks for Wolf?Visual by www.PDIimages.comVisual by www.PDIimages.com

Small Successes EarlySmall Successes Early

Support Conversation at the Support Conversation at the BordersBorders

Christopher Alexander’s Fifteen Christopher Alexander’s Fifteen Properties from The Nature of OrderProperties from The Nature of Order

• 1. Levels of scale. 1. Levels of scale. • 2. Strong centers. 2. Strong centers. • 3. Boundaries. 3. Boundaries. • 4. Alternating repetition. 4. Alternating repetition. • 5. Positive space. 5. Positive space. • 6. Good shape. 6. Good shape. • 7. Local symmetries. 7. Local symmetries. • 8. Deep interlock and ambiguity. 8. Deep interlock and ambiguity. • 9. Contrast. 9. Contrast. • 10. Gradients. 10. Gradients. • 11. Roughness. 11. Roughness. • 12. Echoes. 12. Echoes. • 13. The Void. 13. The Void. • 14. Simplicity and Inner Calm. 14. Simplicity and Inner Calm. • 15. Not-separateness. 15. Not-separateness.

Can these be applied to the Can these be applied to the design of social systems?design of social systems?

* Levels of Scale: Organizations, Divisions, * Levels of Scale: Organizations, Divisions,

Departments, Projects, Teams, Individual.Departments, Projects, Teams, Individual.

* Positive Space: Opposite of “not my job”; * Positive Space: Opposite of “not my job”; better to have contention than gapsbetter to have contention than gaps

* The Void: Need empty space and empty * The Void: Need empty space and empty time; perhaps even roles of peacetime; perhaps even roles of peace

* Roughness: Problems arise when designs * Roughness: Problems arise when designs presume that they have covered every case.presume that they have covered every case.

For more information:For more information:

• www.research.ibm.com/knowsoc/www.research.ibm.com/knowsoc/• www.truthtable.com/websitewelcome_page_index.htmlwww.truthtable.com/websitewelcome_page_index.html• http://www.cpsr.org/conferences/diac02 http://www.cpsr.org/conferences/diac02 • http://www.welie.com/patterns/plml/http://www.welie.com/patterns/plml/• http://www.pliant.org/personal/Tom_Erickson/InteractionPatterns.hthttp://www.pliant.org/personal/Tom_Erickson/InteractionPatterns.ht

mlml• http://http://www.hcipatterns.orgwww.hcipatterns.org//• http://http://www.cpsr.orgwww.cpsr.org/program/sphere/patterns//program/sphere/patterns/• http://http://www.ibm.com/developerWorks/patternswww.ibm.com/developerWorks/patterns//• http://jerry.cs.uiuc.edu/~plop/plop2003/cfp2003.html http://jerry.cs.uiuc.edu/~plop/plop2003/cfp2003.html • http:/www.cs.kent.ac.uk/people/staff/saf/patterns/gallery.htmlhttp:/www.cs.kent.ac.uk/people/staff/saf/patterns/gallery.html• http://www.groupware-patterns.orghttp://www.groupware-patterns.org//• http://www.lmu.ac.uk/ies/comp/research/isle/janetfinlay/http://www.lmu.ac.uk/ies/comp/research/isle/janetfinlay/

• www.truthtable.com/patterns.html/www.truthtable.com/patterns.html/

http://www.research.ibm.com/knowsoc/

http://www.welie.com/patterns/plml/

http://www.pliant.org/personal/Tom_Erickson/InteractionPatterns.html

http://www.pliant.org/personal/Tom_Erickson/InteractionPatterns.html

http://www.hcipatterns.org/



http://www.cpsr.org/program/sphere/patterns/



http://www.ibm.com/developerWorks/patterns/



http://www.groupware-patterns.org/

http://www.groupware-patterns.org/

http://www.truthtable.com/patterns.html/

Education

Human computer interaction research at ibm t