Upload
von
View
45
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Models of Human Performance. CSCI 4800 Spring 2006 Kraemer. Objectives. Introduce theory-based models for predicting human performance Introduce competence-based models for assessing cognitive activity Relate modelling to interactive systems design and evaluation. - PowerPoint PPT Presentation
Citation preview
Models of Human Models of Human PerformancePerformance
CSCI 4800CSCI 4800
Spring 2006Spring 2006
KraemerKraemer
ObjectivesObjectives
Introduce theory-based models for Introduce theory-based models for predicting human performancepredicting human performance
Introduce competence-based models Introduce competence-based models for assessing cognitive activityfor assessing cognitive activity
Relate modelling to interactive Relate modelling to interactive systems design and evaluationsystems design and evaluation
What are we trying to What are we trying to model?model?
Seven Stage Action ModelSeven Stage Action Model[Norman, 1990][Norman, 1990]
Form intentionDevelop plan
Perform action
Object in world
Evaluate against goalInterpret object
Perceive state of object
GOAL OF PERSON
Describing Problem SolvingDescribing Problem Solving
Initial StateInitial State Goal StateGoal State All possible intervening statesAll possible intervening states
– Problem SpaceProblem Space Path ConstraintsPath Constraints State Action TreeState Action Tree Means-ends analysisMeans-ends analysis
Problem SolvingProblem Solving
A problem is something that doesn’t solve easilyA problem is something that doesn’t solve easily
A problem doesn’t solve easily because:A problem doesn’t solve easily because:– you don’t have the necessary knowledge or,you don’t have the necessary knowledge or,– you have misrepresented part of the problemyou have misrepresented part of the problem
If at first you don’t succeed, try something elseIf at first you don’t succeed, try something else
Tackle one part of the problem and other parts may Tackle one part of the problem and other parts may fall into placefall into place
ConclusionConclusion
More than one solutionMore than one solution
Solution limited by boundary Solution limited by boundary conditionsconditions
Representation affects strategyRepresentation affects strategy
Active involvement and testingActive involvement and testing
Functional FixednessFunctional Fixedness
Strategy developed in one version of Strategy developed in one version of the problemthe problem
Strategy might be inefficientStrategy might be inefficient
X ) XXXXX ) XXXX Convert numerals or just ‘see’ 4Convert numerals or just ‘see’ 4
Data-driven perceptionData-driven perception
Activation of Activation of neural structures neural structures of sensory system of sensory system by pattern of by pattern of stimulation from stimulation from environmentenvironment
Theory-driven perceptionTheory-driven perception
Perception driven by memories and Perception driven by memories and expectations about incoming expectations about incoming information. information.
KEYPOINTKEYPOINT
PERCEPTION involves a set of active PERCEPTION involves a set of active processes that impose: processes that impose:
STRUCTURE, STRUCTURE,
STABILITY, STABILITY,
and and MEANING MEANING
on the worldon the world
Visual IllusionsVisual Illusions
Old Woman or Young girl?
Rabbit or duck?
http://www.genesishci.com/illusions2.htm
InterpretationInterpretation
Knowledge of what you are “looking at” Knowledge of what you are “looking at” can aid in interpretationcan aid in interpretation
JAJA CKAN CKAN DJI DJI LLW LLW ENTU ENTU PTH PTH EHIEHI LLTLLT OFEOFETCHTCH APAAPA ILOILOFWAFWA TERTER
Organisation of information is also usefulOrganisation of information is also useful
Story GrammarsStory Grammars
Analogy with sentence grammarsAnalogy with sentence grammars– Building blocks and rules for combiningBuilding blocks and rules for combining
Break story into propositions Break story into propositions
“ “Margie was holding tightly to the string of her beautiful new Margie was holding tightly to the string of her beautiful new balloon. Suddenly a gust of wind caught it, and carried it balloon. Suddenly a gust of wind caught it, and carried it into a tree. It hit a branch, and burst. Margie cried and into a tree. It hit a branch, and burst. Margie cried and cried.”cried.”
Story GrammarStory Grammar
Story
Setting Episode
Event
Reaction
Internal response
Overt response
ChangeOf state
Event
Event
Event
Event Event Event [sadness]
[1]
[2] [3] [4]
[5]
[6]
InferencesInferences
Comprehension typically requires our Comprehension typically requires our active involvement in order to supply active involvement in order to supply information that is not explicit in the information that is not explicit in the texttext
1. Mary heard the ice-cream van coming1. Mary heard the ice-cream van coming
2. She remembered her pocket money2. She remembered her pocket money
3. She rushed into the house.3. She rushed into the house.
Inference and RecallInference and Recall
Thorndyke (1976): recall of Thorndyke (1976): recall of sentences from ‘Mary’ storysentences from ‘Mary’ story– 85% correct sentence85% correct sentence– 58% correct inference – 58% correct inference –
sentence not presentedsentence not presented
– 6% incorrect inference6% incorrect inference
Mental ModelsMental Models
Van Dijk and Kintsch (1983)Van Dijk and Kintsch (1983)– Text processed to extract propositions, Text processed to extract propositions,
which are held in working memory;which are held in working memory;– When sufficient propositions in WM, then When sufficient propositions in WM, then
linking performed;linking performed;– Relevance of propositions to linking Relevance of propositions to linking
proportional to recall;proportional to recall;– Linking reveals ‘gist’Linking reveals ‘gist’
Semantic NetworksSemantic Networks
ANIMAL
Has SkinCan moveEatsBreathes
BIRDCan fly
Has WingsHas feathers
FISHHas finsCan swimHas gillsCANARY
Is YellowCan sing
Collins & Quillian, 1969
Levels and Reaction timeLevels and Reaction time
A canary is a canary
A canary is a bird
A canary is an animal
A canary is a fish
A canary can sing
A canary can fly
A canary has skin
A canary has gills
Collins & Quillian, 1969
0.9
1
1.1
1.2
1.3
1.4
1.5
0 1 2 False
Levels of Sentences
Mean
Reacti
on
Tim
e (
s)
Property
Category
CanariesCanaries
Different times to verify the Different times to verify the statements:statements:– A canary is a birdA canary is a bird– A canary can flyA canary can fly– A canary can singA canary can sing
Time proportional to movement Time proportional to movement through networkthrough network
Scripts, Schema and FramesScripts, Schema and Frames
Schema = chunks of knowledgeSchema = chunks of knowledge– Slots for information: fixed, default, optionalSlots for information: fixed, default, optional
Scripts = action sequencesScripts = action sequences– Generalised event schema (Nelson, 1986)Generalised event schema (Nelson, 1986)
Frames = knowledge about the properties Frames = knowledge about the properties of thingsof things
Mental ModelsMental Models
PartialPartial
Procedures, Functions or System?Procedures, Functions or System?
Memory or Reconstruction?Memory or Reconstruction?
ConceptsConcepts
How do you know a chair is a chair?How do you know a chair is a chair?
A chair has four legs…does it? A chair has a seat…does it?
Prototypes, Typical Features, and Prototypes, Typical Features, and ExemplarsExemplars
PrototypePrototype ROSCH (1973): people do not use feature sets, but ROSCH (1973): people do not use feature sets, but
imagine a PROTOTYPE for an objectimagine a PROTOTYPE for an object
Typical FeaturesTypical Features ROSCH & MERVIS (1975): people use a list of ROSCH & MERVIS (1975): people use a list of
features, weighted in terms of CUE VALIDITYfeatures, weighted in terms of CUE VALIDITY
ExemplarsExemplars SMITH & MEDIN (1981): people use an EXAMPLE to SMITH & MEDIN (1981): people use an EXAMPLE to
imagine an objectimagine an object
Representing ConceptsRepresenting Concepts
BARSALOU (1983)BARSALOU (1983)– TAXONOMICTAXONOMIC
Categories that are well known and can be recalled Categories that are well known and can be recalled consistently and reliablyconsistently and reliably
– E.g., Fruit, Furniture, AnimalsE.g., Fruit, Furniture, Animals Used to generate overall representation of the worldUsed to generate overall representation of the world
– AD HOCAD HOC Categories that are invented for specific purposeCategories that are invented for specific purpose
– E.g., How to make friends, Moving houseE.g., How to make friends, Moving house Used for goal-directed activity within specific event Used for goal-directed activity within specific event
framesframes
Long Term MemoryLong Term Memory
Procedural Procedural – Knowing howKnowing how
DeclarativeDeclarative– Knowing thatKnowing that
Episodic vs. SemanticEpisodic vs. Semantic– Personal eventsPersonal events– Language and knowledge of worldLanguage and knowledge of world
Working MemoryWorking Memory
Limited CapacityLimited Capacity7 7 ++ 2 items (Miller, 1965) 2 items (Miller, 1965)4 4 ++ 2 chunks (Broadbent, 1972) 2 chunks (Broadbent, 1972)Modality dependent capacityModality dependent capacity
Strategies for coping with limitationStrategies for coping with limitationChunkingChunking InterferenceInterferenceActivation of Long-term memoryActivation of Long-term memory
Central executive
Articulatory control process
Auditory word presentation
Visual word presentation
Phonological store
Visual Cache
Inner scribe
Baddeley’s (1986) Model of Working Memory
Slave SystemsSlave Systems
Articulatory loopArticulatory loop– Memory ActivationMemory Activation– Rehearsal capacityRehearsal capacity
Word length effect and Rehearsal speedWord length effect and Rehearsal speed
Visual cacheVisual cache– Visual patternsVisual patterns– Complexity of pattern, number of elements etcComplexity of pattern, number of elements etc
Inner scribeInner scribe– Sequences of movementSequences of movement– Complexity of movementComplexity of movement
TypingTyping
Eye-hand span related to expertiseEye-hand span related to expertise Expert = 9, novice = 1Expert = 9, novice = 1
Inter-key intervalInter-key interval Expert = 100msExpert = 100ms
StrategyStrategy Hunt & Peck vs. Touch typingHunt & Peck vs. Touch typing
KeystrokeKeystroke Novice = highly variable keystroke timeNovice = highly variable keystroke time Novice = very slow on ‘unusual’ letters, e.g., X or ZNovice = very slow on ‘unusual’ letters, e.g., X or Z
Salthouse (1986)Salthouse (1986)
InputInput– Text converted to chunksText converted to chunks
ParsingParsing– Chunks decomposed to stringsChunks decomposed to strings
TranslationTranslation– Strings into characters and linked to Strings into characters and linked to
movementsmovements ExecutionExecution
– Key pressedKey pressed
Rumelhart & Norman (1982)Rumelhart & Norman (1982)
Perceptual processes Perceptual processes – Perceive text, generate word schemaPerceive text, generate word schema
ParsingParsing– Compute codes for each letterCompute codes for each letter
Keypress schemataKeypress schemata– Activate schema for letter-keypressActivate schema for letter-keypress
Response activationResponse activation– Press defined key through activation of Press defined key through activation of
appropriate hand / fingerappropriate hand / finger
Schematic of Rumelhart and Norman’s Schematic of Rumelhart and Norman’s connectionist model of typingconnectionist model of typing
middlering index
little thumb
Left hand
middleindex ring
thumb little
Right hand
Response system
activation
j a z z
jazzWord node, activated fromVisual or auditory stimulus
Keypress node, breakingWord into typed letters;Excites and inhibits nodes
AutomaticityAutomaticity
Norman and Shallice (1980)Norman and Shallice (1980)Fully automatic processing controlled by Fully automatic processing controlled by
SCHEMATASCHEMATA
Partially automatic processing controlled by Partially automatic processing controlled by either Contention Scheduling either Contention Scheduling
Supervisory Attentional System (SAS)Supervisory Attentional System (SAS)
Supervisory Attentional System Supervisory Attentional System ModelModel
Perceptual System
SupervisoryAttentional
System
Effector System
Contentionscheduling
Triggerdatabase
Control schema
Contention SchedulingContention Scheduling
Gear changing when driving involves many Gear changing when driving involves many routine activities but is performed routine activities but is performed ‘automatically’ – without conscious ‘automatically’ – without conscious awarenessawareness
When routines clash, relative importance is When routines clash, relative importance is used to determine which to perform – used to determine which to perform – Contention SchedulingContention Scheduling
e.g., right foot on brake or clutche.g., right foot on brake or clutch
SAS activationSAS activation
Driving on roundabouts in FranceDriving on roundabouts in France– Inhibit ‘look right’; Activate ‘look left’Inhibit ‘look right’; Activate ‘look left’– SAS to over-ride habitual actionsSAS to over-ride habitual actions
SAS active when:SAS active when:Danger, Choice of response, Novelty etc.Danger, Choice of response, Novelty etc.
Attentional Slips and LapsesAttentional Slips and Lapses
Habitual actions become automaticHabitual actions become automatic SAS inhibits habitSAS inhibits habit PerserverationPerserveration
When SAS does not inhibit and habit proceedsWhen SAS does not inhibit and habit proceeds
DistractionDistraction Irrelevant objects attract attentionIrrelevant objects attract attention Utilisation behaviour: patients with frontal lobe Utilisation behaviour: patients with frontal lobe
damage will reach for object close to hand even when damage will reach for object close to hand even when told not totold not to
Performance Operating Performance Operating CharacteristicsCharacteristics
Resource-dependent trade-off Resource-dependent trade-off between performance levels on two between performance levels on two taskstasks
Task A and Task B performed several Task A and Task B performed several times, with instructions to allocate times, with instructions to allocate more effort to one task or the othermore effort to one task or the other
Task DifficultyTask Difficulty
Data limited processesData limited processesPerformance related to quality of data and Performance related to quality of data and
will not improve with more resourcewill not improve with more resource
Resource limited processesResource limited processesPerformance related to amount of resource Performance related to amount of resource
invested in task and will improve with more invested in task and will improve with more resourceresource
POCPOC
Data limited Data limited Resource limitedResource limited
Cost
Task A
Task B
P
M Task A
Task B
P
MCost
Why Model Performance? Why Model Performance?
Building models can help develop theoryBuilding models can help develop theory– Models make assumptions explicitModels make assumptions explicit– Models force explanationModels force explanation
Surrogate user:Surrogate user:– Define ‘benchmarks’Define ‘benchmarks’– Evaluate conceptual designsEvaluate conceptual designs– Make design assumptions explicitMake design assumptions explicit
Rationale for design decisionsRationale for design decisions
Why Model Performance?Why Model Performance?
Human-computer interaction as Applied Human-computer interaction as Applied ScienceScience– Theory from cognitive sciences used as basis Theory from cognitive sciences used as basis
for designfor design
– General principles of perceptual, motor and General principles of perceptual, motor and cognitive activitycognitive activity
– Development and testing of theory through Development and testing of theory through modelsmodels
Types of Model in HCITypes of Model in HCI
SystemSystem ProgramProgram UserUser ResearcherResearcher DesignerDesigner
ProgramProgram XX
UserUser XX XX
ResearcheResearcherr
XX XX XX XX
DesignerDesigner XX XX XX
Whitefield, 1987
Task ModelsTask Models
Researcher’s Model of User, in terms of Researcher’s Model of User, in terms of taskstasks
Describe typical activitiesDescribe typical activities
Reduce activities to generic sequencesReduce activities to generic sequences
Provide basis for designProvide basis for design
Pros and Cons of ModellingPros and Cons of Modelling
PROSPROS– Consistent description through (semi) formal Consistent description through (semi) formal
representationsrepresentations– Set of ‘typical’ examplesSet of ‘typical’ examples– Allows prediction / description of performanceAllows prediction / description of performance
CONSCONS– Selective (some things don’t fit into models)Selective (some things don’t fit into models)– Assumption of invariabilityAssumption of invariability– Misses creative, flexible, non-standard activityMisses creative, flexible, non-standard activity
Generic Model Process?Generic Model Process?
Define system: {goals, activity, tasks, Define system: {goals, activity, tasks, entities, parameters}entities, parameters}
Abstract to semantic level Abstract to semantic level Define syntax / representationDefine syntax / representation Define interactionDefine interaction Check for consistency and completenessCheck for consistency and completeness Predict / describe performancePredict / describe performance Evaluate resultsEvaluate results Modify modelModify model
Device and Task ModelsDevice and Task Models
Device ModelsDevice Models
Buxton’s 3-state device modelBuxton’s 3-state device model
State0
State1
State2
ApplicationApplication
State0
State1
State2
Out of range
Pen on
Pen off Button up
Button down
select drag
Different pointing devicesDifferent pointing devices
DeviceDevice State0State0 State1State1 State2State2
TouchscreeTouchscreenn
XX
PenPen XX XX XX
JoystickJoystick XX XX
MouseMouse XX XX
ConclusionsConclusions
Models abstract aspects of Models abstract aspects of interactioninteraction– User, task, systemUser, task, system
Models play a variety of roles in Models play a variety of roles in designdesign
Hierarchical Task AnalysisHierarchical Task Analysis
Activity assumed to consist of TASKS Activity assumed to consist of TASKS performed in pursuit of GOALSperformed in pursuit of GOALS
Goals can be broken into SUBGOALS, Goals can be broken into SUBGOALS, which can be broken into taskswhich can be broken into tasks
Hierarchy (Tree) descriptionHierarchy (Tree) description
Hierarchical Task DescriptionHierarchical Task Description
1 .0S w itch on O H P
2 .0C h eck p ro jec tion
3 .0P lace fo il on O H P
4 .0F ocu s p ro jec tion
0 .0P resen t O H P s lid es
The “Analysis” comes from The “Analysis” comes from plansplans
PLANS = conditions for combining tasksPLANS = conditions for combining tasks Fixed SequenceFixed Sequence
– P0: 1 > 2 > exitP0: 1 > 2 > exit Contingent Fixed SequenceContingent Fixed Sequence
– P1: 1 > when state X achieved > 2 > exitP1: 1 > when state X achieved > 2 > exit– P1.1: 1.1 > 1.2 > wait for X time > 1.3 > exitP1.1: 1.1 > 1.2 > wait for X time > 1.3 > exit
Decision Decision – P2: 1 > 2 > If condition X then 3, elseif P2: 1 > 2 > If condition X then 3, elseif
condition Y then 4 > 5 > exitcondition Y then 4 > 5 > exit
Reporting Reporting
HTA can be constructed using Post-it notes HTA can be constructed using Post-it notes on a large space (this makes it easy to edit on a large space (this makes it easy to edit and also encourages participation)and also encourages participation)
HTA can be difficult to present in a HTA can be difficult to present in a succinct printed form (it might be useful to succinct printed form (it might be useful to take a photograph of the Post-it notes)take a photograph of the Post-it notes)
Typically a Tabular format is used:Typically a Tabular format is used:
Task Task numbernumber
TaskTask PlanPlan CommentsComments
Redesigning the Interface to a Redesigning the Interface to a medical imaging systemmedical imaging system
Original DesignOriginal Design
Menu drivenMenu driven
Menus accessed by first Menus accessed by first letter of commandletter of command
Menus arranged in Menus arranged in hierarchyhierarchy
Problems with original designProblems with original design
Lack of consistencyLack of consistencyD = DOS commands; Delete; Data file; DateD = DOS commands; Delete; Data file; Date
Hidden hierarchyHidden hierarchyOnly ‘experts’ could useOnly ‘experts’ could use
Inappropriate defaultsInappropriate defaultsSetting up a scan required ‘correction’ of Setting up a scan required ‘correction’ of
default settings three or four timesdefault settings three or four times
Initial design activityInitial design activity
Observation of non-technology workObservation of non-technology workCytogeneticists inspecting chromosomesCytogeneticists inspecting chromosomes
Developed model of taskDeveloped model of taskHierarchical task analysisHierarchical task analysis
Developed design principles, e.g.,Developed design principles, e.g.,Cytogeneticists as ‘picture people’Cytogeneticists as ‘picture people’Task flowTask flowTask mappingTask mapping
Task ModelTask Model
Work flows between specific activitiesWork flows between specific activities
Patient details Administration
Set up
Reporting
Microscope
Cell sample
Analysis
First “prototype”First “prototype”
Layout related to task model
‘Sketch’ very simple
Annotations showmodifications
Second prototypeSecond prototype
Refined layout
‘Prototype’ usingHyperCard
Initial user trials comparedthis with a mock-up of the original design
Final ProductFinal Product
Picture taken from companybrochure
Initial concepts retained
Further modifications possible
Predicting Transaction Predicting Transaction TimeTime
Predicting Performance TimePredicting Performance Time
Time and error are ‘standard’ Time and error are ‘standard’ measures of human performancemeasures of human performance
Predict transaction time for Predict transaction time for comparative evaluationcomparative evaluation
Approximations of human Approximations of human performanceperformance
Unit TimesUnit Times
From task model, define sequence of From task model, define sequence of tasks to achieve a specific goaltasks to achieve a specific goal
For each task, define ‘average time’For each task, define ‘average time’
Quick ExerciseQuick Exercise
Draw two parallel lines about 4cm apart Draw two parallel lines about 4cm apart and about 10cm longand about 10cm long
Draw, as quickly as possible, a zig-zag line Draw, as quickly as possible, a zig-zag line for 5 secondsfor 5 seconds
Count the number of lines and the number Count the number of lines and the number of times you have crossed the parallel of times you have crossed the parallel lineslines
Predicted resultPredicted result
About 70 linesAbout 70 lines
About 20 cross-oversAbout 20 cross-overs
Why this prediction?Why this prediction?
Movement speed limited by biomechanical Movement speed limited by biomechanical constraintsconstraints– Motor subsystem change direction @ 70msMotor subsystem change direction @ 70ms– So: 5000 / 70 = 71 oscillationsSo: 5000 / 70 = 71 oscillations
Cognitive / Perceptual system cycles:Cognitive / Perceptual system cycles:– Perceptual @ 70msPerceptual @ 70ms– Cognitive @ 100msCognitive @ 100ms– Correction takes 70+70+100 = 240msCorrection takes 70+70+100 = 240ms– 5000/240 = 215000/240 = 21
Fitts’ LawFitts’ Law
Paul Fitts 1954Paul Fitts 1954 Information-theoretic account of Information-theoretic account of
simple movementssimple movements Define the number of ‘bits’ processed Define the number of ‘bits’ processed
in performing a given taskin performing a given task
Fitts’ Tapping TaskFitts’ Tapping Task
W
a
Fitts’ LawFitts’ Law
Movement Time = a + b (logMovement Time = a + b (log22 2A/W) 2A/W)
Hits
60
40
20
0 Log2 (2A/W)
1. A = 62, W = 152. A = 112, W = 73. A = 112, W = 21
1 = 5.3 2 = 4.5 3 = 3.2
54
43
21
a
b
a = 10b = 27.5
Alternate VersionsAlternate Versions
MT = a + b logMT = a + b log22 (2A/W) (2A/W)
MT = b logMT = b log22 (A/W + 0.5) (A/W + 0.5)
MT = a + b logMT = a + b log22 (A/W/+1) (A/W/+1)
a and b are “constants”a and b are “constants”
Data derived from plotData derived from plot
Data as predictors?Data as predictors? aa bb
MouseMouse10301030
108108
-107-107
9696
392392
223223
TrackballTrackball7575
282282300300
347347
Potential ProblemsPotential Problems
Data-fitter rather than ‘law’Data-fitter rather than ‘law’
‘‘Generic value’: a+b = 100Generic value’: a+b = 100
Variable predictive power for devices?Variable predictive power for devices?– From ‘mouse data’ we get:From ‘mouse data’ we get:
(assume A = 5 and W = 10) (assume A = 5 and W = 10) loglog22(2A/W) (2A/W) 0.3 0.3339ms, 150.5ms and 34.9ms (!!)339ms, 150.5ms and 34.9ms (!!)
Hick – Hyman LawHick – Hyman Law
William Hick 1952William Hick 1952
Selection time, from a set of items, is Selection time, from a set of items, is proportional to the number of itemsproportional to the number of items
T = k logT = k log22 (n+1), (n+1), Where k = a constant (intercept+slope)Where k = a constant (intercept+slope)
Approximately 150ms added to T for each itemApproximately 150ms added to T for each item
Example of Hick-Hyman LawExample of Hick-Hyman Law
Search Time (s)
4
3
2
1
0 2 3 4 5 6 7 8 10 12
words
numbers
Landauer and Nachbar, 1985
Keystroke Level ModelsKeystroke Level Models
Developed from 1950s ergonomicsDeveloped from 1950s ergonomics
Human information processor as linear Human information processor as linear executor of specified tasksexecutor of specified tasks
Unit-tasks have defined timesUnit-tasks have defined times
Prediction = summing of times for Prediction = summing of times for sequence of unit-taskssequence of unit-tasks
Building a KLMBuilding a KLM
Develop task modelDevelop task model
Define task sequenceDefine task sequence
Assign unit-times to tasksAssign unit-times to tasks
Sum timesSum times
Example: cut and pasteExample: cut and paste
Task Model: Select line – Cut – Select insertion point Task Model: Select line – Cut – Select insertion point – paste– paste
Task One: select lineTask One: select linemove cursor to move cursor to start of linestart of linepress (hold) buttonpress (hold) buttondrag cursor todrag cursor toend of lineend of linerelease buttonrelease button
Times for MovementTimes for Movement
H: homing, e.g., hand from keyboard to mouseH: homing, e.g., hand from keyboard to mouse– Range: 214ms – 400msRange: 214ms – 400ms– Average: 320msAverage: 320ms
P: pointing, e.g., move cursor using mouseP: pointing, e.g., move cursor using mouse– Range: defined by Fitts’ LawRange: defined by Fitts’ Law– Average: 1100msAverage: 1100ms
B: button pressing, e.g., hitting key on B: button pressing, e.g., hitting key on keyboardkeyboard– Range: 80ms – 700msRange: 80ms – 700ms– Average: 200msAverage: 200ms
Times for Cognition / PerceptionTimes for Cognition / Perception M: mental operationM: mental operation
– Range: 990ms – 1760msRange: 990ms – 1760ms– Average: 1350msAverage: 1350ms
A: switch attention between parts of displayA: switch attention between parts of display– Average: 320msAverage: 320ms
R: recognition of itemsR: recognition of items– Range: 314ms – 1800msRange: 314ms – 1800ms– Average: 340msAverage: 340ms
Perceive change:Perceive change:– Range: 50 – 300msRange: 50 – 300ms– Average: 100msAverage: 100ms
Rules for Summing TimesRules for Summing Times
How to handle multiple Mental units:How to handle multiple Mental units:
– M before Ks in new argument stringsM before Ks in new argument strings– M at start of ‘cognitive unit’M at start of ‘cognitive unit’– M before Ps that select commandsM before Ps that select commands– Delete M if K redundant terminatorDelete M if K redundant terminator
AlternativeAlternative
What if we use ‘accelerated scrolling’ What if we use ‘accelerated scrolling’ on the cursor keys?on the cursor keys?– Press Press key and read scrolling numbers key and read scrolling numbers– Release key at or near numberRelease key at or near number– Select correct numberSelect correct number
M H
Pe
P P’ P
Critical Path ModelsCritical Path Models
Used in project managementUsed in project management
Map dependencies between tasks in Map dependencies between tasks in a projecta project– Task X is dependent on task Y, if it is Task X is dependent on task Y, if it is
necessary to wait until the end of task Y necessary to wait until the end of task Y until task X can commenceuntil task X can commence
ProcedureProcedure
Construct task model, taking into account Construct task model, taking into account dependenciesdependencies
Assign times to tasksAssign times to tasks
Calculate critical path and transaction timeCalculate critical path and transaction time– Run forward passRun forward pass– Run backward passRun backward pass
ExampleExample
M H
R
P P’ P
M = 1.35H = 0.32P = 0.2R = 0.34
1 2 3 4
M1.35
H0.32
P0.2 5
P’0.2
R0.34
6
P0.2
Critical Path TableCritical Path Table
ActivityActivity DuratioDurationn
ESTEST LSTLST EFTEFT LFTLFT FloatFloat
MM 1.351.35 00 00 1.351.35 1.351.35 00
HH 0.320.32 1.351.35 1.351.35 1.671.67 1.671.67 00
PP 0.20.2 1.671.67 1.671.67 1.871.87 1.871.87 00
RR 0.340.34 1.671.67 1.731.73 2.012.01 2.072.07 0.060.06
P’P’ 0.20.2 2.072.07 2.072.07 2.272.27 2.272.27 00
PP 0.20.2 2.272.27 2.272.27 2.472.47 2.472.47 00
ComparisonComparison
‘‘Summing of times’ result:Summing of times’ result:– 2.61s2.61s
‘‘Critical path’ result:Critical path’ result:– 2.47s2.47s
R allowed to ‘float’R allowed to ‘float’
Other time-based modelsOther time-based models
Task-network modelsTask-network models– MicroSAINTMicroSAINT– Unit-times and probability of transitionUnit-times and probability of transition
Prompt50ms
Speak word[300 9]ms
System response[1000 30]ms
p
1-p
Models of CompetenceModels of Competence
Performance vs. CompetencePerformance vs. Competence
Performance ModelsPerformance Models– Make statements and predictions about Make statements and predictions about
the time, effort or likelihood of error the time, effort or likelihood of error when performing specific tasks;when performing specific tasks;
Competence ModelsCompetence Models– Make statements about what a given Make statements about what a given
user knows and how this knowledge user knows and how this knowledge might be organised.might be organised.
Sequence vs. Process vs. Sequence vs. Process vs. GrammarGrammar
Sequence ModelsSequence Models– Define activity simply in terms of sequences of Define activity simply in terms of sequences of
operations that can be quantifiedoperations that can be quantified Process ModelsProcess Models
– Simple model of mental activity but define the Simple model of mental activity but define the steps needed to perform taskssteps needed to perform tasks
Grammatical ModelsGrammatical Models– Model required knowledge in terms of Model required knowledge in terms of
‘sentences’‘sentences’
Process ModelsProcess Models
Production systemsProduction systems
GOMSGOMS
Production SystemsProduction Systems
Rules = (Procedural) KnowledgeRules = (Procedural) Knowledge
Working memory = state of the worldWorking memory = state of the world
Control strategies = way of applying Control strategies = way of applying knowledgeknowledge
Production SystemsProduction Systems
Architecture of a production system:Architecture of a production system:
Rule base Working Memory
Interpreter
The Problem of ControlThe Problem of Control
Rules are useless without a useful way to Rules are useless without a useful way to apply themapply them
Need a consistent, reliable, useful way to Need a consistent, reliable, useful way to control the way rules are appliedcontrol the way rules are applied
Different architectures / systems use Different architectures / systems use different control strategies to produce different control strategies to produce different resultsdifferent results
Forward ChainingForward Chaining
A C
AB
C
AB If not C then GOAL
If A then B
If A and B then not C
If A and B then not C
If not C then GOAL
If A then B
Backward ChainingBackward Chaining
A C
AB
C
AB If A then B
If A and B then not C
Need: not C
Need B
If not C then GOAL
Need GOAL
If A and B then not C
If not C then GOAL
If A then B
Production SystemsProduction Systems
A simple A simple metaphormetaphor
Docks
Ships
Production SystemsProduction Systems
Ships must fit the correct Ships must fit the correct dockdock
When one ship is docked, When one ship is docked, another can be launchedanother can be launched
Production SystemsProduction Systems
Production SystemsProduction Systems
Production RulesProduction Rules
IF conditionIF conditionTHEN actionTHEN action
e.g., e.g., IF ship is dockedIF ship is docked
And free-floating shipsAnd free-floating shipsTHEN launch shipTHEN launch shipIF dock is freeIF dock is free
And Ship matchesAnd Ship matchesTHEN dock shipTHEN dock ship
The Parsimonious Production The Parsimonious Production Systems Rule NotationSystems Rule Notation
On any cycle, any rule whose conditions are On any cycle, any rule whose conditions are currently satisfied will firecurrently satisfied will fire
Rules must be written so that a single rule will Rules must be written so that a single rule will not fire repeatedlynot fire repeatedly
Only one rule will fire on a cycleOnly one rule will fire on a cycle
All procedural knowledge is explicit in these All procedural knowledge is explicit in these rules rather than being explicit in the interpreterrules rather than being explicit in the interpreter
Worked Example: Worked Example: The Tower of HanoiThe Tower of Hanoi
3
2
1
A B C
4
5
Possible Steps 1Possible Steps 1
Disc 1 from a to cDisc 1 from a to c
Disc 2 from a to bDisc 2 from a to b
Disc 1 from c to aDisc 1 from c to a
Disc 3 from a to cDisc 3 from a to c
Disc 2 from b to cDisc 2 from b to c
Disc 1 from a to cDisc 1 from a to c
Worked Example: Worked Example: The Tower of HanoiThe Tower of Hanoi
3
2
1
A B C
4
5
Possible Steps 2Possible Steps 2
Disc 4 from a to bDisc 4 from a to b
Disc 1 from c to bDisc 1 from c to b
Disc 2 from c to aDisc 2 from c to a
Disc 1 from b to aDisc 1 from b to a
Disc 2 from a to bDisc 2 from a to b
Disc 3 from a to bDisc 3 from a to b
Worked Example: Worked Example: The Tower of HanoiThe Tower of Hanoi
3
2
1
A B C
45
Possible Steps 3Possible Steps 3
Disc 5 from a to cDisc 5 from a to c
Disc 1 from b to aDisc 1 from b to a
Disc 2 from b to cDisc 2 from b to c
Disc 1 from a to cDisc 1 from a to c
Disc 3 from b to aDisc 3 from b to a
Disc 1 from c to bDisc 1 from c to b
Disc 2 from c to aDisc 2 from c to a
Disc 4 from b to c
Disc 1 from a to c
Disc 2 from a to b
Disc 1 from c to b
Disc 3 from a to c
Disc 1 from b to a
Disc 2 from b to c
Disc 1 from a to c
Simon’s (1975) goal-recursive Simon’s (1975) goal-recursive logiclogic
To get the 5-tower to Peg C, get the 4-tower to Peg B, then To get the 5-tower to Peg C, get the 4-tower to Peg B, then movemoveThe 5-disc to Peg C, then move the 4-tower to Peg CThe 5-disc to Peg C, then move the 4-tower to Peg C
To get the 4-tower to Peg B, get the 3-tower to Peg C, then To get the 4-tower to Peg B, get the 3-tower to Peg C, then movemoveThe 4-disc to Peg B, then move the 3-tower to Peg BThe 4-disc to Peg B, then move the 3-tower to Peg B
To get the 3-tower to Peg C, get the 2-tower to Peg B, then To get the 3-tower to Peg C, get the 2-tower to Peg B, then movemoveThe 3-disc to Peg C, then move the 2-tower to Peg C, The 3-disc to Peg C, then move the 2-tower to Peg C,
To get the 2-tower to Peg B, move the 1-disc to Peg C, then To get the 2-tower to Peg B, move the 1-disc to Peg C, then move move The 2-disc to Peg B, then move the 1-disc to Peg AThe 2-disc to Peg B, then move the 1-disc to Peg A
Production Rule 1Production Rule 1
SUBGOAL_DISCSSUBGOAL_DISCS
IFIF the goal is to achieve a particular configuration of the goal is to achieve a particular configuration of discs discs
AndAnd Di is on Px but should go to Py in the Di is on Px but should go to Py in the configurationconfiguration
AndAnd Di is the largest disc out of placeDi is the largest disc out of place
AndAnd Dj is on PyDj is on Py
And And Dj is smaller than DiDj is smaller than Di
AndAnd Pz is clear OR has a disc larger than DjPz is clear OR has a disc larger than Dj
THENTHEN set a subgoal to move the Dj tower to Pz and Di to set a subgoal to move the Dj tower to Pz and Di to PyPy
Production Rule 2Production Rule 2
SUBGOAL_MOVE_DISCSUBGOAL_MOVE_DISC
IFIF the goal is to achieve a particular configuration of the goal is to achieve a particular configuration of discs discs
AndAnd Di is on Px but should go to Py in the Di is on Px but should go to Py in the configurationconfiguration
AndAnd Di is the largest disc out of placeDi is the largest disc out of place
AndAnd Py is clearPy is clear
THENTHEN move Di to Pymove Di to Py
Goals Operators Method SelectionGoals Operators Method SelectionCard, Moran and Newell, 1983Card, Moran and Newell, 1983
Human activity modelled by Model Human Human activity modelled by Model Human ProcessorProcessor
Activity defined by GOALSActivity defined by GOALS
Goals held in ‘Stack’Goals held in ‘Stack’
Goals ‘pushed’ onto stack Goals ‘pushed’ onto stack
Goals ‘popped’ from stackGoals ‘popped’ from stack
GoalsGoals
Symbolic structures to define desired Symbolic structures to define desired state of affairs and methods to state of affairs and methods to achieve this state of affairsachieve this state of affairs
GOAL: EDIT-MANUSCRIPTGOAL: EDIT-MANUSCRIPT top level goaltop level goal
GOAL: EDIT-UNIT-TASKGOAL: EDIT-UNIT-TASK specific sub goalspecific sub goal
GOAL: ACQUIRE UNIT-TASKGOAL: ACQUIRE UNIT-TASK get next stepget next step
GOAL: EXECUTE UNIT-TASK do next stepGOAL: EXECUTE UNIT-TASK do next step
GOAL: LOCATION-LINEGOAL: LOCATION-LINE specific stepspecific step
OperatorsOperators
Elementary perceptual, motor or Elementary perceptual, motor or cognitive acts needed to achieve cognitive acts needed to achieve subgoalssubgoals
Get-next-lineGet-next-line
Use-cursor-arrow-methodUse-cursor-arrow-method
Use-mouse-methodUse-mouse-method
MethodsMethods
Descriptions of procedures for Descriptions of procedures for achieving goalsachieving goals
Conditional upon contents of working Conditional upon contents of working memory and state of taskmemory and state of task
GOAL: ACQUIRE-UNIT-TASKGOAL: ACQUIRE-UNIT-TASKGET-NEXT-PAGEGET-NEXT-PAGE if at end of manuscriptif at end of manuscript
GET-NEXT-TASKGET-NEXT-TASK
SelectionSelection
Choose between competing Methods, if Choose between competing Methods, if more than one more than one
GOAL:EXECUTE-UNIT-TASKGOAL:EXECUTE-UNIT-TASK
GOAL:LOCATE-LINEGOAL:LOCATE-LINE[select:[select: if hands on keyboard if hands on keyboard
and less than 5 lines to move and less than 5 lines to move USE CURSOR KEYS USE CURSOR KEYS
elseelseUSE MOUSE]USE MOUSE]
ExampleExample
Withdraw cash from ATMWithdraw cash from ATM– Construct task modelConstruct task model– Define production rulesDefine production rules
Task ModelTask Model
Method for goal: Obtain cash from ATMMethod for goal: Obtain cash from ATMStep1: access ATMStep1: access ATM
Step2: select ‘cash’ optionStep2: select ‘cash’ option
Step3: indicate amountStep3: indicate amount
Step4: retrieve cash and cardStep4: retrieve cash and card
Step5: end taskStep5: end task
Production RulesProduction Rules
((GOAL: USE ATM TO OBTAIN CASH)((GOAL: USE ATM TO OBTAIN CASH)
ADD-UNIT-TASK (access ATM)ADD-UNIT-TASK (access ATM)
ADD-WM-UNIT-TASK (access ATM)ADD-WM-UNIT-TASK (access ATM)
ADD-TASK-STEP (insert card in slot)ADD-TASK-STEP (insert card in slot)
SEND-TO-MOTOR(place card in slot)SEND-TO-MOTOR(place card in slot)
SEND-TO-MOTOR (eyes to slot)SEND-TO-MOTOR (eyes to slot)
SEND-TO-PERCEPTUAL (check card SEND-TO-PERCEPTUAL (check card in)in)
ADD (WM performing card insertion)ADD (WM performing card insertion)
ADD-TASK-STEP (check card insertion)ADD-TASK-STEP (check card insertion)
DELETE-UNIT-TASK (access ATM)DELETE-UNIT-TASK (access ATM)
ADD-UNIT-TASK (enter PIN)ADD-UNIT-TASK (enter PIN)
Problems with GOMSProblems with GOMS
Assumes ‘error-free’ performanceAssumes ‘error-free’ performance– Even experts make mistakesEven experts make mistakes
MHP gross simplifies human MHP gross simplifies human information processinginformation processing
Producing a task model of non-Producing a task model of non-existent products is difficultexistent products is difficult
Task Action GrammarTask Action Grammar
GOMS assumes ‘expert’ knows GOMS assumes ‘expert’ knows operators and methods for tasksoperators and methods for tasks
TAG assumes ‘expert’ knows simple TAG assumes ‘expert’ knows simple tasks, i.e., tasks that can be tasks, i.e., tasks that can be performed without problem-solvingperformed without problem-solving
TAG and competenceTAG and competence
CompetenceCompetence– Defines what an ‘ideal’ user would knowDefines what an ‘ideal’ user would know
TAG relies on ‘world knowledge’TAG relies on ‘world knowledge’– up vs downup vs down– left vs rightleft vs right– forward vs backwardforward vs backward
Task-action GrammarTask-action Grammar
Grammar relates simple tasks to Grammar relates simple tasks to actionsactions
Generic rule schema covering Generic rule schema covering combinations of simple taskscombinations of simple tasks
TAGTAG
A ‘grammar’ A ‘grammar’ – mapsmaps
Simple tasksSimple tasks
– Onto Onto ActionsActions
– To form To form an interaction languagean interaction language
– To investigate To investigate consistencyconsistency
ConsistencyConsistency
Syntactic: use of expressionsSyntactic: use of expressions
Lexical: use of symbolsLexical: use of symbols
Semantic-syntactic alignment: order of termsSemantic-syntactic alignment: order of terms
Semantic: principle of completenessSemantic: principle of completeness
ProcedureProcedure
– Step 1: Write out commands and their structuresStep 1: Write out commands and their structures
– Step 2: Determine in commands have consistent structureStep 2: Determine in commands have consistent structure
– Step 3: Place command items into variable/feature Step 3: Place command items into variable/feature relationshiprelationship
– Step 4: Generalise commands by separating into task Step 4: Generalise commands by separating into task features, simple tasks, task-action rule schemafeatures, simple tasks, task-action rule schema
– Step 5: Expand parts of task into primitivesStep 5: Expand parts of task into primitives
– Step 6: Check to ensure all names are uniqueStep 6: Check to ensure all names are unique
ExampleExample
Setting up a recording on a video-Setting up a recording on a video-cassette recorder (VCR)cassette recorder (VCR)
Assume that all controls via front Assume that all controls via front panel and that the user can only use panel and that the user can only use the up and down arrowsthe up and down arrows
Feature list [for a VCR]Feature list [for a VCR]
PropertyProperty Date, Channel, Start, EndDate, Channel, Start, End ValueValue numbernumber FrequencyFrequency Daily, WeeklyDaily, Weekly RecordRecord on, offon, off
Simple tasksSimple tasks
SetDate [Property = Date, Value = US#, Frequency = Daily]SetDate [Property = Date, Value = US#, Frequency = Daily]
SetDate [Property = Date, Value = US#, Frequency = Weekly]SetDate [Property = Date, Value = US#, Frequency = Weekly]
SetProg[Property =Prog, Value = US#]SetProg[Property =Prog, Value = US#]
SetStart[Property = start, Value = US#, Record = on]SetStart[Property = start, Value = US#, Record = on]
SetEnd[Property = start, Value = US#, Record = off]SetEnd[Property = start, Value = US#, Record = off]
Rule SchemaRule Schema
1. Task[Property = US#, Value] 1. Task[Property = US#, Value] SetValue [Value] SetValue [Value]
2. Task[Property = Date, Value, Frequency = US#] 2. Task[Property = Date, Value, Frequency = US#] SetValue SetValue [Value] + press “[Value] + press “ | | ” until Frequency = US#” until Frequency = US#
3. Task[Property = Start, Value] 3. Task[Property = Start, Value] SetValue [Value] + press SetValue [Value] + press “Rec”“Rec”
4. SetValue [Value = US#] 4. SetValue [Value = US#] press “ press “ | | ” until Value = US#” until Value = US#
5. SetValue[Value = US#] 5. SetValue[Value = US#] use “ use “ | | ” until Value = US#” until Value = US#
Architectures for Architectures for CognitionCognition
Why Cognitive Architecture?Why Cognitive Architecture?
Computers architectures:Computers architectures:– Specify components and their Specify components and their
connectionsconnections– Define functions and processesDefine functions and processes
Cognitive Architectures could be Cognitive Architectures could be seen as the logical conclusion of the seen as the logical conclusion of the ‘human-brain-as-computer’ ‘human-brain-as-computer’ hypothesishypothesis
Why do this?Why do this?
Philosophy: Provide a unified understanding of the mindPhilosophy: Provide a unified understanding of the mind Psychology: Account for experimental dataPsychology: Account for experimental data Education: Provide cognitive models for intelligent Education: Provide cognitive models for intelligent
tutoring systems and other learning environmentstutoring systems and other learning environments Human Computer Interaction: Evaluate artifacts and Human Computer Interaction: Evaluate artifacts and
help in their designhelp in their design Computer Generated Forces: Provide cognitive agents Computer Generated Forces: Provide cognitive agents
to inhabit training environments and gamesto inhabit training environments and games Neuroscience: Provide a framework for interpreting data Neuroscience: Provide a framework for interpreting data
from brain imagingfrom brain imaging
General RequirementsGeneral Requirements Integration of cognition, perception, and actionIntegration of cognition, perception, and action
Robust behavior in the face of error, the Robust behavior in the face of error, the unexpected, and the unknown unexpected, and the unknown
Ability to run in real timeAbility to run in real time
Ability to LearnAbility to Learn
Prediction of human behavior and performancePrediction of human behavior and performance
ArchitecturesArchitectures
Model Human Processor (MHP)Model Human Processor (MHP)– Card, Moran and Newell (1983)Card, Moran and Newell (1983)
ACT-RACT-R– Anderson (1993)Anderson (1993)
EPIC EPIC – Meyer and Kieras (1997) Meyer and Kieras (1997)
SOARSOAR– Laird, Rosenbloom and Newell (1987)Laird, Rosenbloom and Newell (1987)
Model Human ProcessorModel Human Processor
•Three interacting subsystems:
•Perceptual•Auditory image store•Visual image store
•Cognitive•Working memory•Long-term memory
•Motor
Parameters of MHPParameters of MHP
CapacityCapacity DecayDecay CycleCycle
Long-term memoryLong-term memory XX XX
Working memoryWorking memory 2.5 – 9 chunks2.5 – 9 chunks 5 – 226s5 – 226s
Auditory image storeAuditory image store 7 – 17 letters7 – 17 letters 70-70-1000ms1000ms
Visual image storeVisual image store 4.4-6.2 letters4.4-6.2 letters 900-900-3500ms3500ms
Cognitive processorCognitive processor 50-200ms50-200ms
Motor processorMotor processor 25-170ms25-170ms
Perceptual processorPerceptual processor 30-100ms30-100ms
Average data for MHPAverage data for MHP
Long-term memory:Long-term memory: ?? Working memory:Working memory: 3 – 7 chunks, 7s 3 – 7 chunks, 7s Auditory image store: 17 letters, 200msAuditory image store: 17 letters, 200ms Visual image store: 5 letters, 1500msVisual image store: 5 letters, 1500ms Cognitive processor: 100msCognitive processor: 100ms Perceptual processor: 70msPerceptual processor: 70ms Motor processor: 70msMotor processor: 70ms
ConclusionsConclusions
Simple description of cognitionSimple description of cognition
Uses ‘standard times’ for predictionUses ‘standard times’ for prediction
Uses production rules for defining Uses production rules for defining and combining tasks (with GOMS and combining tasks (with GOMS formalism)formalism)
Adaptive Control of Thought, Adaptive Control of Thought, Rational (ACT-R)Rational (ACT-R)
http://act.psy.cmu.eduhttp://act.psy.cmu.edu
Adaptive Control of Thought, Rational Adaptive Control of Thought, Rational (ACT-R)(ACT-R)
ACT-R symbolic aspect realised over ACT-R symbolic aspect realised over subsymbolic mechanismsubsymbolic mechanism
Symbolic aspect in two parts:Symbolic aspect in two parts:– Production memoryProduction memory– Symbolic memory (declarative memory)Symbolic memory (declarative memory)
Theory of rational analysisTheory of rational analysis
Theory of Rational AnalysisTheory of Rational Analysis
Evidence-based assumptions about Evidence-based assumptions about environment (probabilities)environment (probabilities)
Deriving optimal strategies (Bayesian)Deriving optimal strategies (Bayesian)
Assuming that optimal strategies reflect Assuming that optimal strategies reflect human cognition (either what it actually human cognition (either what it actually does or what it probably ought to do)does or what it probably ought to do)
Notions of MemoryNotions of Memory
Procedural Procedural – Knowing howKnowing how– Described in ACT by Production RulesDescribed in ACT by Production Rules
DeclarativeDeclarative– Knowing thatKnowing that– Described in ACT by ‘chunks’Described in ACT by ‘chunks’
Goal StackGoal Stack– A sort of ‘working memory’A sort of ‘working memory’– Holds chunks (goals)Holds chunks (goals)– Top goal pushed (like GOMS)Top goal pushed (like GOMS)– Writeable Writeable
Production RulesProduction Rules
Knowing how to do XKnowing how to do X– Production rule = set of conditions and Production rule = set of conditions and
an actionan action
IF it is rainingIF it is rainingAnd you wish to go outAnd you wish to go out
THEN pick up your umbrellaTHEN pick up your umbrella
(Very simple) ACT(Very simple) ACT
Network of propositionsNetwork of propositions
Production rules selected via pattern Production rules selected via pattern matching. Production rules coordinate matching. Production rules coordinate retrieval of chunks from symbolic memory retrieval of chunks from symbolic memory and link to environment.and link to environment.
If information in working memory matches If information in working memory matches production rule condition, then fire production rule condition, then fire production ruleproduction rule
ACT*ACT*
Declarativememory
Proceduralmemory
Working memory
Retrieval Storage Match Execution
OUTSIDE WORLD
Encoding Performance
Addition-Factsix U (4); T (1); H (0)
eight
addend1 sum
addend2
Knowledge Representation
1618 +_____34_____1
Goal buffer: add numbers in right-most columnVisual buffer: 6, 8Retrieval buffer: 14
Symbolic / Subsymbolic levelsSymbolic / Subsymbolic levels
Symbolic levelSymbolic level– Information as chunks in declarative memory, Information as chunks in declarative memory,
and represented as propositionsand represented as propositions– Rules as productions in procedural memoryRules as productions in procedural memory
Subsymbolic levelSubsymbolic level– Chunks given parameters which are used to Chunks given parameters which are used to
determine the probability that the chunk is determine the probability that the chunk is neededneeded
– Base-level activation (relevance)Base-level activation (relevance)– Context activation (association strengths)Context activation (association strengths)
Conflict resolutionConflict resolution
Order production rules by preferenceOrder production rules by preference Select top rule in listSelect top rule in list Preference defined by:Preference defined by:
– Probability that rule will lead to goalProbability that rule will lead to goal– Time associated with ruleTime associated with rule– Likely cost of reaching goal when using Likely cost of reaching goal when using
sequence involving this rulesequence involving this rule
ExampleExample
Activity: Find target and then use mouse Activity: Find target and then use mouse to select target:to select target:
Hunt_FeatureHunt_FeatureIF goal = find target with feature FIF goal = find target with feature FAND there is object X on screenAND there is object X on screenTHEN move attention to object XTHEN move attention to object X
Found_targetFound_targetIF goal = find target with feature FIF goal = find target with feature FAND target with F in location LAND target with F in location LTHEN move mouse to L and clickTHEN move mouse to L and click
ExampleExample
Model reaction time to targetModel reaction time to target– Assume switch attention linearly increases with Assume switch attention linearly increases with
each new positioneach new position– Assume probability of feature X in location y = Assume probability of feature X in location y =
0.530.53– Assume switch attention = 185msAssume switch attention = 185ms
Therefore, reaction time = 185 X 0.53 = Therefore, reaction time = 185 X 0.53 = 98ms per position98ms per position
Empirical data has RT of 103ms per positionEmpirical data has RT of 103ms per position
ExampleExample
Assume target in field of distractorsAssume target in field of distractors– P = 0.42P = 0.42– Therefore, 185 x .42 = 78ms per Therefore, 185 x .42 = 78ms per
positionposition
Empirical data = 80ms per positionEmpirical data = 80ms per position
LearningLearning
Symbolic levelSymbolic level– Learning defined by adding new chunks Learning defined by adding new chunks
and productionsand productions
Subsymbolic levelSubsymbolic level– Adjustment of parameters based on Adjustment of parameters based on
experienceexperience
ConclusionsConclusions
ACT uses simple production systemACT uses simple production system
ACT provides some quantitative ACT provides some quantitative prediction of performanceprediction of performance
Rationality = optimal adaptation to Rationality = optimal adaptation to environmentenvironment
Executive Process Interactive Executive Process Interactive Control (EPIC)Control (EPIC)
ftp://ftp.eecs.umich.edu/people/kierasftp://ftp.eecs.umich.edu/people/kieras
Executive Process Interactive Control Executive Process Interactive Control (EPIC) (EPIC)
Focus on multiple task performanceFocus on multiple task performance
Cognitive Processor runs production Cognitive Processor runs production rules and interacts with perceptual rules and interacts with perceptual and motor processors and motor processors
EPIC parametersEPIC parameters
FIXEDFIXED– Connections and mechanismsConnections and mechanisms– Time parametersTime parameters– Feature sets for motor processorsFeature sets for motor processors– Task-specific production rules and perceptual Task-specific production rules and perceptual
encoding typesencoding types FREEFREE
– Production rules for tasksProduction rules for tasks– Unique perceptual and motor processorsUnique perceptual and motor processors– Task instance setTask instance set– Simulated task environmentSimulated task environment
EPICEPIC
Task environment
Auditory
Visual
Speech
Manual
DISPLAYPERCEPTUALPROCESSORS
Auditory
Visual
Speech
Manual
Long-termmemory
Productionmemory
ProductionRule interpreter
Workingmemory
Tactile
Production MemoryProduction Memory
Perceptual processors controlled by Perceptual processors controlled by production rulesproduction rules
Production Rules held in Production Production Rules held in Production MemoryMemory
Production Rule Interpreter applies Production Rule Interpreter applies rules to perceptual processesrules to perceptual processes
Working MemoryWorking Memory
Limited capacity (or duration of 4s) Limited capacity (or duration of 4s) and holds current production rulesand holds current production rules
Cognitive processor updates every Cognitive processor updates every 50ms50ms
On update, perceptual input, item On update, perceptual input, item from production memory, and next from production memory, and next action held in working memoryaction held in working memory
Resolving ConflictResolving Conflict
Production rules applied to executive Production rules applied to executive tasks to handle resource conflict and tasks to handle resource conflict and schedulingscheduling
Conflict dealt with in production rule Conflict dealt with in production rule specificationspecification– LockoutLockout– InterleavingInterleaving– Strategic response deferentStrategic response deferent
ExampleExample
Task one
Stimulus one
Perceptual process
Cognitive process
Response selection
Memory process
Response one
Task two
Stimulus two
Perceptual process
Cognitive process
Response selection
Memory process
Response two
Executive process
Move eye to S2Enable task1 + task 2
Wait for task1 complete
Task1end
Task2 permission
Trial end
ConclusionsConclusions
Modular structure supports parallelismModular structure supports parallelism
EPIC does not have a goal stack and does EPIC does not have a goal stack and does not assume sequential firing of goalsnot assume sequential firing of goals
Goals can be handled in parallel (provided Goals can be handled in parallel (provided there is no resource conflict)there is no resource conflict)
Does not support learningDoes not support learning
States, Operators, And States, Operators, And Reasoning (SOAR)Reasoning (SOAR)
http://www.isi.edu/soar/soar.htmlhttp://www.isi.edu/soar/soar.html
States, Operators, And Reasoning States, Operators, And Reasoning (SOAR)(SOAR)
Sequel of General Problem Solver (Newell Sequel of General Problem Solver (Newell and Simon, 1960)and Simon, 1960)
SOAR seeks to apply SOAR seeks to apply operatorsoperators to to statesstates within a within a problem spaceproblem space to achieve a to achieve a goalgoal. .
SOAR assumes that actor uses all SOAR assumes that actor uses all available knowledge in problem-solvingavailable knowledge in problem-solving
Soar as a Unified Theory of Soar as a Unified Theory of CognitionCognition
Intelligence = problem solving + Intelligence = problem solving + learninglearning
Cognition seen as search in problem Cognition seen as search in problem spacesspaces
All knowledge is encoded as All knowledge is encoded as productions productions a single type of knowledgea single type of knowledge
All learning is done by chunking All learning is done by chunking a single type of learninga single type of learning
Young, R.M., Ritter, F., Jones, G. 1998 Young, R.M., Ritter, F., Jones, G. 1998
"Online Psychological Soar Tutorial" "Online Psychological Soar Tutorial"
available at: available at: http://www.psychology.nottingham.ac.uk/staff/http://www.psychology.nottingham.ac.uk/staff/Frank.Ritter/pst/pst-tutorial.htmlFrank.Ritter/pst/pst-tutorial.html
SOAR ActivitySOAR Activity
Operators:Operators: Transform a state via some action Transform a state via some action
State:State: A representation of possible stages of A representation of possible stages of progress in the problemprogress in the problem
Problem space:Problem space: States and operators that States and operators that can be used to achieve a goal.can be used to achieve a goal.
Goal:Goal: Some desired situation. Some desired situation.
SOAR ActivitySOAR Activity Problem solving = applying an Operator to a Problem solving = applying an Operator to a
State in order to move through a Problem Space State in order to move through a Problem Space to reach a Goal. to reach a Goal.
Impasse = Where an Operator cannot be applied Impasse = Where an Operator cannot be applied to a State, and so it is not possible to move to a State, and so it is not possible to move forward in the Problem Space. This becomes a forward in the Problem Space. This becomes a new problem to be solved.new problem to be solved.
Soar can Soar can learnlearn by storing solutions to past by storing solutions to past problems as problems as chunkschunks and applying them when it and applying them when it encounters the same problem againencounters the same problem again
SOAR ArchitectureSOAR ArchitectureChunkingmechanism
Production memory
Pattern ActionPattern ActionPattern Action
Decision procedure
Working memoryManager
Preferences Objects
Conflict stack
Working memory
ExplanationExplanation
Working MemoryWorking Memory– Data for current activity, organized into Data for current activity, organized into
objectsobjects Production MemoryProduction Memory
– Contains production rulesContains production rules Chunking mechanismChunking mechanism
– Collapses successful sequences of Collapses successful sequences of operators into chunks for re-useoperators into chunks for re-use
3 levels in soar3 levels in soar
Symbolic – the programming levelSymbolic – the programming level– Rules programmed into Soar that match Rules programmed into Soar that match
circumstances and perform specific actionscircumstances and perform specific actions Problem space – states & goalsProblem space – states & goals
– The set of goals, states, operators, and The set of goals, states, operators, and context.context.
Knowledge – embodied in the rulesKnowledge – embodied in the rules– The knowledge of how to act on the The knowledge of how to act on the
problem/world, how to choose between problem/world, how to choose between different operators, and any learned chunks different operators, and any learned chunks from previous problem solvingfrom previous problem solving
How does it work?How does it work?
A problem is encoded as a current state A problem is encoded as a current state and a desired state (goal)and a desired state (goal)
Operators are applied to move from one Operators are applied to move from one state to anotherstate to another
There is success if the desired state There is success if the desired state matches the current statematches the current state
Operators are proposed by productions, Operators are proposed by productions, with preferences biasing choices in with preferences biasing choices in specific circumstancesspecific circumstances
Productions fire in parallelProductions fire in parallel
ImpassesImpasses
If no operator is proposed, or if there is a If no operator is proposed, or if there is a tie between operators, or if Soar does not tie between operators, or if Soar does not know what to do with an operator, there is know what to do with an operator, there is an impassean impasse
When there are impasses, Soar sets a new When there are impasses, Soar sets a new goal (resolve the impasse) and creates a goal (resolve the impasse) and creates a new statenew state
Impasses may be stackedImpasses may be stacked When one impasse is solved, Soar pops up When one impasse is solved, Soar pops up
to the previous goalto the previous goal
LearningLearning
Learning occurs by chunking the Learning occurs by chunking the conditions and the actions of the conditions and the actions of the impasses that have been resolvedimpasses that have been resolved
Chunks can immediately used in Chunks can immediately used in further problem-solving behaviourfurther problem-solving behaviour
The Switchyard videoThe Switchyard video
ConclusionsConclusions
It may be too "unified"It may be too "unified"– Single learning mechanismSingle learning mechanism– Single knowledge representationSingle knowledge representation– Uniform problem stateUniform problem state
It does not take neuropsychological It does not take neuropsychological evidence into account (cf. ACT-R)evidence into account (cf. ACT-R)
There may be non-symbolic intelligence, There may be non-symbolic intelligence, e.g. neural nets etc not abstractable to the e.g. neural nets etc not abstractable to the symbolic levelsymbolic level
Comparison of ArchitecturesComparison of Architectures
ACT-RACT-R EPICEPIC SOARSOARTypeType HybridHybrid SymbolicSymbolic SymbolicSymbolic
TheoryTheory Rational Rational analysisanalysis
Embedded Embedded cognitioncognition
Problem Problem solvingsolving
BasisBasis Cog. Psy.Cog. Psy. HCIHCI AIAI
LTMLTM Productions; Productions; factsfacts
Productions; Productions; factsfacts
ProductionsProductions
WMWM Goal stackGoal stack Working Working memory; memory; sensory storessensory stores
Working Working memorymemory
LearningLearning YesYes NoNo YesYes
The Role of Models in The Role of Models in DesignDesign
User Models in DesignUser Models in Design
BenchmarkingBenchmarking
Human Virtual MachinesHuman Virtual Machines
Evaluation of conceptsEvaluation of concepts
Comparison of conceptsComparison of concepts
Analytical prototypingAnalytical prototyping
BenchmarkingBenchmarking
What times can users expect to take What times can users expect to take to perform taskto perform task
– Training criteriaTraining criteria– Evaluation criteria (under ISO9241)Evaluation criteria (under ISO9241)– Product comparisonProduct comparison
Human Virtual MachineHuman Virtual Machine
How might the user perform?How might the user perform?
– Make assumptions explicitMake assumptions explicit– Contrast viewsContrast views
Evaluation of ConceptsEvaluation of Concepts
Which design could lead to better Which design could lead to better performance?performance?
– Compare concepts using models prior to Compare concepts using models prior to building prototypebuilding prototype
– Use performance of existing product as Use performance of existing product as benchmarkbenchmark
Reliability of ModelsReliability of Models
Agreement of predictions with Agreement of predictions with observationsobservations
Agreement of predictions by different Agreement of predictions by different analystsanalysts
Agreement of model with theoryAgreement of model with theory
Comparison with TheoryComparison with Theory
Approximation of human information processingApproximation of human information processing
Assumes linear, error-free performanceAssumes linear, error-free performance
Assumes strict following of ‘correct’ procedureAssumes strict following of ‘correct’ procedure
Assumes only way correct procedureAssumes only way correct procedure
Assumes actions can be timedAssumes actions can be timed
KLM ValidityKLM Validity
Predicted values liewithin 20% of observed values
Comparison of KLM predicted Comparison of KLM predicted with times from user trialswith times from user trials
Total time(s)
25
20
15
101 2 3 4 5 6 7
Trial number
CUI: P = 15.84s mean = 15.37sError = 2.9%
GUI: P = 11.05s mean = 8.64sError = 22%
Inter / Intra-rater ReliabilityInter / Intra-rater Reliability
Inter-rater:Inter-rater:– Correlation of several analystsCorrelation of several analysts– = 0.754= 0.754
Intra-rater:Intra-rater:– Correlation for same analysts on several Correlation for same analysts on several
occasionsoccasions– =0.916=0.916
Validity:Validity:– correlation with actual performancecorrelation with actual performance– = 0.769= 0.769
Stanton and Young, 1992
How compare single data How compare single data points?points?
Models typically produce a single Models typically produce a single predictionprediction
How can one value be compared How can one value be compared against a set of data?against a set of data?
How can a null hypothesis be How can a null hypothesis be proved?proved?
Liao and Milgram (1991)Liao and Milgram (1991)
A-D-*sd A-D A-D+*sd A A+D-*sd A+D A+D+*sd D
Defining termsDefining terms
A = Actual values, with observed A = Actual values, with observed standard deviation (sd)standard deviation (sd)
D = Derived valuesD = Derived values = 5% (P < 0.05 to reduce Type I = 5% (P < 0.05 to reduce Type I
error)error) = 20% (P<0.2 for Type II error)= 20% (P<0.2 for Type II error)
Acceptance CriteriaAcceptance Criteria
Accept Ho if: A-D+ Accept Ho if: A-D+ *sd < D< A+D- *sd < D< A+D- *sd*sd
Reject Ho if: D < A-D- Reject Ho if: D < A-D- *sd *sd
Reject Ho if: D > A-D+ Reject Ho if: D > A-D+ *sd *sd
Analytical PrototypingAnalytical Prototyping
Functional analysisFunctional analysis Define features and functionsDefine features and functions Development of design concepts, e.g., sketches and Development of design concepts, e.g., sketches and
storyboardsstoryboards
Scenario-based analysisScenario-based analysis How people pursue defined goalsHow people pursue defined goals State-based descriptionsState-based descriptions
Structural analysisStructural analysis Predictive evaluationPredictive evaluation Testing to destructionTesting to destruction
Analytical PrototypingAnalytical Prototyping
Functional analysisFunctional analysis
Scenario-based analysisScenario-based analysis
Structural analysisStructural analysis
Rewritable RoutinesRewritable Routines
Mental modelsMental models– Imprecise, incomplete, inconsistentImprecise, incomplete, inconsistent
Partial representations of product Partial representations of product and procedure for achieving subgoaland procedure for achieving subgoal
Knowledge recruited in response to Knowledge recruited in response to system imagesystem image
Simple ArchitectureSimple Architecture
Current State
Action to change
machine state
Rewritable Routines
Goal State
Possible States
Relevant State
Next State
Global Prototypical RoutinesGlobal Prototypical Routines
Stereotyped Stimulus-Response Stereotyped Stimulus-Response compatibilitiescompatibilities
Generalisable product knowledgeGeneralisable product knowledge
State-specific RoutinesState-specific Routines
Interpretation of system imageInterpretation of system image– Feature evolutionFeature evolution
Expectation of procedural stepsExpectation of procedural steps
Situated / Opportunistic planningSituated / Opportunistic planning
Describing InteractionDescribing Interaction
State-space diagramsState-space diagrams
Indication of system imageIndication of system image
Indication of user actionIndication of user action
Prediction of performancePrediction of performance
State-space DiagramState-space Diagram
0
Waiting for: Raise lid
Waiting for: Play Mode
Waiting for: Enter
Waiting for: Skip forward
Waiting for: Skip back
Waiting for: Play Waiting for: Stop
Waiting for: Off
Task: Press ‘Play’ Time: 200msError: 0.0004
State 1
• State number•System image•Waiting for…•Transitions
Defining ParametersDefining Parameters
Activity (times)Activity (times) ErrorError P(noviceP(novice))
P(expert)P(expert)
Recall Plan (1380ms)Recall Plan (1380ms) Wrong planWrong plan 0.260.26 0.0030.003
Select (360ms)Select (360ms) Select Select wrong itemwrong item
0.020.02 0.00040.0004
Press (200ms)Press (200ms) Fail to pressFail to press 0.00040.0004 0.00040.0004
Read (180ms)Read (180ms) MisinterpretMisinterpret 0.160.16 0.090.09
Developing ModelsDeveloping ModelsP=0.997P=0.74
P=0.003P=0.26
P=0.9996P=0.9996
P=0.9996P=0.9996
P=0.9996P=0.9996
P=0.0004P=0.0004
P=0.0004P=0.0004
P=0.0004P=0.0004
P=1P=1
P=1P=1
P=1P=1
Recall plan:1380ms
Press play:200ms
Press Playmode:200ms
Wrong plan:1380ms
Cycle through menu:800ms
Switch off:
300ms
Press Enter:0ms
Press Other Key:200ms
Press Playmode:200ms
Press Play:0ms
Start:0ms
ResultsResults
0500
100015002000250030003500400045005000
'novice' t1 t3 t5 'expert'
What is the point?What is the point?
Are these models useful to Are these models useful to designers?designers?
Are these models useful to theorists?Are these models useful to theorists?
Task Models - problemsTask Models - problems
Task models take time to developTask models take time to develop– They may not have high inter-rater They may not have high inter-rater
reliabilityreliability– They cannot deal easily with parallel They cannot deal easily with parallel
taskstasks– They ignore social factorsThey ignore social factors
Task Models - benefitsTask Models - benefits
Models are abstractions – you always Models are abstractions – you always leave something outleave something out
The process of creating a task model The process of creating a task model might outweigh the problemsmight outweigh the problems
Task models highlight task Task models highlight task sequences and can be used to define sequences and can be used to define metricsmetrics
Task Models for TheoristsTask Models for Theorists
Task models are engineering Task models are engineering approximationsapproximations– Do they actually describe how human Do they actually describe how human
information processing works?information processing works?Do they need to?Do they need to?
– Do they describe cognitive operations, Do they describe cognitive operations, or just actions?or just actions?
Some Background ReadingSome Background Reading
Dix, A et al., 1998, Human-Computer Interaction Dix, A et al., 1998, Human-Computer Interaction (chapters 6 and 7) London: Prentice Hall(chapters 6 and 7) London: Prentice Hall
Anderson, J.R., 1983, The Architecture of Cognition, Anderson, J.R., 1983, The Architecture of Cognition, Harvard, MA: Harvard University PressHarvard, MA: Harvard University Press
Card, S.K. et al., 1983, The Psychology of Human-Card, S.K. et al., 1983, The Psychology of Human-Computer Interaction, Hillsdale, NJ: LEAComputer Interaction, Hillsdale, NJ: LEA
Carroll, J., 2003, HCI Models, Theories and Carroll, J., 2003, HCI Models, Theories and Frameworks: towards a multidisciplinary science, Frameworks: towards a multidisciplinary science, (chapters 1, 3, 4, 5) San Francisco, CA: Morgan (chapters 1, 3, 4, 5) San Francisco, CA: Morgan KaufmanKaufman