Upload
oswald-jacobs
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Modeling expert memory search, Modeling expert memory search, knowledge access, and decision knowledge access, and decision making:making: A model of crossword puzzle play A model of crossword puzzle play
Shane T. Mueller & Kejkaew Thanasuan
Department of Cognitive and Learning Sciences
Michigan Technological University
1
Why Study crossword?Why Study crossword? Naturalistic approach to
studying decision making and knowledge-based problem solving
Experts in this task have highly-developed memory encoding and retrieval skills
The knowledge space is well-characterized
Crossword ExpertiseCrossword Expertise
3
Top players are 2-5x faster than good casual players, who are 10x faster than novices
Crossword ExpertiseCrossword Expertise4
Expert Novice
GoalGoal
To understand crossword skill by developing a computational model of crossword solving.
Rely heavily on constraints from a natural corpus.
Understand how 'cues' provide both memory access and constraint
Schematic Model ofSchematic Model ofKnowledge accessKnowledge access
6
Memory search and Access Memory search and Access
7
Any feature in the clue (word stem or hint) will activate a set of answer words according to the relative probability.
Probabilities of multiple features can combine to form activation distribution.
Orthographic RepresentationOrthographic Representation
8
Each answer has associations from orthographic units in the clues that lead to that answer.
Currently, lexical units include:
letters
adjacent letter pairs
length units Based on Mueller & Thanasuan (2014,
JMP) model of word-stem completion.
9
Used (250K token/4M word) Ginsberg database (described later)
Each word coded for 26 letter-features + 27^2 letter-pair features
A bank of 10 (logarithmically-defined) length features.
Orthographic CorpusOrthographic Corpus
250K
26 729 10
Orthographic Search through Orthographic Search through activation spaceactivation space
TU-K--
TUSK TURK TUSKS TUCK TUCKS TURKS TURKEY TIMBUKTU TUCKIN TUTU
---K--
SKATE KOREA OSAKA ANKLE KNEE KOALA KNEES SKEET ASKED AKRON
0.05923629 0.04718652 0.03634697 0.02930575 0.025934 0.0205008 0.01378494 0.01108984 0.01044018 0.01013701
0.0009366936 0.0009117806 0.0009024288 0.0008888983 0.0008640101 0.0008119471 0.0007999176 0.0007830753 0.0007498722 0.0007384729
10
Semantic RepresentationSemantic Representation
11
Each answer has associations from lexical units in the clues that lead to that answer.
Currently, lexical units include:
word
word pairs Association strength increases with
each experience.
No effort made to form semantic associations based on contextual semantics (e.g., LSA) or linguistic analysis
12
Ginsberg's Crossword Clue database 4,000,000+ clues from 50K
puzzles 250K unique answers (rows) With stemming by Celex 2.5,
110K clue words (columns) 550K clue word pair units
(columns) Very sparse matrix (.000024
used)
Semantic Knowledge BaseSemantic Knowledge Base
12
250K
110K 550K word-pairs
Semantic ActivationsSemantic Activations“Thanksgiving Bird”
CARVEPIE BASTE EMU TOM TURKEY TURKEYTROT IBIS EGRET MEALPIES WOODYWOODPECKER
ROC EMUS LOON
“Flop”EDSEL BOMB DUD EDSELS ISHTAR KER SANDAL THONG FIASCO ANEGG EARED UTURN HIT SMASH NERD....TURKEY (25)
0.00760 0.00622 0.00581 0.00551 0.005390.02554 0.02339 0.01639 0.00829 0.00822 0.00539 0.00539 0.00502 0.00470 0.00451
0.2300.086 0.072 0.063 0.057 0.032 0.017 0.017 0.015 0.013 0.013 0.0130.011 0.011 0.010.....0.005713
Memory SearchMemory Search
Any clue can provide both semantic and orthographic cues.
Top options get sampled and evaluated against orthographic constraints and (for orthographic route) semantic cues.
Semantic retrievals must get 'recovered' ala SAM, based on their strength.
Parameters control number of candidates that can be checked and recovery probability,
Hypothesis—search goes on in each domain separately and independently.
Single-route models Single-route models
Crossword ExperimentCrossword Experiment One clue at a time. Stimuli: 56 answers, clued with easy/difficult
semantic and easy/difficult orthographic. Difficult: 1 letter; Easy: 3 blanks Clue difficulty selected subjectively 4-11 letter words Conducted in lab (novice) and ACPT, and
on-line (experts)16
Implemented in the Psychology Experiment Building Language (PEBL). See http://pebl.sourceforge.net
17
Semantic ModelSemantic Model Evaluation Evaluation
43 clues For 24, model and data agreed
with pre-determined difficulty
For 7, model and data agreed, opposite of pre-specified difficulty.
For 3, model agreed with pre-specified difficulty but not data
For 9, humans followed prespecified difficulty, but model did not.
Some failures of Semantic ModelSome failures of Semantic Model
ANSWER BANANA
BRAVADO
FREEZERS
JEWELS
PINECONE
SHOELACE
EASY CLUE
"Good source of potassium"
Swaggering show of courage"
"Food storage places”
"Precious stones"
"Fir tree fruit"
A child's sneaker usually lacks this"
HARD CLUE
"split ingredient”
"Swagger"
"Spoilage slowers"
"Safe deposit"
“Wreath adornment"
“It goes over the tongue"
Experiment ResultsExperiment Results
20
Comparison to noviceComparison to novice
21
Expert resultsExpert results
22
Each uses a generate-and- check scheme
Single route models produce no difficulty effect on non-cued route
Single Route ModelsSingle Route Models
23
Qualitative InterpretationQualitative Interpretation
For Experts, we see both semantic and orthographic difficulty.
Semantic difficulty diminished as more orthographic letters are given.
Consistent with cascading or parallel models search along each rout.
Additional RT data may be needed to understand parallel-serial architecture
SummarySummary Difficulty effects can be produced (usually) by relying
SOLELY on statistics of the environment (no free parameters).
'Cues' provide both memory access and constraint.
Experts have better semantic retrieval which gets them started in the puzzle.
Also, improved orthographic completion that eliminates semantic difficulty effects.
In expert memory, Cues provide both activation and constraint.
25