Embed Size (px)
Citation preview
W H E N D ATA A R E W O R D S : D I G G I N G F O R M E A N I N G I N H U M A N C O M M U N I C AT I O N S
K A R I N V E R S P O O R @ K A R I N V The University of Melbourne
image credit: D Sharon Pruitt
W O R D S A R E E V E R Y W H E R E
simple icons credit: Dan Leech some icons designed by Freepix
A D A M S A N D L E R H A S M A D E S O M E C R A P M O V I E S R E C E N T LY B U T T H E W E D D I N G S I N G E R M A K E S U P F O R T H O S E …
http://redd.it/31zuw9 word cloud made with wordle.net
reddit user speccius 09 April 2015
S E N T I M E N T A N A LY S I S
Subjectivity
neutral: 0.1
polar: 0.9
Polarity
pos: 0.4
neg: 0.6
NEGATIVE
http://text-processing.com/demo/sentiment/ Photo Credit:Eli Ebersole via Compfight .
They poured the wine into the prdsp and drank among friends during the meal.
Example inspired by: http://www.cs.nmsu.edu/ALP/2014/12/semantics-at-scale-when-distributional-semantics-meets-logic-programming/ Photo Credit: BlakJakDavy via Compfight .
N O W O R D I S A N I S L A N D
Photo Credit: Anita via Compfight .
wine
drink
pour
cup
prdsp
θ
They poured the wine into the prdsp and drank among friends during the meal.
V E C T O R S PA C E M O D E L
Photo Credit: madmiked via Compfight .
cup 111
00pour 1
bread 10 0
0wine 1 0
0 10meal
drink 1 01
doc1 doc2 doc3
B A G S O F W O R D S
Mary
John
Janequicker
slower
cup
pour
bread
wine
meal
drink
friends
W O R D S E X P R E S S R E L AT I O N S H I P S
1. Mary is quicker than John.
2. John is quicker than Mary.
3. Mary is slower than John.
4. Jane is quicker than Mary.
Photo Credit: Sangudo via Compfight .
1. Mary is quicker than John.
2. John is quicker than Mary.
1
sent4
1
0
0
1
11Mary 1
slower 10 0
1quicker 1 0
0 00Jane
John 1 11
sent1 sent2 sent3
L E X I C A L A M B I G U I T Y
That bank is covered in flowers.
I have an account at that bank.
The bank is covered in vines.
There was a run on the bank.
Photo Credit: tkw954 via Compfight .
vines 101
11bank 1
flowers 00 0
0account 1 0
0 11money
grass 0 01
doc1 doc2 doc3
Photo Credit: tame_alien via Compfight .
S Y N TA C T I C A M B I G U I T Y
They are hunting dogs.
Police help dog bite victim.
Flying planes can be dangerous.
The man watched the woman with the telescope.
I said I’d be there on Thursday.
Examples courtesy of John Beavers, U Texas and Wikipedia Photo credit: sfkjr via Compfight .
C O N V E R S AT I O N A L I M P L I C AT U R E
Said the elderly woman to her aide,
“Can you roll my wheelchair to the window?”
Said the man to the waiter,
“This coffee is cold!”
Photo Credit: Ondablv via Compfight .
(A looks at dog.)
A: Does your dog bite?
B: No.
(B reaches down to pet the dog.)
(The dog bites him.)
B: I thought you said your dog doesn’t bite!
A: That’s not my dog.
— The Pink Panther
Thanks to John Beavers, U Texas for sharing his linguistic examples. Image credit: http://www.tcm.com/mediaroom/video/288623/Pink-Panther-Strikes-Again-The-Movie-Clip-Does-Your-Dog-Bite-.html
N AT U R A L L A N G U A G E P R O C E S S I N G : Many hard problems, some solutions
• Information Extraction (of Entities, Ontology Concepts/Terms, Relations, Events)
• Word Sense Disambiguation and Sense Induction
• Sentiment Analysis
• Document Summarisation
• Text Classification
• Topic Modeling
• Machine Translation
Image Credit: HikingArtist.com via Compfight .
“This system is designed for physicians to point and click their way through an entire exam quickly and effortlessly.”
— (EMR product review)
April 14, 2007 CHIEF COMPLAINT: Shortness of breath. HISTORY OF PRESENT ILLNESS: This 68-year-old female presents to the emergency department with shortness of breath that has gone on for 4-5 days, progressively getting worse. It comes on with any kind of activity whatsoever. She has had a nonproductive cough. She has not had any chest pain. She has had chills but no fever. EMERGENCY DEPARTMENT COURSE: The patient was admitted. She has had intermittent episodes of severe dyspnea. Lungs were clear. These would mildly respond to breathing treatments and morphine. Her D�dimer was positive. We cannot scan her chest; therefore, a nuclear V/Q scan has been ordered. However, after consultation with Dr. C, it is felt that she is potentially too unstable to go for this. Given the positive D�dimer and her severe dyspnea, we have weighed the risks and benefits of anticoagulation with her heme-positive stools. She states that she has been constipated lately and doing a lot of straining. Given the possibility of a PE, it was felt like anticoagulation was very important at this time period; therefore, she was anticoagulated. The patient will be admitted to the hospital under Dr. C.
April 14, 2007 CHIEF COMPLAINT: Shortness of breath. HISTORY OF PRESENT ILLNESS: This 68-year-old female presents to the emergency department with shortness of breath that has gone on for 4-5 days, progressively getting worse. It comes on with any kind of activity whatsoever. She has had a nonproductive cough. She has not had any chest pain. She has had chills but no fever. EMERGENCY DEPARTMENT COURSE: The patient was admitted. She has had intermittent episodes of severe dyspnea. Lungs were clear. These would mildly respond to breathing treatments and morphine. Her D�dimer was positive. We cannot scan her chest; therefore, a nuclear V/Q scan has been ordered. However, after consultation with Dr. C, it is felt that she is potentially too unstable to go for this. Given the positive D�dimer and her severe dyspnea, we have weighed the risks and benefits of anticoagulation with her heme-positive stools. She states that she has been constipated lately and doing a lot of straining. Given the possibility of a PE, it was felt like anticoagulation was very important at this time period; therefore, she was anticoagulated. The patient will be admitted to the hospital under Dr. C.
April 14, 2007 CHIEF COMPLAINT: Shortness of breath. HISTORY OF PRESENT ILLNESS: This 68-year-old female presents to the emergency department with shortness of breath that has gone on for 4-5 days, progressively getting worse. It comes on with any kind of activity whatsoever. She has had a nonproductive cough. She has not had any chest pain. She has had chills but no fever. EMERGENCY DEPARTMENT COURSE: The patient was admitted. She has had intermittent episodes of severe dyspnea. Lungs were clear. These would mildly respond to breathing treatments and morphine. Her D�dimer was positive. We cannot scan her chest; therefore, a nuclear V/Q scan has been ordered. However, after consultation with Dr. C, it is felt that she is potentially too unstable to go for this. Given the positive D�dimer and her severe dyspnea, we have weighed the risks and benefits of anticoagulation with her heme-positive stools. She states that she has been constipated lately and doing a lot of straining. Given the possibility of a PE, it was felt like anticoagulation was very important at this time period; therefore, she was anticoagulated. The patient will be admitted to the hospital under Dr. C.
O P P O R T U N I T I E S
• Social media analytics
• Predictive analytics from text
• Merging structured and unstructured data
• Anomaly/Fraud detection
Image Credit: HikingArtist.com via Compfight .
Tonnes of Text
Tonnes of Gold
T H E P R O F W A S N O T U N C E R TA I N .
An MIT linguistics professor was lecturing his class.
“In English,” he said, “a double negative forms a positive. However, in some languages, such as Russian, a double negative remains a negative.
But there isn’t a single language, not one, in which a double positive can express a negative.”
Photo Credit: ajschu via Compfight .
A voice from the back of the room piped up:
“Yeah, right.”
Image credit: thems0kitty on DeviantArt
T E X T I S M O R E T H A N J U S T W O R D S
Photo Credit: ElvinW via Compfight .
