Findability and Usability - CS4HS@Unitec workshop 2015cs4hs.unitec.ac.nz/2013/pdf/divoli-findability_and_usability-cs4hs... · Findability and Usability Lessons Learnt from Text Analytics

Findability and Usability Lessons Learnt from Text Analytics

Anna Divoli

@annadivoli

Pingar Research

CS4HS 2013 @ Unitec

twitter, July 2006Google, September 1998

http://www.laurencecaro.co.uk/other/what-the-worlds-biggest-websites-looked-like-at-launch/

http://www.laurencecaro.co.uk/other/what-the-worlds-biggest-websites-looked-like-at-launch/

Yahoo!, March 1995

Facebook, February 2004

http://www.whoisandrewwee.com/facebook/the-new-facebook-hot-or-not/

Often there is resistance!

Anna Divoli CS4HS 2013 @ Unitec

http://imgur.com/gallery/U8C1r

Users adapt & evolve as well!


15

• Familiar Interfaces

• User Centered Design & Usability

• Who am I & who is Pingar

• Text analytics in a nutshell

• Our usability studies & findings• 1. Search interface features: Biomedical Scientists

• 2. Study existing popular systems

• 3. Mock ups of specific features

• Take away message

Outline










Outline


http://blog.lib.umn.edu/torre107/si/pics/superficialintelligence14.jpg

Usability testing is important! Needs to be done right!


User Centered Design

Design

Prototype

User Feedback

Analysis


http://headrush.typepad.com/photos/uncategorized/focusonusersnotcompetitors.jpghttp://headrush.typepad.com/creating_passionate_users/2006/07/

User Centered Design – What should companies focus on!


FOCUS

Users Competitors

• Eye tracking

• Mouse clicks

• Tasks (goal & time)

• Questionnaires

• Observation

• Talk aloud

• Brain scanners

• ….

Usability testing tools

Measurements & Preferences on:

• Aesthetics

• Craftmanship

• Technical performance

• Ergonomics

• Usability


http://gazehawk.com/blog/

Eye tracking










Outline


Who am I?


UKBSc (1999) Biomedical Sciences.

MSc (2001) Biosystems & Informatics.

PhD (2006) Biomedical Text Mining:- Sentence extraction- Automatic protein family database annotation- Document clustering and visualization

USA (2006-2011)Postdoctoral Academic Research:- Text analytics - Information retrieval- User search interfaces- Usability research- Knowledge acquisition- Expert opinions analysis

NZ (2011- )Pingar Research:- Entity Extraction- Automatic Taxonomy Generation- Taxonomy Editor (UI)- Friendly UIs- Usability & UX

Who is Pingar?


A small global company that uses state of the art text analytics to offer solutions!

We analyse unstructured text!

Entity ExtractionPeople, Organizations, LocationsDates, Money amounts, Email addresses…+ Custom Entities

Keyword Extraction

Taxonomy Term ExtractionDomain specific terminology (can plug in any taxonomy/ontology)

SummarizationSelect length and focus

Redaction & SanitizationEnsure anonymization of sensitive information.

In English, German, French, SpanishDutch, Chinese, Japanese









Outline


researchA brief Text Analytics Overview – just Entity Recognition basics…


S. Arlington initiated partnership discussions during his visit to Eureka’s Ltd offices last month.

John Smith went to Washington to see the Smithsonian and also met up with Virginia for a coffee.

Find and classify names…

Information Extraction – Named Entity Recognition


S. Arlington initiated partnership discussions during his visit to Eureka’s Ltdoffices last month.


Find and classify names…





Find and classify names… PeopleLocationsOrganizations

Methods: lexicon-based (gazeteers)grammar-based (rule-based)statistical models (machine learning)hybrids





Find and classify names… People LocationsOrganizations

Methods: lexicon-based (gazeteers)grammar-based (rule-based)

✓ statistical models (machine learning)✓ hybrids





Find and classify names… PeopleLocationsOrganizations

Who? Where?

When?

Dates





Word Sense Disambiguation: identifying which sense/meaning of a word is used in a sentence, when the word has multiple meanings.

Text normalization: transforming text into a single canonical form that it might not have had before.

Disambiguation & Normalization:Word Sense Disambiguation & Text Normalization



Sam Arlington initiated partnership discussions during his visit to Eurekaoffices in September.


J. Smith went to Washington DC to see the Smithsonian Institute and also met up with Virginia Peterson for a coffee.

Word Sense Disambiguation & Text Normalization



Sam Arlington initiated partnership discussions during his visit to Eurekaoffices in September.


J. Smith went to Washington DC to see the Smithsonian Institute and also met up with Virginia Peterson for a coffee.

Word Sense Disambiguation & Text Normalization


What?



Fact & Relationship extraction


What?


John Smith went to Washington to see the Smithsonian and also met up withVirginia for a coffee.

Fact & Relationship extraction


How? Why? How do we feel about it?



Deeper knowledge & Sentiment


How? Why? How do we feel about it?



S. Arlington visited the Eureka’s Ltd offices last month to initiate partnership discussions.

John Smith was delighted to go to Washington to see the Smithsonian and also met up with Virginia for a coffee.

Deeper knowledge & Sentiment


S. Arlington did not initiate partnership discussions during his visit to Eureka’s Ltd offices last month.

John Smith went to Washington to see the Smithsonian and almost also met up with Virginia for a coffee.

Negation… Hedging… Conditional phrases…

Also:

Name boundaries… Disambiguation… Normalization…

Common problems


Eye drops off shelf.

Include your children when baking cookies.

Turn right here.

John saw the man on the mountain with a telescope.

He gave her cat food.

They are hunting dogs.

Human language!


Looking for: interactions between SAF and viral LTR elements

(SAF is a transcription factor, LTR stands for ‘long terminal repeat’)

Gene names:tinman, lilliputian, dreadlocks, lush, cheap date, methuselah, Van Gogh, maggie, brainiac, grim, reaper, cleopatra, swiss cheese, ken and barbie, kenny, out cold, lava lamp, hamlet, sonic hedgehog, werewolf, half pint, fucK, drop dead, chardonnay, agnostic, I’m not dead yet…

Example: Biology…


Knowledge management - Extracting metadata

Search engines

Question Answering

Text mining / knowledge discovery

Analytics & trend analysis

Applications










Outline


Specialized domain study:

1. Search interface features: Biomedical Scientists (HCIR)

Anna Divoli and Alyona Medelyan Search interface feature evaluation in biosciences, HCIR 2011, Google, Mountain View, CA

UI preferred features studies:

2. Study existing popular systems (EuroHCIR)

Matthew Pike, Max L. Wilson, Anna Divoli and Alyona MedelyanCUES: Cognitive Usability Evaluation System, EuroHCIR 2012, Nijmegen, Netherlands

3. Mock ups of specific features (survey)

Our User Studies


1. Search interface features: Biomedical Scientists


Anna Divoli and Alyona Medelyan, Search interface feature evaluation in biosciences, HCIR 2011, Google, Mountain View, CA, USA

Facets – favorite feature for search systems


animal models huntington disease

Facets (in search systems)


Most liked Least liked

animal models huntington disease

Bio-Facets


• Faceted search is the most important stand alone feature in a search interface for bioscientists.

• Few, query-oriented facets presented as checkboxes work best.

• Overly simple aesthetics, although not desirable, do not hurt overall UI score.

• Complex aesthetics turn users away from the systems.

• Bioscientists prefer tools that help them narrow their search, not expand it.

• For generic search: doc-based facets. For domain-specific search: query-based facets.

Facets as search features for biomedical scientists: Findings


Autocomplete

Search expansions★

Facetted refinement

Related searches

Results preview★

+ positive comments - negative comments italics comments on aesthetics

same ranking for both baseline & own query ★ not many systems tested so no low rank ratings

- unhelpful symbols - too complex

- too much info

- hard to read

Semedico

- no diversity - too complex

+ highlighting

- hard to read

NextBio

- too general + font color & size

- unclear presentation

- noisy, many symbols

GoPubMed

+ relevant suggestions + good coverage

- blue font color

- small font size

PubMed

+ diff types of info + simple

+ highlighting

Bing

+ “review” suggestion +/- simple

+ highlighting

+ used to

Google

- too general - not useful

+ overall look

- color

Semedico

+ good functionality - specialized

- unclear functionality

- too complex

PubMed

+ useful - redundancies

+ simple & clean

+ less options

Pingar

+ useful categories - slow functionality

- too complex/busy

- too many colors

Semedico

+ “reviews” category - limited functional.

- poor design

PubMed

+ quick paper access + simple

+ vertical list

- nothing special

Solr

+ “top terms” useful - too many symbols

- too busy

- colors

GoPubMed

+ useful categories + simple

+ vertical list

- not special, colorless

Pingar DB

+ useful categories + simple

+ vertical list

- not special, colorless

Pingar QB

- not scientific + colors

- too small

- too busy

Bing

+ relevant - poor context

- no variety

PubMed

- limited options + clickable

+ font size

+ few options

Pingar

+ good suggestions - redundancies

+ font color /blue links

- too busy

Google

+ specific keywords - snippets lack context

+color

+ font color

Solr

+ helpful keywords + mouseover

- pale

- font size & style

Pingar

positive neutral negative

1 participant

br: browsing ff: fact finding ig: information gathering

br ff ig

br ff ig

br ff ig

br ff ig

br ff ig

Legend

Ranked Last Ranked First

• Useful categories• Simple• Vertical list

• Too complex/busy• Too many colors• Poor design• Limited functionality • Too many symbols• Not special/ Colorless

Facets as search feature: likes & dislikes










Outline


Exploring UI features - Systems Tested: Yippy, Carrot, MeSH, ESD








Boston Oct 2012

A

A B C D E F

B

C

D

E

F

A B C D E F A B C D E F A B C D E F A B C D E F


• Menu highlighting• Hierarchical folder layout• Expand hierarchy with “+” and “–”• Dual view (tree on left, results on right)• Ability to change visualisations of taxonomy• Search function is important• Familiar interface with folders

• Too simple or too much writing - would be nice to have color• Lots of scrolling • Dots in carrot circle – confusing• Double click on foam tree is unintuitive• Too broad taxonomies

Exploring UI features (Yippy, Carrot, MeSH, ESD): likes & dislikes










Outline


60.0%26-40

12.7%41-60

0%61 or older

27.3%25 or younger

Age:

52.7%College/University

43.6%Graduate School

3.6%High School

Highest level of education:

47.3%Yes, but very little

21.8%Yes

30.9%No

Do you have experience using taxonomies?

47.3%Very

47.3%Second nature

5.5%Somewhat

How comfortable you are with computers?

Taxonomy UI preferences: The 51 participants


44.2%popularity (A)

42.3%alphabetically (B)

13.5%no preference

Concept sorting


42.3%A

51.9%B

5.8%no preference

Displaying Counts


72.5%in frames (A)

23.5%with labels (B)

3.9%no preference

Using Labels


47.1%A

37.3%B

15.7%no preference

Plus/minus signs or arrows


11.8%B

70.6%C

3.9%no preference

13.7%A

Search Results Display


74.5%partial

64.7%hidden

2.0%no preference

Search Functionality


Outcome: User-friendly Taxonomy Editor


Outcome: User-friendly Taxonomy Editor


Lessons Learnt


Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.

Antoine de Saint-Exupery, French writer (1900 - 1944)

• Follow basic design principles: alignments, right use of colors, etc

• Avoid clutter! Go for clean and simple always!

• Aim for intuitive User Interfaces.Do not make users think!

• You will never please everybody! Choose the users and use cases you plan to support.

Ignas Kukenys

PhD (2010) in Computer Vision, University of Otago

Previously: NextWindow

Expert in: computer vision, machine learning, classification methods

Alyona Medelyan

PhD (2009) in Natural Language Processing, University of Waikato

Previously: Google, European Media Lab

Expert in: keyword and entity extraction, Wikipedia mining

Ralf Heese.

Diploma in Computer Science, PhD candidate, Humboldt-UniversitätBerlin.Previously: Humboldt-Universität Berlin, Freie Universität Berlin

Expert in: semantic databases, semantic technologies

Andy Chao

BSc (2011) in Computer and Information Sciences, AUT

Hayden Curran

Computer Science, University of Waikato (2008-2010)

Collaborators

Annika Hinze Ian Witten Max Wilson Jeen Broekstra

Alumni

Matthew Pike Steve Manion David Milne Jose Chen Anna Huang

Anna Divoli.

PhD (2006) in Biomedical Text Mining, University of Manchester.Previously: University of Chicago, UC Berkeley, Inpharmatica

Expert in: biomedical text mining, text analytics, user search interfaces, usability

Waikato Uni Waikato Uni Nottingham Uni Rivuli Development

Nottingham Uni PhDc Canterbury Uni PhDc CSIRO postdoc NIH fellow Yangtze Uni Lecturer

Acknowledgements: Pingar Research Team

And all 65+ anonymous

studies participants!

http://farm5.static.flickr.com/4087/5019944289_5e2f0637c9.jpg

Take away message: Involve the user!


Documents

Findability and Usability - CS4HS@Unitec workshop 2015cs4hs.unitec.ac.nz/2013/pdf/divoli-findability_and_usability-cs4hs... · Findability and Usability Lessons Learnt from Text Analytics