Strategies LLCTaxonomy
November 2, 2005 Copyright 2005 Taxonomy Strategies LLC. All rights reserved.
Information Classification in the Workplace
Joseph A. Busch
2Taxonomy Strategies LLC The business of organized information
Agenda
Why information classification is needed What the research says Agricultural economics case study
3Taxonomy Strategies LLC The business of organized information
Information access issues, problems, and concerns
Enormous volumes of information within organizations Diversity of assets
Content and technology
Complex and IT-oriented standards .NET, SOAP, WSDL, etc.
Limited (if any) integration with applications: Search engines Information management applications Back office transaction-based systems Analytical systems …
4Taxonomy Strategies LLC The business of organized information
Finding information should not be about “Feeling Lucky”
5Taxonomy Strategies LLC The business of organized information
Finding information requires multiple approaches
6Taxonomy Strategies LLC The business of organized information
http://www.tesco.com/winestore
Wine shopping by facets
7Taxonomy Strategies LLC The business of organized information
http://www.towerrecords.com
8Taxonomy Strategies LLC The business of organized information
Powered by
http://www.fortunoff.com
9Taxonomy Strategies LLC The business of organized information
Facets on FirstGov.gov site
OrganizationOrganization
Content TypeContent Type
FrequencyFrequency
AudienceAudience
10Taxonomy Strategies LLC The business of organized information
about 3,890,000 results
11Taxonomy Strategies LLC The business of organized information
2,199 results
12Taxonomy Strategies LLC The business of organized information
More than 1,000 relevant
results
13Taxonomy Strategies LLC The business of organized information
14Taxonomy Strategies LLC The business of organized information
15Taxonomy Strategies LLC The business of organized information
16Taxonomy Strategies LLC The business of organized information
Agenda
Why information classification is needed What the research says Agricultural economics case study
17Taxonomy Strategies LLC The business of organized information
Usability research— Taxonomy compared to search results lists
“We found that users preferred a browsing oriented interface for a browsing task, and a direct search interface when they knew precisely what they wanted.”
Marti Hearst (and others)
“The category interface is superior to the list interface in both subjective and objective measures.”
Hao Chen & Susan Dumais
18Taxonomy Strategies LLC The business of organized information
Taxonomy compared to search result lists
0
20
40
60
80
100
120
140
Category List
Me
dia
n S
earc
h T
ime
in
Se
con
ds
In top 20 results
Not in top 20 results
Category is 36% faster
Category is 48% faster
Source: Chen & Dumais
19Taxonomy Strategies LLC The business of organized information
Time saved—Taxonomy compared to search result lists
1 hour per day searching x 36% faster = 22 minutes each day
22 minutes x 250 working days per year = 5500 minutes or 92 hours per year
20Taxonomy Strategies LLC The business of organized information
User interface survey— Which search UI is ‘better’?
Criteria User satisfaction Success completing tasks Confidence in results Fewer dead ends
Methodology Design tasks from specific to
general Time performance Calculate success rates Survey subjective criteria Pay attention to survey
hygiene:– Participant selection– Counterbalancing– T-scores
Source: Yee, Swearingen, Li, & Hearst
21Taxonomy Strategies LLC The business of organized information
User interface survey — Results (1)
Which Interface would you rather use for these tasks?
Google-like Baseline
Faceted Category
Find images of roses 15 16
Find all works from a certain period 2 30
Find pictures by 2 artists in the same media 1 29
…
Overall assessment:Google-like
BaselineFaceted
Category
More useful for your usual tasks 4 28
Easiest to use 8 23
Most flexible 6 24
More likely to result in dead-ends 28 3
Helped you learn more 1 31
Overall preference 2 29
…
Source: Yee, Swearingen, Li, & Hearst
22Taxonomy Strategies LLC The business of organized information
User interface survey — Results (2)
6.06.7
4.7 4.6
5.8 5.56.0
4.0
7.26.3
3.5
7.7 7.4 7.8
4.8
7.6
0123456789
Faceted Category
Google-like Baseline
Source: Yee, Swearingen, Li, & Hearst
23Taxonomy Strategies LLC The business of organized information
Trusted advisers—Taxonomy avoids costs
“The amount of time wasted in futile searching for vital information is enormous, leading to staggering costs …”
Sue Feldman,
Poor classification costs a 10,000 user organization $10M each year—about $1,000 per employee.
Jakob Nielsen, useit.com
24Taxonomy Strategies LLC The business of organized information
Searching
Creating
Commun-icating
Knowledge workers spend up to 2.5 hours each day looking for information …
… But find what they are looking for only 40% of the time.
Source: Kit Sims Taylor
25Taxonomy Strategies LLC The business of organized information
Creating new
content
Recreating existing content
SearchingCommun-icating
25%8%
Knowledge workers spend more time re-creating existing content than creating new content
Source: Kit Sims Taylor (cited by Sue Feldman in her original article)
26Taxonomy Strategies LLC The business of organized information
Agenda
Why information classification is needed What the research says Agricultural economics case study
27Taxonomy Strategies LLC The business of organized information
Task-based test*
15 representative questions were selected Perspective of various organizational units Most frequent website searches Most frequently accessed website content Correct answers to the questions were agreed in advance by team.
15 users were tested Did not work for the organization Represented target audiences
Testers were asked “where would you look for …” “under which facet… Topic, Commodity, or Geography?” Then, “… under which category?” Then, “…under which sub-category?” Tester choices were recorded
Testers were asked to “think aloud” Notes were taken on what they said
Pre- and post questions were asked Tester answers were recorded
* Based on Donna Maurer’s usability
work with the Australian government
28Taxonomy Strategies LLC The business of organized information
Task-based testing: Representative questions
1. How much cotton is imported from China? 2. What are the impacts of “mad cow" disease on U.S. meat production, sales?3. What is the average farm income level in your state?4. How much of our diet comes from fast food?5. How many people receive WIC benefits (Special Supplemental Nutrition
Program for Women, Infants, and Children)?6. How much acreage is planted to genetically engineered corn?7. What is the cost of foodborne illness in the United States?8. What part of food costs go to farmers, retailers?9. Which States produce the most tobacco?10. What percentage of farms in the United States are small farms?11. What are the costs and benefits associated with providing more traceability in
the U.S. food supply?12. How many people in America don’t get enough to eat?13. What is behind the trade balance (surplus or deficit) in agricultural goods?14. What is the extent of conservation compliance? How does that impact
farmer's decisions?15. What are the impacts of foreign trade restrictions on U.S. farmers, U.S. food
prices?
29Taxonomy Strategies LLC The business of organized information
Task-based testing: Closed card sorting
3. What is the average farm income level in
your state?
1. Topics2. Commodities3. Geographic Coverage
1. Topics1.1 Agricultural Economy1.2 Agriculture-Related
Policy1.3 Diet, Health & Safety1.4 Farm Financial
Conditions1.5 Farm Practices &
Management1.6 Food & Agricultural
Industries1.7 Food & Nutrition
Assistance1.8 Natural Resources &
Environment1.9 Rural Economy1.10 Trade & International
Markets
1.4 Farm Financial Conditions
1.4.1 Costs of Production1.4.2 Commodity Outlook1.4.3 Farm Financial
Management & Performance
1.4.4 Farm Income1.4.5 Farm Household
Financial Well-being1.4.6 Lenders & Financial
Markets1.4.7 Taxes
30Taxonomy Strategies LLC The business of organized information
Task based testing: Card sort analysis
Find-it Tasks User 1 User 2 User 3 User 4 User 5
1. Cotton Cotton Cotton Asia Cotton Cotton
2. Mad cow Cattle Food Safety Cattle Cattle Cattle
3. Farm income Farm Income Farm Income US States Farm Income Farm Income
4. Fast foodFood Consumption
Diet Quality & Nutrition
Food Expenditures
Diet Quality & Nutrition
Diet Quality & Nutrition
5. WIC WIC Program WIC Program WIC Program WIC Program WIC Program
6. GE Corn Corn Corn Corn Corn Corn
7. Foodborne illnessFoodborne Disease
Foodborne Disease
Consumer Food Safety
Foodborne Disease
Foodborne Disease
8. Food costs Food Prices Market Structure Market AnalysisFood Expenditures
Retailing & Wholesaling
9. Tobacco Tobacco Tobacco Tobacco Tobacco Tobacco
10. Small Farms Farm Structure Farm Structure Farm Structure Farm Structure Farm Structure
11. Traceability Food System Labeling PolicyFood Safety Innovations
Food Safety Policy Food Prices
12. Hunger Food Security Food Security Food Security Food Security Food Security
13. Trade balanceCommodity Trade
Trade & Intl Markets
Commodity Trade Market Analysis
Commodity Trade
14. ConservationsCropping Practices
Conservation Policy
Conservation Policy
Conservation Policy
Conservation Policy
15. Trade restrictions Trade PolicyFood Safety & Trade WTO Market Analysis
Commodity Trade
31Taxonomy Strategies LLC The business of organized information
Task based testing: Card sort results
In 80% of the trials users looked for information under the categories that we expected them to look for it.
Breaking-up topics into facets makes it easier to find information, especially information related to commodities.
32Taxonomy Strategies LLC The business of organized information
Task based testing: Card sort results
Test Questions%
Correct%
Agree
1. Cotton 91% 82%
2. Mad cow 73% 64%
3. Farm income 100% 55%
4. Fast food 91% 73%
5. WIC 100% 100%
6. GE corn 100% 100%
7. Foodborne illness 82% 82%
8. Food costs 55% 27%
9. Tobacco 100% 100%
10. Small farms 91% 91%
11. Traceability 36% 18%
12. Hunger 100% 73%
13. Trade balance 36% 64%
14. Conservation 91% 91%
15. Trade restrictions 55% 36%
Possible change required.
Change required.
Possible error in categorization of this question because 64% thought the answer should be “Commodity Trade.”
On these trials, only 50% looked in the right category, & only 27-36% agreed on the category.
Policy of “Traceability” needs to be clarified. Use quasi-synonyms.
33Taxonomy Strategies LLC The business of organized information
Task-based testing: User satisfaction survey
Was it easy, medium or difficult to choose the appropriate Topic?
– Easy – Medium– Difficult
Was it easy, medium or difficult to choose the appropriate Commodity?
– Easy – Medium– Difficult
Was it easy, medium or difficult to choose the appropriate Geographic Coverage?
– Easy – Medium– Difficult
34Taxonomy Strategies LLC The business of organized information
User satisfaction survey: Results
-
0.50
1.00
1.50
2.00
Topic Commodity Geography
Facet
Ea
sy
-
->
Dif
fic
ult
EasierMore Difficult
35Taxonomy Strategies LLC The business of organized information
Summary
1. Classifications and classification-like schemes are being used to facilitate information seeking in the workplace.
2. Users take advantage (and prefer) this type of scheme (faceted navigation) when it is made available in the user interface.
3. Hierarchical navigation is guided by the UI.
4. It is best combined with keyword searching. E.g., keyword search followed by faceted navigation of results.
5. What are the challenges they face in navigating classificatory structures?
6. What are the differences between their use of traditional classification schemes such as the Dewey Decimal Classification and navigation of newly developed online directories?
Strategies LLCTaxonomy
November 2, 2005 Copyright 2005 Taxonomy Strategies LLC. All rights reserved.
Questions
Joseph A. Busch+ 415-377-7912
[email protected]://ww.taxonomystrategies.com