Embed Size (px)
Citation preview
Who Uses the Online Tobacco Industry Documents?
Martha Michel1,2, M.S., Ph.D. Lisa Bero1,2, Ph.D.1Graduate Group in Biological and Medical Informatics, UCSF2Center for Tobacco Control Research and Education, UCSF
What are the Tobacco Industry Documents?
As a result of the Master Settlement, millions of internal tobacco industry documents were released onto the Internet (legacy.library.ucsf.edu)The documents contain memos, scientific reports, faxes, emails, budgets, etc…The documents include information about scientific research, manufacturing, marketing, advertising and sales of cigarettes, and more
Example of an Internal Tobacco Industry Document
Document CollectionsLegacy document depository at UCSF
5 million documents About 32 million pages and growing1.5 terabytes
Guilford document depository 8 million British American Tobacco documentsAbout 32-40 million pagesUCSF has 13,000 documents which have been manually indexed.
Industry websites – PM, Lorillard, B&W, RJROther collections – Tobacco Documents Online, CDC tobacco industry documents
Difficulties of searching the documents
No OCR available for searching the full textVariations in spelling and problems when names suddenly changeDuplicatesVast quantities of informationNo or varied indexingUnknown recall and low precisionMalone RE, Balbach ED. Tobacco industry documents: treasure trove or quagmire?
Tobacco Control 2000;9(3):334-8.
Prior Studies of Who Searches
Different types of groups used the paper documents depositories (i.e. lawyers, government officials, researchers, tobacco control advocates, health related fields).We still don’t know who uses the electronic documents or why they search…We are currently conducting an online survey of the UCSF Legacy website to examine the use and barriers to searching the existing websites.
Aim 1: Conduct Online Survey
Purpose of survey:Who uses the documents – (demographics)Purposes for which documents are usedBarriers to searching the documents and Suggestions for improving the archives
MethodsDeveloped and designed survey using Web SurveyorConducted pilot test of survey - N=14Launched surveys in November 20022 surveys – one on TCA, one on Legacy
Tobacco Control Archives (n=50)Legacy Tobacco Control Documents Collection
(n=22)
Results from Tobacco Control Archives Survey
(n=50)
17 United States of America4 United Kingdom of Great Britain and Northern Ireland3 Canada1 Bangladesh1 Brazil1 Republic of Korea1 Spain1 Bulgaria1 Malaysia1 Zimbabw e1 Argentina1 Angola
What country do you live in?
51.5 %
12.1 %
9.1 %3.0 % 3.0 % 3.0 %
3.0 %
3.0 %
3.0 %
3.0 %
3.0 %
3.0 %
Who Uses the documents?
3 New York2 California2 Ohio2 Minnesota1 Virginia1 Missouri1 Indiana1 Kentucky1 Pennsylvania1 Oklahoma1 Montana1 Tennessee1 Colorado
If you live in the US, what state do you live in?
16.7 %
11.1 %
11.1 %
11.1 %
5.6 %
5.6 %
5.6 %
5.6 %5.6 %
5.6 %
5.6 %
5.6 %
5.6 %
Who Uses the documents?
17 Student5 Other2 Librarian2 School employee (K-12)2 Industry2 Community not-for-profit organization2 University or college employee1 Tobacco control advocate1 Law -related
What is your occupation?
50.0 %
14.7 %
5.9 %5.9 % 5.9 %
5.9 %
5.9 %
2.9 %
2.9 %
Who Uses the documents?
21 Other21 Home8 Work4 Public Library
Where do you search the Tobacco Control Archives? (Please check all that apply)
Other Home Work Public Library
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
42.0 % 42.0 %
16.0 %
8.0 %
17 Don't Know8 Cable Modem / DSL6 28.8K to 56K Modem5 T-1 or faster2 Other2 14.4K Modem or slow er1 ISDN
In the location where you access the Tobacco Control Archives the most frequently, what is your primary type of Internet connection?
41.5 %
19.5 %
14.6 % 12.2 %
4.9 %
4.9 %
2.4 %
26 First visit7 Other3 Weekly2 Daily1 Less than once a month1 Monthly
In the past six months, how often have you visited the Tobacco Control Archives?
65.0 %
17.5 %
7.5 %
5.0 %
2.5 %
2.5 %
15 Search engine12 Link from another w ebsite12 Other4 Citation in journal, book or new spaper article4 Colleague/friend told me about it3 Heard about it at a professional conference
How did you find the Tobacco Control Archives? (Please check all that apply)
Search engine Other Colleague/friend told me about it
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
30.0 %
24.0 % 24.0 %
8.0 % 8.0 %
6.0 %
13 Don't know any other place to f ind tobacco industry documents on-line12 Search capabilities11 Other10 Ease of use6 Indexes and abstracts are helpful resources6 Speed of searching5 Most complete tobacco documents resource4 Stability of w ebsite1 Familiarity w ith the interface
Why do you use the Tobacco Control Archives? (Please check all that apply)
Don't know any other place to f ind tobacco industry documents on-line Stability of w ebsite
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
26.0 %
24.0 %
22.0 %
20.0 %
12.0 % 12.0 %
10.0 %
8.0 %
2.0 %
DailyWeeklyMonthlySeldomNeverFirst time
How frequently do you use each of the following Tobacco Control Archives collections?
Legacy Tobacco Documents Library British-American Tobacco Company Tobacco Litigation Documents
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
2.9 % 2.9 % 2.6 % 2.8 %
5.9 %
2.9 %
8.6 % 8.6 %
10.5 %
8.3 % 8.8 %
11.4 %
5.7 % 5.7 % 5.3 %
8.3 %
2.9 % 2.9 %
5.7 % 5.7 % 5.3 % 5.6 %
0.0 % 0.0 %
17.1 %
25.7 %
21.1 %
27.8 %
32.4 %
22.9 %
60.0 %
51.4 %
55.3 %
47.2 % 50.0 %
60.0 %
21 First time6 Never3 Weekly2 Seldom2 Monthly1 Daily
How frequently do you use each of the following Tobacco Control Archives collections? - Legacy Tobacco Documents Library
60.0 %
17.1 % 8.6 %
5.7 %
5.7 %
2.9 %
17 First time10 Never3 Monthly3 Weekly2 Seldom1 Daily
How frequently do you use each of the following Tobacco Control Archives collections? - British-American Tobacco Company
47.2 %
27.8 %
8.3 %
8.3 %
5.6 %
2.8 %
YesNoNot SureWasn't aw are of this option
Some of the collections in the Tobacco Control Archives have a browse option. Do you use the browse option?
Joe Camel Campaign: Mangini v. R.J. Reynolds Tobacco Company Brow n and Williamson Collection
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
13.5 % 13.5 %
11.4 %
16.2 %
18.9 %
17.1 %
21.6 %
18.9 %
22.9 %
48.6 % 48.6 %
48.6 %
22 None of the above6 Center for Disease Control documents collection5 Other4 Tobacco industry w ebsites4 Tobacco Documents Online
Please check other websites you use to search the tobacco industry documents:
None of the above Other Tobacco Documents Online
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
44.0 %
12.0 %
10.0 %
8.0 % 8.0 %
30 I haven't had any problems2 Site w as dow n2 Other2 Slow connection2 Can't f ind documents1 Search capabilities not f lexible
Have you had any problems using the website? (Please check all that apply)
I haven't had any problems Other Slow connection Search capabilities not f lexible
32
30
28
26
24
22
20
18
16
14
12
10
8
6
4
2
0
60.0 %
4.0 % 4.0 % 4.0 % 4.0 % 2.0 %
10 Keyw ord searches10 Brow se collection9 Searching full text of the documents9 Search titles only7 Other
What are the most useful features of the Tobacco Control Archives? (Please check all that apply)
Keyw ord searches Brow se collection Search titles only Other
11
10
9
8
7
6
5
4
3
2
1
0
20.0 % 20.0 %
18.0 % 18.0 %
14.0 %
11 Brow se collection6 Other6 Search by keyw ords6 Searching titles
What features of the website do you find most difficult to use? (Please check all that apply)
Brow se collection Other Search by keyw ords Searching titles
12
11
10
9
8
7
6
5
4
3
2
1
0
22.0 %
12.0 % 12.0 % 12.0 %
Text under “other”I would like more structure on how to work the music sight. (4)some stuffits okaytelling about schoolsdirect assistancelinks to student lead orgainization programs against tobaccomore categorizationI found your site useful. I had to fill out a worksheet from Health class and move along wite the website, but there a few things I could not find. Maybe it was the worksheet not the site, but overall your site helped me out. Thank youlove itWhy do I have do this survey its slowing me down?This site is cool I thinkThe site is wonderful and very usefull. I want to congratulate the authors for the wonderful job.I´d like more if smuggling was in better view.
11 Theme11 Document title5 Search terms4 Date3 Named organizations, corporate authors, recipients2 Other2 Document type2 Named persons, authors, recipients
How do you organize the documents you search online?
Theme Search terms Date Other
12
11
10
9
8
7
6
5
4
3
2
1
0
22.0 % 22.0 %
10.0 %
8.0 %
6.0 %
4.0 % 4.0 % 4.0 %
Results from Legacy Survey (n=22)
12 Academic research10 Public health advocacy5 Personal interest4 Other3 Advertising campaign3 Media story3 Teaching tool2 Litigation or legal research2 Testimony at public hearing
What is your purpose for searching the Legacy Tobacco Documents Library? (Please check all that apply)
Academic research Personal interest Advertising campaign Teaching tool Testimony at public hearing
13
12
11
10
9
8
7
6
5
4
3
2
1
0
54.5 %
45.5 %
22.7 %
18.2 %
13.6 % 13.6 % 13.6 %
9.1 % 9.1 %
15 Work13 Home1 Other1 Public Library
Where do you search the Legacy Tobacco Documents Library? (Please check all that apply)
Work Home Other Public Library
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
68.2 %
59.1 %
4.5 % 4.5 %
7 Weekly7 Daily3 First visit2 Less than once a month2 Monthly
In the past six months, how often have you visited the Legacy Tobacco Documents Library?
33.3 %
33.3 %
14.3 %
9.5 %
9.5 %
12 Stability of w ebsite11 Most complete tobacco documents resource10 Search capabilities10 Ease of use8 Speed of searching5 Indexes and abstracts are helpful resources4 Other4 Familiarity w ith the interface3 Don't know any other place to f ind tobacco industry documents on-line
Why do you use the Legacy Tobacco Documents Library? (Please check all that apply)
Stability of w ebsite Ease of use Other
13
12
11
10
9
8
7
6
5
4
3
2
1
0
54.5 %
50.0 %
45.5 % 45.5 %
36.4 %
22.7 %
18.2 % 18.2 %
13.6 %
16 Searching different tobacco industry collections at the same time 8 Advanced search features 6 Brow se search terms4 Bookbag feature2 Other2 Recording search history
What are the most useful features of the Legacy Tobacco Documents Library? (Please check all that apply)
Searching different tobacco industry collections at the same time Other
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
72.7 %
36.4 %
27.3 %
18.2 %
9.1 % 9.1 %
8 Search capabilities not f lexible6 Other6 I haven't had any problems4 Can't f ind documents2 Retrieved too many documents1 Slow connection
Have you had any problems using the website? (Please check all that apply)
Search capabilities not f lexible Other Can't f ind documents Slow connection
8
7
6
5
4
3
2
1
0
36.4 %
27.3 % 27.3 %
18.2 %
9.1 %
4.5 %
14 Tobacco Industry w ebsites13 Tobacco Documents Online5 Tobacco Control Archives4 None of the above4 Center for Disease Control documents collection2 Other
Please check other websites you use to search the tobacco industry documents.
Tobacco Industry w ebsites Tobacco Control Archives Other
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
63.6 %
59.1 %
22.7 %
18.2 % 18.2 %
9.1 %
4 Search history3 Advanced search features2 Other2 Bookbag features
What features of the website do you find most difficult to use? (Please check all that apply)
Search history Advanced search features Other Bookbag features
4
3
2
1
0
18.2 %
13.6 %
9.1 % 9.1 %
Text from Other responsefull textmanipulation of saved sets: more bookbag features ability to search within retrieved setmore 'popular documents' type oflinks.OCRIt would be great to have a quick way to search only for ads like Philip Morris offers from its advertising archives search enginedon't knowfirst visit, I don't know yetfix the bookbag problemquick search box right at top of home page bypassing other pages.Display list of other bates in a set from which the one comes.Nested searchingSlightly larger fontFull Boolean search capacity;more than six search term limit;feedback on when user errors are syntactical (as PM gives);not having to toggle back and forth between long and short displays;master ID numbers in the display;OCR capacity--not only would it be fantastic to be able to search the text of the document, it would be invaluable to be able to cut and paste text from the documents into a word processor.A better search engine....you seem to have more documents than Tobacco Documents Online, but when I use the same search terms your search engine tells me it doesn't find any....when TDO finds over 100!
16 Organization names13 Dates13 Personal names11 Bates numbers10 Authors6 Other5 Cigarette brands
What attributes of a document are most important to you when searching online? (Please check all that apply)
Organization names Dates Personal names Authors Other Cigarette brands
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
72.7 %
59.1 % 59.1 %
50.0 %
45.5 %
27.3 %
22.7 %
7 Date7 Search terms7 Theme6 Named persons, authors, recipients4 Document title4 Named organizations, corporate authors, recipients2 Other2 Document type
How do you organize the documents you search online? (Please check all that apply)
Date Search terms Document title Other
7
6
5
4
3
2
1
0
31.8 % 31.8 % 31.8 %
27.3 %
18.2 % 18.2 %
9.1 % 9.1 %
Text from Other responseprint them out; download to pc.If there are useful documents in a search, I print out the list, then download and print out the useful documents, numbering them with the number on the list. I file them chronologically by theme.by dateby topic and correspondenceFirst by theme (subject) and then by organization/corporation, and/or date.first visit, I don't know yetI e-mail them to my eudora account and search it when I want a citation.I look for a doc at your site, then go to pmdocs.com or the like, type the bates, and pull the description. Then I type the first bates from master file, and this way I get the set of documents with context. Sometime, the same document is in a few different sets! Then I get back to you to download it, or to cross reference with other collections (say, TI).I'm not sure I understand the difference between this question and the previous one. I use Endnote, if that's the question.I wish I had a consistent way. Can you conduct a seminar showing suggestions?Prefer to collect paper documents and arrange them in files that mirror the files they originally came from and/or dates and or events within a date range
11 Female7 Male
What is your gender?
61.1 %
38.9 %
Who Uses the documents?
1 0 thru 91 10 thru 191 20 thru 294 30 thru 395 40 thru 496 50 thru 59
How old are you?Mean = 40.00, Median = 45.00
0 thru 9 10 thru 19 20 thru 29 30 thru 39 40 thru 49 50 thru 59
6
5
4
3
2
1
0
5.6 % 5.6 % 5.6 %
22.2 %
27.8 %
33.3 %
Who Uses the documents?
6 College graduate5 Ph.D./MD4 Master's degree1 Prefer not to answ er1 Some college1 Less than high school
What is the highest education level you have completed?
33.3 %
27.8 %
22.2 %
5.6 %
5.6 %
5.6 %
Who Uses the documents?
10 University or college employee3 Tobacco control advocate2 Journalist1 Student1 Community not-for-profit organization1 Government-related
What is your occupation?
55.6 %
16.7 %
11.1 %
5.6 %
5.6 %
5.6 %
8 California2 Indiana1 D.C.1 Haw aii1 Wisconsin1 Alaska
If you live in the US, what state do you live in?
57.1 %
14.3 %
7.1 %7.1 %
7.1 %
7.1 %
Who Uses the documents?
Aim 2: Add the British American Tobacco documents to the
Flamenco interface
The Tobacco Flamenco13,000 British American Tobacco documents have been “Flamencoized”More documents are to be indexed as they arrive from Guilford, England and the industry websites
Tobacco ThesaurusThe thesaurus terms are controlled terminology which are described by broader terms, narrower terms, and related terms on the UCSF websiteWe created a hierarchy with 834 terms based on these relationsThe parent-child relationships are 7 levels deep.
ConclusionsMany barriers to searching the documents exist Current searches are characterized by unknown recall and low precision, whether using TCA, Legacy or TDOThere are different searching profiles for the people who use TCA vs. Legacy
Upcoming GoalsCollect more survey data and write up resultsTo create Flamenco server at homeEventually conduct usability trial of modified Tobacco FlamencoTo work on additional BATCo documents
