Literature Literature SearchSearch

Bill Barrett (originator)Bill Barrett (originator)

David W. Embley (editor)David W. Embley (editor)

Why Search the Literature?Why Search the Literature?To address several questions:

• Is the work you are proposing new? (How can you know if you don’t check Prior Art (i.e.what has already been done?)

• To avoid wasting time by repeating Prior Art

• If you are going to research a topic, you need to become knowledgeable about it.

• To make your proposed research better: in what you write about; in how you write about it; and in the ideas used to you create your proposal. If the literature search doesn’t affect, change or improve your proposal, you did it wrong.

Where Do I Look?Where Do I Look?• On the Web

– Web sites of research groups and researcher– On-line Public-Domain indexes

• Google Scholar • Cite Seer • DBLP

– Table of Contents Services – Publisher-specific digital libraries (ACM & IEEE at BYU)– Conference web sites

• At the Library: similar material shelved together• From references of related papers and journals

When Am I Done Looking?When Am I Done Looking?• Never

– Literature search continues throughout the research.– New research results continuously appear.

• But, being practical – here are some rule-of-thumb guidlines– Enough is enough

• No more than about 25-30 references for an MS thesis proposal (perhaps 2-3x that in the final thesis)

• Fewer if an honest search only uncovers very few– When you keep coming across the same papers– When you start hitting cycles when you follow the references in

the papers.– Like coming into a movie half-way. When the “scenes” (papers)

start looking familiar (been there, seen that) you are probably just about done.

• Zobel’s hint (pg. 164): Newer literature has “explored and digested older literature.”

What Do I Do As I Look?What Do I Do As I Look?• Read the abstract and (perhaps) the conclusions.

• Is it related? Does it apply? If not, terminate.

• What are the main claims/results of the paper? You should be able to summarize this (high level) in 2-3 sentences.

• Include your summary in your annotated bibliography. L.I.T. Head and U. Lift, Helium: More than a Balloon, in Proceedings of

Chemical Magic, Vol. 73, Nr. 2, 2007, 123-135.

This paper illustrates the diverse uses of helium and claims that …

– Make your citation complete.– Your annotation should be a good reminder when it comes to

writing your literature review.

How Do I Decide What To How Do I Decide What To Read?Read?

• Categorize your references into a few (< 8) groups and select 2-3 of the best (seminal) papers from that group. Your references do not need to be exhaustive, only representative.

• Choose from the best venues– More competitively refereed– The best researchers publish in the best venues– How do you know what venues are best?

• Check conference rankings and journal rankings• Ask a long-time expert in your area

• Choose from the best researchers (one place to look is Microsoft Academic Search)

How Do I Read What I Find?How Do I Read What I Find?• Some guiding comments (Zobel)

– “There is rarely a need to understand every line.”– “Number of papers that a researcher working on a particular project

has to know well is usually small.”– A brief look is usually sufficient to determine whether a paper is

relevant.

• Some knowledge about CS publications helps– Few papers are perfect

• “While the fact that a paper is refereed is an indicator that it is of value, it is not a guarantee.”

• “Too many people submit work that did not deserve to be written; sometimes it gets published.”

– Be skeptical• Deadlines mean that mistakes are undiscovered and that some issues are

unexplored.• A paper only captures a snapshot of a research program at a moment in

time.– Be forgiving (Your papers won’t be perfect either.)– This knowledge is not an excuse to dismiss past work.

• Do (only) focused reading

What Is Focused Reading?What Is Focused Reading?• Identify contributions and shortcomings rather than simply

read.

• Read by asking questions– What is the main result?– How precise are the claims?– How could the outcomes be used?– What is the evidence?– … (Zobel, pg 167)

• Skim when appropriate– Search only for insights– Ask fewer questions– Check headers and topic sentences – then skip or read in

depth for the insights and answers to your questions

What If I Find the Dreaded What If I Find the Dreaded “Someone Else Has Already “Someone Else Has Already

Done It!”Done It!”

• Be honest (It does little good to redo.)• But

– Some aspects of your work may be novel.– It may be possible to make further

contributions in an area.– Most research (only) incrementally adds to our

knowledge.

BTW, The Secrets for Success BTW, The Secrets for Success Have a Side Benefit. Have a Side Benefit.

• From the vantage point of literature search and review, can you see the reasons for these secrets?– Secret #1: The abstract answering four

questions– Secret #2: Embed the review in the paper

• If everyone did this, how would this simplify your job of searching the literature?