Google versus MERLOT

Jeff Bell

MERLOT

• 12,000 sites indexed

• Over 85% triaged

• Over 1,700 (15%) peer reviewed

• Also has assignments, user comments, and personal collections

Google

• The most popular search engine– 8 billion web pages indexed– Uses a page ranking algorithm that looks at

how sites are linked and how many links point to a site to determine which hits are shown first

– Uses hyper-text matching algorithm to find pages that best match the search query

– Is very fast (less than a half a second for a typical search)

Why compare the MERLOT to Google?

• Google searching is what most faculty and students are most familiar with

• To get users to switch to using MERLOT instead of Google will require showing that MERLOT is clearly superior

• So, is MERLOT superior?

Methodology

• Goal: Compare the effectiveness of MERLOT searches to Google searches

• Use “randomly” chosen recent searches in MERLOT– Most common searches would be better

• Use same term in a Google search and a MERLOT search and measure how many “hits” are in common and how many unique “good” hits are found for each service – look at first 10 and first 50 hits

100 recent MERLOT searches

• Writing• Multimedia• French• learning+styles• Library• Nursing• Density• teacher+education• Media• user+interfaces

• Economics• Physiology• Simulation• Spanish• writing+materials• classroom+organization+exer

cise• Communication• German• Interview• nutrition

• Optics• Perception• Research• Algebra• Astronomy• Chinese• Computer• Eye• heart

• Internet• Java• Management• Network• Phonetique• Physics• Powerpoint• Video• Binary• binomial+norma• lbotany

• computer+architecture

• Computers• Democracy• Dna• Electronics• Esl• evaluating+internet+

sites+101• Evolution• excel

• Forces• Health• human+resources• Law• Literature• Maurici• Microbiology• Mips• Neuroscience• personality

• Philosophy• Planning• Prewriting• programming+languag

e• synaptic+transmission• Test• Tourism• %22user+interfaces

%22

• adult+learning• Airplanes• Analogies• Anatomy• Art• Assessment• Atom• axonal+conduction• bacteria

• Bernoulli+Trials• biology+proteine+tuto

rial• Blueprint• Calculus• Cancer• case-based• centre+of+mass• Child• classroom+organizatio

nal+exercise

• cognitive+ergonomics• Colligative• Composition• computer+math• computer+science• cpu• Database• differentiated+instruction• Education• Energy• Engineering• Environment• Equinox• espanol

• Energy• Engineering• Environment• Equinox• espanol

Observations on Search Terms

• Most searches were very general (“writing,” “DNA,” “Cancer,” “atom,” etc.)

• Less than 10% are specific or compound searches (“axonal conduction,” “centre of mass,” “colligative,” etc.)

• Most speling is OK

DNA, First 10

• Google search page 1– Two good sites, three OK sites, five poor sites

• Poor sites are non-educational sites such as Microsoft.net, DNA lounge, etc.

• Merlot search, page 1– Ten good sites, one in common with Google -

“DNA from the Beginning,” a MERLOT editor’s choice award winner

DNA: First 50

• Google: 15 out of first 50 sites are good

• Merlot: 50 out of first 50 sites are good, 18 have peer reviews

• Similar results are found with other simple searches (botany, anatomy, cancer, etc.)

A Specific Search: Synaptic Transmission

• Google search for "Synaptic transmission”– 487,000 hits

– Seven OK sites in first ten hits, over 25 educational sites in first fifty hits, although only a couple that would get a very high peer review

• MERLOT search for "synaptic transmission”– No hits

• Same general result for most specific searches, thousands of hits in Google, few if any in MERLOT

Conclusions• For instructors doing general searches,

MERLOT is superior - more relevant links, with added information from peer reviews, comments and assignments– New instructors, someone teaching a class for

the first time

• For specific searches Google is better, MERLOT does not have enough breadth– Instructors wanting to add a specific topic to

their course

Suggestions

• Need to identify weak areas in MERLOT and strengthen them - a more systematic approach to adding materials is needed– Search Google using common search terms– Search Google using textbook keywords

(chapter headings, glossaries, etc.)– Other ideas?