Saurabh Sahni
Developer, hacker, evangelist
Build your Own Search Service
Yahoo! HackU IIIT Hyderabad | 22nd Jan 2010
• About BOSS API – What?
– Why? – Features
• What can be built with the BOSS API? • How to use it
– BOSS API – Code example – BOSS Mashup framework
Outline
About BOSS API
• Open Yahoo’s core search features via web services to let 3rd parties revolutionize Search
What?
http://developer.yahoo.com/search/boss
Usage
Opening the search technology stack
CRAWL
EXTRACT
SPAM <-> Gold
Analyze
Index
Rank Assist
Index
Web Map
Retrieve
Usage
Opening the search technology stack
CRAWL
EXTRACT
SPAM <-> Gold
Analyze
Index
Rank Assist
Index
Web Map
Retrieve
WEB API
Your App here
Why?
• Removes entry barriers
• Asset to Innovate – Develop new relevance models – Change presentation style
• Search anywhere – Improve Vertical Quality with Web comprehensiveness
• No branding or attribution
• Ability to change presentation style • Ability to re-order results and blend-in additional
content
• Access to multiple verticals (web search, image, news)
• Keyword suggestions, spell checks • Semantic data, in-links, abstracts • Ability to monetize
BOSS API features
What can be built with the BOSS API?
middlespot.com
10
Inquisitor: Browser Extenstion
11
Cluuz: http://cluuz.com
Keyword finder - http://keywordfinder.org/
askBOSS: http://ask-boss.appspot.com/
askBOSS: http://ask-boss.appspot.com/
askBOSS: http://ask-boss.appspot.com/
Webmeme: http://www.webmeme.in
17
How to use it?
Get Started
• Register for an application id http://developer.yahoo.com/wsregapp/
• Documentation http://developer.yahoo.com/search/boss/boss_guide/
• Code samples: Javascript, PHP and Python http://www.saurabhsahni.com/boss-examples.zip
Get Started…
• {query}: term to look for (url-encoded) • {vert} := {web, news, images, spelling}
• @ required – appid
• @ optional – start, count, lang, region, format, callback,
sites, view
BOSS Search API Interface
http://boss.yahooapis.com/ysearch/{vert}/v1/{query}
(Source: http://en.wikipedia.org/wiki/File:Slumdog_Millionaire_poster.jpg)
BOSS API
• Search for avatar: – http://boss.yahooapis.com/ysearch/web/v1/avatar?appid=xyz&format=xml
BOSS API: XML response
h"p://boss.yahooapis.com/ysearch/web/v1/avatar?appid=xyz&format=xml
Site Restrict Search
• Search for avatar on selected movie sites – Add param
sites=indiatimes.com,movies.yahoo.com,imdb.com
– http://boss.yahooapis.com/ysearch/web/v1/avatar?appid=xyz&sites=imdb.com%2Cmovies.yahoo.com&format=xml
Site Restrict Search
Search images
• http://boss.yahooapis.com/ysearch/images/v1/avatar?dimensions=large
http://boss.yahooapis.com/ysearch/images/v1/ avatar
Search News
• http://boss.yahooapis.com/ysearch/news/v1/avatar?age=15d
Code Example Movie Search
Movie Search Code Example
http://www.saurabhsahni.com/boss-examples.zip
More with BOSS API
Add parameter view=keyterms – http://boss.yahooapis.com/ysearch/web/v1/avatar?
appid=xyz&view=keyterms&format=xml
Related keywords
http://boss.yahooapis.com/ysearch/web/v1/avatar &appid=xyz&view=keyterms&format=xml
Semantic Data
• Access structured data acquired through SearchMonkey
Semantic Data
view=searchmonkey_feed view=searchmonkey_rdf
http://developer.yahoo.com/search/boss/stuctureddata.html
• Add parameter abstract=long – get up to 300 characters instead of 130
Long abstracts
http://boss.yahooapis.com/ysearch/spelling/v1/avataar?format=xml
Response
Spell Check
• Get page inlinks – http://boss.yahooapis.com/ysearch/se_inlink/v1/
{URL}?appid={APPID}
• Page data: collection of subpages in a domain – http://boss.yahooapis.com/ysearch/se_pagedata/v1/
{URL}?appid={APPID}
Site Explorer
• Python (v2.5+) library
• BOSS Search SDK plus …
• SQL for remixing arbitrary XML/JSON sources
BOSS Mashup Framework
http://developer.yahoo.com/search/boss/mashup.html
• Enhanced version of BMF to GAE platform
• http://zooie.wordpress.com/2008/08/04/yahoo-boss-google-app-engine-integrated/
• Enables quick deployment of BOSS applications online
BMF + Google App Engine
• http://mashable.com/boss/ • http://delicious.com/tag/bossmashup
• Add yours by tagging it with “bossmashup” on
Del.icio.us!
More BOSS Implementations
One more thing… One more thing…
• The biggest dataset available on web • Very useful for Web-mining research
experiments – Natural language processing – Semantic extraction
– Related keywords – Similarity detection – Clustering algorithms – Spelling corrections
BOSS in Academic Research
Learn more at: http://developer.yahoo.com/search/boss/
Slides at: http://www.slideshare.net/saurabhsahni/
Questions?
Appendix
BOSS Mashup Framework simplifies aggregating and presenting multiple data sources
Search UI Templates in BOSS Mashup Framework
• select, group, sort, union, joins, udfs, where • Text normalization and duplicate removal
• All-in-memory storage and retrieval operations
• Ability to join lists of tables via an arbitrary predicate function (map-like)
• Search UI template framework • Single search function provides total access to
BOSS REST API
BMF Features
Recommended