Embed Size (px)
DESCRIPTION
Rising of social networks like Twitter and Facebook has been flooding the web with user-generated contents in the amount that was never seen before. As we become more depending on the social media for daily information gathering and social interaction, we have started to demand more from search engines, such as real time conversations and social connections. Such new environment is forcing search engine technology to evolve. This presentation will first review the traditional search engine technologies and architecture. Then, we will talk about some of the new architectures for supporting real time search and social search. The last part of the presentation is devoted to discussion of the emerging real time web monitoring system, the cousin of search engine. Social media monitoring is predicted to become a mainstream tool for consumers and businesses because the inherent collective intelligece has the potential to further increase the efficiency of finding information and making decision as what search engine has done.
Citation preview
SDForum Event
Building Search Engine in the Social Media Era
twitter: @web2expresshttp://web2express.org
3/24/2010
Agenda
1. The changing search environment 2. Traditional search engine architectures3. New architectures for real time search
and social search4. Semantic search 5. Emerging real time web monitoring6. Q/A
The Changing Search Environment
The Changing Search Environment
The Changing Search Environment
Agenda
1. The changing environment 2. Traditional search engine
technologies and architecture3. New architectures for supporting real
time search and social search4. Semantic search 5. Emerging real time web monitoring
system6. Q/A
Open Source Lucene/Solr
from Grant Ingersoll
Lucene/Solr Search for Web App
Lucence Indexing/Searching
Solr Server Architecture
from Yonik Seeley
Scaling Lucene and Solr
from Mark Miller
Agenda
1. The changing environment 2. Traditional search engine technologies
and architecture3. New architectures for supporting
real time search and social search4. Semantic search 5. Emerging real time web monitoring
system6. Q/A
Near Real Time SearchLucene 2.9: IndexReader (old + new records) <-- IndexWriter Zoie (LinkedIn): Swap Indexes in memory
Google Real-time Search
Google Social Search
Google Aardvark Social Q/A Engine
Agenda
1. The changing environment 2. Traditional search engine technologies
and architecture3. New architectures for supporting real
time search and social search4. Semantic search 5. Emerging real time web monitoring
system6. Q/A
Bing/Powerset NLP Search Engine
from Barney Pell
Sindice: Retrieving semantic data
index API:sindice.com search engine:sig.ma
Google Semantic Search
Hybrid Search Engine
Hybrid (Universal) Search
Agenda
1. The changing environment 2. Traditional search engine technologies
and architecture3. New architectures for supporting real
time search and social search4. Semantic search 5. Emerging real time web
monitoring system6. Q/A
Social Media MonitoringBenefits for businesses:
understand what people are talking about the company, brands, products, and competitions, etc.identify leads for marketing and sales;engage with customer and community conversations;support customers;cultivate product advocates;use customer feedback to improve products and brands;
Differences from search engine:
search engine for filtering real-time text analysis and semantic analysisauto-discovery of trends/patternscollective intelligencerule engine for routing right information to right person
Basic Monitoring
Scaling up Social CRMProblem: Today's social media program does not scale
Solution:Build a pipeline of social context flow and then apply advanced text analytics to automate routing of social contexts.
Intelligent Monitoring Pipeline
Social CRM Integration in the Clouds
Turning CRM into Social CRM
Agenda
1. The changing environment 2. Traditional search engine technologies
and architecture3. New architectures for supporting real
time search and social search4. Semantic search 5. Emerging real time web monitoring
system6. Q/A
