Building Conversational Search with Fusion

Andrew Oliver March 1, 2017

Based in San Francisco

Offices in Bangalore, Bangkok, New York City, Raleigh, Munich

Over 300 customers across the Fortune 1000

Fusion, a Solr-powered platform for search-driven apps

Consulting and support for organizations using Solr

Produces the world’s largest open source user conference dedicated to Lucene/Solr

Lucidworks is the primary sponsor of the Apache Solr project

Employs over 40% of the active committers on the Solr project

Contributes over 70% of Solr's open source codebase

40%

70%

• Over 50 connectors to integrate all your data

• Robust parsing framework to seamlessly ingest all your document types

• Point and click Indexing configuration and iterative simulation of results for full control over your ETL process

• Your security model enforced end-to-end from ingest to search across your different datasources

ExperienceManagement

Operational Simplicity

SECURITY BUILT-IN

Shards Shards

Apache Solr

Apache Zookeeper

ZK 1

Leader Election

Load Balancing

Shared Config Management

Worker Worker

Apache SparkCluster

Manager

RE

ST A

PI

Admin UI

Lucidworks View

LOGS FILE WEB DATABASE CLOUD

HD

FS

(Op

tio

nal

)

Core Services

• • •

ETL and Query Pipelines

Recommenders/Signals

NLP

Machine Learning

Alerting and Messaging

Security

Connectors

Scheduling

Fusion Simplifies the Deployment

Core Services

• • •

NLP

Recommenders / Signals

Blob Storage

Pipelines

Scheduling

Alerting / Messaging

Connectors

RE

ST A

PI

Admin UI

Lucidworks View

LOGS FILE WEB DATABASE CLOUD

• Seamless integration of your entire search & analytics platform

• All capabilities exposed through secured API's, so you can use our UI or build your own.

• End-to-end security policies can be applied out of the box to every aspect of your search ecosystem.

• Distributed, fault-tolerant scaling and supervision of your entire search application

Conversational Search: “The principle behind conversational search

is that a user can speak a sentence into a device, and that device can respond with a

full sentence.” -Technopedia

A Pathway to Better Search

•Better Query Parsers

•Similarity

•Collaborative Recommendations

•Personalization

Dismax / Extended Dismax

•Searches and scores multiple fields

•Scores aren’t simply added

Similarity / More Like This

• Fusion Pipeline Stage takes a document as an argument, returns like documents

• BM25 Similarity

• https://en.wikipedia.org/wiki/Okapi_BM25

Collaborative Recommendations

• Boost results users commonly click on

• Based on “Signals” - built into Fusion

• https://www.youtube.com/watch?v=0AwLSdcrJSc&sns=tw - Demonstrated in last Webinar

Recommendations

Personalization

•Signals on a more per-user basis

•Boost things that I click on often

•Boost results like the ones I’ve clicked on

•More on this next time!

Language

• Easier and more accurate than Natural Language

• Domain phrases PE Ratio, Stock Price, SKU

• Entities: GE, IBM, etc

Domain Specific Language

Spoken Language

•Speech to text

•Text-to-speech

•Mainly the latter is just UI.

• Donald Bot

• Alexa

• Deep Drumpf

Conversational

https://www.messenger.com/t/thedonaldbot/

• https://twitter.com/deepdrumpf

• http://karpathy.github.io/2015/05/21/rnn-effectiveness/

Alexa

Cortana

Assistant

Etc

•Speech to text

•Search Engine

•UI Answers

• Tagging

• Faceting

• Domain Language (Finance)

Open Calais

Demo

• Speech to Text enabled Search

• Plugged into Fusion Query Pipeline

• https://lucidworks.com/2016/10/13/lucidworks-integrates-ibm-watson-into-fusion-enterprise-discovery-platform/

Watson

Demo

RTFMBot

•Based on Chatterbot

•NLTK

•Train on Slack

•Answer Questions generally with Fusion search results

RTFMBot

Questions?

Building Customer-Targeted Search: March 15 @ 11am PThttps://lucidworks.com/webinar/building-customer-targeted-search-with-fusion/

Upcoming Webinars: