Use O365 and Azure Cognitive Services for intelligent search

Jeff FriedCTO, BA Insight

Use Office 365 and Azure Cognitive Services for Intelligent Search

October 2017

Longtime

Search Nerd

• CTO, BA Insight

• Senior PM, Microsoft

• VP, FAST

• SVP, LingoMotors

Jeff Fried

Passionate About

• Search

• Search-driven applications

• Information Strategy

• Text Analytics

• Machine Learning

Email: jeff.fried@bainsight.com

Twitter: @jefffried

Blog: www.DoMoreWithSearch.com

Outline

Wait, isn’t search

ALREADY ‘intelligent’?

Matching IntentContent

Content Intent

LookupLearnExploreMessy

Human

Language

A new wave of Intelligent Search Engines?

Machine Learning (ML), Natural Language Processing (NLP),

Semantics, Cognitive, Artificial Intelligence (AI), Self-Learning, …

http://www.cmswire.com/information-management/intelligent-cognitive-ai-based-search-separating-hype-from-reality/

Forrester: “Keyword search of the past has evolved into Cognitive Search”

Forrester, June 2017

Evolution of enterprise search technology

TODAYYESTERDAY TOMORROW

Relevancy based on general criteria that

compare how well a single item matches a

query, compared to the rest of the index.

Matching Content & Popularity

By leveraging your personal activity, we

can signal highly relevant content that

would otherwise drown in the noise

Personalized & behavior based Drive actionable insights

Take action on data and empower

decision making by understanding

intention across many facets

Scope & scale of Microsoft Graph

Use machine learning to create

insights from your data

Construct pairs for questions & answers

Query match quality in content or page

Popularity of document or page

Content metadata such as created date

Bing entity extraction

Understanding organizational priorities

as well as your extended network

Using AI for semantic understanding

+ +

Microsoft, September 2017

11

Semantics! Machine Learning! AI!

Latest Enhancements to SPO search

Making O365 search intelligent using Azure Cognitive Services

more

ˇ

Start with understanding how it works

Matching IntentContent

Context

ContentEnrichment

UX

Connections

QueryAugmentation

Insight Data Sets

Trending Documents

People I Work With

UsedDocuments

Email

Files

Contacts

Groups

Meetings

Org

Tasks

File Enrichments

ItemMetadata

# ActivityHistory

myGraph

Sites I Use

Suggested Sites

Usage Statistics

How does the search box work?

Used

Documents

Trending

Documents

Sites

I Use

Suggested

Sites

Beautiful

Bergen

Contoso

Norway

Kathrine

Hammervold

Metadata Drives Great User ExperiencesDocuments from many sourcesAll client or matter-relevant documents are integrated. Rich MetaData

Content annotated automatically – concepts,

categories, citations, matters, clients, etc

Navigation Controls

Explore, Discover, Drill-down

Where is the Metadata in Delve?

So….I don’t need metadata, right?

Delve doesn’t have refinement

Chart courtesy of xkcd

Outline

The Cognitive Arms Race

Agent Bots Cognitive Machine Learning Augmented RealityMicrosoft Cortana Bot Framework Cognitive Services

- Vision- Speech- Language- Knowledge- Search- Labs

Azure MLCognitive Toolkit

Hololens

Google Google Assistant API.AI Cloud Vision APICloud Video IntelligenceCloud Speech APICloud Natural Language APIGoogle Knowledge GraphGoogle Custom SearchML Advanced Solutions Lab

Cloud ML EngineTensorFlow

Glass at WorkDaydream/Tango

IBM Watson Virtual Agent Watson Conversation Watson APIs- Vision- Speech- Language- Data Insights

Watson ML ServiceApache SystemML

Amazon Alexa Amazon Lex RekognitionPolly

Amazon ML

Apple Siri SiriKitSpeech

Core ML ARKit

Facebook Facebook M ParlAI FastTextCommAI

Torch Oculus Rift

The Cognitive Arms Race

Matching IntentContent

Context

Content

Enrichment

UXConnections

Query

Augmentation

Making SharePoint search intelligent with Cognitive Services

more

ˇ

BA Insight Product Portfolio

Smart Analytics

AutoClassifier

Content from 60+ Enterprise Systems

Cognitive Services (AI, Machine Learning)

SearchIndex(es)

Search forDynamics

Connectors

ExpertiseLocator

SmartPreviews

Sm

art

Hub

Content

ContentProcessing

SearchIndex

/ Graph

Search UI

Enhancing Content with Cognitive Services

QueryProcessing

Content

Enrichment

AutoClassifierrules-driven core, combined with learning-based modules

Enhanced Content

Enriched with

Metadata and

Content Types

Search Visualization Workflow

Adding Text Analytics to ContentUser

Uploads Content Event

Fires

Content Text

Analysis

Sentiment - Is text positive or negative?

Key Phrases - What are people discussing in a single article?

Topics - What are people discussing across many articles?

Language - What language is text written in?

Add metadata to item

Shows in O365 search

Adding Translation to ContentUser

Uploads Content Event

Fires

Content Text

Analysis

Translation – Copy existing content and translate to specified language

save in SharePoint

Shows in O365 search

Create new file/content for specific language

Image SearchUser

Uploads Content Event

Fires

Computer Vision

Image Processing – recognize objects and text inside images

Shows in O365 search

Add metadata to item

Extract image(s)

from documents

Image Classifier and Search

project libraries

and

issue detection

Search Visualization Workflow

Feed content from over 60 enterprise systems

SearchIndex(es)

Search UI

Connectors

SmartAnalytics

Extensiblecontent processing

Pluggable query and results pipelines

Auto Classifier

Cognitive Services

SmartHub

“Architectural” View

37

Content is spread across lots of systems• Aderant

• Amazon S3

• Alfresco

• Box

• Confluence

• CuadraSTAR

• Elite / 3E

• EMC Documentum

• EMC eRoom

• Google Drive

• HP Consolidated Archive

• (EAS, aka Zantaz)

• HPE Records Manager/HP TRIM

• IBM Connections

• IBM Content Manager

• IBM DB2

• IBM FileNet P8

• IBM Lotus Notes

• IBM WebSphere

• iManage Work

• Jive

• LegalKEY

• LexisNexis Interaction

• Lotus Notes Databases

• Microsoft Dynamics CRM

• Microsoft Exchange

• Microsoft Exchange Public Folders

• Microsoft SQL Server

• MySQL

• NetDocuments

• Neudesic The Firm Directory

• Objective

• OpenText LiveLink/RM

• OpenText eDOCS DM

• Oracle Database

• Oracle WebCenter

• Oracle WebCenter Content (UCM/Stellent)

• PLC/Practical Law

• ProLaw

• Salesforce.com

• SAP ERP

• ServiceNow

• SharePoint Online

• SharePoint 2016

• SharePoint 2013

• SharePoint 2010

• SharePoint 2007

• Sitecore

• Any SQL-based CRM system

• Veeva Vault

• Veritas Enterprise Vault

(Symantec eVault)

• West km

• Xerox DocuShare

• Yammer

List of standard connectors, plus a proven architecture and process for creating new connectors to complex systems

Feed content from over 60 enterprise systems

SearchIndex(es)

Search UI

Connectors

SmartAnalytics

Extensiblecontent processing

Pluggable query and results pipelines

Auto Classifier

Cognitive Services

SmartHub

Bots

AR (hololens) apps

Bots and AR

Better Search- data enrichment

- query augmentation

- relevance

- recommendations

New Types of Search

and Discovery- multi-media search

- multi-lingual search

- spoken language search

- search bots

- search in AR

AI for Intelligent Search & Discovery

Content

ContentProcessing

SearchIndex

Search UI

ML applied to search

QueryProcessing

Learning about content structure Learning about user intent

Analytics

Behavioral data to drive learning

Learning to rank

BA Insight Product Portfolio

Smart Analytics

AutoClassifier

Content from 60+ Enterprise Systems

Cognitive Services (AI, Machine Learning)

SearchIndex(es)

Search forDynamics

Connectors

ExpertiseLocator

SmartPreviews

Sm

art

Hub

www.BAinsight.com

Jeff.Fried@BAinsight.com

@jefffried