Embed Size (px)
Citation preview
Hybrid SharePoint with the new Cloud Hybrid Search
Jeff Fried CTO, BA Insight
Cloud Search Service Application
• Unified index with on-premises and cloud content
• Feeds Office Graph/Delve experiences
• Supports Search as a Service
• Reduces search server footprint
Audio textAnd search
indexpropertiessignalsMetadata extraction and processing
2 Hybrid SharePoint sessions on Monday 4-5:15 – pick one, review the other later
SharePoint Hybrid: The Sure Path Forward - Ben Curry
Debunking the Hybrid SharePoint Infrastructure Dilemma - Jill Hannemann & Adam Levithan
2 Hybrid search sessions on Tuesday – go to one or both
11:45am The Future of Microsoft Search is Here! Cloud SSA - Jeff Fried & Ben Curry
3:45pm Hybrid SharePoint with the new Cloud Hybrid Search - Jeff Fried
Hybrid Sessions at
Focused on Search and
SharePoint since 2004
Longtime
Search Nerd
• CTO, BA Insight
• Senior PM, Microsoft
• VP, FAST
• SVP, LingoMotors
About Jeff Fried
Passionate About
• Search
• SharePoint
• Search-driven
applications
• Information Strategy
Blog:
DoMoreWithSearch.com
Technet Column
“A View from the
Crawlspace”
About BA Insight
– Connectivity
– Applications - Im
– Classification -
– Analytics
–
–
–
–
–
Why Hybrid SharePoint?
7
Approaches to Hybrid – by Workload
Split Workload
different tools in
different places
Split User
task uses content or
sites across ‘the divide’
Exchange, SharePoint, Lync
OneDrive, Yammer, PowerBI, Delve
Extranet, Mysites, Team Sites, Project Sites
Portals, Intranet, Services/Applications
Links Search
Online
On-Prem
Cloud Hybrid Search
Cloud
SSA
Text &
Metadata
Online
On-Prem
Logical Architecture: Crawling
Cloud SSACloud SSA
ParseCrawl
SCS
ACL Map Process
Blob
storequeue
Online
On-Prem
Query
processing
Logical Architecture: Query
Cloud
SSA
Online
On-Prem
Logical Architecture: Query
Cloud SSA
Query
processing
Online
On-Prem
Failure mode: what if you can’t reach the cloud?
Cloud
SSA
Online
On-Prem
Combination: double crawling
Cloud
SSA
Text &
Metadata
Mechanisms
Cloud SSA
Remote Result Source
Cloud App model
Add-ins
External Content
Federation
Identity and Directory Sync
Benefits of Cloud Hybrid Search
External Content
(on-premises and/or
in the cloud)
SharePoint Server
(On-premises or Hosted)Office 365
SharePoint Online Content
Onedrive for Business Content
Co
nnect
ors
SharePoint Content
Adding External Content
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
BA Insight ConnectorsMailbox and Archiving Systems• Microsoft Exchange
• Microsoft Exchange Online
• IBM Lotus Notes
• Symantex Evault
• Autonomy EAS / (Zantaz)
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
ERP and Portal Systems•
•
•
•
•
•
•
•
•
•
•
•
Plus a proven architecture and process for creating new
connectors to complex systems
External Content in O365 UX
Unified view across all content
- on-premises and on-line
- inside and outside SharePoint
Scaling
External Content
(on-premises and/or
in the cloud)
Custom
Processing
CEWS
Bottlenecks:
1) Source systems
2) Content Processing
3) Indexer
….
External Content
(on-premises and/or
in the cloud)
Bottlenecks:
1) Uplink
2) Source systems
….
24
Performance
External Content
(on-premises and/or
in the cloud)
CEWS
Custom
Processing
Bottlenecks:
1) Uplink
2) Source systems
3) Content Processing
….
Performance Monitoring and Bandwidth (Get-Counter -ListSet "Search Gatherer Azure Plugin - SharePointServerSearch").counter
500K items crawled on an Azure D3
50 DPS 100 DPS
1 hour
Less servers is OK
•
•
•
•
•
Directory Synchronization
SID S-1-5-21-1212121212-1212121212-1212
msOnline-
OnPremiseSecurity
Identifier
S-1-5-21-1212121212-1212121212-1212
PUID PUID-XXXX-XXXXXXXXXX
Mapping of Access Control Lists
Allow: S-1-5-21-1212121212-1212121212-1212
PUID-XXXX-XXXXXXXXXX
• User SIDs are mapped to PUIDs
• Group SIDs are mapped to Object IDs
• «Everyone» and «Authenticated users» are mapped to
«Everyone except external users»
SUPPORTED
– Custom IFilter
– BCS connectors
– Partner connectors
Customizations: Supported & Unsupported
SUPPORTED
– Tenant level schema mapping
– Query rules
– Result sources
On-premises In the cloud
NOT SUPPORTED
• Content that requires custom security trimming
NOT SUPPORTED
• Site collection level schema mapping
• Custom security trimming
• Custom entity extraction
• Content enrichment web service
1)
2)
3)
Cloud Hybrid Search Limitations + Workarounds
33
Feature OOB Limitation BA Insight
CEWS not available with Cloud SSA available via connector framework
Entity Extraction not available with Cloud SSA available via autoclassifier
Custom Security Trimming not available with O365 index can 'map down' to AD groups
ThesaurusSharePoint Online doesn't support a thesaurus
can use Federator - with SP server-
based search center
Removal of on-premises
search results not available with Cloud SSA (could provide a custom solution)
External Content
(on-premises and/or
in the cloud)
SharePoint Server
(On-premises or Hosted)
SPO Content
OneDrive Content
Co
nnect
ors SharePoint Content
Connector
Framework
Office 365
AutoClassifier
(app version)
CEWS
Custom
Processing
External Content
(on-premises and/or
in the cloud)
SharePoint Server
(On-premises or Hosted)
SPO Content
OneDrive Content
Co
nnect
ors SharePoint Content
Connector
Framework
AutoClassifier
Office 365
AutoClassifier
(app version)
CEWS
DLP Sensitive Data Search works with hybrid
Search for sensitive data across on-premises and SharePoint Online
All Built-in sensitive types
Identification and export
Extends to data in OneDrive
Sensitive Information type detection through KQL searches
Get instant statistics
Preview & export results
Right now: only when you query for it
A global single index solution
Cloud SSA
Cloud SSA
Cloud SSA
Cloud SSA
Cloud SSA
–
–
–
NOT OOB …. but there’s a way to handle them all
Scenarios
40
41
Connectors
Federator
OOB Federated Search User Experience
Results from Cloud
Results from SharePoint On-Premise
Refiners from Cloud only
No termset
synchronization
Result Blocks
(not interleaved)
BA Insight Federator
45
Full Range of Hybrid Search Configurations
ScenarioMost systems and portals
hosted on-premises
Most systems and portals
hosted in the cloud
Must work across borders but
maintain data residency
Single Single MultiSearch
Search Index Search Index Across Multiple
in SP Server in SP Online * Search Indices
How it works
Crawls SP Online and
other sources from SP
Server
Crawls SP Server and other
sources from Cloud SSA, pushes
text & metadata to SP Online
Searches SP Online and SP
Server simultaneously;
combines the results
AdvantagesSimplest approach;
best search experience
Low footprint on-premises;
can use online features
(Delve, DLP)
Only solution for some
scenarios
BA Insight
Improvement over OOBno OOB solution
Adds content outside SP
Preview content outside SP2013
Supports content enrichment
Provides single interleaved
result set and refiners
* requires Microsoft Cloud SSA
Approaches for Hybrid SharePoint
Configuration
Should I run index reset?
NO!
Best Practice:Content Source Naming & Deletion
{ { { {
Action CrawlDB state Office 365 index
state
User’s view
Create contentsource1_v1
Crawl doc1
doc1 in crawldb Doc1 indexed Doc1 is searchable
“index reset” <empty> Doc1 indexed Doc1 is searchable
Create a result source to exclude
contentsource1_v1 from the tenant &
search center site collections
<empty> Doc1 indexed Doc1 is no longer searchable
Tenant Admin opens SR to delete ALL
cloud SSA content.
<empty> <empty> All external content has been
removed
Create contentsource1_v2
Crawl doc1
doc1 in crawldb Doc1 indexed Doc1 searchable again
Orphaned Content
50
Customer Example: ACE
Built on SharePoint 2013 – but couldn’t run as-is in O365
SharePoint Server in Azurein hybrid configuration with O365 Tenant
Virtual Network
Cloud Service
Availability Set
Active Directory & DNS
Cloud Service Cloud Service
Availability Set
Front End
Availability Set
App server
Availability Set
Database
Microsoft Azure
Gatewaysubnet
Active VPN
On-premises environment
Optional!
Example: Using Search-First Migration with Hybrid
Cloud Service
Availability Sets
SharePoint Services Farm
Microsoft Azure
SharePoint Online
Site collections
Office 365 Tenant
SharePoint 2013 Content Farms
SharePoint 2010 Farm(s)
2) Migrate / UpgradeContent Farms
Each site collection can be moved independentlyCan be on-premises, in O365, or hosted in Azure
3) Decommission old farm(s)
1) Establish Search Service(using Azure IaaS)
Key Considerations for Hybrid: Workloads, Environment, Data, Customizations
Availability of features Online versus
On-Premises on particular workloads
Significant investments in
customization of On-Premises
workloads
Concerns over global network
performance with remote sites
Regulatory
considerations
Manageability concerns
References
http://technet.microsoft.com/en-us/library/dn197172(v=office.15).aspx
http://sp2013searchtool.codeplex.com/
https://github.com/OfficeDev/PnP-Tools/tree/master/Scripts/SharePoint.Hybrid.Search.Configuration
References - Blogs
http://blogs.msdn.com/b/spses/archive/2015/09/15/cloud-hybrid-search-service-application.aspx
http://blogs.msdn.com/b/spses/archive/2013/10/22/office-365-configure-hybrid-search-with-directory-synchronization.aspx
http://blogs.msdn.com/b/spses/archive/2014/01/05/office-365-configure-hybrid-search-with-directory-synchronization-password-sync-part2.aspx
http://blogs.msdn.com/b/spses/archive/2014/01/07/identity-federation-amp-single-sign-on-deployment-for-hybrid-search-in-office-365-sharepoint-online-part3.aspx
http://blogs.msdn.com/b/spses/archive/2015/03/19/configuring-microsoft-web-application-proxy-server-for-inbound-hybrid-topology-with-office-365-and-microsoft-sharepoint-server-2013-part7.aspx
https://www.youtube.com/watch?v=JWEZx9SHDb0&list=PLvmwu6WYeFdjNbiy7SISJAZd1HjzIJoz5
https://azure.microsoft.com/en-us/documentation/articles/active-directory-aadconnect/
https://azure.microsoft.com/en-us/documentation/articles/active-directory-aadconnect/
http://blogs.msdn.com/b/spses/archive/2015/09/15/cloud-hybrid-search-service-application.aspx
References – Installing with SP2016
Tools
62
New Sites to bookmark
63
