Upload
mike-maadarani
View
264
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Learn about the SharePoint 2013 Search Topology and how it differs from 2010. In this presentation, I explain the details of all search components, how to properly configure your search topology, and your options to extend your search farm in a hybrid “cloud/on-premise” scenario. This session will explain what you need to consider to design your search, in order to handle your organization's needs. I dive into scripting a high availability search topology, keeping it healthy and manage your day-to-day search operations. Learn about how to optimize your search for best performance and search relevancy, to support reliable search applications.
Citation preview
Search Topology and OptimizationMike MaadaraniCanada, -UTC 5:00 EST
April 16th /17th, 2014
App Dev and Architecture for over 18 years (15 Years Microsoft, 3 Years with the “Other Guys”)
Business focused on Enterprise Content Management, Publishing Sites, & Search
Technology focused on SharePoint, SQL Server and SharePoint Integration
Architect, trainer, and presenter
Mike MaadaraniSharePoint ArchitectNavigatusOttawaCanada
Contact
SP24 Search SessionsSession
Time (GMT)
Time (EST) Title Level Speaker Twitter Handle
#SP24S090 5:00 1:00Using the Content Enrichment Web Service with SharePoint Server 2013 Search
Technical - Advanced
Sezai Komur - NEC Australia
@sezai
#SP24S021 6:00 2:002013 Search, Display Templates, Query Rules, Result Types
Technical - Beginner
Omer Zubair - CSC
@Omer_Zubair
#SP24S047 8:00 4:00Real-life experience building search-driven applications for product-centric sites
Technical - Advanced
Marius Constantinescu - blue-infinity SA
@c_marius
#SP24S032 10:00 6:00Custom Indexing Connectors - How to integrate external system into your SharePoint Enterprise Search
Technical - Advanced
Heinrich Ulbricht - Communardo Software GmbH
@h_ulbricht
#SP24S083 11:00 7:00Search First Migration with SharePoint 2013 – Benefits / Disadvantages
Technical - Intermediate
Max Melcher - Alegri International Service GmbH
@maxmelcher
#SP24S034 14:00 10:00The Search Immaturity Cycle, and How to Create a Search Strategy
Business - Intermediate
Jeff Fried - BA Insight
@jefffried
#SP24S009 16:00 12:0010 Things to Know about Search in SP2013 and Office 365
Business - Beginner
Agnes Molnar - Search Explained
@molnaragnes
#SP24S019 18:00 14:00 Search Topology and OptimizationTechnical - Advanced
Mike Maadarani - MCM Consulting
@mikemaadarani
#SP24S050 22:00 18:00Search Driven Application Development in SharePoint 2013
Technical - Intermediate
Matt Youngstrom - Magenic
@spguru
Configuring SSA and PS
Topology ScenariosAgenda
Relevancy, Query Builder, &Optimization
SharePoint 2013 Search Overview
Architecture and Resource Utilization
Search in SharePoint 2010
Crawl Component
Query Component
SharePoint 2010 Search Service Application
Crawl Indexing Engine
Query Engine
Search Admin
Property Store (SQL)
Content
UserWFE
FAST Search for SharePoint 2010
FAST Content SSA
FAST Query SSA
FAST back-end components(managed separately)
Extensibility:• Sandbox• Entity
Extraction
Crawl Indexing Engine
Query Engine
Content Pipeline
Analysis Engine
Query Pipeline
Search AdminContent
UserWFE
… In SharePoint 2013
SharePoint 2013 Search Service Application
Index Component
Query Engine
Content Pipeline
Content ProcessingComponent
CrawlComponent
Query ProcessingComponent
AnalyticsProcessingComponent
Query Pipeline
Search Admin
Admin Component
Entire index on local disk
Property Store (SQL)
Content
UserWFE
Analysis Engine
Crawl Indexing Engine
Link/query analysis & recommendations
Separate crawl and indexing
Extensibility:• Web
callout• Entity
Extraction
SharePoint 2013 Search Architecture
SearchAdmin
Content UserCrawlContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
Crawl
Search Admin
Link
Analytics Reporting
FAST Search Index
SharePointSP AppsDevicesNon-SP UX
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
Public API
Search topology components
Content Query
Why Search is so important?
I just uploaded a document. Make it searchable, quick!
FAST
Why Search is so important?EASY
Why Search is so important?EASY
Why Search is so important?Search Driven Applications
noderunner.exe noderunner.exe noderunner.exe noderunner.exe
Where is Search?
Windows servicesSharePoint Search Host Controller service
Runtime/lifecycle control of search components (except crawler) hostcontrollerservice.exe
SharePoint Server Search service
Crawl Componentmssearch.exemssdmn.exe
ProcessesNoderunner.exe
Runtime environment for search components (except crawler)
msseearch.exemssdmn.exe
CrawlComponentnoderunner.exe
Search Runtime Environment
hostcontrollerservice.exe
Host Controller
Sh
are
Poin
t A
pp
Serv
er
Admin entitiesSearch Service Instance: Provisioning of the search service on each boxSearch Service Application: SharePoint Configuration entity
Still there, but only Crawl Component
AdminComponent
Query ProcessingComponent
Content ProcessingComponent
IndexComponent
Analytics ProcessingComponent
CPU loadDriving factors
QPS
Query transformations
Network loadDriving factors
Number of index partitions
Size of queries and results
Example:
20 index partitions @ 20 qps => 200/100 Mbit/s in/outbound
Query processing component (QPC)
Item count
DPS QPS
Load impact (relative)
CPU NetworkDisk
CPU loadDriving factors
QPS and item count
Guidelines per index component @ 2 GHz CPU
1M items: 5 QPS per CPU core
5M items: 2 QPS per CPU core
10M items: 1 QPS per CPU core
Disk loadDriving factors
QPS and item count
New content invalidates caches
Disk size: 500GB @ 10M items per index component
Index component
Item count
DPS QPS
Load impact (relative)
CPU NetworkDisk
Crawl component
CPU loadDriving factors
Documents per second
Link discovery
Crawl management
Network loadDriving factors
Downloading items from content sources
Passing items on to CPC
Disk loadAll documents are temporarily stored in data folder
Item count
DPS QPS
Load impact (relative)
CPU NetworkDisk
Content processing component
CPU loadDriving factors
Documents per second
Document size and complexity
Feature extractionEstimate: 5-10 DPS per CPU core
Network loadDriving factors
Documents per second
Document size
Item count
DPS QPS
Load impact (relative)
CPU NetworkDisk
Analytics processing component
CPU loadDriving factors
Number of items
Site activity
Disk loadLocal disk used for temporary storage
Bulk load, primacy concern is load isolation
Network loadSame as for CPU load
PLUS: Network traffic increases when distributing APC across multiple machines
Item count
DPS QPS
Load impact (relative)
CPU NetworkDisk
Search administration component
Low CPU and network load
Load increase with more components in the search topology Item
countDPS QPS
Load impact (relative)
CPU NetworkDisk
Create your SSA$SSADB = "SharePoint_Demo_Search"$SSAName = "Search Service Application SP24"$SVCAcct = "mcm\sp_search"$SSI = get-spenterprisesearchserviceinstance -local
#1. Start the search services for SSIStart-SPEnterpriseSearchServiceInstance -Identity $SSI
#2. Create the Application Pool$AppPool = new-SPServiceApplicationPool -name $SSAName"-AppPool" -account $SVCAcct
#3. Create the search application and set it to a variable$SearchApp = New-SPEnterpriseSearchServiceApplication -Name $SSAName -applicationpool $AppPool -databaseserver SQL2012 -databasename $SSADB
#4. Create search service application proxy$SSAProxy = new-SPEnterpriseSearchServiceApplicationProxy -name $SSAName" Application Proxy" -Uri $SearchApp.Uri.AbsoluteURI
#5. Provision Search Admin ComponentSet-SPEnterpriseSearchAdministrationComponent -searchapplication $SearchApp -searchserviceinstance $SSI
#6. Create the topology$Topology = New-SPEnterpriseSearchTopology -SearchApplication $SearchApp
#7. Assign server(s) to the topology$hostApp1 = Get-SPEnterpriseSearchServiceInstance -Identity "SP2013"New-SPEnterpriseSearchAdminComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1New-SPEnterpriseSearchCrawlComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1New-SPEnterpriseSearchContentProcessingComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1New-SPEnterpriseSearchAnalyticsProcessingComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1New-SPEnterpriseSearchIndexComponent -SearchTopology $Topology -SearchServiceInstance $hostApp1 –IndexPartition 0
#8. Create the topology$Topology | Set-SPEnterpriseSearchTopology
Small Search Topology
Fault tolerant small search topology
Host
VM
Index QPC
VM
Admin
Crawl
CPC
APC
Host
VM
Index QPC
VM
Admin
Crawl
CPC
APC
Other SharePoint applications
Web front end
Admin
Crawl
CPC
APC
Index
QPC
Small search farm (up to 10M items)
Resources @ 10M items8x CPU cores24 GB RAM800 GB disk
Sized independently
Separate disk
for index
Scaling to medium search topology
Adm
Crawl
Index Index IndexIndex QPCCPC
APC
Adm
Crawl
Index Index Index IndexQPCCPC CPC
APC
QPC
Extend your SSA#2. Extend the Search Topology:$hostApp1 = Get-SPEnterpriseSearchServiceInstance -Identity "SP2013"$hostApp2 = Get-SPEnterpriseSearchServiceInstance -Identity "SPSearch"Start-SPEnterpriseSearchServiceInstance -Identity $hostApp1Start-SPEnterpriseSearchServiceInstance -Identity $hostApp2
#3. Keep running this command until the Status is Online:Get-SPEnterpriseSearchServiceInstance -Identity $hostApp1 Get-SPEnterpriseSearchServiceInstance -Identity $hostApp2
#4. Once the status is online, you can proceed with the following commands:$ssa = Get-SPEnterpriseSearchServiceApplication$active = Get-SPEnterpriseSearchTopology -SearchApplication $ssa –Active$newTopology = New-SPEnterpriseSearchTopology -SearchApplication $ssaNew-SPEnterpriseSearchAdminComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1New-SPEnterpriseSearchCrawlComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1New-SPEnterpriseSearchContentProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1
New-SPEnterpriseSearchAnalyticsProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp1 –IndexPartition 0New-SPEnterpriseSearchAdminComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2New-SPEnterpriseSearchCrawlComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2New-SPEnterpriseSearchContentProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2New-SPEnterpriseSearchAnalyticsProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2New-SPEnterpriseSearchQueryProcessingComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $hostApp2 –IndexPartition 1
#5. Activate the topology:Set-SPEnterpriseSearchTopology -Identity $newTopology
Medium Search Topology
Challenges: IntentWhere is my talk
Project Plan?
Are Documents held at the same place?
I wonder if there are references from
previous projects?Different people have different intentsQuery Rules help you handle intents
There is rarely a single right answer
Infrastructure Project
Conceptual Relevance Flow
For all queries:
Authorities: Level 1: http://employment
Ranking model: {incorporate user ratings}
Query:HR Employmentquarterly report
Search Web Part
Query Processing Engine
Document Collection
Thesaurus: HR Human ResourcesBest bets: HR Employment /HR/employment
(WORDS HR, Human Resources) AND(WORDS employees, employed) AND (WORDS quarterly, quarterlies) AND(WORDS report, reports, reported)
Mixed Results for:• HR Employment best bet• HR Employment quarterly
report• HR Employment
ContentType=reports
Dynamic Reordering Rules: Quarterly Report {prefer docs from http://reports}
Query Rule: {Terms} Quarterly Report {Terms} ContentType=“reports”
Authorities: SSA-level configuration
Sites that are important
Sites with low intrinsic relevance
Takes ~24hrs to propagate
Authorities: Connected
1
0
1
1
2
4
3
2
∞
4
Setting an authority affects all sites connected through hyperlinks
Sites are weighted by distance to the authority
Tune Search Results
Created at the SSA, Tenant, Site Collection or SiteSSA
Site Collection
Site
Query Rules
ConditionWhen Do I apply the rule?
ActionWhat to do when the rule is matched?
PublishingWhen should the rule be active?
Query Rules
Exact match, beginning or end Ad-hoc or term store dictionary Match a regex (advanced) Is this query more likely aimed at
the following source…? Do people mostly click on result of
the following type…?
Conditions Show a promoted result Show a block of results Replace the core results
with a different query
Actions
Query Rules
Dynamically Ranking Change
Part of the query
Results Ranking
Query Builder
High Availability and Performance
Better Search Quality
Better management
Friendly results and tools
Session Objective and Takeaways
...the anchor will pick some of them up!
Questions?Post your questions in the chat...
SP24 Search SessionsSession
Time (GMT)
Time (EST) Title Level Speaker Twitter Handle
#SP24S090 5:00 1:00Using the Content Enrichment Web Service with SharePoint Server 2013 Search
Technical - Advanced
Sezai Komur - NEC Australia
@sezai
#SP24S021 6:00 2:002013 Search, Display Templates, Query Rules, Result Types
Technical - Beginner
Omer Zubair - CSC
@Omer_Zubair
#SP24S047 8:00 4:00Real-life experience building search-driven applications for product-centric sites
Technical - Advanced
Marius Constantinescu - blue-infinity SA
@c_marius
#SP24S032 10:00 6:00Custom Indexing Connectors - How to integrate external system into your SharePoint Enterprise Search
Technical - Advanced
Heinrich Ulbricht - Communardo Software GmbH
@h_ulbricht
#SP24S083 11:00 7:00Search First Migration with SharePoint 2013 – Benefits / Disadvantages
Technical - Intermediate
Max Melcher - Alegri International Service GmbH
@maxmelcher
#SP24S034 14:00 10:00The Search Immaturity Cycle, and How to Create a Search Strategy
Business - Intermediate
Jeff Fried - BA Insight
@jefffried
#SP24S009 16:00 12:0010 Things to Know about Search in SP2013 and Office 365
Business - Beginner
Agnes Molnar - Search Explained
@molnaragnes
#SP24S019 18:00 14:00 Search Topology and OptimizationTechnical - Advanced
Mike Maadarani - MCM Consulting
@mikemaadarani
#SP24S050 22:00 18:00Search Driven Application Development in SharePoint 2013
Technical - Intermediate
Matt Youngstrom - Magenic
@spguru