Network Topology for Search Service
DeploymentSunita Shrivastava
Protection with AuthZ in AT with Load Balancers sending requests to AT
AT
Job
ES
Load Balancer
Private NetworkLoad
Bal
ance
r
Questions : Is a Load Balancer in front of ES Required?NEST Client seems to support load balancing of requests across a set of nodes
Protection with a Gateway (App Insight Model?)
AT
Job
ES
Load Balancer
Gateway
Load
Bal
ance
r
Questions : Gateway Availability, AT and Job in Separate Private Network
Private Network
References• http://azure.microsoft.com/blog/2014/01/07/new-windows-azure-ne
twork-security-whitepaper/
Shared Elastic Search Cluster across VSO Services• Requirements for supporting agility/isolation
• Separate Data Nodes• Separate Query/Aggregation Nodes for each Service• Separation of workload/capacity planning
• Motivation• Required for on-prem• Some of the work/analysis is mandated by ALM Search Search• Belief that it will simplify monitoring
• We need to ensure this • What are the key metrics that each workload would require monitoring?
• Aggregations across queries will be facilitated, in certain scenarios
• Concern • ALM Search Uses plugins
• How do we deal with fixes in plugins• Are our operational models really the same• Impact of capacity planning or index management excercises• Shared asset would be the network
• Do we understand the implications of the Test Results traffic (or bandwidth use)• If we can build some understanding of the network bandwidths consumed by our workloads?
ES Inside the ALM Search VNET (Opt 1)
AT
Job
ES
Load Balancer
Private Network
Inte
rnal
Loa
d Ba
lanc
erALM Search Service
AT
Job
VSO Service (Test Results Service)
TR Data Nodes
AT
Job
AT
Job
Search Data
Nodes
TR Query/Indexin
g Nodes
ALM Search Query/Indexing
Nodes
Shared Master Nodes
Load
Bal
ance
r
TR Query/Indexin
g NodesTRS
Query/Indexing Nodes
TR Data NodesTR Data
Nodes
ALM Search Query/Indexing
NodesALM Search
Query/Indexing Nodes
Search Data
Nodes
Search Data
Nodes
Option 1• Search• Authentication for user in ALM Search Service• Authorization for indexes is in Auth Checks in ALM Search Service AT
• Test Results• Authentication of the user
Securing the ES cluster[TBD] Authentication at ES Cluster
Comments
Jetty SSL/Basic Auth Replacement for NettyHttpServerTransport,Caller is authenticated, Certs are required
NGinx A Reverse Proxy soln used by other ES customers
VNET to VNET Gateway
None 1) Allow only nodes within the vnet in which TRS Service resides access to ES nodes
VSSF Based Proxy Use S2S calls between clients and gateway
Salyh/Elastic Search Security Plugin
If we use ALM Search indexing pipelines for TRS as well
AT
Job
ES
Load Balancer
Private Network
Inte
rnal
Loa
d Ba
lanc
er
ALM Search Service
AT
Job
TR Data Nodes
Search Data
Nodes
TR Query/Indexin
g Nodes
ALM Search Query/Indexing
Nodes
Shared Master Nodes
TR Query/Indexin
g NodesTRS
Query/Indexing Nodes
TR Data NodesTR Data
Nodes
ALM Search Query/Indexing
NodesALM Search
Query/Indexing Nodes
Search Data
NodesSearch Data
Nodes
VSO Service (Test Results Service)
AT
Job
AT
Job
Crawl
VSO Service (TFS)
AT
Job
AT
Job
Crawl Test Results