Copyright 2014 Splunk Inc.
Michael de Buin, Schuberg Philis [email protected] Gert Kremer, Schuberg Philis [email protected] Dani Flexer, Splunk [email protected]
Islands of Splunk MulJple Splunk as a Service Architecture and ImplementaJon
Disclaimer
2
During the course of this presentaJon, we may make forward looking statements regarding future events or the expected performance of the company. We cauJon you that such statements reflect our current expectaJons and
esJmates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements,
please review our filings with the SEC. The forward-looking statements made in the this presentaJon are being made as of the Jme and date of its live presentaJon. If reviewed aTer its live presentaJon, this presentaJon may not contain current or accurate informaJon. We do not assume any obligaJon to update any forward looking statements we may make. In addiJon, any informaJon about our roadmap outlines our general product direcJon and is subject to change at any Jme without noJce. It is for informaJonal purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligaJon either to develop the features or funcJonality described or to
include any such feature or funcJonality in a future release.
Agenda
! The MulJ-Splunk-as-a-Service (MSaaS) framework ! MSaaS implementaJon @ Schuberg Philis ! DemonstraJon
3
MSaaS Architecture
Background ! Splunk administrators are increasingly required to provision Splunk as a service offering to mulJple customers Commonly requires provisioning a Splunk instance to each customer
! MSaaS is a conceptual framework designed to help deliver such an offering
! Schuberg-Philis have implemented this framework using Chef on Apache CloudStack
5
FuncJonal Requirements
6
! AutomaJc deployment and update of mulJple Splunk instances ! Packaged binaries, scripts and configuraJons ! Modular ! Each instance can scale from very small to as large as necessary ! Each instance is customized as needed ! System must funcJon independently without external resources
Except for authenJcaJon, a datacentre automaJon (DCA) tool and an opJonal license manager to fulfill its purpose
! Archiving, backup and resilience requirements are defined per customer
FuncJonal Requirements ! Resilience
Disaster recovery (DR), no single point of failure, indexing Jer resilience, storage resilience
! Strict data segregaJon ! Different network jurisdicJons isolated from each other with no shared resources nor shared informaJon
! Cross-jurisdicJon access possible when explicitly enabled ! JurisdicJon hierarchy supported
A jurisdicJon can include other jurisdicJons
7
FuncJonal Requirements ! Indexed data can be shared, subject to jurisdicJon, and indexes copied between instances
! MulJple licensing models supported central license manager and license pools per-Island license manager and mulJple keys assigned on deployment global license
! Roles defined independently for each instance ! All credenJals maintained in the enterprise idenJty management system and allocated at deployment
8
Architecture Concepts: Island ! A complete Splunk deployment ! No informaJon sharing between Islands ! Own set of users and roles ! Manages a set of Forwarders ! Forwarders can send data to many Islands but are managed by a single Island
9
Architecture Concepts ! Bridges
Islands without indexing capabiliJes that enable search on mulJple Islands
! Deployment unit An independently deployed collecJon of Splunk
components ! Customer
An independent user of Splunk a business unit or customer
! Island service agributes ReplicaJon factor, search factor, DR requirements,
security, backup, storage Jer, performance, retenJon plan, daily volume
! AdministraJon Island monitors the other Islands
10
MSaaS Deployment Server ! A centralized system responsible for installing, configuring, and updaJng the Islands
! Maintains all binaries, applicaJons, configuraJons, apps and rouJng informaJon in a version control system (VCS)
! Updates the Islands binaries when necessary
! Each Splunk DS maintains its Islands components ConfiguraJon files supplied by the MSaaS
DS and propagated by Island DS
11
ApplicaJons Each Island has any number of Splunk applicaJons a.k.a. apps Apps are
Managed centrally and deployed with the Island Versions are maintained in the VCS Customized for the MSaaS as a whole or for a subset of the Islands
! MSaaS administraJon apps reside on a dedicated Island Monitors the other Islands for usage, security, charge-back, health and need
for maintenance
! Standard apps deployed with each Island S.o.S Splunk on Splunk App for Unix or for Windows Infrastructure as appropriate
12
MSaaS ImplementaJon @SBP
My Company and I Our customers:
Gert Kremer Mission CriJcal Engineer since 2007
Engineering the MSaaS Architecture
15
! Dual datacenter setup, no data backups (RF=SF=2) ! Maximum 100 GB/day per Island ! Centralized license server ! AcJve-standby search heads (rsync-ed)
! Not implemented: Splunk Deployment Server AdministraJon Island Dedicated Job Servers
Ingredients
16
Requirements & use cases M-SaaS Meta Architecture DescripJon
Believers
Splunk Chef Cloud(Stack) Github enterprise
Will it blend?
17
Configura)on Indexing: Avg KBPS
Indexing: Avg EPS
Search: Avg First Event (sec)
Search: Avg Search (sec)
HP DL380G7; CPU: 26 Xeon 2.67GHz; Memory: 12GB; OS: Linux 64-bit, Fedora 14 (*)
22,400 79,057 2.48 20.18
Linux on EC2: c1.xlarge 800 pIOPS (*)
12,410 43,639 2.12 27.37
SBP: 4 CPU/16GB 12,449 43,865 2.82 18.24
SBP: 8 CPU/32GB 14,715 51,959 1.37 17.24
(*) hgp://blogs.splunk.com/2013/06/06/splunkit-v2-0-2-results-ec2-storage-comparisons/
Prior art on Splunk, Chef and Cloud
18
! Best Buy Splunk cookbook hgps://github.com/bestbuycom/splunk_cookbook
! OpsCode Splunk cookbook hgps://github.com/opscode-cookbooks/chef-splunk
! Splunk Storm (Splunk as a Service) hgp://www.getchef.com/customers/splunk/
Scope of AutomaJon
19
! Island creaJon ! Server instanJaJon ! Splunk soTware installaJon
! Cluster configuraJon ! Data disk and indexes: creaJon and management
! Data replicaJon between Search Heads ! Security (firewall rules, SSL setup) ! Monitoring (process, connecJvity, cluster health, Splunk alerts)
! Single-sign-on (SSO) ! Splunk applicaJons
Splunk Enterprise at Schuberg Philis
Search head Search head
Indexer Indexer Cluster master License server
Datacenter 1
Datacenter 2
SupporJng Systems
Splunk Enterprise at Schuberg Philis
Bridge (SIEM)
Search head Search head AcJve Directory
License server
SupporJng Cloud
Infrastructure
AuthenJcaJng proxy
Island Customer
Search head Search head
Indexer Indexer Cluster master
Island Customer
Search head Search head
Indexer Indexer Cluster master
Island Customer
Search head Search head
Indexer Indexer Cluster master
License Server
Island InstanJaJon
Search Head
Indexer Cluster Master
Proxy Servers
Indexer
Search Head
Datacenter 1
Datacenter 2
1. Configure island 2. Deploy island 3. Integrate island
1. Configure island 2. Deploy island 1. Configure island
License Server
Island InstanJaJon
Search Head
Indexer Cluster Master
Proxy Servers
Indexer
Search Head
Datacenter 1
Datacenter 2
1. Configure island 2. Deploy island 3. Integrate island
1. Configure island 2. Deploy island 1. Configure island
Returning the favor
24
! Generalize available Chef Cookbooks ! Splunk monitoring Nagios plugin available in Nagios Exchange ! Splunk deployment best pracJces and tools
DemonstraJon by Michael
THANK YOU
[email protected], [email protected]
Process Requirements
27
! Service Request When customers request the service, a process is triggered that results in a
deployed instance of Splunk that implements the customers use-cases and the other agributes of the service requested.
! Charge Back The cost of the service to its operators can be charged back to its customers
based on the actual cost of provisioning the service.
! Easy to Onboard The process of incorporaJng data sources into the system is well defined
and simple
Data RouJng ! Islands use-cases can overlap requiring them to share data and data-sources
! Data rouJng is maintained in a global rouJng table ! On update, the rouJng table is converted into Splunk configuraJon elements suitable for inserJon into the transforms.conf file that is then propagated to the data collecJon Jer by the Master Deployment Server and Isla
