dlux - Splunk Technical Overview

splunk> Overview

Your Guide David Lutz

Copyright © 2011, Splunk Inc. Listen to your data.2

splunk> Lunch’n Learn Agenda

splunk> Overview· What is splunk>?· Use cases· ArchitectureUsing splunk>· Installation and setup (v4.3)· Adding data· Search with keywords, boolean operators and statistical commands· Alerts, reports, and dashboards

splunk> Public Assets· Splunkbase (splunk> repository of over 200 free applications)· Answers – community-driven Q&A· Online Documentation· Enterprise and Global Supportsplunk> App demo· A demonstration of the splunk> for Application Management App and more

Copyright © 2011, Splunk Inc. Listen to your data.

Agenda

What is Splunk? splunk> explainedSearching, Alerting & Reporting explained

Universal Indexing explainedDeployment Options explained

Demonstration

3


What is it?


Collects, indexes and harnesses your machinedata to identify problems, patterns, risks and opportunities and drive better decisions for

IT and the business.


Wait. What?


splunk> is software

splunk> creates a key : value index

splunk> retains pristine copies of all data

splunk> searches that index for data


splunk> is the engine for machine dataNo predefined schema, no custom connectors, no RDBMS, no need to filter/forward.

Web logsLog4J, JMS, JMX.NET eventsCode and scripts

ConfigurationssyslogSNMPnetflow

ConfigurationsAudit/query logsTablesSchemas

HypervisorGuest OS, AppsCloud

ConfigurationssyslogFile systemps, iostat, top

RegistryEvent logsFile systemsysinternals

Logfiles Configs Messages Traps Alerts

Metrics Scripts TicketsChanges

Linux/UnixWindows NetworkingDatabasesApplicationsVirtualization & Cloud

Click-stream dataShopping cart dataOnline transaction data

Customer Facing Data

Outside the Datacenter

Manufacturing, logistics…CDRs & IPDRsPower consumptionRFID dataGPS data


splunk> eliminates finger pointing

13


What It’s Like In The Trenches

Service DeskApplication

SupportSystems

AdministratorApplication Developer

Application Developer

Database Administrator

Log call. The console says everything is

green.

Java monitoring tools don’t show anything either.

Call the developer.

Stop working on new code to

troubleshoot. Need production

logs!

Stop what they’re doing to identify

and gather production logs for developer.

Manual investigation

establishes not application problem.

DBA analyzes audit logs which

points to bad query.

Escalate. Escalate. Escalate. Respond. Escalate. Now what?


Harnessing Your Machine Data

Search and Investigate

ReactiveIT silo chaos

Investigate, find and fix problems dramatically faster across your organization

Proactive






Proactive




Proactive Monitoring

Reactive

Automatically monitor to identify issues,problems and attacks before they impact your customers and services

IT silo chaos

Proactive





Reactive

Automatically monitor to identify issues,problems and attacks before they impact your customers and services

IT silo chaos

Proactive





Operational Visibility

Reactive

Gain end-to-end visibility to track and deliver on IT KPIs and make better-informed IT decisions

IT silo chaos

Proactive






Reactive


IT silo chaos

Proactive






Real-time Business Insights

Proactive


Gain real-time insight from operational data to make better-informed business decisions







Proactive




splunk> Supports Diverse Apps/Solutions

Security

IronPort WSA

CDR

Universal Indexing


key : value

Universal Indexing Indexes Unstructured Data

It’s that simple.


Every wordEvery valueEvery character … even punctuation … even white spaces

Universal Indexing Indexes Unstructured Data


New Approach to Heterogeneous Data

29

Universal Indexing Search-time Knowledge Flexibility and Fast Time to Value

• No data normalization• Automatically handles

timestamps• Parsers not required• Index every term &

pattern “blindly”• No attempt to

“understand” up front

• Knowledge applied at search-time• No brittle schema to

work around• Multiple views into the

same data• Splunk helps find

transactions, patterns and trends

• Normalization as it’s needed• Faster implementation• Easy search language• Multiple views into the

same data

Deployment Options


A splunk> Installation Has 4 Functions Searching and Reporting (Search Head)

Indexing and Search Services (Indexer)

Data Collection and Forwarding (Forwarder)

Local and Distributed Management (Deployment Server)

35

A splunk> installation can be one or all of these …


Single splunk> Indexer

Get started with a single splunk> Indexer. You can index, search, alert, report, correlate within 15 minutes.

Send in syslog data Pull in local data via File or Dir monitoring, WMI, and/or Scripted Inputs

ScriptsLogs

Messages

Servers & Desktops

Firewalls, Routers, Switches


splunk> Indexer with Forwarders

splunk> Forwarders can be used to send data to your splunk> Indexer reliably and securely


splunk> Scales Across the Datacenter

Send data from 1,000s of servers using combination of splunk> Forwarders, syslog, WMI, message queues, or other remote protocols

Auto load-balanced forwarding to as many splunk> Indexers as you need to index, up to TBs/day

Offload search with dedicated splunk> Search Head(s)


splunk> Runs Across Datacenters

39

Distributed search unifies the view across locations

Role-based access controls how far a given user's search will span


splunk> Scales to TBs/day and 1,000s of Users

40

Automatic load balancing linearly scales indexing

Distributed Search with MapReduce linearly scales search and reporting

Demonstration

Executive and Domain-level Dashboards


Remember ‘Harnessing Your Machine Data’?



Proactive

Reactive









Proactive








Reactive


IT silo chaos

Proactive

Root Cause Analysis


Is There A Problem?


Where Is The Problem?


What Is The Problem?


Remember ‘Harnessing Your Machine Data’?




Proactive


The IT Search Company

Your Guide David Lutz

email: [email protected]: @dlux_at_splunk

skype: dluxatsplunkdotcom

mailto:[email protected]

Technology

dlux - Splunk Technical Overview