Chef Analytics

James Casey

Engineering Lead – Analytics

james@getchef.com

Outline

• Framing the problem

• Chef analytics

• Demo

• Roadmap

• Questions

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

Enterprises Have Nearly Unlimited Computing Resources

Virtual Nodes

Physical Hardware

1980Mainframe

1990Client/Server

2000Datacenter

2010+Web-Scale

20

40

60

80

100

120

Mill

ion

s

Mil

lio

ns o

f S

erv

ers

Exponential Increase in Size Leads toOperational Complexity

Web Servers

Application Servers

Database

Exponential Increase in Size Leads toOperational Complexity

Web Servers

Application Servers

Database

Add 1 server

20+ Changes

12+ New

Dependences

Speed of Execution Requires Visibility

• Change tracking

• Security logs

• Auditing

• Performance monitoring

Chef Analytics Keeps Your Finger on the Pulse of Your Infrastructure

Chef Analytics Provides Three Core Components

• Actions and Run History

• Record any policy or administrative changes to any object managed by Chef Server

• Track changes through all sources including management console, knife command or

direct application of chef-client

• Real-time Reporting

• Browse events in a friendly web UI with search, filters and sorting options

• Integrate with existing tools via API

• Notifications

• Alert teams of every change through built-in messaging and email integration

• Extend notifications to existing systems with simple webhook architecture

Chef Analytics History

• Launched in May 2014

• Builds on Reporting (Run history) feature shipped in 2013

• Adds new fundamental data collection components

• Actions – track policy modifications on the Chef Server

• Compliance – assert controls on changes to infrastructure and policy

• Pluggable analytics pipeline

• Chef Actions component available now!

Chef Analytics Architecture

Chef Analytics Data Flow

Reporting - Who did what on your Chef Server?

• Single view of what is changing in your infrastructure

• Success/Failure status of individual Chef Client runs

• Rollups of success/failure counts

• Rollups of run durations

• Drill-down detail to individual resource convergence

• State before/after

• Diffs (e.g. for templates, files)

• Errors

Reporting – what’s happening on chef-client runs ?

Actions – The Real-time Event Stream

• Provide a read-only view of what happened

• Can be customized to meet audit and compliance reporting

requirements

• Allow administrators to react to events as they happen or

after the fact investigation

• “What happened just before nodes started failing runs?”

• “When did our systems gets patched for Heartbleed?”

DemoResolving Infrastructure Problems with Chef Analytics

knife cookbook delete collectd-plugins

> knife cookbook delete collectd-plugins

Which version(s) do you want to delete?

1. collectd-plugins 1.2.0

2. collectd-plugins 1.0.15

3. All versions

1

Deleted cookbook[collectd-plugins][1.2.0]

Roadmap

Analytics Roadmap

• To fill in

Notifications

• Adds a language which allows you to express rules on

• Run Start

• Run End

• Run Resource convergence

• Actions

• Extensible for your business requirements

• “When someone not in the ‘siteops’ group modifies the DNS cookbook, alert the siteops team via email to

siteops@example.com”

• “When the /etc/ssh/ssh_config file is modified, raise audit rule 24.1”

• Send to different external data sinks

• Messaging systems, e.g. Hipchat, Slack, SMTP

• Generic Webhook

Notification Rules

rule (action) when

set($siteops_members, [“james”, “allen”, “prajakta”]),

when entity_type = “cookbook”

and entity_name = “dns”

and array:contains($siteops_members, requestor_name) != true

Then

notify(“smtp”,

“siteops_mail”,

“DNS Cookbook modified by {{requestor_name}}”),

audit(“Rule 3.2 – DNS Cookbook modification”, false)

How You Can Get Analytics

• Available as a Premium Feature of Chef Server

• Supported on Enterprise Chef 11.2 and Chef Server 12

• Included in Chef Subscription

• Also available free for installations less than 25 nodes

• Installation instructions

• https://docs.getchef.com/install_analytics.html

• Download from https://downloads.getchef.com/

Questions?