Upload
librato-inc
View
765
Download
1
Tags:
Embed Size (px)
Citation preview
hi.
github: djosephsen
Who put Stream Processing in my Monitoring Conference?
github: djosephsen
Graphing NagiosLet’s talk about signal processing
github: djosephsen
Graphing NagiosLets talk about signal processing
RealTime Audio Processing
github: djosephsen
Graphing NagiosLets talk about signal processing
Graphing NagiosLets talk about signal processing
github: djosephsen
Graphing NagiosLets talk about signal processing
github: djosephsen
Graphing NagiosLets talk about signal processing
github: djosephsen
Graphing NagiosLets talk about signal processing
github: djosephsen
Graphing NagiosLets talk about signal processing
github: djosephsen
Lets talk about signal processing
WhisperDB
WhisperDB
WhisperDB
WhisperDB
github: djosephsen
Lets talk about signal processing
github: djosephsen
Graphing NagiosLets talk about signal processing
github: djosephsen
Graphing NagiosLets talk about signal processing
github: djosephsen
github: djosephsen
Nagios
Zabbix
Collectd
DAC (transform and normalize the signal)
(e.g., translate input to a standard json blob)
github: djosephsen
Zabbix
Filters (de-duplication, noise reduction)
(e.g., compute percentiles to filter outliers)
github: djosephsen
Multiplex (aggregation)
(e.g., aggregate http 200’s from multiple hosts)
github: djosephsen
Effects (ad-hoc external signal processing)
??
github: djosephsen
Effects (ad hoc external signal processing)
??
Say “Exponential Smoothing”
again
Disk Latency
Queue Latency
Model A
Model B
ing
MultiVariate Data Science Menu
Specials Gaussian
Fuzzy
Discrete
Hybridized
SmoothingExponentialSmoothingHolt-Winters
Chicken Fried Exponential SmoothingNaieve Baysian
linear regression
Markov Chains
extreme studentized
Kung-Pow distance vector groupingstream decompisiton
Organic, free-range Ansecomb Quartet
really really smooth Exponential Smoothingxtra seasonality available on request
Downsampling
Statistical Convergence
Lacunary Sequence
Hashsum Distance
Lambda Convergence
Lambda ANYTHINGSeasonal Hybridized Whatever
HALF OFF!
Anything Named after a turn-of-the-century Member of the Russian Intelligencia
Spacial Forecasting
deviant TestQuantile isolation
Probablistic merging
Hybridized/Fuzzy Combo platter!
Collaborative n-gram cosine matching
Robust Sequential Map-Reduce
Micro-lot N-variate chi-square
Ask about our locally-sourced quantile robustness
Roasted Daily!
Scientist gonna science
Scientist gonna scienceEngineer gonna.. yeah.
Scientist gonna scienceData Engineer gonna ??
Making a model is just the beginning
Nagios
GraphiosNagios
Graphios StatsdNagios
Graphios Statsd GraphiteNagios
Graphios Statsd Graphite Grafana
Graphios Statsd Graphite Grafana
Math thingy ???
github: djosephsen
Effects (ad hoc external signal processing)
??
Stuff Statsd Graphite Grafana
Stuff Graphite GrafanaHeka
Stuff Graphite GrafanaHeka
Model A
Model B
Stuff Graphite GrafanaHeka
Model A
Model Bcarbon relay
carbon relay
carbon relay
carbon relay
carbon relay
carbon relay
carbon relay
carbon relay
Stuff Graphite GrafanaHeka
Model A
Model Bcarbon relay
carbon relay
carbon relay
carbon relay
carbon relay
carbon relay
carbon relay
carbon relay
Elastisearch
Elastisearch
Data in Truth out
X
XXX
XXX X X
XX
Kafka
SSA
Kafka
SSA
Kafka
SSAr60
Kafka
SSAr60
r900
Kafka
SSA
SSA
r60
Raw Writer
SSA
r60
Raw Writer
r900 r3600
SSA
r60
Raw Writer
Alerts
SSA
r60
Raw Writer
Alerts
SSA
MySQL
SSA
Alerts
MySQL
SSA
Alerts
MySQL
SSA
Alerts
MySQL
{ {
SSA
Alerts
MySQL
Storm is “pretty OK”
Lacks visibility into flow, performance, and failures
Netty? JVM Hops? Ser/Der?
Samza? Spark?
NIH? ( kafka + zk + mezos + disruptor+? )
Questions?