20
Can Skynet Beat Humans in Signal Detection? Kyle Redinger, Co-Founder @VividCortex

Man vs Machine: Signal Detection Theory and Big Data

Embed Size (px)

DESCRIPTION

Humans are much better than computers at detecting unknown patterns in a visual data set, but, like computers we have our shortcomings. We need to optimize inputs for humans so they are fast and efficient in detecting patterns, especially with the coming data tsunami.

Citation preview

Page 1: Man vs Machine: Signal Detection Theory and Big Data

Can Skynet Beat Humans in Signal Detection?

Kyle Redinger, Co-Founder @VividCortex

Page 2: Man vs Machine: Signal Detection Theory and Big Data

Background

Database Performance Management

Measure ‘All the Things’ in 1-Second Detail

Page 3: Man vs Machine: Signal Detection Theory and Big Data

Your Mandate

Manage 300x more data

with 1.5x more people.

Page 4: Man vs Machine: Signal Detection Theory and Big Data

Signal Detection Theory

Humans process up to 500GB of data per second.

Page 5: Man vs Machine: Signal Detection Theory and Big Data

Find the Outliers

Page 6: Man vs Machine: Signal Detection Theory and Big Data

Signal Detection

Easy Hard

Page 7: Man vs Machine: Signal Detection Theory and Big Data

Easy Signal Detection

Easy signals are always easy, regardless of set size.

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

200

300

400

500

600

700

800

900

1000

% Correct Response Time

Set Size Set Size

Page 8: Man vs Machine: Signal Detection Theory and Big Data

Hard Signal Detection

Hard signals get harderas set size increases.

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

200

300

400

500

600

700

800

900

1000

% Correct Response Time

Set SizeSet Size

Page 9: Man vs Machine: Signal Detection Theory and Big Data

Signal Detection in Use

Statistical Process Controls are “Easy”

Upper Control Limit

Lowe Control Limit

Page 10: Man vs Machine: Signal Detection Theory and Big Data

Signal Detection in Systems

1-Second CPU over 5 Minutes

What’s the cause?

Page 11: Man vs Machine: Signal Detection Theory and Big Data

Looking at Work

In a database, a query is work.

But, I have 300 query classes.

Crap.

Page 12: Man vs Machine: Signal Detection Theory and Big Data

Graph Everything

Query Execution Time

CP

U

Page 13: Man vs Machine: Signal Detection Theory and Big Data

Reduce DataZoom & Remove the Obvious

1/10th of Data Points

Query Execution Time

CP

U

Page 14: Man vs Machine: Signal Detection Theory and Big Data

Man vs MachineMan Machine

Page 15: Man vs Machine: Signal Detection Theory and Big Data

Man vs MachineMan Machine

Non-linear increasing CPU Statistically significant

Page 16: Man vs Machine: Signal Detection Theory and Big Data

Man vs MachineMan Machine

Non-linear increasing CPU Statistically significant

Pattern!

Not model friendlyPattern!

Pattern!

Page 17: Man vs Machine: Signal Detection Theory and Big Data

Shortcomings

Elephantsin the room?

Page 18: Man vs Machine: Signal Detection Theory and Big Data

Shortcomings

Humans lose with a

known pattern

Page 19: Man vs Machine: Signal Detection Theory and Big Data

Key Thoughts

Our brains do much better with fewer distractors, so eliminate them!

With unknown patterns, humans find signals faster than computers.