Upload
realkyleredinger
View
623
Download
4
Embed Size (px)
DESCRIPTION
Humans are much better than computers at detecting unknown patterns in a visual data set, but, like computers we have our shortcomings. We need to optimize inputs for humans so they are fast and efficient in detecting patterns, especially with the coming data tsunami.
Citation preview
Can Skynet Beat Humans in Signal Detection?
Kyle Redinger, Co-Founder @VividCortex
Background
Database Performance Management
Measure ‘All the Things’ in 1-Second Detail
Your Mandate
Manage 300x more data
with 1.5x more people.
Signal Detection Theory
Humans process up to 500GB of data per second.
Find the Outliers
Signal Detection
Easy Hard
Easy Signal Detection
Easy signals are always easy, regardless of set size.
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
200
300
400
500
600
700
800
900
1000
% Correct Response Time
Set Size Set Size
Hard Signal Detection
Hard signals get harderas set size increases.
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
200
300
400
500
600
700
800
900
1000
% Correct Response Time
Set SizeSet Size
Signal Detection in Use
Statistical Process Controls are “Easy”
Upper Control Limit
Lowe Control Limit
Signal Detection in Systems
1-Second CPU over 5 Minutes
What’s the cause?
Looking at Work
In a database, a query is work.
But, I have 300 query classes.
Crap.
Graph Everything
Query Execution Time
CP
U
Reduce DataZoom & Remove the Obvious
1/10th of Data Points
Query Execution Time
CP
U
Man vs MachineMan Machine
Man vs MachineMan Machine
Non-linear increasing CPU Statistically significant
Man vs MachineMan Machine
Non-linear increasing CPU Statistically significant
Pattern!
Not model friendlyPattern!
Pattern!
Shortcomings
Elephantsin the room?
Shortcomings
Humans lose with a
known pattern
Key Thoughts
Our brains do much better with fewer distractors, so eliminate them!
With unknown patterns, humans find signals faster than computers.
Thanks
@kyleredinger
http://www.flickr.com/photos/danielproulx/3524826318/http://www.sciencedirect.com/science/article/pii/S0896627301003920
http://www.himandus.net/elefunteria/eday/puzzles/hunt_24.htmlhttp://www.federaljack.com/terminator-robots-a-reality/
Remember: Less is more.