Upload
chrissanders88
View
513
Download
0
Embed Size (px)
Citation preview
COPYRIGHT © 2016, FIREEYE, INC. ALL RIGHTS
RESERVED.
Chris Sanders (@chrissanders88)
BBQ Pit Master
FireEye/Mandiant
Former DoD &
InGuardian
Founder, Rural Tech Fund
Author
PhD Researcher
Copyright © 2016 Chris Sanders
Disclaimer
Copyright © 2016 Chris Sanders
I’m going to talk about matters of the brain, not just the normal tech stuff.
My research for this presentation involved consultation with psychologists.
I, however, am not one,
….yet.
Learning Objectives
Increase awareness of:
Metacognitive gap
Investigation process
So you can:
Become a better analyst
Approach investigations in a more systematic way
Get better at training new analysts
Accelerate the effects of experience
Appreciate the value of teaching and learning
Copyright © 2016 Chris Sanders
The Metacognitive Gap
Copyright © 2016 Chris Sanders
Perception vs. Reality
Copyright © 2016 Chris Sanders
Perception
A way of regarding, understanding, or interpreting
something.
Reality
The state of things as they actually exist.
Perception RealityLearning
How do we do it?
Copyright © 2016 Chris Sanders
How did you learn to catch bad guys?
Experimentation
Observation / OJT
Mentorship
KSU SOC Anthropological Study:
“SOC analysts often perform sophisticated
investigations where the process required to
connect the dots is unclear even to themselves.”
Metacognition
Thinking about thinking
“Why did I do this?”
Understanding your own thought process
Relationship between metacognitive
awareness and performance.
Two Components:
Knowledge of Cognition (Understand It)
Regulation of Cognition (Apply It)
Copyright © 2016 Chris Sanders
Mapping the Investigation
Process
Copyright © 2016 Chris Sanders
Experiment Design
Research Questions:
Are experts more metacognitively aware?
What separates novice and expert analysts?
Sample:
Novice and expert analysts
Methodology:
30 case studies
Stimulated recall interviews
Focus on individual investigations of varying types
Perform key phrase analysis
Copyright © 2016 Chris Sanders
Key Phrase Mapping
Copyright © 2016 Chris Sanders
Intuition
Experimentation
Restructuring
Imagination
Incubation
Metacognition
Evaluation
Goal Setting
Making Plans
Reflection
Analytically Viewing
Data
Rule-Based
Reasoning
Considering
Alternatives
Dual Process Theory
Intuition: Implicit, unconscious, fast
Reflection: Explicit, controlled, slow
Results
Copyright © 2016 Chris Sanders
Novices Experts
Intuition Metacognition Reflection
Findings
Copyright © 2016 Chris Sanders
1. Experienced analysts rely on rule-based
reasoning to a much larger extent.
2. Experienced analysts are more
metacognitively aware than novice analysts.
Closing the Gap
Copyright © 2016 Chris Sanders
Novice: “How do I do
this job?”
Expert: “Here, watch
me.”
Expert: “Study this way
of thinking. Then,
come try it for
yourself.”
Goal Setting
Making Plans
Evaluation
How can we train analysts to be more
metacognitively aware, and provide them with the
tools to apply that knowledge?
Rule-Based Reasoning
Copyright © 2016 Chris Sanders
Rule-Based Reasoning
Copyright © 2016 Chris Sanders
Humans think in if-
then-else
statements
Rules are heuristics
Shortcuts for solving
problems
Derived from
experience
Investigation Heuristics
If the process name is made to look like a
legitimate system process but isn’t
Then it’s probably malware
If the domain has a bunch of random
characters
Then it might have been created by a DGA
Else it’s just a coincedence
If the host is beaconing externally
Then it might be command and control
Else it’s a normal service I should remember for next
time Copyright © 2016 Chris Sanders
Documenting Heuristics
Copyright © 2016 Chris Sanders
We need an industry wide effort to document these…
If - Then - Else Format
Store in narrative and structured format
Use estimative language
Bonus: You can use these in IR playbooks
Metacognition and the
Investigation Process
Copyright © 2016 Chris Sanders
The Investigation Process
Copyright © 2016 Chris Sanders
“An investigation is the systematic inquiry and
examination of evidence and observations in an
effort to gain an accurate perception of whether an
incident has occurred, and to what extent.”
Question
Hypothesis
Answer
Observation
Conclusion
Goal-Driven Questioning
You should be able to articulate what question you’re trying to answer at any given time.
Focus questioning around uncovering relationships
Questioning is driven by rule-based reasoning
Experience really shines here due to a larger library of heuristics
Question
HypothesisAnswer
Copyright © 2016 Chris Sanders
Hypothesis Generation
You already do this, but
it’s a passive process.
Expose and Attack Bias
Form an educated guess
about the answer to your
questions
Consider your “Because”
statement
I believe X because Y
Copyright © 2016 Chris Sanders
Question
HypothesisAnswer
Seeking Answers
Key processes:
Finding and Filtering Data
Performing open source intel research
Reviewing evidence
Uncovering additional questions
Hypothesis validation/invalidation
Copyright © 2016 Chris Sanders
Question
HypothesisAnswer
Investigation Scenario 1
Question
• Was this done maliciously?
Copyright © 2016 Chris Sanders
Discovery
• SIEM AlertUser account added to domain admin group
Hypothesis
• No – Normal admin activity
Answer
• Yes
Question
• What did the user account do afterwards?
Hypothesis
• Normal admin activities
Answer
• Accessed mail server and mounted exec staff mailboxes
Investigation Scenario 2
Question
• Did the host get infected?
Copyright © 2016 Chris Sanders
Discovery
• IDS Alert
Angler EK Landing Page
Hypothesis
• Yes
Answer
• No – exploitation failed
Question
• What type of payload was downloaded?
Hypothesis
• Flash exploit due to SWF file alert evidence
Answer
• Hypothesis Confirmed
Question
• Is a vulnerable version of flash installed?
Hypothesis
• It’s Flash, so probably
Answer
• No – Flash is not installed
Further Research
Copyright © 2016 Chris Sanders
More case studies
Supporting
whitepaper +
dissertation
Further
experimentation in
identified areas
Practical applications
Teaching case
studies
Action Items
Copyright © 2016 Chris Sanders
Identify and document your rules/heuristics
Start framing through the investigative process
Use the process as a teaching tool
Think about thinking – applied thought has
power
Try to teach this stuff to someone
Thank You!
Web:
http://www.chrissanders.org
E-Mail:
Twitter:
@chrissanders88
Copyright © 2016 Chris Sanders