Retaining Collective Intelligence

in Incident Response and Controls

Effectiveness through Gamification

Mark Jaster, Founder & CEO mark@418intelligence.com

(610) 742-9366

www.418intelligence.com

RADICAL TRANSPARENCY

BELIEVABILITY WEIGHTED DECISION MAKING

BACKTESTED DECISIONS WITH OUTCOMES

• Incident Responders’ and inputs were

“Cyber Believability Weighted?”

• Detection & Mitigation Methods were

Back Tested against outcomes?

• Responders and stakeholders were

rewarded for the value of their inputs?

What if we tried this in IR?

Continuous Collective Intelligence Calibrated Up-to-Date Answers on Call

Can we turn IR into a game?

• IARPA developed and tested

• Gamifies, scores & retains collaborative

intelligence in a Bayes Net Model

• Merges human and machine intelligences

through the language of probabilities

FOURSight Technology

As covered by …

Get Points Analyze Incidents Bet Outcome Probabilities Gamified Rewards

How FOURSight works

Trends Predictive Analytics

Prototype User Experience & Design

8

FOURSight Game Board

FOURSight Game Board

• Timed Rounds

• Currency Updated Continuously

• Running EV Score (not shown)

FOURSight Game Board

• All primary navigation occurs here

• Assess risk factors

• Collaboratively analyze attack TTPs

• Estimate effectiveness of IR options

FOURSight Game Board

• Background

Information Window

FOURSight Game Board

• Probabilities

Assessments Window

FOURSight Game Board

• Top Ten

• Player’s 1 over, 1 under

FOURSight Game Board

• Achievements

(Badges)

FOURSight Game Board

• Player Submissions

• IOCs

• Playbooks

Let’s Begin!

17

Each Round has a new briefing FOURSight

Artifacts served up for context FOURSight

Sim artifacts revealed by Rounds FOURSight

Users capture IOCs FOURSight

• This is a test!

(Sorry no

partial credit!)

Find the Easter Eggs to unlock

bonus content!

Assess Macro Situation

Now that you’ve seen the scenario

and some initial artifacts…

• What is at risk?

• How severe?

• What do we know from priors?

• What forecasts can we make on

the outcomes?

FOURSight

Careful, things may change next round!

Show us what you know! FOURSight

• What is at risk?

• How severe?

• Who else knows

something?

(Hint – Check the

Info window)

• What will happen

next?

Take your long positions early

when the “price” is cheap.

Assess the Target FOURSight

• What is the Target’s Security

Baseline?

• What is their maturity?

Assess the Threat FOURSight

• STIX Threat Factors

• These can modulate Detection and

Mitigation Efficacies

(eg. NIDS should perform differently

in APTs than in SQL Injection attacks)

• And they can be inferred from

TTPs

Analyze Incident TTPs FOURSight

• ATT&CK Model

• All 11 Tactics

• Prototype has 20% of Techniques

Show you can spot a red herring,

and profit from it!

Which Techniques are present? FOURSight

Going long, and going short can be

just as profitable!

How confident are you, and when?

What is the community consensus?

FOURSight

The community says Automated

Collection was used to Collect Data.

What Detection methods are best? FOURSight

• Each Technique is mapped to

between 5 and 20 Methods

• Post Prototype, the Techniques

and Methods will be chosen

dynamically from Pick Lists

What Detection tools would you bet on? FOURSight

How to Detect Automated

Collection? That is the question!

Do threat factors change your bets? FOURSight

Well, maybe it depends… (This is a side-bet that

pays extra if APT is True, else costs nothing).

What do your peers think, and why? FOURSight

Deeper insights for better actions FOURSight

• Because this is likely an

APT type of attack…

• Centralized Logging

moves to the top

Detection method

• The insight is retained

for new cases –

Community Memory!

Which Mitigation methods are best? FOURSight

• Same mappings as Detection in

the prototype

How would you mitigate this TTP? FOURSight

The community knows something new FOURSight

• Share emerging best

practices

Use Cases FOURSight

• Skills assessment and development –

Individuals and Teamwork

• Community brain bank and leverage –

All that knowledge at your back!

• Tools investment decisions – Base

your reco’s on proven tool experts.

FOURSight Collective IR Analysis Platform

Gamified Information Market

Incident Analysis Playbooks & Countermeasures

The End Game