Games We Play Defenses and DisincentivesAllison Miller

OverviewOverview of econ & game theory concepts

Game theory games

Infosec issues as games

Designing games to win

Walk-through a defense built on disincentives

Wrap-up

Economics applied to security

Utility theory

Externalities

Information Asymmetries

Signaling

Marginal cost

Game theoryBranch of applied mathematics

Studies decisions made by players interacting (or competing)- Scenarios have rules and pay-offs

- Costs & benefits dependent on decisions of other players

Used as a framework in economics, comp sci, biology, & philosophy- Also business, negotiation, and military strategy

Discussing GamesMechanics of a payoff matrix

Player 2

A B

Player 1

A A1, A2 A1, B2

B B1, A2 B1, B2

Discussing Games

Mechanics of decision trees

UP

DOWN

CIRCLE

RED

BLUE

MARIO

LUIGI

KIRBY

GIZMO

10, 3

2, 10

10

2, 5

-3, 3A

B

B

A

A

A

Typical game theory "games"

Chicken / Brinkmanship- Push it to the edge

Volunteer’s Dilemma- For the greater good

Tragedy of the Commons- Share and share alike (cumulative effect of

cheating)

Prisoner’s Dilemma

Discussing GamesPrisoner’s Dilemma

Player 2

Keep quiet Confess

Player 1

Keep quiet

-1, -1Mutual cooperation

0, -10Individual defection

Confess -10, 0Individual defection

-3, -3Mutual punishment

Predicting outcomes

Cooperation

Defection

Dominant strategies

Equilibrium

Nash Equilibrium

Equilibrium is reached when:- Players in a game have selected a strategy

- Neither side can change it’s strategy independently & improve position

Optimal solution in games with limited outcomes

Discussing GamesPrisoner’s Dilemma

Player 2

Keep quiet Confess

Player 1

Keep quiet

-1, -1Mutual cooperation

0, -10Individual defection

Confess -10, 0Individual defection

-3, -3Mutual punishment

Setting up risk problems as games

Identify players in the game

Clarify the “rules”

Show me your moves

Describe payoffs

Single move or repeated game

Discussing GamesTragedy of the Commons: Spam, Bandwidth usage

Everyone else’s choices

> n choose wise usage

Less than n choose wise usage

Individual choice

Use resource

wisely

Cost, but social benefit

Mutual cooperation

Cost(Subsidize social

use)

Overuse resource

Social benefit(Benefit w/o cost)

0Resources depleted

Discussing GamesChicken/Brinkmanship: Vulnerability Disclosure

Vulnerability Researcher

Report Exploit

Asset Owner

Reward / Respond

0, 0Responsible disclosure

-2, +2Early disclosure

Ignore / Deny

+2, -2Defer vulnerability

-10, -100-day go boom

Discussing GamesVolunteer’s Dilemma: Data breach cost info sharing

All other victims

At least one shares All keep quiet

Victim

Share 0 0Cost, limited benefit

Keep quiet

1Benefit w/o cost

-10Everyone’s in the

dark

How games are won

Clarify dominant strategies

Find equilibrium

Pursue equilibrium or change the payoffs

MovesCurrent game-play- Controls are layered or chained until we're satisfied that for some set of attackers,

the cost of the attack is higher than the utility associated with their payoff

Reputation requirements for participation

Role requirements for participation (access control)

Incremental authentication

Content/context based filtering

Blacklisting / whitelisting

Rate limiting

Bot limiters (Captcha)

Obfuscation/Encryption

Counter-moves

For every move there is a counter-move

Putting the pieces on the board

The amount of friction inserted into the system depends on:- Value of asset to the owner

- Value of the asset to potential attackers

- Number of attackers expected

- Portion of attacks that must be averted

- Disincentive value of each layer of friction for an attacker

Now it’s time to play our game

Does this sound familiar?

Managing DecisionsGame Theory is a framework for studying decisions- Since payoffs depends on the choices of other players, moves

are risky

- Players play based on their risk appetite

- Risk management = decision management

Defenders design control systems that make decisions- Where risks manifest in observable behavior

- That make moves/counter-moves depending on the context and understanding of an actor’s identity or intent

- Where system or individual costs/payoffs depend on the outcome of an actor’s actions

SHALL WE PLAY A GAME?(SINCE WE CAN’T PLAY “CLUE” FOR EVERY LOGIN

TRANSACTION NEW USER MESSAGE

FRIEND REQUEST ATTACHMENT

PACKET WINK POKE CLICK

BIT

WE BUILD RISK MODELS)

Applying Decisions

Risk management is decision management

ACTOR ATTEMPTS

ACTIONSUBMIT

WHAT IS THE REQUEST

HOW TO HONOR THE REQUEST

SHOULD WE HONOR?

RESULT ACTION OCCURS

Not all risk decisions have a competitive element, but all competition / games have risks

Create account using fake identity

Script completion of verifications

Outsource captcha

Create accounts across virtual devices

Distribute creation of accts using botnet

Scrape identities from public sites

Age accounts, then reactivate

Use stolen credentials

Defraud verification process

...

Require email verification

Test for human behind keyboard

Rate limit by device ID

Rate limit by IP/location

Look for similarities across accounts

Require reputation level to proceed

Filter for content / context, add auth challenge

Require manual verification

Manual review of account/event

...

Except one small thing...

...what kind of game is this?

Multi-player Mode

Offense

Attempt Success

Defense

Deflect 4, 4 0, 10

Ignore 10, 0 1, 1 Offense

Attempt Success

Defense

Deflect 4, 4 0, 10

Ignore 10, 0 1, 1Offense

Attempt Success

Defense

Deflect 4, 4 0, 10

Ignore 10, 0 1, 1

Offense

Attempt Success

Defense

Deflect 4, 4 0, 10

Ignore 10, 0 1, 1

Attackers are not the only players in the game

Legitimate users that are also affected by added friction

Team DynamicsSo this adds another factor into the appropriate level of friction question, which is:- Disincentive value of each layer of friction for

an innocent

- Likelihood the disincentive will be incorrectly applied to an innocent

- Likelihood the disincentive value > payoff value for the innocent (go find a new game)

Decisions, Decisions

Authorize Block

Good false positive

Bad false negative

RESPONSE

POPULATION

Incorrect decisions have a cost Correct decisions are free (usually)

Good Action Gets

Blocked

Bad Action Gets

Through

Downstream Impacts

GAME OVER

1-UP?

Why are we still playing?

Economic/mathematical models depend on rational participants

Free will doesn’t imply rationality

Economics studies what should happen, behavioral economics studies what does happen

Example of rational irrationality

Ultimatum Game- Player A given $1000

Player A needs to split the $ with Player BPlayer A gets to choose the split

- Player B receives offerIf B accepts, both get $If B rejects, both get 0

Take it or leave itOutcomes- Player A’s usually offer ~50%

- Player B’s often reject if offered <30%

- This behavior occurs across cultures, levels of wealth

Emotions matter- Heightened brain activity in

Bilateral antierior insula (disgust) w/low offersDorsolateral prefrontal cortext (cognitive decision making) w/high offers

- Fairness, Fear, Punishing the mean

Therefore: Winning strategies depend on understanding behavior

Both attackers and defenders may exhibit bias when making decisions - about the game and other players

Retrofit conceptual models to actual experiences

Fill in the blanks on player costs/payoffs

Risk controls still either need to- Change friction (cost), or

- Change expected value of pay-off

Continue to analyze game dynamics over time - Low-risk, high frequency interactions (data)

- High-risk, low frequency interactions (negotiation)

Prediction is very difficult, especially about the future

Niels Bohr

Allison Miller @selenakyle

Some referencesAxelrod, Robert. The Evolution of Cooperation.

Dixit, Avinash and Nalebuff, Barry. The Art of Strategy: A Game Theorist’s Guide to Success in Business and in Life.

Fisher, Len. Rock, Paper, Scissors: Game Theory in Everyday Life.

Gibbons, Robert. Game Theory for Applied Economists.

Meadows, Donella. Thinking in Systems: A Primer.

Wikipedia’s sections on Game Theory, Economics, & Probability.