NIST Computer Security Research, Development, and Outreach

NIST Computer Security Research, Development, and

Outreach

12/22/2011

Rick Kuhn

Talk Overview

• Who we are, what we do, including some examples of NIST contributions to computer security

• In-depth review of technology that saves a lot of companies a lot of money

12/22/2011● Computer Security Division ●2

Please keep the dull stuff to a minimum!

Conduct research, development and outreach necessary to provide

standards and guidelines, mechanisms, tools, metrics and practices to protect our nation’s

information and information systems.

12/22/2011

3

COMPUTER SECURITY DIVISION (CSD)

● Computer Security Division ●

NIST Computer Security- examples• Cryptographic standards

– Data Encryption Standard (DES) – publicly available encryption algorithm; developed by IBM to meet NIST requirements

– Advanced encryption standard (AES)– SHA family of hash functions– FIPS 140 Cryptographic Module Validation Program

• Role Based Access Control (RBAC) – Used in most large organizations; NIST formal model and standard

• Personal Identity Verification (PIV) and CAC card specs– Standard development and interoperabilty testing

• Security automation, SCAP standard – managing security updates• National Vulnerability Database – publicly known vulnerabilities• Quantum cryptography research – single photon source and detector• Quantum-resistant crypto – resist cryptanalysis using quantum computer


Computer Security Division Groups• CRYPTOGRAPHIC TECHNOLOGY GROUP (773.01): Research, develop,

engineer, and standardize cryptographic algorithms, methods, and protocols.

• SECURITY COMPONENTS AND MECHANISMS GROUP (773.02): Research, develop and standardize foundational security mechanisms, protocols and services.

• SECURE SYSTEMS AND APPLICATIONS GROUP (773.03): Integrate and apply security technologies, standards and guidelines for computing platforms and information systems.

• SECURITY OUTREACH AND INTEGRATION GROUP (773.04): Develop, integrate and promote the mission-specific application of information security standards, guidelines, best practices and technologies.

• SECURITY TESTING, VALIDATION AND MEASUREMENT GROUP (773.05): Advance information security testing, measurement science, and conformance.

12/22/2011

5● Computer Security Division ●

Core Focus Areas

• Research, Development, Standards, Guidelines, Reference Materials– Security Mechanisms (e.g. protocols, cryptographic, access control,

auditing/logging)– Security Mechanism Applications

• Confidentiality• Integrity• Availability• Authentication• Non-Repudiation• Accessibility• Safety

• Secure System and Component configuration• Conformance validation and assurance of security properties of

products and systems

12/22/2011

6● Computer Security Division ●

Delivery Mechanisms

1) Standards – FIPS, International Consensus, National Consensus2) Guidelines – NIST SPs, IRs3) Journal & Conference papers4) Reference Materials5) Workshops & Conferences – hosting and participation in6) Consortia & Forums7) Training8) Reference Implementations & Demonstrations9) Conformance Verification Activities10) Test, Tools and other conformance determination tools11) Standards Development Organization Participation


Community Engagement

• Industry- Accessing Expertise and Leveraging Resources- Coordinating Standards and Initiatives

• Academia- Accessing Expertise and Leveraging Resources- Representative Institutions and Consortia

• International- Formal Standards Groups- Accessing Expertise and Leveraging Resources

• Federal, State, and Local Government- Interdepartmental- Department of Commerce- State and Local Governments


● Chief Information Officers (CIO) Council

● Federal Systems Security Governance Board Member

● National Cyber Study Group (NCSG) Member

● Cyber Security and Information Assurance Interagency Working Group

● Information Security Research Council

● Common Terrorism Information Security Standards Working Group

● Committee for National Security Systems (Observer)

● Information Sharing Environment Enterprise Architecture Security Working Group

● Supply Chain Risk Management Working Group

● Federal Information Systems Security Educators' Association

● Software Assurance Forum

● IT Entrepreneurs' Forum

● Governance Coordinating Council

● Federal Enterprise Architecture Security and Privacy Profile Working Group

● Interagency C&A Transformation Working Group

● Internet Engineering Task Force (IETF) Security Chair

● International Organization for Standardization (Chair/Convener several Committees, Work Groups, and Task Forces)

● American National Standards Institute

● International Committee for Information Technology Standards (Biometrics Chair)

● Biometrics Consortium Co-Chair

● National Science &Technology Council Committee on Biometrics and Identity Management (Co-Chair)


Community Engagement Examples

NIST Responsibilities for Cybersecurity• NIST is responsible for developing standards and guidelines, including minimum requirements, that

provide adequate information security for all agency operations and assets in furtherance of its statutory responsibilities under the Federal Information Security Management Act (FISMA) of 2002, Public Law 107-347, but such standards and guidelines shall not apply to national security systems.

• Under FISMA NIST shall “conduct research, as needed, to determine the nature and extent of information security vulnerabilities and techniques for providing cost-effective information security.”

• NIST develops guidelines consistent with the requirements of the Office of Management and Budget (OMB) Circular A-130, Section 8b(3), Securing Agency Information Systems, as analyzed in A-130, Appendix IV: Analysis of Key Sections. Supplemental information is provided in A-130,

• In accordance with the Cyber Security Research and Development Act, The National Institute of Standards and Technology

develops, and revises as necessary, checklists setting forth settings and option selections that minimize the security risks associated with each computer hardware or software system that is, or is likely to become, widely used within the Federal Government.

• Homeland Security Presidential Directive 7; “The Department of Commerce will work with private sector, research, academic, and government organizations to improve technology for cyber systems and promote other critical infrastructure efforts, including using its authority under the Defense Production Act to assure the timely availability of industrial products, materials, and services to meet homeland security requirements.”

• Homeland Security Presidential Directive 12: “The Secretary of Commerce shall promulgate in accordance with applicable

law a Federal standard for secure and reliable forms of identification (the "Standard")”


NIST Responsibilities for Cybersecurity• Cloud Computing

– America COMPETES Reauthorization Act of 2010– Federal Cloud Computing Strategy (February 2011)

• Healthcare– American Recovery and Reinvestment Act of 2009

• Identity Management– National Strategy for Trusted Identities in Cyberspace

• Smart Grid– Energy Independence and Security Act (EISA) of 2007 – American Recovery and Reinvestment Act of 2009

• Voluntary Voting System Standards– Military and Overseas Voter Empowerment (MOVE) Act of 2009– Help America Vote Act

• Cyber Security Education– National Initiative for

Cyber Education (NICE)


When is the part about saving money?

Combinatorial Methods in Software Testing

What is NIST and why are we doing this?• US Government agency, whose mission is to support US industry

through developing better measurement and test methods

• Project goals – reduce testing cost, improve cost-benefit ratio for testing

• Over 26,000 downloads of tutorial

• Users include hundreds of major tech organizations

Need• Exploitable vulnerabilities often arise from

implementation flaws • Analysis of reports in the NVD shows more than half are

buffer errors, memory management, numeric errors, other flaws that are not uniquely security related.

• Testing is typically half of software cost

buffe

r erro

r

SQL in

jectio

n

resou

rce m

gmt

perm

issions,

acces

s ctl

input v

alida

tion

code i

njectio

n

numeri

c erro

rsoth

er

authen

ticati

on

path tra

versal

desig

n error

OS comman

d injec

tion

crede

ntials m

gmg

cryptog

raphic

issue

config

uration

race c

onditio

ns

format

string

inform

ation

leak

cross

site r

eque

st forg

e

link f

ollowing

cross

site s

cript

0

200

400

600

800

1000

1200

Our approach - benefits• Cooperative R&D Agreement w/ Lockheed Martin• 2.5 year study, 8 Lockheed Martin pilot projects in

aerospace software• Results announced Feb. 2013 • Results: “Our initial estimate is that this method

supported by the technology can save up to 20% of test planning/design costs if done early on a program while increasing test coverage by 20% to 50%.”

Our approach - benefits(cont.)• Rockwell Collins• “used this approach to automate parts of the unit and

integration testing of a 196 KSLOC avionics system.” • “The goal was to see if it might cost-effectively reduce

rework by reducing the number of software defects escaping into system test”

• “Overcoming scalability issues required moderate effort, but in general it was effective – e.g., generating 47,040 test cases (input vectors, expected outputs) in 75 seconds, executing and analyzing them in 2.6 hours. It subsequently detected all seeded defects, and achieved nearly 100% structural coverage.”

How did we get here?• NIST studied software failures in 15 years of FDA

medical device recall data• What causes software failures?

• logic errors? calculation errors? inadequate input checking? interaction faults? Etc.

Interaction faults: e.g., failure occurs ifpressure < 10 && volume>300 (interaction between 2 factors)

Example from FDA failure analysis:

Failure when “altitude adjustment set on 0 meters and total flow volume set at delivery rate of less than 2.2 liters per minute.”

Examples from the National Vulnerability Database

• Single variable, 1-way interactionexample: Heap-based buffer overflow in the SFTP protocol handler for Panic Transmit … allows remote attackers to execute arbitrary code via a long ftps:// URL.

• 2-way interactionexample: single character search string in conjunction with a single character replacement string, which causes an "off by one overflow"

• 3-way interactionexample: Directory traversal vulnerability when register_globals is enabled and magic_quotes is disabled and .. (dot dot) in the page parameter


What does a 2-way fault look like in code?How does an interaction fault manifest itself in code?Example: altitude_adj == 0 && volume < 2.2 (2-way interaction)

if (altitude_adj == 0 ) {

// do something

if (volume < 2.2) { faulty code! BOOM! }

else { good code, no problem}

} else {

// do something else

}A test with altitude_adj == 0 and volume = 1 would find this

How are interaction faults distributed?• Interactions e.g., failure occurs if pressure < 10 (1-way interaction) pressure < 10 & volume > 300 (2-way interaction) pressure < 10 & volume > 300 & velocity = 5 (3-way interaction)• Surprisingly, no one had looked at interactions beyond 2-way before • The most complex medical device failure reported required 4-way

interaction to trigger.

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4

Interaction

% d

etec

ted

Interesting, but that's just one kindof application!

Number of factors involved in faults

What about other applications? Server (green)

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

These faults more complex than medical device software!!

Why?


Others? Browser (magenta)

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted


Still more?NASA Goddard distributed database (light blue)

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted


Even more?FAA Traffic Collision Avoidance System module

(seeded errors) (purple)

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted


FinallyNetwork security (Bell, 2006) (orange)

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6

Interactions

% d

etec

ted

Curves appear to be similar across a variety of application domains.



• New algorithms make it practical to test these combinations• We test large number of combinations with very few tests

• Number of factors involved in failures is small

• Refers to how many parameters are involved in faults: Interaction rule: most failures are triggered by one or two parameters, and progressively fewer by three, four, or more parameters, and the maximum interaction degree is small.

• Maximum interactions for fault triggering was 6• Popular “pairwise testing” not enough • More empirical work needed• Reasonable evidence that maximum interaction strength for

fault triggering is relatively small

Interaction Rule

How does it help me to know this?

How does this knowledge help?

If all faults are triggered by the interaction of t or fewer variables, then testing all t-way combinations can provide strong assurance.

(taking into account: value propagation issues, equivalence partitioning, timing issues, more complex interactions, . . . )

Still no silver bullet. Rats!

Let’s see how to use this knowledge in testing. A simple example:

How Many Tests Would It Take?

There are 10 effects, each can be on or off All combinations is 210 = 1,024 tests What if our budget is too limited for these tests? Instead, let’s look at all 3-way interactions …

There are = 120 3-way interactions. Naively 120 x 23 = 960 tests. Since we can pack 3 triples into each test, we need

no more than 320 tests. Each test exercises many triples:

Now How Many Would It Take?

OK, OK, what’s the smallest number of tests?

103

0 1 1 0 0 0 0 1 1 0

A covering array

Each row is a test:Each column is a parameter:

• Developed 1990s• Extends Design of Experiments concept• Difficult mathematically but good algorithms now

All triples in only 13 tests, covering 23 = 960 combinations 103

Suppose we have a system with on-off switches. Software must produce the right response for any combination of switch settings:

A larger example

34 switches = 234 = 1.7 x 1010 possible inputs = 1.7 x 1010 tests

How do we test this?

• 34 switches = 234 = 1.7 x 1010 possible inputs = 1.7 x 1010 tests• If only 3-way interactions, need only 33 tests• For 4-way interactions, need only 85 tests

What if we knew no failure involves more than 3 switch settings interacting?

33 tests for this range of fault detection

85 tests for this range of fault detection

That’s way better than 17 billion!


Example: Network Deadlock Detection

• “Simured” network simulator• Kernel of ~ 5,000 lines of C++ (not including GUI)

• Objective: detect configurations that can produce deadlock:

• Prevent connectivity loss when changing network• Attacks that could lock up network

• Compare effectiveness of random vs. combinatorial inputs

• Deadlock combinations discovered

Detection rates Deadlocks Detected:

combinatorial

t Tests500 pkts

1000 pkts

2000 pkts

4000 pkts

8000 pkts

2 28 0 0 0 0 03 161 2 3 2 3 34 752 14 14 14 14 14

Average Deadlocks Detected: random

t Tests500 pkts

1000 pkts

2000 pkts

4000 pkts

8000 pkts

2 28 0.63 0.25 0.75 0. 50 0. 753 161 3 3 3 3 34 752 10.13 11.75 10.38 13 13.25

ResultsDetected 14 configurations that can cause deadlock: 14/ 31,457,280 = 4.4 x 10-7

Combinatorial testing found more deadlocks than random, including some that might never have been found with random testing

Why do this testing? Risks:• accidental deadlock configuration: low• deadlock config discovered by attacker: much higher

(because they are looking for it)

Example: Laptop application testing

Problem: connect many peripherals, order of connection may prevent proper operation

Sequence Covering Array• With 6 events, all sequences = 6! = 720 tests• Only 10 tests needed for all 3-way sequences, even better for larger numbers of events;

• Results: problems detected with far fewer tests• Example: .*c.*f.*b.* covered. Any such 3-way seq covered.

Test Sequence1 a b c d e f2 f e d c b a3 d e f a b c4 c b a f e d5 b f a d c e6 e c d a f b7 a e f c b d8 d b c f e a9 c e a d b f

10 f b d a e c

What tools are available?• Covering array generator – basic tool for test input or

configurations;

• Sequence covering array generator – new concept; applies combinatorial methods to event sequence testing

• Combinatorial coverage measurement – detailed analysis of combination coverage; automated generation of supplemental tests; helpful for integrating c/t with existing test methods

• Domain/application specific tools:• Access control policy tester• .NET config file generator

• Smaller test sets faster, with a more advanced user interface• First parallelized covering array algorithm• More information per test

126001070048>1 dayNA47011625>1 dayNA65.03109416

1549313056>1 dayNA43.544580>1 dayNA18s42265

12764696>21 hour14763.541536540014843.0513634

3.079158>12 hour4720.71413102023880.364003

2.75101>1 hour1080.0011080.731200.81002

TimeSizeTimeSizeTimeSizeTimeSizeTimeSize

TVG (Open Source)TConfig (U. of Ottawa)Jenny (Open Source)ITCH (IBM)IPOGT-Way

New algorithms

Traffic Collision Avoidance System (TCAS): 273241102

Times in seconds

ACTS - Defining a new system

Constraints

Variable interaction strength

Covering array output

ACTS Users: > 1,000 organizations

Information Technology

Defense

Finance

Telecom

Output optionsMappable values

Degree of interaction coverage: 2Number of parameters: 12Number of tests: 100

-----------------------------

0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 1 1 1 1 2 0 1 0 1 0 2 0 2 2 1 0 0 1 0 1 0 1 3 0 3 1 0 1 1 1 0 0 0 1 0 0 4 2 1 0 2 1 0 1 1 0 1 0 5 0 0 1 0 1 1 1 0 1 2 0 6 0 0 0 1 0 1 0 1 0 3 0 7 0 1 1 2 0 1 1 0 1 0 0 8 1 0 0 0 0 0 0 1 0 1 0 9 2 1 1 1 1 0 0 1 0 2 1 0 1 0 1 Etc.

Human readableDegree of interaction coverage: 2Number of parameters: 12Maximum number of values per parameter: 10Number of configurations: 100-----------------------------------Configuration #1:

1 = Cur_Vertical_Sep=2992 = High_Confidence=true3 = Two_of_Three_Reports=true4 = Own_Tracked_Alt=15 = Other_Tracked_Alt=16 = Own_Tracked_Alt_Rate=6007 = Alt_Layer_Value=08 = Up_Separation=09 = Down_Separation=010 = Other_RAC=NO_INTENT11 = Other_Capability=TCAS_CA12 = Climb_Inhibit=true

Rick Kuhn Raghu Kacker [email protected] [email protected]

http://csrc.nist.gov/acts

Please contact us if you are interested.

Documents

NIST Computer Security Research, Development, and Outreach