CSCE 548 Secure Software Development Risk-Based Security Testing

CSCE 548 CSCE 548 Secure Software Secure Software

DevelopmentDevelopment

Risk-Based Security TestingRisk-Based Security Testing

CSCE 548 - Farkas 2

ReadingReading

This lecture: – Risk-Based Security Testing, McGraw: Chapter 7

Next lecture:– Security Operations, McGraw: Chapter 9

CSCE 548 - Farkas 3

Application of TouchpointsApplication of Touchpoints

Requirement and Use cases

Architecture and Design

Test Plans Code Tests andTest Results

Feedback fromthe Field

5. Abuse cases

6. Security Requirements

2. Risk Analysis

External Review

4. Risk-Based Security Tests

1. Code Review(Tools)

2. Risk Analysis

3. Penetration Testing

7. Security Operations

CSCE 548 - Farkas 4

Software TestingSoftware Testing

Running a program or system with the intent of finding errors

Evaluating capability of the system and determining that its requirements are met

Physical processes vs. Software processes Testing purposes

– To improve quality– For Verification & Validation (V&V) – For reliability estimation

CSCE 548 - Farkas 5

Quality AssuranceQuality Assurance

External quality: correctness, reliability, usability, integrity

Interior (engineering) quality: efficiency, testability, documentation, structure

Future (adaptability) quality: flexibility, reusability, maintainability

CSCE 548 - Farkas 6

Correctness TestingCorrectness Testing

Black box: – Test data are derived from the specified

functional requirements without regard to the final program structure

– Data-driven, input/output driven, or requirements-based

– Functional testing – No implementation details of the code are

considered

CSCE 548 - Farkas 7

Correctness TestingCorrectness Testing

White box:– Software under test are visible to the tester – Testing plans: based on the details of the

software implementation – Test cases: derived from the program structure – Glass-box testing, logic-driven testing, or

design-based testing

CSCE 548 - Farkas 8

Performance TestingPerformance TestingGoal: bottleneck identification, performance

comparison and evaluation, etc.Explicit or implicit requirements"Performance bugs" – design problems Test: usage, throughput, stimulus-response

time, queue lengths, etc. Resources to be tested: network bandwidth

requirements, CPU cycles, disk space, disk access operations, memory usage, etc.

CSCE 548 - Farkas 9

Reliability TestingReliability Testing

Probability of failure-free operation of a system

Dependable software: it does not fail in unexpected or catastrophic ways

Difficult to test

CSCE 548 - Farkas 10

Security TestingSecurity Testing

Test: finding flaws in software can be exploited by attackers

Quality, reliability and security are tightly coupled

Software behavior testing– Need: risk-based approach using system

architecture information and attacker’s model


Risk-Based TestingRisk-Based Testing

Identify risksCreate tests to address identified risksSecurity testing vs. penetration testing

– Level of approach– Timing of testing


Penetration TestingPenetration Testing

Performed after the software is completed– Evaluate operational environment– Dynamic behavior

Outside in activity – defending perimetersCursory



Can be applied before the product is completed

Different levels of testing (e.g., component/unit level vs. system level)

Testing environmentDetailed


Risk Analysis Risk Analysis

Design phase analysis:– Identify and rank risks– Discusses inter-component assumptions

Component/unit testing– Test for:

Unauthorized misuse of and access to the target assets Violations of assumptions

– Breaking system into a number of discrete parts– Risk can be mitigated within the bounds of contextual

assumptions


System-Level TestingSystem-Level Testing

Focus on the properties of the integrated software system

Penetration testing = Security testingUsing data flow diagrams, models, and

inter-component documentations, identify– Inter-component failures– Design level security risks

Use misuse cases to enhance test plan


Behavior in the Presence of Behavior in the Presence of Malicious AttackMalicious Attack

What happens when the software fails?– Safety critical systems

Track risk over timeSecurity relative to

– Information and services protected– Skills and resources of adversaries– Cost of protection

System vulnerabilities


VulnerabilitiesVulnerabilities

Design-level– Hardest to detect– Prevalent and critical– Requires great expertise to detect – hard to

automateImplementation specific

– Critical– Easier to detect – some automation



Functional security testing: testing security mechanisms for functional capabilities

Adversarial security testing: risk-based security testing– Understanding and simulating the attacker’s approach

Both approaches must be used Security attacks may ignore the security

mechanism to exploits of the software defects


Who Should Perform the Test?Who Should Perform the Test?

Standard testing organizations– Functional testing

Software security professionals– Risk-based security testing– Important: expertise and experience


How to Test?How to Test?

White box analysis– Understanding and analyzing source code and design– Very effective finding programming errors– Can be supported by automated static analyzer– Disadvantage: high rate of false positives

Black box analysis– Analyze a running program– Probe the program with various input (malicious input)– No need for any code – can be tested remotely


Malicious InputMalicious Input

Software: takes inputTrust input?

– Malformed or malicious input may lead to security compromise

– What is the input? Data vs. control

Attacker toolkit


What Else?What Else?

Testing for malicious input: necessary but NOT sufficient

Risk-based security testing – Planning tests (use forest-level view)– Need operational aspects

System state vs. applications used Multithread system – time-based attacks


Next ClassNext Class

Security Operations

Documents

CSCE 548 Secure Software Development Risk-Based Security Testing