Introduction to Application Security Testing

Security Verified

© 2012 iCode information security All rights reserved

Introducing Security Testing

Mohamed Ridha Chebbi, CISSP [email protected] Mobile : +216 26 88 10 11

mailto:[email protected]�

Security Verified


Analyzing Applications with a Security Mindset

Testing the security of software requires a certain mindset, as a security test engineer needs to look at software from the perspective of someone who wants to attack it.

Adopting this mindset involves understanding the main information sources for security testing and how input can affect software.

Testing the security of a given piece of software often entails searching a very large set of inputs in order to find issues that can compromise that security.

Automation plays an important role in this search.

Security Verified


The Value of Security Testing

In the past, Microsoft products used to be the primary targets for attackers. As a result of their prominent position in the industry, Microsoft operating systems received high scrutiny from the security community. This coupled with the fact that Microsoft products did contain a fair number of vulnerabilities, hackers and vulnerability finders had little need and interest in searching other platforms. The ubiquity of Microsoft products exacerbated this situation, as viruses and worms were able to cause epidemics by taking advantage of vulnerabilities in commonly used server and client applications. The Security Development Life Cycle (SDL) introduced at Microsoft in 2002 has raised the bar. Vulnerability finders have to work harder to penetrate Microsoft applications, and the focus of the security research community is now shifting to those vendors and platforms once considered to be secure. Well-engineered security testing plays a key role in the effectiveness of the SDL. Vulnerabilities found during the SDL have less chance of damaging customer confidence in Microsoft, and removing critical flaws before shipping leaves attackers with fewer reasons to focus on Microsoft.

Security Verified


Security Testing and the SDL

Anecdotal evidence indicates that engineers spend less time testing for security bugs in products covered by the SDL. As the design and coding for these products is more secure in the first place, security testing in the SDL is more concerned with verifying that the product is secure, rather than with finding and eliminating security bugs. The purpose of security testing in the SDL is to allow the product team to perform a thorough assessment of the security and privacy of their application. Performing security testing during the verification phase of the SDL allows the product team to:

Perform adequate penetration testing activities on new and legacy code. Verify that the application is adequately reviewed for security vulnerabilities. Ensure that threats against the application are properly mitigated. Document the evidence indicating that users can trust the application. Perform a security push for those legacy components that were not covered by the SDL.

Security Verified


Functional Testing vs Security Testing

Security testing and functional testing differ in four main areas:

Identifying helpful sources of information for test planning : Functional testing involves testing an application under realistic user scenarios, while in security testing the test engineer frequently deals with scenarios that may not be realistic for the common user.

Choosing an appropriate set of inputs : During functional testing the test engineer will most commonly use expected inputs derived from use cases, whereas during security testing the test engineer is required to use unexpected and malicious inputs that may expose a security problem.

Automating security tests. Interfaces that are not visible to the human eye are most interesting for security testing. In order to be viewed, such interfaces often require the use of specialized tools. Security test engineers must be familiar with these tools and their limitations. During functional testing it is simpler to automate a testing process, as results for a specific test are easily verified. Establishing whether a security defect has been triggered during security testing is much harder.

Deciding when testing is complete : When carrying out functional testing, it is straightforward to determine when testing is complete. When carrying out security testing, however, deciding when to stop is more complicated because the test engineer needs to help to quantify the risk and decide whether enough testing has been done to declare the product "safe".

Security Verified


Gathering Information for Security Testing

The User Interface as a Source of Information :

The user interface is the first place to investigate when planning security testing. The information on the screen is primarily directed at legitimate users, but it is important to remember that hackers are users as well, and they can learn quite a bit from this information.

Identify the Field :

When testing a Web application, testers should be aware of how their interactions with the client will affect the behavior of the server application

Error Messages as a Source of Information :

Many applications output error messages to their users to inform them on what the expected format of input is. This can provide very useful information to the tester on the possible assumptions that developers made on the type of data that are expected through a given input vector.

Abuse Cases as a Source of Information :

Whereas functional test engineers consider use cases, security test engineers must consider abuse cases.

Abuse cases can often be gleaned from sources of historical information such as:

CERT and Bugtraq, which contain security advisories and discussions about existing and new vulnerabilities.

Black-hat literature.

http://www.cert.org/�

http://www.securityfocus.com/�

Security Verified


Considering Inputs with a Security Testing Mindset

Identifying Entry Points of the Application :

Not all inputs come from the keyboard or mouse. From a security perspective, the Graphical User Interface (GUI) is often the least of concerns, and testers need to worry more about input sources that are not quite so visible. In fact, a security tester has to learn not to think like a GUI tester. There are four sources of input that need to be considered when planning security testing :

OS/RTE : Input from the OS and the Run-time Environment (RTE) includes input from the OS APIs and any run-time environment that is supporting the application, such as the Common Language Run-time (CLR). User Interface (UI) : Examples of user interfaces are a local GUI and a Web front-end. External Resource : A Web server and a database are examples of resources that are external to the application. File System : Input from the file system includes temporary files, registry values, and configuration files.

Security Verified


Categorizing Inputs

Categorizing Dangerous Inputs

Lists of dangerous inputs are a key asset for security test engineers.

Within an organization, such lists are often kept secret for fear that they may be used against a system by attackers.

A group should be diligent about doing the necessary research to ensure it has a complete list, as well as ensuring that everyone in the organization is aware of such a list and utilizes it.

Dangerous input can be categorized into the following groups:

Long strings

Format strings

Numeric boundaries

Scripts

Code

OS commands

Control characters

Error codes

Return values

Note that such lists are never static

Security Verified


Automating Security Testing

The Value of Automated Security Testing Larger coverage : overage refers to the percentage of security-critical functions that are exercised by different inputs during a security test. Regression testing : During regression testing an application is tested for newly introduced bugs, usually by re-running previous tests and checking whether previously fixed faults have re-emerged Efficiency : Automated security tests are more efficient than manual testing. Automated tests are repeatable with no human intervention, as opposed to manual testing, which is far more time-consuming.

Automation allows for more targeted manual testing as they disclose areas of potential weakness in various execution, trust, and I/O boundaries, which means the test engineer need not try to find all weaknesses manually, but can focus on key areas and provide feedback to threat modeling and other phases of SDL.

Security Verified


Generating the Input

In order to generate inputs that are dangerous to your application, a list detailing all possible inputs must be created. This list might be generated in real time as the software is being tested, manually or by using a fuzz test automation suite, or it may already be available to you from other security tests applied to similar products. If the latter, you must keep in mind that often a list is not cross-applicable, and you may need to modify it or create your own list.

Some interfaces, such as the network and the file system, lend themselves well to automation. For other interfaces it might be more difficult to automate testing because the instrumentation of them is more difficult and/or because their outputs are harder to observe.

Security Verified


Software you Actually Build

The actual behavior of an application, as coded by developers, rarely matches perfectly with the intended software behavior, as specified in the design documents. Most security bugs are found in the area where behavior that was not part of the application’s intended behavior has been inadvertently implemented by developers.

Security Verified


Quantifying the Risk

Quantifying the Risk : To help quantify the risk, you have to ask the following: Was the process which was established to prevent and detect security flaws fully implemented?

How many unmitigated vulnerabilities are currently extant?

Is additional testing necessary?

What can we do to make this product safer?

Assigning Severity: To help test engineers assign severity to defects, the SDL has established four different levels of severity:

Critical : Remote, anonymous user escalation of privilege

Arbitrary code execution

Important DoS (low bandwidth attack, blue-screen, or long duration)

Local elevation of privilege

Information disclosure with privacy implications

Tampering with user data

Spoofing a user or computer

Moderate Temporary DoS

Information disclosure, general

Low Non-persistent or hard-to-replicate issues

Security Verified


Review the Process

Collecting the Proper Metrics: To make a well-informed decision on whether to stop testing requires relevant metrics. questions need to be asked :

How well did the team: - Create required documentation?

- Adhere to best practices? - Use all appropriate tools? - Respond to problems that arose?

How well did the testing team test: - Untrusted interfaces? resources? protocols? files? How well did the testing team cover the:

-Threat model? -Set of possible attacks that apply to the application?

Security Verified


Security Assurance

Mohamed Ridha Chebbi, CISSP

Security Verified


Security Assurance Maturity

Low

Low

High

High

People & Process

Tool

s &

Tec

hnol

ogy

Typical Progression Curve

Pit of Despair Security as Core Business Process Panic Scramble

Security Verified


Stage 1 : Reactionary

Security Verified


Stage 2 : Apply Band Aids

Security Verified


Stage 3 : Beyond Badness-Ometer

Security Verified


Stage 4 : Teach a Man to Fish

Security Verified


Stage 5 : Homo Securous

Security Verified


Typical Level 5 SAMM Assessments in SDLC

Pen Test

Software Development Life Cycle

Decision Support & Process Optimization

Code Review

Design Build Deploy Operate

Black Box Testing

White Box Testing

Build & Test Automation

Security Verified


Security Testing Simplified

Mohamed Ridha Chebbi, CISSP

Security Verified


Design Build Test Deploy

Static Testing

PEN Testing

Dynamic Testing

Internal Review

New Versions/Releases

Automated Testing During SDLC

Annually

Operate

…

…

Security Verified


Design Build Test Deploy

Static Assessment

PEN Testing

Dynamic Assessment

Internal Review

New Versions/Releases

Application Security Assurance

Annually

Operate

…

…

Security Verified


Static Testing Vectors

Security Verified


CWE ID CWE Name Directory Traversal

73 External Control of File Name or Path Untrusted Search Path

114 Process Control 426 Untrusted Search Path 427 Uncontrolled Search Path Element

Buffer Overflow 121 Stack-based Buffer Overflow 122 Heap-based Buffer Overflow

Buffer Management Errors 129 Unchecked Array Indexing 135 Incorrect Calculation of Multi-Byte String Length 170 Improper Null Termination 249 Often Misused: Path Manipulation 125 Out-of-bounds Read

Format String 134 Uncontrolled Format String

Numeric Errors 190 Integer Overflow or Wraparound 191 Integer Underflow (Wrap or Wraparound) 192 Integer Coercion Error 195 Signed to Unsigned Conversion Error 196 Unsigned to Signed Conversion Error 197 Numeric Truncation Error

API Abuse 234 Failure to Handle Missing Parameter 243 Failure to Change Working Directory in chroot Jail 193 Off-by-one Error 245 J2EE Bad Practices: Direct Management of Connections 560 Use of umask() with chmod-style Argument

Static Assessment Vectors

Security Verified


Information Leakage 201 Information Leak Through Sent Data ✔ 209 Error Message Information Leak ✔ 215 Information Leak Through Debug Information ✔ 497 Information Leak of System Data ✔ 498 Information Leak through Class Cloning ✔

Dangerous Functions 242 Use of Inherently Dangerous Function ✔

Credentials Management 256 Plaintext Storage of a Password ✔ 259 Hard-Coded Password ✔

Authentication Issues 247 Reliance on DNS Lookups in a Security Decision ✔

Cryptographic Issues 311 Failure to Encrypt Sensitive Data ✔ 316 Plaintext Storage in Memory ✔ 321 Use of Hard-coded Cryptographic Key ✔ 326 Weak Encryption ✔ 331 Insufficient Entropy ✔ 347 Improper Verification of Cryptographic Signature ✔ 354 Improper Validation of Integrity Check Value ✔

614 Sensitive Cookie in HTTPS Session Without 'Secure' ✔

310 Cryptographic Issues ✔

CWE ID CWE Name


Security Verified


Race Conditions 364 Signal Handler Race Condition ✔

367 Time-of-check Time-of-use (TOCTOU) Race Condition ✔ Time and State

377 Insecure Temporary File ✔

378 Creation of Temporary File With Insecure Permissions ✔ 382 J2EE Bad Practices: Use of System.exit() ✔ 557 Concurrency Issues ✔ 708 Incorrect Ownership Assignment ✔

Session Fixation 384 Session Fixation ✔

Error Handling 391 Unchecked Error Condition ✔

Code Quality 111 Direct Use of Unsafe JNI ✔

401 Failure to Release Memory Before Removing Last Reference ('Memory Leak') ✔

404 Improper Resource Shutdown or Release ✔ 415 Double Free ✔ 416 Use After Free ✔

470 Use of Externally-Controlled Input to Select Classes or Code ('Unsafe Reflection') ✔

479 Unsafe Function Call from a Signal Handler ✔ 489 Leftover Debug Code ✔ 597 Use of Wrong Operator in String Comparison ✔

CWE ID CWE Name


Security Verified


OS Command Injection

78 Failure to Preserve OS Command Structure ('OS Command Injection') ✔

SQL Injection 89 Failure to Preserve SQL Query Structure ('SQL Injection') ✔

564 SQL Injection: Hibernate ✔ Cross-Site Scripting (XSS)

80 Improper Sanitization of Script-Related HTML Tags in a Web Page (Basic XSS) ✔

CRLF Injection 93 Failure to Sanitize CRLF Sequences ('CRLF Injection') ✔

113 Failure to Sanitize CRLF Sequences in HTTP Headers ('HTTP Response Splitting') ✔

117 Improper Output Sanitization for Logs ✔ Encapsulation

494 Download of Code Without Integrity Check ✔ 501 Trust Boundary Violation Malicious ✔

Malicious Code 506 Embedded Malicious Code ✔ 511 Logic/time bomb ✔

656 Reliance on Security through Obscurity Untrusted Initialization ✔

Untrusted Initialization 15 External Control of System or Configuration Setting ✔

Insufficient Input Validation 103 Struts: Incomplete validate() Method Definition ✔ 104 Struts: Form Bean Does Not Extend Validation Class ✔ 112 Missing XML Validation ✔ 601 URL Redirection to Untrusted Site ('Open Redirect') ✔

CWE ID CWE Name


Security Verified


Automated Scanning •Cross Site Scripting (XSS) •Injection Flaws •Malicious File execution •Insecure Direct Object Reference •Information Leakage and Improper Error Handling •Insecure Cryptographic Storage

Dynamic/Manual Testing Required to identify •Cross Site Request Forgery (CSRF) •Broken Authentication and Session Management •Insecure Communications •Failure to restrict URL access

OWASP Top 10 Compliance

Security Verified


Automated Scanning CWE-20: Improper Input Validation CWE-89: Failure to Preserve SQL Query Structure ('SQL Injection') CWE-78: Injection Flaws Failure to Preserve OS Command Structure (‘OS Command Injection’) CWE-209: Error Message Information Leak CWE-426: Untrusted Search Path CWE-494: Download of Code Without Integrity Check CWE-259: Hard-Coded Password CWE-330: Use of Insufficiently Random Values CWE-116: Improper Encoding or Escaping of Output CWE-79: Failure to Preserve Web Page Structure ('Cross-site Scripting') CWE-362: Race Condition CWE-119: Failure to Constrain Operations within the Bounds of a Memory Buffer CWE-404: Improper Resource Shutdown or Release CWE-682: Incorrect Calculation CWE-327: Use of a Broken or Risky Cryptographic Algorithm CWE-73: External Control of File Name or Path

SANS Top 25 Compliance

Security Verified


Dynamic/Manual Testing Required to identify CWE-319: Cleartext Transmission of Sensitive Information CWE-665: Improper Initialization CWE-285: Improper Access Control (Authorization) CWE-642: External Control of Critical State Data CWE-352: Cross Site Request Forgery (CSRF) CWE-94: Failure to Control Generation of Code (aka 'Code Injection‘) CWE-732: Insecure Permission Assignment for Critical Resource CWE-602: Client-Side Enforcement of Server-Side Security CWE-250: Execution with Unnecessary Privileges

SANS Top 25 Compliance

Security Verified


Dynamic Testing Vectors

Security Verified


06-01 Application Path Disclosure 06-02 Browse HTTP from HTTPS 06-03 Brute Force Login 06-04 Auth Over HTTP 06-05 Credit Card Disclosure 06-06 Document Caching 06-07 Find Pages 06-08 Flush Permissions 06-09 Form Caching 06-10 HTTP Parameter Pollution 06-11 Login Redirect 06-12 Non-masked Password 06-13 Plateform Path Disclosure 06-14 Privacy Notification 06-15 Private IP Disclosure 06-16 Remote File Inclusion 06-17 Social Insurance Disclosure 06-18 Social Security Disclosure 06-19 Username & Password in HTTP request

Internal Security

Internal Security Vector

Security Verified


07-01 Application Exception 07-02 Application Path Disclosure 07-03 Blind SQL Injection 07-04 Buffer Overflow 07-05 Cross-Site Scripting 07-06 Format String 07-07 Frame Injection 07-08 GET for POST 07-09 HTTP Parameter Pollution 07-10 HTTP Response Splitting 07-11 Integer Overflow 07-12 LDAP Exception 07-13 LDAP Injection 07-14 Open Redirect 07-15 Parameter Addition 07-16 PHP & Perl Code Injection 07-17 Remote File Inclusion 07-18 SQL Disclosure 07-19 SQL Error Message 07-20 SSI Injection 07-21 Unix Command Injection 07-22 Unix Relative Path 07-23 Windows Command Injection 07-24 Windows Relative Path

Entry Validation

Entry Validation Vector

Security Verified


09-01 Acquire Session ID 09-02 Authentication Bypass 09-03 Authorization Boundary 09-04 Clickjacking 09-05 Cross Site Request Forgery 09-06 Ineffective Session Termination 09-07 Javascript Hijacking 09-08 Lockout 09-09 Password Change 09-10 Privilege Escalation 09-11 Register Password 09-12 Run Traversal Only 09-13 Session Fixation 09-14 Session Hijacking 09-15 Session ID Identification 09-16 Session ID in URL 09-17 Session ID Length 09-18 Session ID Randomness 09-19 Weak Password

Session Mgmt

Session Management Vector

Security Verified


OWASP Top 10

10-A01-01 Blind SQL Injection 10-A01-02 LDAP Exception 10-A01-03 LDAP Injection 10-A01-04 SQL Error Message 10-A01-05 SQL Disclosure 10-A01-06 Unix Command Injection 10-A01-07 Windows Command Injection 10-A02-01 Cross-Site Scripting 10-A03-01 Authentication Bypass 10-A03-02 Cookie Vulnerabilities 10-A03-03 Login Redirect 10-A03-04 Non-masked Password 10-A03-05 Non-SSL Password 10-A03-06 Password Autocomplete 10-A03-07 Session ID Randomness 10-A04-01 Frame Injection 10-A04-02 Remote File Inclusion 10-A06-01 Cross Site Request Forgery 10-A06-01 Application Exception 10-A06-02 Application Path Disclosure 10-A06-03 HTTP Methods 10-A06-04 Directory Browsing 10-A06-05 Plateform Path Disclosure 10-A06-06 Web Server Vulnerabilities 10-A08-01 File & Directory Discovery 10-A08-02 Unix Relative Path 10-A08-03 Windows Relative Path 10-A09-01 Browse HTTP from HTTPS 10-A09-02 Non-SSL Form 10-A09-03 SSL Checks 10-A10-01 Cross-Frame Scripting 10-A10-02 HTTP Response Splitting 10-A10-03 Open Redirect

OWASP TOP 10 Compliance

Security Verified


Thanks

Mohamed Ridha Chebbi, CISSP [email protected] Mobile : +216 26 88 10 11

mailto:[email protected]�

Technology

Introduction to Application Security Testing