Testing for Software Security

ECEN5053 Software Engineering of Distributed Systems

University of Colorado, Boulder

Testing for Software Security, Hebert Thompson, James Whittaker, Dr. Dobb’s Journal, November, 2002, pp. 24-34

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

When is a security bug not like a bug? Traditional non-security bugs -- often defined as a

violation of a specification. Security bugs -- additional behavior, not

originally intended– Meanwhile, it is doing what it is supposed to do– Traditional techniques not good at finding– Even in inspections, tend to look for

• missing behavior• incorrect behavior

– Neglect to look for ... undesirable side-effects

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Intended vs. Implemented Behavior

Traditional faults

Intended Functionality

Actual Software Functionality

Unintended, undocumented, unknown functionality

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Traditional faults Incorrect

– Supposed to do A but did B instead

Missing– Supposed to do A and B but did only A.

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Security Bugs Side effects

– Supposed to do A, and it did.– In the course of doing A, it also did B

Monitoring for side effects and their impact on security can be challenging– Side effects can be subtle and hidden– Examples: file writes, registry entries, extra

network packets with unencrypted data

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Tools Commercially available tools

– Mutek’s AppSight

http://www.identify.com/products/appsightsuite.html– Holodeck Lite

http://se.fit.edu/holodeck/

(freeware developed by the authors using techniques similar to those in the handout Listing One to help easily monitor and obstruct common system calls)

– Write your own customized monitoring solution

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Ways in / sources of interaction with the environment Human interface -- UI

– Set of API’s get input from kbd, mouse, etc.– Concerns: unauthorized access, privilege

escalation and sabotage File system

– Provides data stored in binary or text format– Trusted to store sensitive data– Test how stored, retrieved, encrypted, managed

API -- input in form of return values of API calls Operating system kernel -- memory, file pointers,

time and date functions, etc.

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Memory is vulnerable Any information an app uses passes through

memory eventually– Encrypted -- usually ok– Decrypted and stored -- at risk of being read– Encryption keys, CD keys, passwords and other

sensitive information are eventually used in an unencrypted form• Must protect their exposure in memory

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Stress Stress testing for low memory and other faulty

operating conditions that may cause an application to crash

App’s tolerance to environmental stress can prevent – denial of service– situations where app may crash before

completing an important task like encrypting passwords• Once it crashes, the state of stored data is ... ?

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

What won’t work Look at each method of input delivery Bombard that interface with input

Why that won’t work well enough Most revealing attacks require you to apply inputs

through multiple interfaces

Search said these categories can be used to expose vulnerabilities– Dependency attacks– Design-and-implementation attacks

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Attacking Dependencies Apps rely heavily on their environment to work

properly Not as overt as human input but there are lots of

sources of input– Like any input, if software receives a value

outside of its expected range, it can fail. Environment failures lead to calls on error-

handling code (if it exists)– Error handlers are the security weak point of an

application

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Testing error handlers Failures in the sw’s environment that exercise

these code paths are difficult to produce in a test lab situation– Tests that involve disk errors, memory failures,

and network problems are only superficially explored

– Illusion of security but ...• Servers do run out of disk space• Network connectivity can be intermittent• File permissions can be set improperly

Need to integrate failures into tests to evaluate

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Just do the impossible ... Create environmental failures

– Tamper with the application code– Simulate specific failing responses from the o.s.– Yeah, right

• Takes huge amount of time, effort, and expertise needed to simulate such a failure

• Determine where in the code the app uses these resources

• How to make appropriate changes to simulate a real failure

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Alternative approach Run-time fault injection

– Simulating errors to the app in a black-box fashion at run time

Advantage– Nonintrusive– Lets you test production binaries, not contrived

versions of the app that have phony return values hard-coded

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

HOW Overwrite the first few bytes of the actual function

to be called in the process space and insert a JMP statement to fault injection code in its place.

Modify import address tables (For doing this in Windows environment, authors recommend Jeffrey Richter, Programming Applications for Microsoft Windows, 4th Edition, Microsoft Press, 1999)

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

WHY? You redirect a particular system call to your own

impostor function Can log events

– Watch the application for file, memory, and registry activity

In control, you can forward a system request to the actual OS function or deny the request by returning any error message you choose.

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

What about registry bugs? Security problem with the “registry” is trust When developers read from the registry, assume

– values are accurate– haven’t been tampered with maliciously– especially, if their code wrote those values in

the first place “Try and buy” -- users have limited functionality

or a time limit in which to try the software.– App can be unlocked if purchased/registered– App may check a registry key at startup

• protected with weak encryption or it’s a 1/0

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Test app’s use of corrupted files & names App reads/writes hundreds of files

– Handle bad data gracefully without• exposing sensitive information• allowing unsafe behavior

Test with the approach that can happen but is often not tested for– Change a file in a way not covered in

requirements• Numerical data --> add text characters

– “Successful” test results in denial of service• Crashes the application or the whole system• May expose data during the crash

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Test low memory/disk/network availability Deprive the app of resources to see robustness

under stress Rule of thumb -- block a resource when an app

seems most in need of it– memory when in an intense computation– disk errors -- introduce faults when doing

writes/reads (modify code in Listing One to intercept appropriate system functions)

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Attack (Test) Design and Implementation There are subtle security implications made during

the design phase Even if design is secure, choices made during

implementation can impact security Tests that expose these:

– Force all error messages– Seek unprotected test APIs– Overflow input buffers– Connect to all ports

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Force all error messages Purposes

– Try values that should result in error msgs to see how many are handled properly

– Make sure error msgs do not reveal unintended information

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Seek unprotected APIs Difficult to test complex apps by relying on the

APIs extended for normal users alone.– Many designers therefore include additional

hooks (extended but not published API) • These often bypass normal security because

they are intended for use by the good guys• Intended for custom test harnesses• They mean to remove them ... but then ...

they’ll be handy for the next release• So integrated into the code and testing

process, removal may destabilize the code If purposely left in, verify they are not trouble if

found

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Overflow input buffers Test ability to handle long strings in input fields

– Especially where long strings are entered into fields that have an assumed, but not enforced, length, e.g. ZIP codes, state names

API calls notorious for unconstrained inputs– GUI can filter inputs as they are entered– API parameters dealt with internally

• checks to ensure values are appropriate before they are used

– Most vulnerable -- seldom used or supporting legacy functionality

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Connect to all ports Sometimes apps open custom ports to connect

with remote servers– Create maintenance channels to automatic

updates– Relic from test automation– http://www.ntbugtraq.com/ for many cases

where these are left open and unsecured If this is done in the app

– Test the port in the release version– Check what kind of data flows through it

November 18, 2002

Testing for SW Security, ECEN5053, University of

Colorado, Boulder

Conclusion These attacks by testers can expose vulnerabilities

before release Only part of complete security-testing methods This area of research is in the infant stages