Enhancing Availability and Security Through Failure-Oblivious Computing

Enhancing Availability and Security Through Failure-Oblivious ComputingMartin Rinard, Cristian Cadar, Daniel

Dumitran, Daniel Roy,

and William Beebee, Jr.

Introduction

Memory errors are a common source of program failures ML and Java use dynamic checks to eliminate

such errors Assumption:

Invalid memory access unsafe to continue the execution

Failure-Oblivious Computing

Instead of throwing an exception or terminating Ignores any memory access errors and continue Read (an out of bounds array element)

Just read a manufactured value Write (an out of bounds array element)

Discard the value

Wrong Results?

Many programs can continue to run As long as errors do not corrupt the program’s

address space or data structures Failure-oblivious computing can improve the

availability, robustness, and security of such programs

Shouldn’t We Stop at the First Error? Debugging may not be an option

No source code Not enough time

Failure-oblivious computing can still provide acceptable service Better than no service

Servers and Buffer-Overrun Attacks When a program allocates a fixed-size buffer

Then fails to check that each input string fits in the buffer

A long input string containing executable code can overwrites the stack contents Can coerce the server into running arbitrary code

Servers and Buffer-Overrun Attacks Failure-oblivious computing discards the

excess characters, preserving the integrity of the stack Server detects invalid request and returns an error Converts a dangerous attack into an invalid input

Multiple Items or Outputs

Many programs (e.g. mail readers) process multiple items

Some applications generate multiple outputs Some outputs are more important than others

Without failure-oblivious computing Failure to process one can prevent the program to

process the rest

Benefits and Drawbacks

+ Increased resilience Graceful degradation and continue to operate

successfully on most of its inputs

+ Increased security Can survive stack overruns

+ Reduced development costs Pressured to find and eliminate all disruptive bugs

+ Reduced administration overhead Reduce the success rate of attacks

Benefits and Drawbacks

+ Safer integration Lowers the risks to use foreign components

- May generate unacceptable results Inevitable consequence for better resiliency Need to convert unanticipated states into

anticipated error states

Scope

Interactive computing environments Mailers Servers System administration tools Operating systems Document processing systems

Mission critical applications Halting is not an option

Scope

Less appropriate for programs No easy way to determine whether the output is

correct Safety-critical applications

Safer to terminate the computation

Example

A Mutt procedure Takes an input string Returns an encoded output string Fails to allocate sufficient space

With standard compilers Writes succeed, corrupt the address space, and

program segfaults With safe-C compilers

Mutt exits before presenting the GUI

Example

With the failure-oblivious compiler The returned string is incorrect Server responds with an error

Failure oblivious approach works Mostly correct programs

With subtle errors

Implementation

Failure oblivious compiler Generate two kinds of additional code

Checking code Discard erroneous writes Manufactures values for erroneous reads

Continuation code Executes when checking code detects an attempt to

perform illegal access

Checking Code

Jones and Kelly’s Scheme Track the locations to structs, arrays, variables Each data item is padded with an extra byte

Initialized to ILLEGAL Check the status of each pointer before

dereferencing it

Continuation Code

Write continuation code Discards the value

Read continuation code Redirects the read to a preallocated buffer of

values Iterates through all small integers Increasing the chance to exit loops

To avoid nontermination Mostly 0s and 1s

Continuation Code

Optional logging Can be used to track down errors

Failure-oblivious computing Can also reduce the inventive to eliminate errors

Case Studies

Recompiled widely-used open-source programs with known memory errors Pine (mail user agent) Midnight commander (file manager) Sendmail (mail transfer agent) Mutt (mail user agent) Samba (file server) WsMp3 (mp3 server) Apache (http server)

Methodology

Compare each program compiled differently By a standard C compiler By the CRED safe-C compiler By the failure-oblivious compiler

Workloads Contain inputs that exploit known security

vulnerabilities

Pine 4.44

Fails to correctly parse certain legal From fields Possible to execute arbitrary code

Standard version: crashed Safe version: terminated with an error

message Failure oblivious version: continued to run

Was able to forward the read and forward the message with the problematic From field

Midnight Commander

Problems with symbolic links in tgz files Standard version: segfaulted Safe version: terminated with an error

message Failure-oblivious version: continued to run

Sendmail 8.11.6

Allows root privilege to execute arbitrary code on the machine running the Sendmail server

Standard version: vulnerable to an attack to gain the root shell

Safe version: exited with an error message Failure-oblivious version: not vulnerable to

the attack

Mutt 1.4

Memory error in the conversion from UTF-8 to UTF-7 string formats

Standard version: crashed Safe version: exited with an error message Failure oblivious version: continued to

execute 6x slow down Took about 1 second to load 3,000 messages

Samba 2.2.5

Memory corruption error A remote user can obtain the root shell

Standard version: vulnerable to an attack to gain the root shell

Safe version: functional until the attack The child process exited

Failure oblivious version: continued to run Similar performance compared to the safe version

WsMp3 0.0.5

Memory-error vulnerability Standard version: segfaulted Safe version: crashed the entire server

Single threaded Failure-oblivious version: survived the attack

Apache 2.0.47

mod_alias contains a memory-error vulnerability

Standard version: child process segfaulted Safe version: child process exited properly Failure-oblivious version: child process

redirected the attacking request to a nonexistent URL The child process stayed alive and processed

subsequent requests correctly

Gzip 1.2.4a

Memory error in its file name processing code An attacker can run arbitrary code

Standard version: segfaulted Remaining files were not processed

Safe version: exited at the problematic file Failure-oblivious version: prompted an error

message for the problematic files Proceeded to process all remaining files 10x slow down (1.2 MB/sec)

Discussion

Failure oblivious versions survived all memory-corruption attempts Work well for this class of applications

One input has a minimal effect on the next input Unless it corrupts the data structures or address space

Little performance degradation for interactive programs

Safe versions are prone to DoS attacks Tend to terminate prematurely

Related Work

Any safe-C compiler can be modified to implement a failure-oblivious compiler Discard writes Manufacture values for unsafe reads

Typically < 2x slow down Occasionally 8x slow down Does not perceptibly degrade the response times

of interactive programs Also I/O-bound programs

Safe Languages

Jave and ML Modify the exception handling code

Discard illegal writes Return manufactured values for illegal reads

Traditional Error Recovery

Traditional approaches Reboot Checkpointing Partial system restarts Hardware redundancy

Failure-oblivious computing reduces down time and vulnerabilities to persistent errors Restarting Pine will not solve the problem

Other Approaches

Data structure repair Failure-oblivious approach is preventive

Statically detect all buffer-overrun errors May conservatively reject almost working code

Buffer-overrun detection tools Detect overwriting the return address Detect overwriting function pointers Failure-oblivious approach prevents the attack

from corrupting the address space

Conclusion

Failure-oblivious computation enhances availability, resilience, and security Converts dangerous unknown system states to

known error cases