Upload
keith
View
48
Download
1
Embed Size (px)
DESCRIPTION
A Binary Agent Technology for COTS Software Integrity. Richard Schooler Anant Agarwal InCert Software. Operating System. Input. COTS Binary. COTS Binary. SAP. Output. The Mission Critical Environment. The development environment. The deployment environment. COTS Binary. - PowerPoint PPT Presentation
Citation preview
A Binary Agent Technology for COTS Software Integrity
Richard Schooler Anant Agarwal
InCert Software
The Mission Critical Environment
Output
Input OperatingSystem
SAP
COTSBinary
The developmentenvironment The deployment
environment
COTSBinary
Objective
COTSBinary
Input
Output
OperatingSystem
SAP
To improve the integrity of thedeployment environmentwith COTS softwarein the presence ofattacks, bugs
Our Focus
COTSBinary
Input
Output
OperatingSystem
SAP
Outer security defences will be breached by attackers
Use a practical, systems level approach – execution-time monitoring
On COTS program or data corruption, rapidly
d- detect problemsa- trigger an alarmp- try to protectr- recover
Our Approach: Execution-Time Monitoring of COTS through Binary Instrumentation
The developmentenvironment
The deploymentenvironment
COTSNewMissing
sourceLegacy
COTS Binary
COTSNewMissing
sourceLegacy
d- Policy specs for detectiond- Heartbeat insertiond- Argument range checksd- Rare code execution/sigs.a- Alarm messages to consolep- Defaults for fault tolerancep- Access constraints, redund.r- Logging
COTSNewMissing
sourceLegacy
Our Approach to Integrity A pragmatic system level approach Works with COTS binaries, even legacy codes Allows a user to establish desired security levels and
to some extent modify policy on the fly Allows monitoring of all user code execution, not
selected OS calls Can work completely at the user’s deployment site
Drawbacks of Binary Insertion Specific to a single platform, needs new technology
development for different platform Challenging to relate low-level observable events
back to high-level user actions hard to detect some types of intrusions that only
affect data corruption hard to protect or correct problems at higher
semantic levels
Three Major Components in the Prototype,Three Major Tasks
Core technology for customizable agent insertion into PC/NT, PC/Linux
Anomaly detection and reporting Rapid recovery and problem pinpointing
Selected Challenges and Ideas Core technology for agent insertion into binary
How to deal with unknown relocations, e.g., for dusty decks – incremental control and dataflow analysis; an integrated static and dynamic method
Dealing with multithreading, in particular, time syncing and monitoring events in a distributed environment
Anomaly detection Runtime comparison against execution path signatures? State machines for control flow checks (e.g., Abraham) How to minimize runtime overhead – borrow compiler optimization
techniques (e.g., steal registers, inline code, sampling, multilevel checks) Rapid recovery and problem pinpointing technology
Instrumentation records trace in circular buffer to pinpoint problem and facilitate recovery from crash/alert following an actual or suspected attack
Can we get data values? Use dataflow analysis and offline simulation to obtain intermediate data values
Measures of Success Core technology for agent insertion into binary:
Can we handle all binaries, even dusty decks? Target: Performance degradation to be under 1 percent
Anomaly detection What fraction of injected problems can we detect
automatically with user spec
Rapid recovery technology Performance degradation to be under 1 percent Can we cut recovery time significantly? We will measure
recovery time with and without As a bonus, can we catch problems before system goes down?
We are building a prototype system, work with real users, and measure
Current Progress Work on NT binary insertion prototype ongoing Demo of early capability showing
instrumentation simple recovery log detecting application has crashed taking control and writing out log
Handling multithreading, DLLs imminent Ongoing thinking on detection capability
AGENTtest al,0x3jnz 0x1143
AGENTadd ebx,ecxjc 0x1101
AGENTshr edx,0x1add ebx,edx
AGENTtest al,0x3jnz 0x1143
AGENTinc eaxadd ecx,ediadd edx,esicmp eax,0xa
1
2
3 4
5
... 1 2 4 5
while ((c = ++ci)) { INSTRUCTION_ITERATOR ii = c->Instructions(); while ((inst = ++ii)) inst->Lift(null_state); while ((inst = ++ii)) inst->Lift(null_state);
while ((c = ++ci)) { INSTRUCTION_ITERATOR ii = c->Instructions(); while ((inst = ++ii)) inst->Lift(null_state); while ((inst = ++ii)) inst->Lift(null_state);
Instrumentation for Recovery: Basic Idea At each program block, record the progress of program
execution.
Instrumentation Runtime
Naïve implementation:
•Write block id
•Increment index (in memory)
•Check for wrap-around
Too big/slow: many instructions per block; large buffer
DAG Header
•setup new trace record
DAG Block 1
•set bit in current record
DAG id
2
3 4
5
DAG id
DAG id
Instrumentation Example 1
test al,0x3jnz 0x1143add ebx,ecxjmp short 0x1156shr edx,0x1add ebx,edxinc eaxadd ecx,ediadd edx,esicmp eax,byte 0xajc 0x1101
Original
Instrumentation Example 2
mov eax,[0x300c]mov ecx,[byte eax+0xc]mov [byte eax+0xc],dword 0x1mov edx,[byte eax+0x8]mov [dword eax+edx+0xffe],ecxadd edx,0x4and edx,[byte eax+0x4]mov [byte eax+0x8],edxtest al,0x3jnz 0x1143or [byte ebp+0xe],byte 0x2add ebx,ecxjmp short 0x1156or [byte ebp+0xe],byte 0x1shr edx,0x1add ebx,edxor [byte ebp+0xe],byte 0x4inc eaxadd ecx,ediadd edx,esicmp eax,byte 0xajc 0x1101
Instrumentation
Original
Instrumentation Example 3
mov eax,[0x300c]mov ecx,[byte eax+0xc]mov [byte eax+0xc],dword 0x1mov edx,[byte eax+0x8]mov [dword eax+edx+0xffe],ecxadd edx,0x4and edx,[byte eax+0x4]mov [byte eax+0x8],edxtest al,0x3jnz 0x1147or [byte ebp+0xe],byte 0x2add ebx,ecxjmp short 0x1164or [byte ebp+0xe],byte 0x1shr edx,0x1add ebx,edxor [byte ecp+0xe],byte 0x4inc eaxadd ecx,ediadd edx,esicmp eax,byte 0xajc 0x1103
Instrumentation
Displacement Fixup
Original
Instrumentation Example 4
mov eax,[0x300c]mov ecx,[byte eax+0xc]mov [byte eax+0xc],dword 0x1mov edx,[byte eax+0x8]mov [dword eax+edx+0xffe],ecxadd edx,0x4and edx,[byte eax+0x4]mov [byte eax+0x8],edxtest al,0x3jnz 0x1147or [byte ebp+0xe],byte 0x2add ebx,ecxjmp short 0x1164or [byte ebp+0xe],byte 0x1shr edx,0x1add ebx,edxmov ecp,ebpor [byte ecp+0xe],byte 0x4inc eaxadd ecx,ediadd edx,esicmp eax,byte 0xajc 0x1103
Instrumentation
Displacement Fixup
Original
Register Transfer
Related Approaches Applied at source level during development
e.g., type based safety; work of Lee et al. Applied at link time with special object formats
e.g., software fault isolation; work of Pandey et al. Applied through interpretor
e.g., safe Java interpreters Applied during program execution – works with existing COTS
e.g., sandboxing through middleware software e.g., modify OS with extensions to catch OS calls,
Ghosh e.g., wrap application to steer OS calls to
intermediary, Ghosh
The Current Commercial Solution: Middleware
COTSBinary
Input
Output
OperatingSystem
SAP
Middlewarewrappers
•Slow•Cannot deal with viruses•Cannot improve faulttolerance of COTS package itself
Sandboxing through OS Call Interception(OS extensions or wrapping executable)
COTSBinary
Input
Output
OperatingSystem
SAP
•This is the right first step, and less intrusive than binary instrumentation
•Cannot improve fault tolerance of COTS package itself
•Cannot handle communication through shared memory
•Cannot observe internal app behavior
•OS extensions can slow OS for others
Summary A systems approach to COTS Integrity Approach based on execution-time
monitoring using binary insertion We have an early prototype version of NT
binary insertion implemented