Anti-Unpacking techniques

Peter FerrieSenior Anti-virus Researcher

1 May, 2008

1

Classes of unpackers

Memory dumperDebuggerEmulatorCode-bufferW-X interceptor

2Peter Ferrie, Microsoft Corporation

Anti-dumping

SizeOfImage Field in PEB When altered, prevents debugger from attaching Causes incorrect number of pages to be dumped

Erasing the header Removes section information Not obvious how many pages should be dumped Used by Yoda’s Crypter

Nanomites Introduced in Armadillo Replace branches with int 3 and encoded pointers

3Peter Ferrie, Microsoft Corporation

Anti-dumping (cont.)

Stolen bytes Introduced in ASProtect Copy bytes elsewhere and execute from there Optionally destroy original bytes

Guard pages Allows on-demand decryption or decompression Used by Armadillo (decryption) Used by Shrinker (decompression)

Imports Table is often removed, encrypted or compressed Sometimes erased after first use Functions can be redirected to another location

4Peter Ferrie, Microsoft Corporation

Anti-dumping (cont.)

Virtual machines Uses p-code instead of native code P-code can be polymorphic (Themida) P-code can contain anti-debugging (HyperUnpackMe2) Interpreter can be obfuscated (Virtual CPU)

5Peter Ferrie, Microsoft Corporation

Anti-debugging

NtGlobalFlag Field in PEB Debugger typically causes certain flags to be set Very common method Very commonly incorrect (cmp instead of test) Very unreliable (several ways to set/clear flags) Used by ExeCryptor

Heap flags Pointer is field in PEB Two sets of flags Debugger typically causes certain flags to be set Also very unreliable (flags controllable like NtGlobalFlag)

6Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Heap Tail markers appear if certain flag is set Debugger typically causes flag to be set Marker is constant and easy to find Used by Themida

IsDebuggerPresent() Introduced in Windows 95 Reads field in PEB One of the most common methods Returns true if a debugger is present

7Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

CheckRemoteDebuggerPresent() Introduced in Windows XP SP1 Wrapper around NtQueryInformationProcess (ProcessDebugPort) Returns true if a debugger is present

NtQueryInformationProcess() Can be called with ProcessDebugPort class Returns EPROCESS->DebugPort value Non-zero if a debugger is present Used by MSLRH

8Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Debug objects Another use for NtQueryInformationProcess() Introduced in Windows XP Can be called with ProcessDebugObjectHandle class Returns non-zero if a debugger is present Used by HyperUnpackMe2

More debug objects Yet another use for NtQueryInformationProcess() Also introduced in Windows XP Can be called with ProcessDebugFlags class Returns inverse of EPROCESS->NoDebugInherit bit Returns zero if a debugger is present Used by HyperUnpackMe2

9Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Kernel debugger Final use for NtQueryInformationProcess() Can be called with SystemKernelDebuggerInformation class Not supported by any version of Windows(?) Returns non-zero in AH register if a debugger is present Used by SafeDisc

NtQueryObject() Supported since Windows NT Extended in Windows XP to include debug object Request all objects Search for "DebugObject" Check associated handle count Non-zero if a debugger is present

10Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Thread hiding NtSetInformationThread() method Introduced in Windows 2000 Can be called with HideThreadFromDebugger class Debugger receives no further events for that thread Used by HyperUnpackMe2

OpenProcess() Csrss.exe process can be opened If debug privilege is obtained first Even though it's a system process Process can be opened with full access rights Allows thread injection and system-wide DoS

11Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

CloseHandle() Different behaviour if debugger is present Kernel raises EXCEPTION_INVALID_HANDLE For invalid handle if debugger is present Exception can be intercepted by debugged process

OutputDebugString() Different behaviour if debugger is present Kernel clears thread error code If debugger is present

ReadFile() Indirect breakpoint removal Read data from file over location of breakpoint

12Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

WriteProcessMemory() Alternative breakpoint removal Write data from memory over location of breakpoint Used by NsAnti

UnhandledExceptionFilter() Called as last resort if no other handlers succeed Not called if a debugger is present Exception is passed to debugger instead

BlockInput() Blocks mouse and keyboard events Used by Yoda's Protector

13Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

SuspendThread() Can be used on remote processes Such as debuggers Used by Yoda's Protector

Guard pages Certain debuggers might not support them properly Access should cause EXCEPTION_GUARD_PAGE OllyDbg uses guard pages for memory breakpoints Consumes breakpoint and allows execution Used by PC Guard

14Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Alternative desktop Sessions support multiple desktops A new desktop can be created and made active No obvious way to switch back Used by HyperUnpackMe2

Prefetch queue Queue is not flushed when rep instruction is used Allows for some old self-overwriting tricks Two known variations of the trick One is used by Invius The other is used by Obsidium

15Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Hardware breakpoints Exception handler receives context structure Structure contains all CPU debug registers Non-zero values suggest presence of debugger Handler can also remove breakpoints and resume

Instruction counting Handler can insert own hardware breakpoints Then count number of exceptions that occur Debugger might skip some breakpoints

Execution timing Single-stepping through code take a long time Delay can be detected Several time sources available

16Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Exceptions Obfuscated method to move EIP around Infinite variety of possible implementations

Header entrypoint Some debuggers don't use VirtualProtectEx() So header remains write-protected Cannot set initial breakpoint Code runs freely

Parent process Should be shell process if launched by clicking Usually Explorer.exe, unless debugger is involved Used by Yoda's Protector

17Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Self-execution Simple method to escape from debugger Synchronisation object prevents infinite executions Used by MSLRH

Process name Process names can be enumerated Names can be compared to "watch list" List commonly includes anti-malware software

18Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Threads Multiple threads are hard to trace Some anti-malware emulators don't support them One thread is often devoted to debugger detection Can also be used to detect code changes Used by PE-Crypt32

Self-debugging Effective anti-debugging method Only one debugger can attach to a process Defeated by zeroing EPROCESS->DebugPort Or call DebugActiveProcessStop() on Windows XP Used by Armadillo

19Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Disassembly Can be used to steal instructions Safer method to detect breakpoints Can also be used to detect detours Problematic if routine is hot-patched Since that will be a detour

TLS callback Relatively little-known method to execute code Code executes before main entrypoint Also executes before inserted breakpoint Debuggers typically don't handle it Used by ExeCryptor

20Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

Device names A way to communicate with drivers Also a way to detect presence of drivers List of names often target SoftICE, RegMon and FileMon

SoftICE-specific Driver names can be enumerated Version information can be queried Product and copyright strings can be compared to "watch list" Harder to change than renaming the driver List of strings are usually SoftICE-related

21Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

SoftICE-specific (cont.) Interrupt 1 is normally not invokable from ring 3 SoftICE hooks interrupt 1 and allows ring 3 access So wrong exception when SoftICE is running Used by SafeDisc

OllyDbg-specific Cannot handle unusual NumberOfRvaAndSizes value Some unchecked fields allow memory allocation DoS Initial ESI register value is -1 on Windows XP Looks like a detection method It's just a coincidence

22Peter Ferrie, Microsoft Corporation

Anti-debugging (cont.)

OllyDbg-specific (cont.) Passes user-defined data directly to _vsprintf() Leads to DoS condition Debugger window can be found by calling FindWindow("OLLYDBG")

HideDebugger-specific Plug-in for OllyDbg Detectable by far jump at OpenProcess()+6

ImmunityDebugger-specific Based on OllyDbg Shares many of the same vulnerabilities

WinDbg-specific Debugger window can be found by calling FindWindow("WinDbgFrameClass")

23Peter Ferrie, Microsoft Corporation

Anti-emulating

Interrupt 3 Size of instruction is not checked Using "cd 03" instead of "cc" instruction -> EIP points to "03" Prefixed "cc" instruction works, too Except perhaps in some emulators Used by TryGames

Time-locks Effective anti-emulation technique Causes all iterations to be consumed before anything happens Some loops can be skipped arbitrarily Some loops cannot (results might be used for decryption keys) Used by Tibs

24Peter Ferrie, Microsoft Corporation

Anti-emulating (cont.)

Invalid API parameters Some emulators do not support invalid parameters in APIs An expected error code can be used for decryption keys No error code means no decryption Used by Tibs

GetProcAddress There are too many possible APIs for emulators to handle So some emulators always return a value for GetProcAddress() Sometimes GetProcAddress() should return 0 A non-zero value reveals the emulator Used by NsAnti

25Peter Ferrie, Microsoft Corporation

Anti-emulating (cont.)

"Modern" CPU instructions Some emulators do not handle modern CPU instructions Some malware use FPU and MMX as do-nothing instructions Emulators might give up early as a result Used by NsAnti

Undocumented instructions Some emulators do not handle undocumented CPU instructions Some instructions more undocumented than others Example: multi-byte NOP Example: fully allocated PREFETCH

26Peter Ferrie, Microsoft Corporation

Anti-emulating (cont.)

Selector verification Some emulators pretend to be Windows NT-based While using Windows 9x-based selector values A CS selector value larger than 0xff reveals this Used by MSLRH

Memory layout Certain in-memory structures are at predictable memory locations Example: RTL_USER_PROCESS_PARAMETERS at 0x20000 If they are missing, the program might not run Used by TryGames

27Peter Ferrie, Microsoft Corporation

Anti-emulating (cont.)

File-format tricks Non-aligned SizeOfImage

Windows will silently round up the value Overlapping structures

Tools such as IDA have a problem with this Non-standard NumberOfRvaAndSizes

SoftICE and OllyDbg have a problem with this Non-aligned SizeOfRawData

Windows will silently round up the value Non-aligned PointerToRawData

Windows will silently round down the value No section table

Allowed when SectionAlignment is less than 4kb Header becomes writable and executable

28Peter Ferrie, Microsoft Corporation

Anti-intercepting

Write->Exec Some interceptors watch for write-then-exec Executing dummy just-written instruction can fool them Used by ASPack, but probably for multi-processor support

Write^Exec Some interceptors prevent write and exec on the same page Change can be detected indirectly Kernel functions return error when writing to read-only pages VirtualQuery() and VirtualProtect() return old page attributes

29Peter Ferrie, Microsoft Corporation

Thank you

More techniques in the paperhttp://pferrie.tripod.comQuestions?

30Peter Ferrie, Microsoft Corporation