HES2011 - Eloi Vanderbeken - Hackito Ergo Sum Crackme

IntroductionVerification algorithm

ObfuscationEncryption layers

Direct native API callAnti-X

How to break itPossible evolutions

Hackito Ergo Sum Crackme

Eloi Vanderbeken

eloi.vanderbeken (at) ens-cachan.fr

09 April 2011

Eloi Vanderbeken Hackito Ergo Sum 2011





Plan

1 Introduction

2 Verification algorithm

3 ObfuscationInstruction mutationControl Flow Graph (CFG) obfuscation

4 Encryption layers

5 Direct native API call

6 Anti-X

7 How to break it

8 Possible evolutions






About me

Last year of master’s degree in cryptology and softwaresecurity at Ecole Normale Superieure of Cachan

Reversing software protection for 6 years

Participate to several Capture The Flags in my spare time






Plan

1 Introduction



4 Encryption layers


6 Anti-X

7 How to break it







Some facts and numbers

First crackme

2 months

6000 lines of python

Use TTASM and BeaEngine

Randomly generated

Only 3 valid solutions






Plan

1 Introduction



4 Encryption layers


6 Anti-X

7 How to break it







Algorithm

Custom RC4 (initial table changed, starts with ”HESFTW”)

Serial is used as a 256-byte key

Generate the first 7 DWORDs

Combine them with 0xDEADBEEF to generate a DWORD

Use it as a constant to calculate the length of the last layers






Instruction mutationControl Flow Graph (CFG) obfuscation

Plan

1 Introduction



4 Encryption layers


6 Anti-X

7 How to break it








Plan

1 Introduction



4 Encryption layers


6 Anti-X

7 How to break it








Instruction mutation

Make a static analysis to get freely modifiable registers

Use them to mutate instructions

Main rule: if your code use an instruction, it has to be usedeverywhere ⇒ no difference between the useful code and thegarbage one







Plan

1 Introduction



4 Encryption layers


6 Anti-X

7 How to break it








Control Flow Graph (CFG) obfuscation

Before instruction mutation:

Duplicate some piece of code

Add a fake or non determinist test to choose which block willbe executed

After:

Make a static analysis to know which flags are set/unset foreach instruction

Add conditional jump which are never/always taken accordingto the flags

Shuffle blocks


Figure: (incomplete) CFG of a layer made by IDA





Plan

1 Introduction



4 Encryption layers


6 Anti-X

7 How to break it







Layers

How to make a layer the most annoying possible?

Hide the end address of the encryption layer

Hide the size of the encrypted code






Layers (continued)

Use Linear Feedback Shift Register (LFSR) to encrypt n timesboth end address and a dword counter

At execution, decrypt them one time at each loop execution

When the dword counter has been decrypted, jump to thedecrypted address

Use indirect jump to avoid breakpoint:cmp counter, CST ‖ setz cl ‖ jmp [ecx +

addresses]

Bonus: LFSR having the biggest possible period, you can addas many fake comparison as you want






Plan

1 Introduction



4 Encryption layers


6 Anti-X

7 How to break it







Direct native API call

Get syscall numbers at initialisation

Use sysenter (do not work on 64bit systems)

Make random invalid or non invasives (ZwYieldExecution)syscalls to prevent conditional breakpoints onKiFastSystemCallRet






Plan

1 Introduction



4 Encryption layers


6 Anti-X

7 How to break it







Classical anti-debuggers tricks

Well known anti-debuggers trick made difficult to bypass becauseof direct use of sysenter:

NtQueryInformationProcess + ProcessDebugPort

NtSetInformationThread + ThreadHideFromDebugger

NtSetContextThread to delete hardware breakpoints






Anti-instrumentation framework

Detect hook on KiUserExceptionDispatcher andKiUserCallbackDispatcher

Detect stack reuse by instrumentation code: place a constantin esp-4, execute a sysenter, check if esp-4 has been changed






Anti data-tainting

After execution of each part of the algorithm, result is copied usingindirect dependance

AntiTaintMemcpy (BYTE* source, BYTE *dest, int size)

{

for (int t =0 t < size ++t) {

BYTE val = 0

do {

BOOL isDiff = FALSE

if (source[t] != val)

isDiff = True

if (! isDiff)

dest[t] = val

++val

} while (val != 0)

}

}






Plan

1 Introduction



4 Encryption layers


6 Anti-X

7 How to break it







So how to solve it?

API init Set memory breakpoint on ntdll PE

Anti-Dbg Don’t use debugger, inject a DLL and useVectoredExceptionHandler to set HBP

Anti-HBP Replace ZwSetThreadContext syscall number by anharmless or invalid one

ReadFile Use kernel debugger or replace syscall numbers by aGDI one which will call KiUserCallbackDispatcher

Obfuscation Step the code :)






How to find a valid serial

Bruteforce (JB Bedrune)

Use of the RC4 internal table created with the key published(Kyriakos Economou and Mark Wodrich)

Generate a valid keystream, use a backward algorithm to findan internal table which generate this keystream, find a keywhich generate this table (Me :) )






Plan

1 Introduction



4 Encryption layers


6 Anti-X

7 How to break it







Some ideas for the next year ;)

Recode all the crackme generation in C

Use XED (by intel) to disassemble and reassemble instruction

Multi-thread

Virtualisation

More mutation rules, more complex instructions (FPU, MMX,SSE etc.)

More anti-X






Thanks

Thank you !






Questions ?

Questions ?


Technology

HES2011 - Eloi Vanderbeken - Hackito Ergo Sum Crackme