Introduction to Fuzzing...3 Background In 2016, DARPA hosted fully automated hacking challenge Software components become more larger and complicated. → Many people try to automate

Introduction to FuzzingRen Kimura

[email protected]://ricsec.co.jp

https://ricsec.co.jp/

Ren Kimura

2

CVE-2019-14247, CVE-2019-14248, CVE-2019-14249, CVE-2019-14250, CVE-2019-16161, CVE-2019-16162, CVE-2019-16163, CVE-2019-16164, CVE-2019-16165, CVE-2019-16166, CVE-2019-16167, CVE-2019-19725

Ricerca Security, Inc. CEO

twitter: @RKX1209mail: [email protected]

#

#

#

#

#

#

3

Background

In 2016, DARPA hosted fully automated hacking challenge

Software components become more larger and complicated.

→ Many people try to automate analyzing process.

In 2016, DARPA hosted Cyber Grand Challenge (CGC)

→ Almost all winners used Fuzzing technique in the vulnerability detection process.

#

#

#

#

#

#

4

VulnerabilityFinding

TriageEvaluation

ExploitGeneration

“Fuzzing”, “Static Analysis”

“Symbolic Execution”

“Triage”, “Exploitability”

“Bug Reproduction”

“Automatic Exploit Generation”

(AEG)

Crash Inputs Triaged / POC Exploit Code

How to automate Hacking?

#

#

#

#

#

#

Mutation based Black box FuzzingVulnerability Finding

6

They randomly generates large amount of inputs and execute program with it.

Black box Fuzzing (Synopsys defensics, zzuf)

“a”

“kqeqert”

“G\x13\x02”

“iohbpofi9qnpiof”

“3i129074g”

They don’t observe program

behavior, learn nothing.

They continue dumb generation forever

#

#

#

#

#

#

#

#

#

7

Fuzzer Target Program(PUT)

Initial Seed

Mutator

Black box Fuzzing

They mutates initial seed, generate inputs and execute program with it.

#

#

#

#

#

#

#

#

#

8

Initial Seed

・・・

Mutation

They mutate initial seed to generate new random inputs

Input Generation (defensics, zzuf)

Initial.png, Initial.jpeg...

#

#

#

#

#

#

#

#

#

9

Bitflip mutationFlip n-th bit of input. Many fuzzers (AFL, zzuf..) use it.

0 0 1 0 1 0 0 1

0 0 1 0 1 1 0 1

BitFlip

#

#

#

#

#

#

#

#

#

10

Byteflip mutation

0 0 1 0 1 0 0 1

0 0 1 0 1 1 0 1

Byteflip

1 1 0 1 0 1 1 0

Flip n-th byte of input. Many fuzzers (AFL, zzuf..) use it.

#

#

#

#

#

#

#

#

#

11

Arithmetic mutationAdd, Subtract, Multiply or Divide crazy integer on n-th byte

0 0 1 0 1 0 0 1

0 0 1 0 1 1 0 11 1 0 1 0 1 1 0

Arithmetic operation +-*/ 256, U32_MAX, U32_MIN

#

#

#

#

#

#

#

#

#

12

Insert/Delete mutationInsert extra bytes at n-th offset. Delete n bytes subset of input

0 0 1 0 1 0 0 1

0 0 1 0 1 0 0 1

Insert/Delete

0 0 1 0 1 1 0 11 1 0 1 0 1 1 0

#

#

#

#

#

#

#

#

#

13

Hands-on 1 (zzuf)

“Using zzuf directly“ https://fuzzing-project.org/tutorial1.html

zzuf -s 0:1000000 -c -C 0 -q -T 3 objdump -x win9x.exe

zzuf -s <range> -T <timeout> <program> <initial seed>sudo apt-get install zzuf

Try other combination, like readelf -a /bin/ls, file, ...

#

#

#

#

#

#

https://fuzzing-project.org/tutorial1.html

#

#

#

Mutation based Grey box FuzzingVulnerability Finding

15


Initial Seed

Mutator

Feedback

Grey box Fuzzing

They generates large amount of inputs in the smart way with feedback.

#

#

#

#

#

#

#

#

#

16

How does fuzzer get feedback?


Initial Seed

Mutator

Feedback

They get some kind of feedback from program execution.

#

#

#

#

#

#

#

#

#

17

Feedback mechanism (edge coverage)

if (input[0] == ‘G’)

if (input[1] == ‘E’)

if (input[2] == ‘T’) if (isinteger(input))

if (!strcmp(input, “HTTP”))

2536

124

6210

8147

297

4010

Instrument unique random numbersto every basic blocks by compiler.

#

#

#

#

#

#

#

#

#

18

Feedback mechanism (edge coverage)


if (isinteger(input))


2536

124

297

4010

Calculate hash keys based on random numbers on program path.

Map hash keys to memory

key = nextBB ^ prevBB >> 1

#

#

#

#

#

#

#

#

#

19https://tunnelshade.in/blog/2018/01/afl-internals-compile-time-instrumentation/

Hands-on 2 (afl-gcc on binutils)

cd AFL && make git clone https://github.com/google/AFL

wget https://ftp.gnu.org/gnu/binutils/binutils-2.26.tar.gztar xf binutils-2.26.tar.gzcd binutils-2.26CC=/path/to/AFL/afl-gcc ./configure --disable-werrormake

#

#

#

#

#

#

https://tunnelshade.in/blog/2018/01/afl-internals-compile-time-instrumentation/

#

#

#

20https://tunnelshade.in/blog/2018/01/afl-internals-compile-time-instrumentation/

Hands-on 2 (afl-gcc on binutils)gdb -q binutils-2.26/binutils/nm-new(gdb) disas mainDump of assembler code for function main: 0x0000000000031ba0 <+0>: lea -0x98(%rsp),%rsp 0x0000000000031ba8 <+8>: mov %rdx,(%rsp) 0x0000000000031bac <+12>: mov %rcx,0x8(%rsp) 0x0000000000031bb1 <+17>: mov %rax,0x10(%rsp) 0x0000000000031bb6 <+22>: mov $0xab10,%rcx 0x0000000000031bbd <+29>: callq 0x3caf8 <__afl_maybe_log>

#

#

#

#

#

#

https://tunnelshade.in/blog/2018/01/afl-internals-compile-time-instrumentation/

#

#

#

21





2536

124

6210

8147

297

4010

Feedback mechanism (edge coverage)If we only have a binary executable...

Instrument unique random numbersto every basic blocks by emulator.

#

#

#

#

#

#

#

#

#

22

Black box vs Grey box Fuzzer

Unconditional Branch

Conditional Branch

Vunlerable

Control Flow Graph(CFG) of target program

“GET”

“HTTP”

“501”

“HTTP1.?”





#

#

#

#

#

#

#

#

#

23

Black-box Fuzzer (zzuf)Initial Seed

“a”

“GET”

“HTTP”

“501”

“HTTP1.?”





・・・・・

“kq” “\xfeXp\x2a”

Mutation

#

#

#

#

#

#

#

#

#

24

Black-box Fuzzer (zzuf)

“HTTP”

“501”

“HTTP1.?”





“a”

・・・

“kq” “G” “G”

Generated “G” from initial seed.

Initial Seed

Mutation

“GET”

#

#

#

#

#

#

#

#

#

25


“HTTP”

“501”

“HTTP1.?”





“a”

・・・

“kq” “GE”

“GE”

Initial Seed

Mutation

Generated “GE” from initial seed.

“GET”

#

#

#

#

#

#

#

#

#

26


“HTTP”

“501”

“HTTP1.?”





“a”

・・・

“kq”

“GET”

“GET”

Initial Seed

Mutation

Generated “GET” from initial seed.

#

#

#

#

#

#

#

#

#

27

Initial Seed

“a”

・・・

“kq”

Mutation

“GET”

Coverage information + Genetic Algorithm (GA)

Initial Seed

“a”

Mutation

“G”

“GE”

“GET”

Generation

Selection and Mutation based on Fitness Function

Grey-box Fuzzer (AFL, libfuzzer)

Mutation

Mutation

#

#

#

#

#

#

#

#

#

28

Mutator


Initial Seed

Mutator

Feedback

They generate inputs by mutating initial seed or parent seeds.

Initial Seed

“a”

Mutation

“G”

“GE”

“GET”

Mutation

Mutation

#

#

#

#

#

#

#

#

#

29

Initial Seed

“a” “G”

“GE”

“GET”

Fitness Function

Seed Scheduler (Selection)

Mutation

If generated input leads new program coverage, F(input) = 1.

Which inputs should be mutated?

Generate next inputs by mutatting parent inputs .

Grey-box Fuzzer (AFL, libfuzzer)Coverage information + Genetic Algorithm (GA)

Generation

Mutation

Mutation

Mutation

#

#

#

#

#

#

#

#

#

30

“GET<\x00”

“HTTP”

“501”

“HTTP1.?”





“GET”

“NULL” (Initial Seed)

“G” “HTTP” “501”

“GE”

“GET”

“GET<\x00”

“HTTP1.x”

“a”

“GE”

“G”

Many state-of-art fuzzers use genetic algorithm with coverage information.

Grey-box Fuzzer (AFL, libfuzzer) Coverage information + Genetic Algorithm (GA)

#

#

#

#

#

#

#

#

#

31

Hands-on 3 (AFL)

https://github.com/google/AFL

./afl-fuzz -i <initial seed dir> -t <time out> -m <memory limit> -o <output dir> -- <command line> @@

cd AFL && make git clone https://github.com/google/AFL

ls <output dir>/queue # Show all saved seedsls <output dir>/crashes # Show crash inputs

#

#

#

#

#

#


#

#

#

32

Hands-on 3 (AFL example, nm)


./afl-fuzz -i initial/ -t 10000 -m 1024 -o output -- binutils-2.26/binutils/nm-new @@

cp /bin/ls initial/mkdir initial


#

#

#

#

#

#


#

#

#

Generation based FuzzingVulnerability Finding

34

Highly structured inputs

Fuzzer

JavaScript

HTML

Firefox, Chrome(Target Program)

#

#

#

#

#

#

#

#

#

35

Rule

・・・

Generation

They generate large amount of inputs from rule

Generation based Fuzzing

Fileformat (PIT)

Grammar

#

#

#

#

#

#

#

#

#

36


Rule

Generator

Generation based Fuzzing

They generate inputs and execute program with it.

#

#

#

#

#

#

#

#

#

37

Hands-on 4 (PEACH)

Read: https://github.com/MozillaSecurity/peach

Run PEACH on Firefox

./peach.py -pit Pits/<component>/<format>/<name>.xml -target Pits/Targets/firefox.xml

-run Browser

#

#

#

#

#

#

https://github.com/MozillaSecurity/peach

#

#

#

38

They infer data dependency between API calls and generate valid stub code.

XNU kernel fuzzing with API Inferring

HyungSeok Han IMF: Inferred Model-based Fuzzer [CCS17]

#

#

#

#

#

#

#

#

#

White box FuzzingVulnerability Finding

40


Initial Seed

Executor

SMT solver

White box FuzzingThey generate inputs by solving constraints, using SMT solver.

Patrice Godefroid Automated Whitebox Fuzz Testing [NDSS08]

#

#

#

#

#

#

#

#

#

41

How to work?

Conditional Branch

Unconditional Branch

Vunlerable

Control Flow Graph(CFG) of target program

“GET<\x00”

“HTTP”

“501”

“HTTP1.?”





#

#

#

#

#

#

#

#

#

42

White box Fuzzer (SAGE)Initial seed

“GET<\x00”

“HTTP”

“501”

“HTTP1.?”




“a”(1) Execute program with initial input “a”

(2) Build constraints from trace

(input[0] != “G”) & (strcmp(input, HTTP)) &

(!isinteger(input))

(3) Negate one of the constraints

(input[0] == “G”) & (strcmp(input, HTTP)) &

(!isinteger(input))

(4) Solve constraints by SMT solver

→ next input “G”

#

#

#

#

#

#

#

#

#

43

“GET<\x00”

“HTTP”

“501”

“HTTP1.?”


if (input[2] == ‘T’)

“G”


Next seed

“G”(1) Execute program with initial input “G”


(input[0] == “G”) & (input[1] != ‘E’) &

(input[2] != “T”)


(input[0] == “G”) & (input[1] == ‘E’) &

(input[2] != “T”)


→ next input “GE”

White box Fuzzer (SAGE)

#

#

#

#

#

#

#

#

#

44

“GET<\x00”

“HTTP”

“501”

“HTTP1.?”



“GE”


Next seed

“GE”(1) Execute program with initial input “GE”


(input[0] == “G”) & (input[1] != ‘E’) &

(input[2] != “T”)


(input[0] == “G”) & (input[1] == ‘E’) &

(input[2] == “T”)


→ next input “GET”


#

#

#

#

#

#

#

#

#

45

“GET<\x00”

“HTTP”

“501”

“HTTP1.?”



“GET”


Next seed

“GET”

Congratulation!

We found 3 program path lead by

“G”, “GE” and “GET” inputs.


It’s also called

“Dynamic Symbolic Execution” (DSE)

#

#

#

#

#

#

#

#

#

46

White box vs Grey box FuzzerWhite box fuzzer entails significant computational cost (Build constraints, SMT solve)

{ C1 & C2 & C3 & C4 & C5 …

C100 & C101 & C102 …

….

C1000 && C1001 && C1002

….

C2360 & C2361 & C2362

…. }

C1

C2

C3

C2360

C2361

Thousand of Constraints….

SMT query (over 2000 queries)

PerformanceOverhead

#

#

#

#

#

#

#

#

#

47

White box vs Grey box FuzzerGrey box fuzzer is hard to overcome the long magic number comparison.

if (input == 0xdeadbeefcafebabe) {

crash();

}

Grey box way

White box way

Mutation “ 0x0” “0xdeadbeefcafebabe ”

P(crash) = 1/(2^64)

SMT solve“ 0x0” “0xdeadbeefcafebabe ”

#

#

#

#

#

#

#

#

#

48

Hybrid Fuzzing (Driller)Control Flow Graph(CFG) of target program

if (input == 0xdeadbeefcafebabe) if (input[0] == ‘A’’)

if (input[2] < 10)

Nick Stephen Driller: Augmenting Fuzzing Through Selective Symbolic Execution [NDSS16]

Grey-box fuzzingDynamic Symbolic Execution (DSE)

#

#

#

#

#

#

#

#

#

Feedback MechanismVulnerability Finding

50

Edge Coverage by Compiler, Emulator/Intel PT (AFL/kAFL)




2536

124

297

4010

Calculate hash keys based on random numbers on program path.

Sergej Schumilo kAFL: Hardware-Assisted Feedback Fuzzing for OS Kernels[UESNIX17]

#

#

#

#

#

#

#

#

#

51

Build CFG from binary

by Static Analysis

Sanjay Rawat VUzzer: Application-aware Evolutionary Fuzzing [NDSS 17]

Markov model on static analysis (VUzzer)




1.0

0.5 0.5

0.5 0.5

0.5

0.5

0.5

1.0

1.0

0.5

0.5

0.5

0.5

P(path) = 1.0*0.5*0.5*0.5*0.5 = 0.0625

F(path) = 1/P(path) = 1/0.0625 = 16

Binary executable formatMarkov Model

#

#

#

#

#

#

#

#

#

52

Dynamic Binary Rewriting (Chizpurfle, RetroWrite)

Stalker server rewrites the code block to instrument additional instructions, reveal basic block coverage.

Antonio Chizpurfle: A Gray-Box Android Fuzzer for Vendor Service Customizations [ISSRE17]Sushant Dinesh RetroWrite: Statically Instrumenting COTS Binaries for Fuzzing and Sanitization[S&P20]

They use frida,rewrite Android system services

#

#

#

#

#

#

#

#

#

State-of-art FuzzingVulnerability Finding

54


Rule

Mutator

Feedback

Smart Grey box Fuzzing (AFLSmart, Nautilus)

They generates large amount of inputs based on Rule with feedback.

#

#

#

#

#

#

#

#

#

55

Smart mutation with ruleMutate inputs with knowledge about grammar or file format.

Semantics Mutation

int f(int arg) { return g(arg); }

short f(int arg) { return h(arg); }Grammar File (EBNF)

Van-Thuan Pham Smart Greybox Fuzzing [TSE20]

File Format (PIT)

Cornelius Aschermann NAUTILUS: Fishing for Deep Bugs with Grammars [NDSS19]

#

#

#

#

#

#

#

#

#

56

Hands-on 6 (AFLSmart)

Read: https://github.com/aflsmart/aflsmart

Find Crash input of WavPack by AFLSmart

./afl-fuzz -h -i <initial seed> -o <output dir> -w peach -g <input model file>

-x <dictionary file> -- <command line> @@


#

#

#

#

#

#

https://github.com/aflsmart/aflsmart

#

#

#

57

Group work (Bug hunting in Real World)

Choose your favorite real world programs.

Try to find bug or vulnerabilities from them.

You can use any fuzzers.

You should try many (program, fuzzer) combinations!

#

#

#

#

#

#

#

#

#

58

Group work (Bug hunting in Real World)

afl-qemu (AFL against binary) https://github.com/google/AFL

VUzzer (Fuzzing with tant analysis) https://github.com/vusec/vuzzer

kAFL (Fuzzing linux kernel) https://github.com/RUB-SysSec/kAFL

Nautilus (Grammar fuzzing) https://github.com/RUB-SysSec/nautilus

Driller (Hybrid fuzzing) https://blog.grimm-co.com/post/guided-fuzzing-with-driller/

#

#

#

#

#

#


https://github.com/vusec/vuzzer

https://github.com/RUB-SysSec/kAFL

https://github.com/RUB-SysSec/nautilus

https://blog.grimm-co.com/post/guided-fuzzing-with-driller/

#

#

#

https://ricsec.co.jp

https://ricsec.co.jp/

Documents

Introduction to Fuzzing...3 Background In 2016, DARPA hosted fully automated hacking challenge Software components become more larger and complicated. → Many people try to automate