fighting the cyber threatsQasim Zaidi

Text

We were DDos’edwe must be doing something right !

Text

Denial of ServiceLegitimate users are denied service

Types

Volumetric (UDP Floods)

State Exhaustion (TCP Syn Attacks)

Application Layer Attacks (HTTP, DNS query flood)

Application 15%

State Exhaustion 20%

Volumetric 65%

Reflection AttacksDo not directly attack the Target.

Forge Reply to Address

Send request to normal servers

Trick them to reply to the Target

Makes it distributed and harder to deal with.

AmplificationA new class of reflection

Amplification attacks

Because a small question can have a big answer.

Why? How?

; <<>> DiG 9.8.3-P1 <<>>;; global options: +cmd;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 64739;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:;dig. IN ANY

;; AUTHORITY SECTION:. 73193 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2016041601 1800 900 604800 86400

;; Query time: 80 msec;; SERVER: 8.8.8.8#53(8.8.8.8);; WHEN: Sun Apr 17 10:19:42 2016;; MSG SIZE rcvd: 96

;; Got answer:;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39944;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 6, ADDITIONAL: 10

;; QUESTION SECTION:;yahoo.com. IN A

;; ANSWER SECTION:yahoo.com. 1762 IN A 98.139.183.24 yahoo.com. 1762 IN A 206.190.36.45 yahoo.com. 1762 IN A 98.138.253.109

;; AUTHORITY SECTION:yahoo.com. 17439 IN NS ns3.yahoo.com. yahoo.com. 17439 IN NS ns5.yahoo.com. yahoo.com. 17439 IN NS ns2.yahoo.com. yahoo.com. 17439 IN NS ns1.yahoo.com. yahoo.com. 17439 IN NS ns6.yahoo.com. yahoo.com. 17439 IN NS ns4.yahoo.com.

;; ADDITIONAL SECTION:ns1.yahoo.com. 1197500 IN A 68.180.131.16 ns1.yahoo.com. 66008 IN AAAA 2001:4998:130::1001 ns2.yahoo.com. 1197500 IN A 68.142.255.16 ns2.yahoo.com. 85955 IN AAAA 2001:4998:140::1002 ns3.yahoo.com. 1197585 IN A 203.84.221.53 ns3.yahoo.com. 73296 IN AAAA 2406:8600:b8:fe03::1003 ns4.yahoo.com. 1198687 IN A 98.138.11.157 ns5.yahoo.com. 1197585 IN A 119.160.247.124 ns6.yahoo.com. 160785 IN A 121.101.144.139 ns6.yahoo.com. 1762 IN AAAA 2406:2000:108:4::1006

;; Query time: 27 msec;; SERVER: 8.8.8.8#53(8.8.8.8);; WHEN: Sun Apr 17 10:19:42 2016;; MSG SIZE rcvd: 391

dig ANY yahoo.com @8.8.8.8

(64 bytes)

A (391 bytes)

6x amplification

The D in DDos

SSDP

Simple Service Discovery Protocol (UPnP)

Example: Used to discover printers on your network

SSDP Discovery - HTTP over UDP sent to a multicast address.

1. Recruiting Zombies

2. Flooding the victim

First Attack

Happened at 6 PM on a Monday

Website seemed slow

SSH to servers even slower

public ips private ips

dmesg outputUDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 318UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 318UDP: bad checksum. From 190.129.30.190:1900 to 182.253.224.184:80 ulen 347UDP: bad checksum. From 190.129.169.3:1900 to 182.253.224.184:80 ulen 291UDP: bad checksum. From 190.129.169.3:1900 to 182.253.224.184:80 ulen 291UDP: bad checksum. From 200.87.245.44:1900 to 182.253.224.184:80 ulen 311UDP: bad checksum. From 190.129.81.203:1900 to 182.253.224.184:80 ulen 281UDP: bad checksum. From 190.129.6.33:1900 to 182.253.224.184:80 ulen 301UDP: bad checksum. From 73.201.211.248:1900 to 182.253.224.184:80 ulen 253UDP: bad checksum. From 190.129.199.12:1900 to 182.253.224.184:80 ulen 347UDP: bad checksum. From 200.87.155.100:1900 to 182.253.224.184:80 ulen 285UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 300UDP: bad checksum. From 190.129.182.57:1900 to 182.253.224.184:80 ulen 280UDP: bad checksum. From 190.129.165.180:1900 to 182.253.224.184:80 ulen 237UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 306UDP: bad checksum. From 190.129.81.26:1900 to 182.253.224.184:80 ulen 283UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 300UDP: bad checksum. From 172.97.240.102:1900 to 182.253.224.184:80 ulen 343UDP: bad checksum. From 190.129.195.29:1900 to 182.253.224.184:80 ulen 246UDP: bad checksum. From 172.97.240.102:1900 to 182.253.224.184:80 ulen 237UDP: bad checksum. From 190.129.165.171:1900 to 182.253.224.184:80 ulen 301UDP: bad checksum. From 172.97.240.102:1900 to 182.253.224.184:80 ulen 237UDP: bad checksum. From 182.215.214.137:1900 to 182.253.224.184:80 ulen 302UDP: bad checksum. From 190.129.30.176:1900 to 182.253.224.184:80 ulen 289

First Response

sudo iptables -A INPUT -p udp -sport 1900 -j DROP

Drops all incoming packets with source Port 1900

saves some resources, but remember that packets still have to be processed by NIC card, and the pipe is still clogged.

dmesg output goes away, but recovery isn’t complete.

GEO IP Lookup

But we knew we haven’t yet

During Q4 (2015), repeat DDoS attacks were the norm, with an average of 24 attacks per targeted customer in Q4. Three targets were subject to more than 100 attacks each and one customer suffered 188 attacks – an average of

more than two per day. Source: Akamai

Attacker’s persist, especially if they don’t get

what they wanted.

Attack 2

The very next day, at 2 PM

Same attack vector, but more distributed

Lot’s of Indonesian IP addresses

Attacked all of our public IP’s, not DNS based.

identify

netstat

dmesg

iptraf netstat -iKernel Interface tableIface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flgem1 1500 0 266063410705 0 327198 0 269121217381 0 2 0 BMRUem2 1500 0 19266620548 0 197 0 20700650229 0 0 0 BMRUlo 16436 0 79744956 0 0 0 79744956 0 0 0 LRU

iptables/netfilter/tuning

kernel parameters tuning

NIC TX/RX Buffer tuning

sudo iptables -A INPUT -p udp --sport 1900 -j DROP

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

iptables -I INPUT -s <ipaddress> —j DROP

tcpkill / cutter

synproxy (against syn flood attacks)

sudo ethtool -g em1Ring parameters for em1:Pre-set maximums:RX: 2047 RX Mini: 0 RX Jumbo: 0TX: 511 Current hardware settings:RX: 200 RX Mini: 0 RX Jumbo: 0TX: 511

Know who to call @ ISP

tc / firehol

Ensure you can ssh to the server when your network is congested

Limit bandwidth

class ssh commit 2Mbit server ssh client ssh

class rsync commit 2Mbit max 10Mbit server rsync client rsync

private net

Minimize Attack Surface

private net

under attack

normalwhois tokopedia.com

1

2

2

Use a WAF / hide origin

–Johnny Appleseed

“Type a quote here.”