Advanced Netfilter Iptables - linuxwall.info Firewall Talk Julien... · Advanced Netfilter & Iptables Understand the Linux Firewall and make it do magical tricks Julien Vehent - PLUG

AdvancedNetfilter

&Iptables

Understand the Linux Firewall and make it do magical tricks

Julien Vehent - PLUG - September 2012

1Tuesday, September 11, 12

Julien Vehent - jve.linuxwall.info - 2012

Who am I ?

• Julien Vehent http://jve.linuxwall.info

• Security Engineer, Linux Sysadmin, and a bit of a Dev

• Love Networks, Packets, Routers, Firewalls, etc...

• I work for in the Philly suburb

• I write and publish at http://wiki.linuxwall.info• http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:networking:traffic_control

• http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:dspam

• http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:postfix:dkimproxy


http://jve.linuxwall.info

http://jve.linuxwall.info

http://wiki.linuxwall.info

http://wiki.linuxwall.info

http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:networking:traffic_control

http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:networking:traffic_control

http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:dspam

http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:dspam

http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:postfix:dkimproxy

http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:postfix:dkimproxy


Everything is packet

Physical

Data Link

Network

Transport

App

TCP/IP Model

T +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+R | Source Port | Destination Port |A +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+N | Sequence Number |S +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+P | Acknowledgment Number |O +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+R P| Data | |U|A|P|R|S|F| |T R| Offset| Reserved |R|C|S|S|Y|I| Window | O| | |G|K|H|T|N|N| |C T+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+O O| Checksum | Urgent Pointer |N C+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+T O| Options | Padding |R L+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+O | data |L +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+I +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+N |Version| IHL |Type of Service| Total Length |T +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+E | Identification |Flags| Fragment Offset |R +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+N | Time to Live | Protocol | Header Checksum |E +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+T | Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+P | Destination Address |R +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+O | Options | Padding |T +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+E +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+T | DESTINATION MAC ADDRESS |H +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+E | DEST MAC | SOURCE MAC ADDRESS |R +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+N | SRC MAC ADDRESS | T Y P E |E +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+T



Linux and sk_buff

struct sk_buff { /* These two members must be first. */ struct sk_buff *next; struct sk_buff *prev; struct sock *sk; struct net_device *dev; struct nf_conntrack *nfct; struct nf_bridge_info *nf_bridge; union { __u32 mark; __u32 dropcount; __u32 avail_size; }; __u16 vlan_tci; sk_buff_data_t transport_header; sk_buff_data_t network_header; sk_buff_data_t mac_header; sk_buff_data_t tail; sk_buff_data_t end;

}; sample from <include/linux/skbuff.h>

Linux stores each ingress and egress packet into an instance of the sk_buff structure.

Netfilter applies packet filters on the sk_buff structures.



Iptables 101

Stateless firewalliptables -A INPUT -p tcp --dport 80 -j ACCEPTiptables -A OUTPUT -p tcp --sport 80 -j ACCEPT

Stateful versioniptables -I INPUT -p tcp -m conntrack \ --ctstate ESTABLISHED -j ACCEPTiptables -A INPUT -p tcp --dport 80 -m conntrack \ --ctstate NEW -j ACCEPT

Loggingiptables -A INPUT -p tcp --dport 80 -m conntrack \ --ctstate NEW -j LOG --log-prefix “In HTTP ”



Netfilter Packet Flow(sane version)

Network InterfaceP P

PREROUTING POSTROUTING

FORWARD

INPUT OUTPUT

Socket

applicationread write



Introducing Tables

filter: defaultnat: only traversed on state==NEW

mangle: for packet alterationraw: to disable conntrack



Tables

FILTER: implied when nothing specifiediptables -A INPUT -p tcp --dport 80 -j ACCEPTiptables -t filter -A INPUT -p tcp ...

RAW: don’t waste ressources on conntrackiptables -t raw -I PREROUTING -i lo -j NOTRACKiptables -t raw -I OUTPUT -o lo -j NOTRACKiptables -t raw -I PREROUTING -p udp --sport 53 \-s 8.8.8.8 -j NOTRACK

MANGLE: manipulate packets right before transmitiptables -t mangle -A POSTROUTING -o eth0 \-p tcp --tcp-flags SYN SYN --dport 443 \-j CONNMARK --set-mark 300



Stateful (conntrack) vs Stateless• Conntrack knows the state of each TCP & UDP

connection on the system

• When a new packet arrive, conntrack can tell if it is part of an existing connection

• Downside: maintaining state information can be expensive for very high traffic system (start worrying around 10,000 packets per second)

# grep "dport=22 " /proc/net/ip_conntrack

tcp 6 299 ESTABLISHED src=10.1.0.145 dst=10.1.0.25 sport=54656 dport=22 packets=819 bytes=62669 src=10.1.0.25 dst=10.1.0.145 sport=22 dport=54656 packets=436 bytes=251335 [ASSURED] mark=0 secmark=0 use=2

see linux/net/netfilter/nf_conntrack_proto_tcp.c9Tuesday, September 11, 12


Some fun with modules• Mangle connection based on their “size”

iptables -t mangle -A POSTROUTING -o eth1 -p tcp \-m connbytes --connbytes 10000000: --connbytes-mode bytes \--connbytes-dir both -j CONNMARK --set-mark 999

$ nc -l 1664 < /dev/zero $ nc 192.168.1.222 1664 > /dev/null

# iptables -t mangle -L POSTROUTING -v Chain POSTROUTING (policy ACCEPT 115K packets, 4715K bytes)pkts bytes target prot opt in out source destination 111K 4452K CONNMARK tcp -- any eth0 anywhere anywhere connbytes 10000000:18446744073709551615 connbytes mode bytes connbytes direction both CONNMARK set 0x3e7

# grep mark=999 /proc/net/ip_conntracktcp 6 299 ESTABLISHED src=10.0.2.15 dst=192.168.1.222 sport=45234 dport=1664 packets=371355 bytes=14854220 src=192.168.1.222 dst=10.0.2.15 sport=1664 dport=45234 packets=415324 bytes=606265294 [ASSURED] mark=999 use=2



Some fun with modules• Mangle packets based on ASCII strings in the payloads

iptables -t filter -A INPUT -i eth1 -p tcp --dport 80 \-m string --string "get /admin http/1.1" --icase --algo bm \-m conntrack --ctstate ESTABLISHED -j DROP

$ nc -l 80get /hello+world http/1.1

$ nc 192.168.1.222 1664get /hello+world http/1.1get /admin http/1.1X

# iptables -t filter -L INPUT -v Chain INPUT (policy ACCEPT 93 packets, 6531 bytes)pkts bytes target prot opt in out source destination 13 936 DROP tcp -- eth1 any anywhere anywhere tcp dpt:http STRING match "get /admin http/1.1" ALGO name bm TO 65535 ICASE ctstate ESTABLISHED



Some fun with modules• Filter SSH on Christmas

• Mark the packets of a particular user, and Log

iptables -t filter -A INPUT -p tcp \--dport 22 -m time \--datestart "2012-12-25T00:00:00" \--datestop "2012-12-25T23:59:59" \--utc -j DROP

iptables -t filter -o eth0 -A OUTPUT -p tcp \--tcp-flags SYN SYN -m owner --uid-owner 1000 \-j CONNMARK --set-mark 1664

iptables -t filter -A OUTPUT -p tcp \--tcp-flags SYN SYN -m connmark --mark 1664 \-j LOG --log-prefix "Mark 1664 trigerred "

Aug 11 04:29:15 firewall1 kernel: [ 9895.696205] Mark 1664 trigerred IN= OUT=eth0 SRC=10.0.2.15 DST=173.194.75.106 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=7270 DF PROTO=TCP SPT=50940 DPT=443

WINDOW=14600 RES=0x00 SYN URGP=0



Hooks Magic• libnetfilter_queue provides hooks to divert packets from

the normal Netfilter flow, into Userland

• Use case: record outbound traffic from suspicious user “spongebob” (using https://github.com/jvehent/)

iptables -I OUTPUT -m owner --uid-owner $(id -u spongebob) -j NFQUEUE

root@ossec:~# ./nfqueue_recorder -o spongebob_record.pcappcap recording into spongebob_record.pcapopening pcap file at spongebob_record.pcapopening library handleunbinding existing nf_queue handler for AF_INET (if any)binding nfnetlink_queue as nf_queue handler for AF_INETbinding this socket to queue '0'setting copy_packet mode-- New packet received --hw_protocol=0x0000 hook=3 id=0 outdev=2 payload_len=60 bytesIP{v=4; ihl=20; tos=0; tot_len=60; id=15388; ttl=64; protocol=17; saddr=10.0.2.15; daddr=10.0.2.2}UDP{sport=45374; dport=53; len=10240}

-- New packet received --hw_protocol=0x0000 hook=3 id=1 outdev=2 payload_len=60 bytesIP{v=4; ihl=20; tos=0; tot_len=60; id=15391; ttl=64; protocol=17; saddr=10.0.2.15; daddr=10.0.2.2}UDP{sport=47435; dport=53; len=10240}


https://github.com/jvehent/

https://github.com/jvehent/


Hooks Magic

Network InterfaceP

POSTROUTING

OUTPUT

Socket

applicationwrite nfqueue

recorder

spongebob_

record.pcap



Better than TCPDump• TCPDump duplicates traffic at the NIC level.

• Dump incoming traffic and see packets entering but never reach the application

• Place your hook anywhere within you ruleset to verify that packets are coming through

NICP

PREROUTING

INPUT

Socket

application

P P

tcpdumpduplicates

DROP

nfqueuerecorder

display:packet packet packet .....

display:nothing !



Recording traffic on /admin at night

iptables -t filter -A INPUT -i eth1 -p tcp --dport 80 \-m string --string "get /admin http/1.1" --icase --algo bm \-m time --timestart "18:00" --timestop "10:00" --utc \-m conntrack --ctstate ESTABLISHED -j CONNMARK --set-mark 666

# ./nfqueue_recorder -o night_admin.pcappcap recording into night_admin.pcapopening pcap file at night_admin.pcapopening library handleunbinding existing nf_queue handler for AF_INET (if any)binding nfnetlink_queue as nf_queue handler for AF_INETbinding this socket to queue '0'setting copy_packet mode

iptables -t filter -A INPUT \-m connmark --mark 666 -j NFQUEUE

# nc -l 80 $ nc 10.16.64.11 80get /spongebobsquarepants http/1.1get /admin http/1.1get /spongebobblaaaahhh fooo



IPSets• Netfilter uses linear lookups. Each rule is tested against

each packet, until a terminal rule matches the packet.

• Lookup time grows linearily.

• IPSets provides constant time hash lookup in Netfilter. apt-get install ipset && modprobe ip_setipset -N droplist nethashipset --add droplist 192.168.1.0/24iptables -A INPUT -m set --match-set droplist src \-j DROP

#! /bin/bashipset -N droplist nethashwget -q http://www.spamhaus.org/drop/drop.txt -O drop.lasso.$(date +%s)if [ -e drop.lasso.$(date +%s) ]; then ipset --flush droplist for i in $(grep -v -E "^;|^$" drop.lasso.$(date +%s) | awk {'print $1'}); do echo "insert $i to droplist" ipset --add droplist $i donefi


http://www.spamhaus.org/drop/drop.txt

http://www.spamhaus.org/drop/drop.txt


Ipsets: accounting per country• Get the list of IP blocks for each country from https://

www.countryipblocks.net/country_selection.php

• Load each country blocks into a separate Ipset

Chain INPUT (policy DROP 20 packets, 5942 bytes) pkts bytes target 0 0 COUNTRIES [...] match-set nigeria src 367 25797 COUNTRIES [...] match-set france src 10 440 COUNTRIES [...] match-set china src


https://www.countryipblocks.net/country_selection.php





Atomic (and fast) Restore

• Most people write their firewall rules in a Bash script. It’s the best way to start, until you have thousands of rules.

• [ USE IPTABLES-RESTORE ]

# iptables -L INPUT -v -n |grep ACCEPT|wc -l62511 <= that’s 62,511 individual INPUT rules

# time iptables-restore < iptables-rules.saved real 0m1.204suser 0m0.450ssys 0m0.720s

# time bash rules.iptables.shreal 41m28.277s <= 42 minutesuser 12m55.300ssys 32m17.290s <= 32 minutes spent in the kernel



Aweber FireWall• Moving away from the massive entry gate logic

• To something a lot more modularnode node

node node

node

node node

node node

node



Aweber FireWall



Aweber FireWall• Completely dynamic. Chef creates inbound/outbound

rules for nodes automatically. Convergence takes time.

• Whitelist for outbound connections per system user.

• Rules are stored in /etc/firewall/rules.iptables-A INPUT -i eth0 -p tcp --dport 80 -s provision -m state --state NEW -j ACCEPT-A INPUT -i eth0 -p tcp --dport 80 -s 10.1.0.228 -m state --state NEW -j ACCEPT-A INPUT -i eth0 -p tcp --dport 80 -s 10.1.0.229 -m state --state NEW -j ACCEPT

:www-data - [0:0]-A OUTPUT -m owner --uid-owner 33 -m state --state NEW -j www-data-A www-data -j LOG --log-prefix "AFW_www-data_OUTPUT_DROP " --log-uid-A www-data -j DROP

:ntp - [0:0]-A OUTPUT -m owner --uid-owner 104 -m state --state NEW -j ntp-A ntp -o eth0 -p udp --dport 123 -d ntp1 -m state --state NEW -j ACCEPT-A ntp -o eth0 -p udp --dport 123 -d ntp2 -m state --state NEW -j ACCEPT-A ntp -j LOG --log-prefix "DROP_AFW_OUTPUT_ntp " --log-uid --log-tcp-sequence-A ntp -j DROP



Aweber FireWalloverride_attributes( :afw => { :rules => { 'MongoDB App Entry Point' => { :protocol => 'tcp', :direction => 'in', :user => 'mongodb', :source => '(roles:nodejs-app OR roles:*python-worker-node) AND SAMETAG', :dport => '27017' }, 'MongoDB Cluster Inbound Replication' => { :protocol => 'tcp', :direction => 'in', :user => 'mongodb', :source => 'shard_name:#{node[:mongodb][:shard_name]}', :dport => '27017' }, 'MongoDB Staging accessible from Production Workers' => { :protocol => 'tcp', :direction => 'in', :user => 'mongodb', :source => 'roles:worker-node AND SAMETAG AND chef_environment:production', :dport => '27017', :env => 'staging', :options => ['disable_env_limit'] },



AnyQuestion ?



Netfilter Packet Flow(for real men)

http://upload.wikimedia.org/wikipedia/commons/3/37/Netfilter-packet-flow.svg




Documents

Advanced Netfilter Iptables - linuxwall.info Firewall Talk Julien... · Advanced Netfilter & Iptables Understand the Linux Firewall and make it do magical tricks Julien Vehent - PLUG