An Anomaly-Based Approach for Intrusion Detection in Web Traffic

Carmen Torrano-Gimenez, Alejandro Perez-Villegas and Gonzalo Alvarez

Instituto de Física Aplicada,Consejo Superior de Investigaciones Científicas,

Madrid, Spain

Journal of Information Assurance and Security 5 (2010)

Mike Hsiao 2010.06.11

2

References• Carmen Torrano-Gimenez, Alejandro Perez-Villegas and Gonzalo Alvarez,

“An Anomaly-Based Approach for Intrusion Detection in Web Traffic,” in Journal of Information Assurance and Security, vol. 5, 2010.

• C. Torrano-Gimenez, A. Perez-Villegas and G. Alvarez, “A Self-learning Anomaly-Based Web Application Firewall,” in 2nd International Workshop in Computational Intelligence in Security for Information Systems (CISIS 09), vol. 63 of , 85-92, Springer-Verlag, 2009.

• A. Liu, Y. Yuan, D. Wijesekera, and A. Stavrou, “SQLProb: A Proxy-Based Architecture toward Preventing SQL Injection Attacks,” in Proc. of the 2009 ACM Symposium on Applied Computing (SAC’09), 2009.

• Fredrik Valeur, Giovanni Vigna, Christopher Kruegel, and Engin Kirda, “An Anomaly-Driven Reverse Proxy for Web Applications,” in Proc. of the 2006 ACM Symposium on Applied Computing (SAC’06), 2006.

3

Outline• Introduction• Web Applications and Web Attacks

– Web Applications– Web Attacks– Web Vulnerabilities

• System Overview– Architecture (WAF: Web Application Firewall)– Normal Behavior Description– Detection Process

• Experiments: Case Study (Web Shopping)– XML/Training/Testing– WAF Protection Mechanism– Performance/Results

• Comments

An Anomaly-Based Approach for Intrusion Detection in Web Traffic

4

Introduction

• Web applications handle large amounts of sensitive data, which makes web applications even more attractive for malicious users.– Identity supplanting, sensitive data hijacking, unauthorized

information, web content modification, command execution, etc.

• Conventional firewall (operating at network and transport layers) are usually not enough to protect against web-specific attacks.– To be really effective, the detection is to be moved to the

application layer.

5

Traditional Firewall (Layer 3/4)

Application layer (7) E.g., HTTP, FTP, PRC

Transport layer (4) E.g., TCP, UDP

Network layer (3) E.g., IP

valuable server attacker

Packet Inspection!

6

Traditional Firewall (Layer 3/4)

Traditional Firewall: it can inspectmessages and headers carrying in layer 3 and 4.

Some firewall may extend itscapability to capture layer 2information.

Most of the IDS can inspect theapplication layer messages, butthey basically fall into “misuse”based category, which only captureknown attacks.

7

Traditional Firewall (Layer 3/4)• Netfilter/iptables (L3/L4)

– iptables -A INPUT -p TCP -i $RED_DEV --dport 135 -s 0/0 -j DROP– Such rules can not distinguish attacks from norms. All network traffic to TCP port 135 will be

dropped.• Snort (L3/L4 + L7 signature)

– alert tcp $EXTERNAL_NET any -> $HOME_NET 135 (msg:"NETBIOS DCERPC ISystemActivator path overflow attempt little endian unicode"; flow:to_server,established; content:"|05|"; within:1; byte_test:1,&,16,3,relative; content:"|5C 00 5C 00|"; byte_test:4,>,256,-8,little,relative; flowbits:isset,dce.isystemactivator.bind; classtype:attempted-admin; sid:2351; rev:10;)

– Such rule is much more precise than the previous one, but it relies on “syntactic” analysis, not “semantics” analysis.

• Shield (L3/L4 + L7 Semantic signature, Vulnerability-based IDS)–

These tools areuseful for detecting

known attacks, but theysimply block the malicious traffic.

They are simply temporary solutions.The vulnerable software need to be fixed

as soon as possible. And also, these rules may not be robust to detect the attack and its variants.

8

IDS – Intrusion Detection System

• Signature Detection System (Misuse)– Negative approach– Known attacks/exploitations– String Matching Techniques

• Anomaly Detection System (Anomaly)– Positive approach– Normal/Common Behavior– Irregular behavior will be tagged as intrusive

• Hybrid

9

Traditional IDS shortcoming

• Signature Detection System (Misuse)– Fragmentation, pattern changing, …– False positive/negative problem

• Anomaly Detection System (Anomaly)– Complex environments (large network with

multiple servers and operating systems)• Up-to-date normal? Feasible normal?

– FP/FN problem

10

UTM - Unified Threat Management (2003)

• UTM 是利用單一或簡單的介面設定來管控以及保護公司網路進出的安全的閘道式設備。

• UTM 依照不同機型，可以將以下幾種主要的功能選擇幾項整合於單一主機中，路由、防火牆、垃圾郵件過濾、防毒（包含病毒、惡意軟體、網路釣魚等 ... ）、 IDS 或 IPS （入侵偵測或防禦系統）、網頁過濾、 proxy 、 VPN 、 VOIP 、 NAT 、抵禦 DoS / DDoS ( 阻斷服務 / 分散式阻斷服務 ) 。

• UTM systems must– Be an appliance– Include multiple security features– Have a hardened OS– Be able to perform:

• Network firewalling• Intrusion prevention (IPS)• Gateway anti-virus

11

WAF – Web Application Firewall

• WAF analyzes the HTTP traffic (application layer) in order to detect malicious behaviors that can compromise the security of web application.

• This paper relies on an XML file to describe what a normal web application is.

12

Web Applications and Vulnerabilities

• Application– Presentation, application logic, storage (see next page)

• IIS/Apache, Tomcat, MSSQL/MySQL

– Web content are dynamic• CGI in Perl, Python, C/C++; JSP, PHP, ASP; Java, VB, C#

• Attack– Static attack looks for security vulnerabilities in the web application

platform: web server, application server, database server, firewall, OS, and third-party component such as shopping cart, crypto modules, …

– Dynamic web attacks only request legal pages of the application but they subvert the expected parameters.

• Vulnerability – OWASP Top 10

13

How WAF works?

Attacker

User

Web Server

App Server

DB Server

Media Server

14

Application Security Risk

OWASP (The Open Web Application Security Project)OWASP Top 10 – 2010 (rc1)

15

2007 vs. 2009

16

Architecture(reverse)

ModSecurity is a popular open source signature-based WAF.

17

Armorize SmartWAFTM

18

Reverse Proxy (+ Load Balance)

Internet

WAF

Web Server

Cash Flow 1

Cash Flow 2

WebMail

Media Server

WAF 代替網頁伺服器回應 response:可在收到 request 時 , 檢查內容 ,若正常 , 則向網頁伺服器請求內容 , 並回應 .

But sometimes WAF is not enough.

19

Web Site Design

[*] Fredrik Valeur, Giovanni Vigna, Christopher Kruegel, and Engin Kirda, “An Anomaly-Driven Reverse Proxy for Web Applications,” in Proc. of the 2006 ACM Symposium on Applied Computing (SAC’06), 2006.

(a) an e-commerce web site implemented with a single server that relies on a single back-end database and that accesses a credit card processing server.

f1: function 1X: table X

(c) The database is modified to create two different users u1 and u2, where u1 is allowed to access table x only and u2 is able to access both table x and table y. User u1 is associated with server B and user u2 is associated with server C.

20

• A web site could be made more resilient to attacks if it would be possible to design both the server and the database infrastructure so that different levels of access to the database and the hosts running the server processes could be clearly enforced.

• Design (b)– (i) non-sensitive, static information about the e-commerce company (e.g.,

company contacts and support information) is accessible through one server;– (ii) the non-sensitive, dynamic information about product availability is

accessible through a second server that accesses a product database; and, finally,

– (iii) the sensitive information about users is accessible through a third server that relies on a user database, which is separated from the product database.

– This last server has also access to the credit card processing server.

[*] Fredrik Valeur, Giovanni Vigna, Christopher Kruegel, and Engin Kirda, “An Anomaly-Driven Reverse Proxy for Web Applications,” in Proc. of the 2006 ACM Symposium on Applied Computing (SAC’06), 2006.

21

Normal Behavior Description

• The XML file contains rules regarding to the correctness of HTTP verbs, HTTP headers, accessed resources (files), arguments, and values for the arguments.

• Verbs.– The verbs node simply specifies the list of allowed HTTP verbs. Requests using any

other verb will be rejected.• Headers.

– The headers node specifies a list of some HTTP headers and their allowed values. Different values will not be accepted.

• Directories.– Each directory in the web application space is represented in the XML file by a

directory node.– Each file in the web application space is represented by a file node.– Input arguments are represented by argument nodes within the corresponding file

node.• Legal values for arguments should meet some statistical rules.

22

The XML file is generated by training/testing method.

23

Example of rules: PrefixPrefix

/taiwan/content/imageView\.asp

/C2M21/manager_citation(_acts)?\.php.*

/F/[A-Z0-9]{51}\-[0-9]{5}

/cgi/openfile{3_0}?

(/[\w\-]*)*/hypage\.cgi

/ttscgi/ttsweb([0-9]|new)?

/cgi-bin/(?.Count.cgi|counter)

(/cgi-bin/file-upload\.cgi|eduArea/|cgin1110\.asp)

/saweb/pc|f\.file.*\.[Pp][Dd][Ff]

Snort rule (Blaster):alert tcp $EXTERNAL_NET any -> $HOME_NET 135 (msg:“NETBIOS DCERPC ISystemActivator path overflow attempt little endian unicode”; flow:to_server,established; content:“|05|”; within:1; byte_test:1,&,16,3,relative; content:“|5C 00 5C 00|”; byte_test:4,>,256,-8,little,relative; flowbits:isset,dce.isystemactivator.bind; classtype:attempted-admin; sid:2351; rev:10;)

24

Example of rules in Armorize SmartWAF

All Regular Expression!!

25

WAF vs. Fortify RTA

WAF

Fortify RTA

26

A. Liu, Y. Yuan, D. Wijesekera, and A. Stavrou, “SQLProb: A Proxy-Based Architecture toward Preventing SQL Injection Attacks,” in Proc. of the 2009 ACM Symposium on Applied Computing (SAC’09), 2009.

27

Comments

• In order to achieve the deeper packet inspection, the proxy design is used to mitigate the effort in message extraction.

• Input validation is important task all kind of web applications.

• Proxy (in front of different application server) can focus on only checking the attacks that related to the server.

• Misuse or anomaly approach?