A Case Study on Web Application Security Testing

with Tools and Manual Testing

LaShanda Dukes, Xiaohong Yuan, Francis Akowuah

Department of Computer Science

North Carolina Agricultural & Technical State University

Greensboro, United States

ldukes@aggies.ncat.edu, xhyuan@ncat.edu, fakowua@aggies.ncat.edu

AbstractWeb application security has become a big issue because of common vulnerabilities found in web applications.

This paper illustrates a case study on conducting security testing

on an example application, Tunestore. The example application

was tested using a number of tools such as Paros, WebScarab,

JBroFuzz, Acunetix, and Fortify. Manual testing was also

conducted. The testing results of different tools and manual

testing are compared and discussed. Our case study shows

manual testing is very important since some vulnerability types

can only be found through manual testing and testers observations, and it is important to utilize a variety of tools as

well as conduct careful manual testing in order to find the most

number of vulnerabilities in a web application. Based on this case

study, hands-on labs can be developed for teaching web security, software security testing, and other topics.

I. INTRODUCTION

Web applications today are highly functional, and rely upon a two-way flow of information between the server and browser. Security becomes a big issue because no one wants to use a web application if they believe their information will be disclosed to unauthorized parties [1]. Vulnerabilities commonly found in web applications include injection, cross-site scripting, cross-site request forgery, security misconfiguration, broken authentication and session management, and more.

Penetration testing and static code analysis may be used to assess the vulnerabilities of web applications. Penetration testing is a method of security testing through the simulation of an attack. Static code analysis, also known as source code analysis is a code review process that examines the softwares source code for common coding errors and defects without execution [3].

This case study uses an example web application, Tunestore to conduct security testing. It illustrates web application security testing using tools and manual testing. The tools used to conduct testing on Tunestore are Paros, WebScarab, JBroFuzz, Fortify, and Acunetix. Our case study shows manual testing is very important since some vulnerability types can only be found through manual testing and testers observations, and it is important to utilize a variety of tools as well as conduct careful manual testing in order to find the most number of vulnerabilities in a web application. Based on this case study, hands-on labs can be developed to

teach about web application vulnerabilities, software security testing, and other topics.

This paper is organized as follows: Section II discusses some common web application vulnerabilities. Section III describes the tools we used for web application security testing, Section IV gives a description of the example application used to conduct security testing, and Section V and VI discuss the results from testing the example application using tools as well as through manual testing. Section VII provides further discussion on the testing results and Section VIII concludes the paper.

II. COMMON WEB APPLICATION VULNERABILITIES

This section discusses common vulnerabilities of web applications. They include SQL injection, cross-site scripting, cross-site request forgery, broken authentication and session management. These vulnerabilities were listed in OWASPs Top 10 Project as among the Top 10 critical web application security risks [4]. They also appear significantly in the Web Hacking Incident Database (WHID) [5]. A brief discussion is provided here. A more detailed discussion can be found in [6].

SQL injection is a vulnerability type where an application accepts malicious scripts that are later parsed and executed. The fundamental way of attack involves the direct insertion of code into user-input variables that are concatenated with SQL commands and executed. A rare and indirect way of attack is where the malicious code is injected into strings that are destined for storage in a table or as metadata. Subsequently when the stored string is concatenated into a dynamic SQL command the malicious code is executed. An attacker can anonymously read and modify data contained in a database as well as take full control of the server by exploiting the SQL vulnerability.

An application is vulnerable to cross-site scripting (XSS) vulnerabilities when it takes untrusted data and sends it to a web browser without proper validation and escaping. This allows attackers to embed malicious JavaScript, VBScript, ActiveX, HTML or Flash into a dynamic page to trick the user, executing the script on his machine in order to gather data, hijack sessions, redirect users to a malicious site or deface the website. Furthermore, a successful attack may result in the compromising of private information, manipulation or theft of cookies, creation of requests that can be mistaken for those of a

This work is partially supported by Department of Education under grant

P120A090049 and NSF under grant HRD-1137516. Any opinions, findings,

and conclusions or recommendations expressed in this material are those of

the author(s) and do not necessarily reflect the views of the National Science

Foundation and Department of Education.

978-1-4799-0053-4/13/$31.00 2013 IEEE

valid user. It might also cause the execution of malicious code on the end user systems. Three types of XSS exist. The first one is Reflected XSS, in which dynamic web pages take text submitted by a client or user and simply renders this text back to user within its response. The second type is Stored XSS. It occurs where some malicious user-submitted data is stored in a database to be used in the creation of pages that will be served to other users later. Thus, visitors of the web page fall victim for this attack. Lastly, Local XSS exploit targets vulnerabilities that exist within the webpage code itself. Usually, this happens when Document Object Model (DOM) is wrongly used in JavaScript. As a result, opening another web page with malicious JavaScript code in it at the same time might actually alter the code in the first page on the local system.

Broken authentication and session management is the vulnerability whereby the application does not implement its authentication and session management functions correctly. The attacker uses leaks or flaws such as exposed accounts, session IDs and passwords in the authentication or session management function to impersonate users. When successful, an attacker is able to do anything the victim could do. The situation worsens when the administrator is the victim.

Cross-site request forgery usually occurs where HTTP cookies are primarily used to transmit session tokens. Where precaution is not taken against the misuse of the token, an attacker can trick an authenticated user to perform actions of his (attacker) choosing. Victims are forced to change data and perform functions they are allowed or authorized to do

Insecure direct object reference is a common web application vulnerability regarding access control. This occurs when an application wrongfully exposes a reference to an internal implementation object. An attacker, usually an authorized user, exploits this vulnerability by simply changing a parameter value that references a system object. This enables him to access unauthorized objects.

III. TOOLS USED IN THE CASE STUDY

This section briefly describes the tools used in this case study

which are: Paros, WebScarab, JBroFuzz, Acunetix and

Fortify.

A. Paros

Paros [7] is a free web application vulnerability assessment

tool written in Java by ProofSecure.com. It can be used as a

web spider, vulnerability scanner, proxy, and a fuzzer. The web

spider function is used to discover the content of a website by

parsing a webpage for links to other content, requesting these

pages, and continuing this process recursively. The scanner

function scans the web application to identify common

vulnerabilities such as cross-site scripting, SQL injection, forms with autocomplete enabled, old versions of files, etc.

Paros can also trap HTTP and HTTPS requests and responses

so they can be modified manually [1, 8].

B. WebScarab

WebScarab [9] is an open-source web application security testing tool written in Java. The tool is a part of the Open Web Application Security Project (OWASP) [10]. It can be used for

analyzing applications that communicate with HTTP and HTTPS protocols. WebScarab operates as an intercepting proxy, in which the user is allowed to review and modify intercepted requests that are created by the browser and responses from the server. Similar to Paros, it can be used as a web spider to discover the content of a website. The fuzzer feature of WebScarab allows a user to test a web page by taking a list of test strings and trying them in the input fields automatically [11].

C. JBroFuzz

JBroFuzz is a web application fuzzer written in Java and is

a part of the OWASP Project. The fuzzer is for requests made

over HTTP and HTTPS protocols [12]. The fuzzing feature is

used to create a HTTP request to send to the web application.

A fuzzer can be added to a parameter field to add attack

payloads. The payload feature has attack strings of several

vulnerability categories such as Injection, XSS, Integer Overflow, XPATH Injection, SQL Injection, and more. HTTP

responses are displayed in an output window showing the

payload and the HTTP response status code. The responses of

the web application to the attack strings are also shown. The

graphing feature can generate different types of graphs to

visualize the fuzzing results. There are six types of

graphs/charts: Status Code Pie Chart, Response Time Bar

Chart, Response Size Bar Chart, Jaccard Index, Hamming

Distance, and Response Header.

D. Acunetix

Acunetix is a product of the Acunetix Company [13]. It is a

web vulnerability scanner created to replicate hackers methodologies to find vulnerabilities in web applications.

Acunetix crawls through the websites to analyze the content

including flash content, SOAP and AJAX. The scanner

includes features such as advanced and in-depth SQL injection

and cross-site scripting testing, penetration tools which include HTTP Editor and HTTP Fuzzer, and others. The cost

of the tool ranges from $1,000 - $12,995. It has a free trial

version that can only scan for cross-site scripting

vulnerabilities in a web application. In this case study, the free

trial version was used to test the example web application.

E. Fortify

Fortify is a static analysis tool used to find root causes of

security vulnerabilities in source code. Fortify supports a

range of languages, platforms, build environments and

software component APIs. Fortify can detect more than 480

types of software security vulnerabilities among 20 different

languages. The tool displays results sorted by severity of risk

and guidance on how to mitigate the risks [14]. The

demonstration version of the software that comes with the

book Software Security: Building Security In [3] only scans for SQL injection, buffer overflow, system information leak,

and HTTP response splitting vulnerabilities. In this case study, the demonstration version of Fortify was used to test the

example web application.

Table 1 summarizes the automated tools used for the case

study.

IV. THE EXAMPLE APPLICATION

The application used to conduct security testing is the Tunestore web application developed by Dr. Bei-Tseng Bill

Chus project team from the University of North Carolina at Charlotte. The Tunestore web application is a .war file installed

on a server on an Ubuntu Virtual Machine to conduct security

testing. Tunestore is an online retail store of songs. A user has

to create an account to gain access to the store and its database

of songs, although users may view the list of songs without

being a registered user. Registered users are able to select

several options within the store: add balance, friends, profile, CDs, and log out. The add balance option allows the user to add money to their account by using a credit

card to purchase songs. The friends option allows the user to add and view friends, as well as accept friend requests. The

profile option provides the user with their username and their current account balance. The user may also change their

password under the profile option. The CDs option displays a list of songs the user can buy, send as a gift, or make

a comment under the CD. This option also shows the users purchased CDs. The log out option logs the user out of their profile and redirects them to the homepage of the store.

V. SECURITY TESTING WITH TOOLS

The Tunestore web application was tested using the five

tools described in Section III. This section describes the testing results of using these tools.

A. Testing Results with Paros

Using the scanner feature of Paros, 37 potential

vulnerabilities were discovered in the Tunestore web

application. These vulnerabilities are listed below. The

number following the vulnerability refers to the number of occurrences of the vulnerability.

Obsolete File (7): It indicates that miscellaneous items including files, backup, unused, or obsolete

files exist. If these files contain program source,

information such as server logic or ODBC/JDBC

user IDs and passwords may be revealed since these

file extensions may not be processed by the web server.

Obsolete File Extended Check (7): It also indicates that miscellaneous items including files, backup,

unused, or obsolete files exist.

Password Autocomplete in Browser (7): This vulnerability means the AUTOCOMPLETE attribute

is not disabled in the HTML FORM/INPUT element

containing password type input. Passwords may be

stored in browsers and retrieved.

Session ID in URL rewrite (7): URL rewrite is used to track user session IDs. The session ID may be disclosed in the referer header. Besides, the session

ID can be stored in the browsers history or server logs.

SQL Injection (4): SQL injection is possible. User parameters submitted will be formulated into a SQL

query for database processing. If the query is built by

simple string concatenation, it is possible to modify

the meaning of the query by carefully crafting the

parameter.

SQL Injection Fingerprinting (4): SQL injection may be possible.

Tool Functionalities Cost Operating System

Paros [7] Web proxy

Web spider

Automated vulnerability scanner

Manual request with proxy, fuzzer

Free Linux, Apple Mac OS X, Microsoft Windows

WebScarab [8] Web proxy

Web spider

Manual fuzzer

Manual vulnerability scanner

Manual request with web proxy, spider,

fuzzer, history

Free Linux, Apple Mac OS X, Microsoft Windows

JBroFuzz [11] Automated fuzzing

Graphing

Built-in attack payloads

Free Linux, Apple Mac OS X, Microsoft Windows

Acunetix [12] Automatic client script analyzer SQL Injection and Cross-site scripting

testing

Fast scanner crawls

Port scans on web servers

Commercial $1,000 -

$12,995

Microsoft Windows, Microsoft Windows Server, Microsoft Windows 2008 Server

Fortify [13] Automated vulnerability scanner

Source code analyzer

Commercial Linux, Microsoft Windows, Mac OS, Oracle

Solaris, HP-UX

Table 1. Automated Tools used in the Case Study

Cross-Site Scripting (1): Malicious scripts may be injected into the browser which can appear to be

genuine content from the original site. These scripts

can be used to execute arbitrary code or steal

customers sensitive information such as user passwords or cookies.

B. Testing Results with WebScarab

Using the fuzzer and the proxy features of WebScarab, we discovered potential SQL injection and cross-site scripting vulnerabilities:

SQL Injection: An input attack file was created which included SQL injection attack strings. This attack file was submitted to the WebScarab fuzzer to test the login page of the Tunestore web application. Some of the attacks were successful.

Reflected Cross-Site Scripting: Using WebScarabs fuzzer feature, an input attack file was created which included cross-site scripting attack strings to test the login page of the Tunestore web application. The scripts were executed showing that the login page has XSS vulnerabilities.

Stored Cross-Site Scripting: Using WebScarabs proxy feature, comments submitted by the user was intercepted and changed to malicious scripts. The malicious scripts were then saved on the server, and would be activated when someone views the comment on the CD.

Other input fields such as those related to the registration, friend, and add balance options could also be tested using WebScarab. However, in this case study, those fields were tested using the manual approach instead of WebScarabs fuzzer feature.

C. Testing Results with Fortify (the Demonstration Version)

Using Fortify Source Code Analyzer, 30 potential

vulnerabilities were discovered. These vulnerabilities are

listed below. The number following the vulnerability refers to the number of occurrences of the vulnerability.

SQL Injection (27).

System Information Leak (2): It means system data or debugging information is revealed, which

can help attackers learn about the system and plan

attacks.

HTTP Response Splitting (1): It means unvalidated data in HTTP response headers can

enable cache-poisoning, cross-site scripting,

cross-user defacement or hijacking attacks.

D. Testing Results with JBroFuzz and Acunetix

JBroFuzz was first used to test for SQL injection and XSS

vulnerabilities. However, a bug in the tool was discovered. For

some valid SQL injection attack strings, JBroFuzz generated

invalid requests to the web application, and thus could not

identify the SQL injection vulnerability. Because of this bug,

JBroFuzz was no longer used to test for other vulnerabilities in

the Tunestore web application. However, the payloads

embedded in JBroFuzz were used in manual testing to

discover some of the vulnerabilities in Tunestore.

Using Acunetix (free edition), cross-site scripting

vulnerabilities were discovered.

VI. MANUAL TESTING

Using manual testing, authentication, access control, injection, cross-site scripting, cross-site request forgery, and integer overflow vulnerabilities were discovered. Some of the vulnerabilities confirm those found through automated tools. Others were not reported by the tools.

The vulnerabilities found through manual testing which confirm those reported by the automated tools are the following:

SQL Injection This allows illegitimate users to login as a legitimate user.

Reflected Cross-Site Scripting Malicious scripts may be injected into input fields of the login page and be executed.

Stored Cross-Site Scripting Using WebScarabs proxy feature, a malicious script can be stored in the comment field.

Using manual testing, we were able to confirm some of the vulnerabilities reported by some of the tool. Acunetix indicated cross-site scripting vulnerabilities in the login page of the Tunestore web application. Although, the tool reported true positives, manual testing also revealed cross-site scripting vulnerabilities on the comment page. Malicious scripts were stored into the comment field and stored in the database. This indicates a typical case of stored cross-site scripting (XSS) vulnerability.

The vulnerabilities found through manual testing which are not reported by the automated tools are the following:

The application allows users to register with weak passwords. It does not have a minimum or maximum length for passwords. It does not give users feedback on password strength. This vulnerability can cause an attacker to carry out brute force attacks and enumerate existing users on the system.

The application does not have a lockout policy for failed login attempts. This makes the application have a high risk of brute force attacks.

Unencrypted credentials are transmitted over HTTP protocol.

Forgot password feature was not implemented. Registered users will not be able to access the system if they do not remember their passwords.

The application implements a stay logged in feature. This makes the application have a high risk of session hijacking.

Insecure storage of credentials. Username and passwords stored in the database are not encrypted. As a result, an attacker can eavesdrop on the network and gain access users unauthorized information. Thus, an attacker is able to gain access to the database to view users data.

Horizontal privilege escalation The application allows users to view friends CDs and view comments on CDs without logging into the application.

Cross-Site Request Forgery Malicious scripts may be injected into the comment field of Tunestore which includes a crafted link. When a victim clicks on the link, it causes some actions to occur, such as logging the victim out, etc.

Integer Overflow The submission of large numbered values will overflow the balance field

on the account. The add balance option will no longer operate correctly.

An attacker can add an amount to the add balance feature in their profile by using SQL injection techniques. Hence, an attacker can

successfully create a new user account with a

substantial amount of money added to their account without providing valid credit card

information. As a result, the attacker is able to

purchase CDs for free. This vulnerability was

discovered through examining the source code.

Insecure Direct Object Reference. Requests for protected resources, in this case the music files,

are made directly to the static resources, which

are located within the web root of the server. The

music filenames are predictable. The last names

of the artists are used. Changing the values in the

download link permits a user to download any song from Tunestore without purchasing them.

VII. DISCUSSION

Our case study shows that Tunestore is a good example

application for teaching students about web security. It can be

used to demonstrate various web application vulnerabilities to

the students. Limited by our academic environment, we do not

have the full versions of Fortify and Acunetix. Therefore the

testing results from the demonstration or the trial version of

the tools are very limited in this study. Universities can apply

for the Hewlett-Packard Company Educational Software

License Agreement to receive the full version of Fortify for

free for educational purposes. However, we could not get the full version of Fortify because our university did not agree

with the user agreement for the license for the Fortify

software.

The web spidering and proxy features of Paros and

WebScarab are very useful for web application testing. The

web spidering functions allow testers to understand the

content and structure of the web application. The proxy allows

testers to intercept and modify requests and responses

transmitted between the client and server to conduct attacks.

The scanner feature of Paros is quite limited. It does report

some main vulnerabilities such as SQL injection, cross-site

scripting, and session ID in URL rewrite, etc. WebScarabs fuzzer feature can be very useful because a large number of

attack strings can be submitted to test the application

automatically. Some of the manual testing conducted in this

case study, such as testing for buffer overflow, integer

overflow vulnerabilities, could have been done using

WebScarabs fuzzer feature. JBroFuzz does not have good documentation on how to use the tool. From our use of the

tool, we discovered a bug in the tool, and did not think the tool

worked properly as it is supposed to. However, the attack

types and payloads embedded in the tool can be useful. The

payloads can be used for manual testing, or testing with

WebScarabs fuzzer feature.

Manual testing discovered most of the vulnerabilities

especially the design flaws. Some of them are the same as

those found using Paros, Fortify, Acunetix, for example, SQL

injection, and cross-site scripting vulnerabilities. Most of the

authentication and access control vulnerabilities can only be

found through manual testing based on testers observations. Therefore it is important to utilize a variety of tools as well as

conduct careful manual testing in order to find the most

number of vulnerabilities in a web application.

VIII. CONCLUSION

This paper describes a case study of testing for security vulnerabilities of an example application Tunestore. It provides an overview of web application security testing tools

such as Paros, WebScarab, JBroFuzz, Acunetix, and Fortify. It

presents the security vulnerabilities found within an

application by using these tools, and by manual testing. While

commercial tools are not easily available in an academic

environment, open source tools can be used to conduct

security testing with limited results. Our case study shows

manual testing is very important since some vulnerability

types can only be found through manual testing and testers observations. It is important to utilize a variety of tools as well

as conduct careful manual testing in order to find the most number of vulnerabilities in a web application. Based on this

case study, hands-on labs can be developed for teaching web

security, software security testing, and other topics.

REFERENCES

[1] Studdard, D., & Pinto, M. The Web Application Hackers Handbook: Discovering and Exploiting Security Flaws,

Wiley Publishing: Indianapolis, Indiana, 2008.

[2] The Open Web Application Security Project, Testing: Introduction and Objectives, 2012

https://www.owasp.org/index.php/Testing:_Introduction_

and_objectives, retrieved on November 6, 2012. [3] McGraw, G. Software Security: Building Security In.

Pearson Education: Boston, Massachusetts, 2006.

[4] The Open Web Application Security Project, OWASP, Top Ten 2010 Main, 2010, www.owasp.org/index.php/Top_10_2010-Main, retrieved

on January 13, 2013.

[5] Web-Hacking-Incident-Database. Retrieved on Jan 6, 2013 http://projects.webappsec.org/w/page/13246995/Web-

Hacking-Incident-Database

[6] Top 10 2010. Retrieved on January 10, 2013: https://www.owasp.org/index.php/Top_10_2010-Main

[7] Paros, Paros for web application security assessment, 2004, http://www.parosproxy.org/, retrieved on

November 6, 2012.

[8] Paros, User Guide for Paros v2.x, 2003, http://www.parosproxy.org/paros_user_guide.pdf,

retrieved on November 6, 2012.

[9] The Open Web Application Security Project, Category: OWASP WebScarab Project, 2012,

https://www.owasp.org/index.php/Category:OWASP_We

bScarab_Project, retrieved on November 6, 2012.

[10] The Open Web Application Security Project, OWASP, 2012, www.owasp.org, retrieved on November 6, 2012.

[11] Dustin, E., Nelson, L., Wysopal, C., Zovi, D. The Art of Software Security Testing: Identifying Software Security Flaws. Symantec Corporation: Upper Saddle River, New

Jersey, 2007.

[12] The Open Web Application Security Project, JBroFuzz, 2012, https://www.owasp.org/index.php/JBroFuzz,

retrieved on November 6, 2012.

[13] Acunetix Ltd., Acunetix Web Application Security, www.acunetix.com/vulnerability-scanner/, retrieved on

November 6, 2012.

[14] HP Enterprise Security, HP Fortify Static Code Analyzer, 2012, http://www.hpenterprisesecurity.com/products/hp-

fortify-software-security-center/hp-fortify-static-code-

analyzer/, retrieved on November 6, 2012.