Modern Computer Networks: An Open Source Approach …speed.cis.nctu.edu.tw/~ydlin/course/cn/mcn_writeup/1-in-1/chapter7.pdf · Modern Computer Networks: An Open Source Approach Chapter

Modern Computer Networks: An Open Source Approach Chapter 7

1

Chapter 7 Network Security

Problem Statement

Network security is an overall consideration problem. Network hackers may

make use of weakness or security holes to the attack system if there is any problem

within system. For example, virus of “Code Red” and “Nimda” adopt the Distributed

Denial of Service to attack the system hosts and thus paralyze the network and stop

the service of the target. The key problem of the example is due to the Microsoft IIS

system has security hole to make network hacker to attack it. Nowadays, since the

information networks develop rapidly, it is necessary to understand the terms of

System Security or Network Security, then further work for system protection.

Consequently, making the network system more robust and safer to avoid the hackers

intrude systems. Although, we make sure the system is complete protection by

firewall or security-hole free, the system may suffer from attacking of Denial of

Service and result in it can't continue to provide the service normally. That reveals the

importance and the necessity of the network security. This goal of this chapter is how

to protect the security of computer systems in such a complicated environment in

Today’s Internet and networks. Therefore, we describe that in detail in three aspects,

including data security, fire wall system, and intrusion detection system (IDS).

7.1 General Issues

As the number of E-Transactions via networks increasing, the security issue of

sending sensitivity data, including banking account, password, credit card number,

and secure content of E-Transaction, becomes more and more attentative and critical.

For instance, someone wants to intercept these sensitivity data for recording,

analyzing, reproducing, or spoofing. Then the network security problem will suffer to

be challenged. After all, the network only provides people another media with the

purpose for exchanging information, data commutation, and electronics trade. If the


2

network security can't guarantee completely, the amount of E-Transaction by

networks will be restricted.

In network security, we always explain the data flow with three virtual persons:

Alice (represented as sender A), Bob (represented as receiver B) and Trudy

(represented as intruder T). For example, the Alice wants to send data to Bob under

without any the protection networks. The transmitted plaintext between Alice and Bob

may be read and collected easily by the intruder Trudy. If Trudy has the greed and

techniques, Trudy may be reproduced, modified, and spoofed these data. Because

network has the characteristic of remote site invisible. When the falsification data

arrives at Bob, Bob receives the data undoubtly and normally as it comes from Alice.

To avoid the situation occurs, some prevention processes have to adopt to secure the

senstitive data before sending it. Several processing of network security will be

described in detail in the following sections in this chapter. Now we first introduce

some emphases of each section.

Cryptograph Theory

Firstly, we begin with the traditional theory of data cryptograph. In crytograph

theory, a common key is used to encrypt and decrypt data, which is called Symmetric

Encryption or Single-key encryption system. Since the private key and the public key

are the same one, how to distribute this key efficient and secure is an important issue.

As a consequence, in 1976, Diffie and Hellman proposed the encrypted method of

Asymmetric Encryption. The goal of such an asymmetric encryption is adopted

different key to encrypt and decrypt data. Therefore, the key distribution in networks

becomes more easy and secure. There are several representive systems for the two

kinds of encryption systems in nowadays networks. For instance, Data Encryption

Standard (DES) and International Data Encryption Algorithm (IDEA) is based on the

symmetric encryption, on the other hand, RSA is based on the asymmetric encryption.

Authentication

In network communication, since the Sender Alice and Receiver Bob are located

at different site, they can't be identified the other like by face to face confabulation

and can not be recognized the other like by phone talking can recognize the other

party with the voice. This is also a special characteristic of remote site un-visible in


3

networks. Therefore, how to authenticate both of sender and receiver correctly is an

important issue in network transaction. We will have detail description about digital

authentication in this chapter.

Data Integrity

Even though both of the sender Alice and receiver Bob are authenticated

normally, they still can not ensure that the original data did not be modified, spoofed,

and malicious forged. In Section 7.2.2 we will introduce the technique of how to

ensure the characteristic of data integrity.

Secure Socket Layer Protocol (SSL) and Secure Electronic Transaction

standard (SET)

How to achieve the secure network transactions if all of above mentioned

security processes are satisified? In Section 7.2.3.1, we first explain how the Secure

Socket Layer Protocol (SSL) works for the security mechanism of transferring

encrypted data. Since SSL can not provide fully mechanism of exchanging secure data,

we have to introduce the Secure Electronic Transaction standard (SET) and explain

the operations of SET. Both of SSL and SET are the required technique of security

mechanisms for the application of Electronic Commerce.

IP Security (IPSec)

Since the beginning operation of Internet from 70’s, the Internet users are most of

some specified organizations, including Acadmeics, Governments, and Organizations,

and the amount user of Internet are very stable. Nevertheless in the beginning of 90’s,

the amount user of Internet increases significantly due to the new era opening of

World Wide Web (WWW). Hence, so-called Internet Security Protocol (IPSec) was

proposed by IETF for supporting two types of security protocols, which are based on

IP Network Layer and listed as follows.

Authentication Header (AH) Protocol

Encapsulation Security Payload (ESP) Protocol

The AH protocol provides the authentication of source node and data integrity.

On the other hand, but on the other hand the ESP protocol supports complete


4

authentication, data integrity, and security mechanism; relatively, the processing

complexity of ESP is more complicated than that of AH. The description of the IPSec

protocol and its application, Virtual Private Network (VPN), will be explained in

detail in Section 7.2.4.

Firewall

For achieving the purpose of network security in a Local Area Networks (LAN),

a good method is to impose the mechanisms of access control onto the border node

that is located between the outside networks and this LAN. With the access control

mechanism, the forwarding frames are monitored by the border node. More specificed,

the goal of Firewall is to set some rules for allowing/denying networks. This is the

simpleset concept to protect the internal network. Therefore, in Section 7.3, the

description of Firewall is introduced in two aspects, including concepts of Firewall

and the components of it. There are two types of Firewall system, which are

Packet Filter-based Firewall

Application Gateway-based Firewall

In the packet filter-based firewall, it filters and routes packets based on the header of IP header or the filter rules of management, hence it operates at the network layer in the OSI reference model. In the application gateway-based firewall, it filters and routes packets based on the filter function at the application layer. Two kinds of firewall systems are introduced in Section 7.3. One is NetFilter, which is a packet filter-based firewall. The other is Trusted Information System (TIS), which is an application gateway-based firewall.

Intrusion Detection System

Since routing in Internet is based on the TCP/IP protocol, the protocol security

holes and some defects of TCP/IP maybe result in intrusion or denial of service of

service providers, for instance, the attack events of Yahoo, Amazon and e-Bay servers

in recent years. Nevertheless, it is necessary to understand the attacking technologies

of network hackers then propose several protection mechanisms to against such

attacks. Therefore, we describe some attacking technologies and protection skills in

Section 7.4.


5

7.2 Data Security According to the consideration in data security, the important data should be

encrypted before transmitting by Alice. Even though the encrypted data is intercepted

by Trudy, Trudy still can not get the original plaintext. As a result, data encryption

protects the original plaintext and prevents monitoring attacks. After Bob receiving

the encrypted data, Bob can obtain the plaintext from Alice by using the decryption

key to decrypt it. The procedure of data encryption and decryption is shown in Fig.

7-1.

In the principles of cryptograph, there are two main systems including, symmetric

and asymmetric key systems. The difference between them is that the symmetric key

system adopts the same key to encrypt and decrypt the plaintext; nevertheless, key is

different for encryption and decription in asymmetric key system. These two key

systems will introduce in the next two subsections, and then describe the issues of key

distribution and related authentication in section 7.2.3. Finally, the security

mechanisms of transport and network layers are described, respectively.

Encryption key

Plaintext

Encrypteddata

Plaintext

Decryption keyE D

if (Encryption key = = Decryption key) "It is a symmetric key system."else "It is an asymmetric key system."

Alice Bob

Figure 7-1 Data encryption and decryption


6

7.2.1 Principles of Cryptograph

7.2.1.1 Symmetric Key System

Although the theory of cryptograph was proposed in very early ago, the US

government firstly adopted the Data Encryption Standard (DES) to secure data in

1977. DES is a 56-bit symmetric key system, which uses a single key to encrypt and

decrypt the plaintext. Moreover, the International Data Encryption Algorithm (IDEA)

also adopts the symmetric key system. In present, the 56-bit DES algorithm still

extensively used in the world, nevertheless a more secure symmetric key system, the

112-bit DES algorithm, can be used in USA only.

In 56-bit DES, it encrypts each 64-bit data lock unit via a 56-bit key, then

produces the monoalphabetic result, that is, the DES obtains the same encrypted data

of a plaintext if it uses the same key to do the operation of encryption. The operations

of DES are based on the transposition ciphers, substitution ciphers, and sixteen

iterations computation. The principle of DES operation is shown in Fig. 7-2 and

described as follows. Firstly, a plaintext is partitioned into several 64-bit data blocks.

Each block, T = t1 t2…..t64, is performed the initial transposition to obtain T0, where

T0 is t58 t50 t42……t23 t15 t7 , then forms two 32-bit blocks, i.e. R0 and L0, which is

shown as follows,

T0 = L0 R0,

where

L0 = t58 t50 t42 … t16 t8

R0 = t57 t49 t41 … t15 t7.

The data blocks of L0 and R0 are to be inputs for next i terat ion of

encrypting, respect ively.

L1 = R0

R1=L0♁ f(R0, K1).


7

After that, the result becomes T1 = L1R1，where K1 is computed from 56-bit key. The

56-bit key is pre-computed as sixteen 48-bit keys: K1, K2,…, K16. The process

procedure of f(R0, K1) is shown in Fig. 7-3, which adopts that the 32-bit R0 and the

48-bit K1 as encryption inputs. Firstly, the 32-bit R0 is expanded to get a 48-bit result

by the operation of E(R0). Secondly, both of the 48-bit E(R0) and 48-bit K1 is

peformed XOR operation to obtain a 48-bit result, which will be partition into eight

6-bit inputs, B1, B2…..B8, for the following computation of substitution.

Li=Ri-1 Ri=Li-1 f(Ri-1,Ki)

64 bits

InputT=t1 t2 ... t64

Initial TranspositionIP

T0

KeySelection

16 keys:K1,...,K16

Key

IP-1

Output

16 Iterations 48

64 bits

64 bits64 bits

64 bits

64 bits

Fig 7-2 Encryption Procedure of DES


8

Ri-1 Key

KiE(Ri-1)

E KS

+

S1 S2 S3 S4 S5 S6 S7 S8

32 bits

48 bits

64 bits

48 nits

p

f(Ri-1,Ki)

32 bits

B1 B2 B3 B4 B5 B6 B7 B8

S1(B1) S2(B2) S3(B3) S4(B4) S5(B5) S6(B6) S7(B7) S8(B8)

Fig 7-3 Computation Process of f(Ri-1，Ki)

After the computation of substitution, Si, eight 4-bit blocks, Si(Bi), are obtained,

then performs a 32-bit transportation to get f(R0，k1) and finally the R1 can be also

obtained by the operation of L0♁f(R0，k1).

By doing the same iteration 16 times, i.e., LiRi →Li+1Ri+1，i=0,…,1, T16=L16R16

can be obtained, then performs the inverse initial transposition to get a 64-bit

encrypted data. On the other hand, the plaintext can be obtained from decrypting the

encrypted data by performing the inversion procedure of encryption.

How about the characteristic of security that DES has? No one can guanantee that.

Since the RSA Data Security company provide USD 10000 for whom can decrypt the

plaintext with “Strong cryptography makes the world a safer place.”, which is

encrypted by the 56-bit DES algorithm in 1997, the encrypted data is decrypted by a

team at less than four monthes. Moreover, a person who decrypted the last challenge

of DES challenge Ⅲ in 22 hours in 1999. Therefore, if we feel that the DES is not

secure enough in application, the several times computation of DES algorithm can

support more secure that single DES system. For instance, Triple-DES (3DES) and

128-bit DES algorithms have been proposed by the US government as the standard

for encryption and decryption in USA.


9

7.2.1.1 Asymmetric Key System

From the principle of the symmetric key system, we knew that uses the same key

to both operations of encryprtion and decryption. There are two issues should be

addressed before using it. First, how to identify the sender Alice and receiver Bob in

the first time data exchange. Second, how to distribute the secret key secure from

sender Alice to receiver Bob. Therefore, two different keys are propsed to encrypt and

decrypt data, respectively. Such a system is called Asymmetric Key System or Public

Key System, which uses a public key to perform encryption the plaintext and uses

anyther private key to perform decryption as shown in Fig. 7-4. In Fig. 7-4, Alice and

Bob use two different keys, in which Alice uses Bob’s public key to encrypt and Bob

uses his private key to decrypt the encrypted data. Consequently, Bob can distribute

public key to anyone in networks more secure and convenient.

Bob's public key

Plaintext, m

Encrypteddata

Plaintext, m

Bob's private keyE D

Alice Bob

m=DBob(EBob(m))

c=EBob(m)

c=EBob(m)

Fig 7.4 Asymmetric Key Cryptography

In asymmetric key system, RSA is the most famous algorithm, which was

proposed by three professors in MIT including Ronald Rivest, Adi Shamir, and

Leonard Adleman in 1977. RSA uses a pair keys to encrypt and decrypt data;

furthermore, it has been extensively adopted for the application of digital signature.

Nevertheless, RSA still has a primary disadvantage of high computation complexity,

which results in low efficient and can not be apply to vast amount data encryption. In

morden networks, it always adopts RSA to distribute keys and performs the operation


10

of decryption in digigal signature, and the vast amount data is encrypted by DES

algorithm.

In addition, the procedure of selecting public and private keys in RSA is

described in Fig. 7-5.

1. Select two large enough primes p and q. Larger primes are less crack, but the

computation time will increase significantly. RSA Lab. Suggests that the

selected primes should be larger than 10 1 0 0 .

2. Compute n by p*q and z by (p-1)*(q-1), i.e., n=p*q and z=(p-1)*(q-1) .

3. Choose a value e as the public key, which is less than n and that is prime to

the value of z.

4. Compute a value d as the private key, where e*(d-1) should be divisible by

the value of z.

Therefore, Bob can be distributed the public key (n,e) to Alice or anyone in the

network, then Alice can use the public key to encrypt data and Bob can use his private

key (n,d) to decrypt data.

For instance, Alice transmits a number or a bit pattern m to Bob, where m is less than

n. Alice first computes me and divid it by n to get the remainder c, which c is the

cipher or encrypted data. Once Bob receiving the encrypted data c, he computes cd

and divids it by n to get the remainder m, where m is the original plaintext, which is

shown as following equations.

c = me mod n //use the (n,e) public key to encrypt plaintext and get

encrypted data c

m = cd mod n // use the (n,d) private key to decrypt the encrypted

data then get plaintext m.


11

Select two very largeprime values, p and q

Select two very largeprime values, p and q

n = p x qz = (p-1) x (q-1)

n = p x qz = (p-1) x (q-1)

Choose a number, e, lessthan n, which has no

common factors with z

Choose a number, e, lessthan n, which has no

common factors with z

Find a number, d, s.t. ed-1is exactly divisible by z

Find a number, d, s.t. ed-1is exactly divisible by z

Getpublic key (n,e)

andprivate key (n,d)

Getpublic key (n,e)

andprivate key (n,d)

1.

2.

3.

4.

5.

Fig 7-5. Procedures of public key and private key section by RSA

Next, we give an example to describe key selection procedures in the RSA algorithm.

First, Bob selects p=11 and q=17, then computes n by p*q (n=187) and computes z by

(p-1)*(q-1) (q=160). Second, Bob selects 23 as e, where e is prome to z. Finally, Bob

computes (z+1)/e to obtain d=7. Therefore, Bob distributes the public key (n=187,

e=23) to Alice. Once Alice uses the public key to encrypt a plainytext m and gets the

encrypted data c. After Bob receives the encrypted data c, he decrypts it by his private

key (n=187, d=7).

Assume that Alice sends a plaintext of “clap” to Bob, Alice first maps characters

a~z to numbers 1~26 and obtaining ‘c’=3, ‘l’=12, ‘a’=1, and ‘p’=16. The encryption

procedure with public key (n=187, e=23) is shown in Fig. 7-6(a) and the decryption

procedure with secret key (n=187, d=7) is shown in Fig. 7-6(b)

Plaintext m me c = me mod n ‘c’ 3 94143178827 181 ‘ l’ 12 6.6247E+24 177 ‘a’ 1 1 1


12

‘p’ 16 4.9517E+27 169

Figure 7-6 (a) Procedure of Alice encrypt ing plain text “clap” by using public key

(n=187, e=23)

Encrypted text , c cd m = cd mod n Plaintext 181 6.3642E+15 3 ‘c’ 177 5.4426E+15 12 ‘ l’

1 1 1 ‘a’ 169 3.9373E+15 16 ‘p’

Figure 7-6 (b) Procedure of Bob decrypt ing by using secret key (n=187, d=7)

From above explanation of the RSA algorithm, we know that both of encryption

and decryption procedures are computed by exponential operation, which results in

high computation complexity. As RSA Lab. declarates that the efficiency is 21.6 kb/s

for 512-bit and 7.4 kb/s for 1024-bit, respectively. Nevertheless, efficieny of DES is

100 times faster by software computation and 103~104 times faster by hardware

computation than that of RSA. It is clearly, the RSA algorithm suffers from

computing of vast amount data. Another issue is how to select an extreme large

numbers to satisy RSA efficiently.

Today most applications are combined the symmetric and asymmetric key

systems in reality. Sender Alice randomly generates a session (symmetric) key to

encrypt plaintext into ciphertext C. Then sender Alice uses Bob’s public key

(asymmetric) to encrypt the session key, and sends it with ciphertext C to receiver

Bob. After Bob receiving them, Bob first uses his private key (asymmetric) to decrypt

the encrypted session key. Second Bob uses the session key to decrypt the ciphertext

C into plaintext successfully. By using the advantage of cominding symmetric and

asymmetric key systems, both of sender Alice and receiver Bob are unnecessary to

know the same session key before data transmission. Hence, the key distribution

procedure is under safe operation and results in efficient data transmission.

Therefore, RSA has two primary functions. One is using RSA to encrypt session

key of DES for convient distribution of symmetric session key, and the other function

is using RSA to authenticate network users. Authentication is an important issue in

network security, which will be described in next subsection.


13

7.2.2 Digital Signature

The problem of invisible under long distance communication has been mentioned

in the beginning of this chapter. Once Bob receives a message from Alice, how to

identify that the message is sent by Alice. If an intrusion Trudy pretends Alice to send

it that reveals authentication is essential and important. There are three methods to

achieve authenticate including, secret information, possession of object, and

characteristic. Password and encryption authentication are based on secret information.

Checking password is the most popular method to identify network user, but it’s easy

to attack by hacker, network intruder and network monitor. Hence, checking password

is not a good method to achieve authentication. Another encryption has described in

previous section. In symmetric key system, the main problem of key distribution

should be addressed. In asymmetric key system, “Digital Signature” is the most

popular authentication. Like passport checking while boarding a flight. Consequently,

there are three advantages of applying digital signature onto transmission data

including, to identify this data is sent by sender Alice, sender Alice can not deny that

he sent the data before, and receiver Bob can not modify the received data.

By using asymmetric key system and hash function to achieve the technique of

digital signature. As shown in Fig. 7-7 and 7-8, sender Alice adopts digital signature

to perform authentication with sending plaintext to receiver Bob. In Fig. 7-7, sender

Alice first computes the plaintext via a hash function to get a unique hash value of

“12340782”, and then Alice encrypts the hash value by his private key and obtains

encrypted text of “??!!??!!”. Finally, Alice sends the encrypted text, i.e. Alice’s digital

signature, with plaintext to Bob. After Bob receiving the plaintext with digital

signature of Alice, two separate processes are applied. First Bob decrypts the digital

signature of “??!!??!!” by Alice’s public key to get the hash value of “12340782”.

Second, Bob computes the plaintext by the same hash function to obtain the hash

value of “12340782”. If these two hash values are the same, it certifies that the

plaintext is sent by Alice. Finally, above mentioned three functions of digital signature

are satisfied as follows.

Alice can not deny sending this document before, for the reason that Alice

encrypts hash value via his private key.

Bob can not modify the received document; otherwise these two hash values


14

will not be the same.

The document is identied that is sent by Alice because of having the same

value of “12340782”.

Plaintext Hashfunction

Hashfunction 12340782

unique hash value

??!!??!!

Alice'sprivate key

??!!??!!

Plaintext

Encrypted text

Plaintext with"Digital Signature"

Plaintext

+

a) Alice can NOT deny sendingthis document, for the reasonthat Alice encrypts hash valuevia his private key

Figure 7-7 Alice sends documentat ion with “Digi tal Signature”

Hashfunction

Hashfunction

12340782

unique hash value

??!!??!!Alice's

public key

??!!??!!

Plaintext

Plaintext

c) The document is identifiedthat is sent by Alice because ofhaving the same value"12340782"

12340782

b) Bob can NOT modify thereceived document, otherwisethese two output values will notbe the same.

The document is sent byAlice, if these two outputvalues are the same.Otherwise, Bob can NOTidentify it is sent by Alice.

Figure 7-8 Bob ident ify the received documentat ion with “Digi tal Signature”,

whether i t is sent by Alice or not

We have described that Alice should be generated a hash value of the corresponding

plaintext before sending it. The hash value is so called the Message Digest (MD). The

function of message digest is to keep data integrity for achieving three characteristics

of authentication. Popular hash functions include MD4, MD5, and Secure Hash

Algorithm (SHA), etc, where MD4 and MD5 proposed by Ron Rivest in 1992 and

MD5 is the most useful algorithm to generate a 128-bit message digest. Furthermore,

a similar to MD4 hash function, SHA-1, is adopted by the US Federal government,


15

which generates a 160-bit message digest.

7.2.3 Transport Layer Security

Cryptograph theory and authentication techniques have been described in the

beginning of this section. Now we explain how to combind these two parts for

achieving secure function in transport layer.

For the increasing demand to access mobile information via heterogeneous

wireless networks, the wireless mobile networks and Internet play an important role to

achieve the goal, in which there are several key areas: the wireless mobile networks,

the personal mobile communications, and network security. In the area of network

security, how to build a secure and reliability e-transaction or m-transaction between

client and server hosts and to protect private information of client are very important.

One of good solution is the mechanism of Secure Socket Layer (SSL).

Nevertheless, in the e-transaction procedure of E-commerence, a more secure

mechanism is needed. Security Electronic Transaction (SET) was proposed for this

purpose. Both of SSL and SET will be introduced in Section 7.2.3.1 and Section

7.2.3.2, respectively.

7.2.3.1 Secure Socket Layer (SSL)

In web browser, a small lock icon is always in unlocked status and sometimes in

locked status while sending secure data. That is an example operation of SSL while

lock is in locked status. SSL was proposed by Netscape to support data encryption and

authentication of data exchange between web client and sever. Also SSL is one of

popular web secure mechanism, which adopts the Transport Layer Security protocol

(TLS) and operates between Transport and Application layers. Now that is defined in

RFC 2246, which is shown in Fig.7-9.

Before performing SSL, client and server should be negotiated with data

encryption algorithms such as, DES or IDEA, and both authentication certification.

After completing the negotiation procedure, the key encryption and decryption

processes can be started to confirm secure of data transmission. The transaction flow

of SSL protocol is shown in Fig. 7-1, which is explained as follows.


16

Client send “SSL Client Hello” message to construct encryption mechanism

with Server.

Server replys “SSL Server Hello” message to Client to confirm it, then it sends

its certification back to Client to request Client’s certification.

Client sends its certification to Server.

After that, Server and Client perform the negotiation of key exchange, in which

session key is encrypted by Server’s public key. Finally, Client and Server

obtain session key and peform data encryption and data exchange.

ApplicationApplication

SSLSSL

TCPTCP

IPIP

Figure 7-9 SSL layer

SSL Client Hello

SSL Server Hello

Server Certification

Request Client Certification

ClientKeyExchange (RSA)

Certificate Verify

ChangeCipherSpec

Client Server

Finished

Encrypted data stream (DES)

SSL Handshake

Encrypted Data

Client Certification

DigitalSignature

Figure 7-10 SSL Transact ion f low


17

Lack of data integrity in SSL

SSL supports protocol of data encryption between Client and Server, but it lacks

of integrity of secure payment mechanism in backend, for example, secure payment of

credit card. Assume that Alice orders some merchandise from Bob and pays it by

credit card. The credit card information is secure to send to Bob. Since Bob has key to

decrypt the encrypted information of Alice’ credit card, we can’t make sure that Bob

will be abused of Alice’s credit card information. This is the reason of SSL lacking of

data integrity and fully secure. Moreover, SSL also lacks of the certification of

Client’s credit card. Once hacker gets someone’s credit card number, he may be abuse

it. Furthermore, the transmission data is encrypted between Client and Server,

Intrusion Detection System (IDS) will not filter the encrypted information, which

results in security holes of host. Since SSL lacking of data integrity and having

security holes, the Security Electronic Transaction (SET) was propsed to overcome it.

SET supports fully secure electronic transactions between frontend and backend hosts

which is described in next subsection.

7.2.3.2 Secure Electronic Transactions (SET)

Secure Electronic Transactions (SET) is a secure payment

protocol, whichis proposed by Visa, MasterCard, IBM, Microsoft, and

HPcooperarions in February 1996. And the Secure Electronic Transaction LLC

(or called the SETC) organization established by July of 1997 is responsible for the

management and promotion SET protocol in the world. Basicly, the characteristic of

SET is shown as follows.

SET only provides to encrypt the related information in payment,

rather than SSL can encrypt information between the Client

and Server.

SET combines the buyer, selling party and selling party bank,

and provides encryption high sensitivity data, which is

transferring among these three parties. At the same time, these


18

three are required to have the Digital Certification.

The main difference between SET and SSl is that SET will not

give the creditcard number of buyer to the seller. That prevents

abnormal using buyer ’s creditcard by the seller; hence, it

keeps the payment in secure enviorment.

SET is to apply in the finance system, so it does not be

restricted to use shorter key.

Then we describe the operation flow of SET by using the Fig.

7-11. In SET, there are four main roles including buyer Bob, e-shop

seller Alice, crad holder ’s bank, and e-shop’s bank. Bob’s public key

(EB) and private key (DB), Alice’s public key (EA) and private key (DA), and both

Certifications need to be process in the operation of SET. The main data flow of

SET is that the order information and creditcard number of cardholder Bob should

be sent to the crad holder ’s bank and e-shop’s bank safely.

Consequently, the order flow of ordering some products from buyer

Bob to Alice’s e-chop via SET security protocol is shown as

follows.

1. Bob selects some interesting products from Alice’s e-shop and infors Alice that

he will be paid by creditcard.

2. Alice returns transaction ID of this order to Bob.

3. Alice sends his certification, public key, and public key of his

bank to Bob.

4. Bob receives above messages at step 3.

5. Bob makes an order from network and has Order Information (OI)

and Purchase Information (PI). Bob encrypts OI by Alice’s

public key and sends it to Alice. At the same time, Bob encrypts

PI by the public key of Alice’s bank and sends it to Alice’s

bank.

6. Alice sends “Request to Certificate” message to Bob’s credit card

bank with the order ID.

7. Alice uses the public key of his bank to encrypt the following

messages, including the encrypted PI from Bob, Alice’s

Certification and “Request to Certificate” message and sends


19

them to his bank.

8. Alice’s bank decrypts these encrypted messages and checks that

were modified before or not.

9. Alice’s bank uses the original exchange mechanism of creditcard

to process the related operation.

10. Bob’s bank replys the result of certification to Alice’s bank.

11. If Alice’s bank receives “successful certificated”, then it replys

the message to Alice.

12. If everything is OK, Alice sends the reply of order message to

Bob for making sure that the transaction is done.

From the operation flow of SET, each pair procedures of request or

response should need two parties. This is to protect any third party to

modify or gather secure information. Furthermore, the creditcard

number of Bob (within PI) has been encrypted by the public key of

Alice’s bank, Alice can not obtain the original creditcard number of

Bob. As a result, SET can provide secure mechanism for e-transaction

to ensure secure transaction environment through networks.

Merchant Server

Merchant

Payment Gateway

CA

P.S 2341Credit Card

Acquirer (Bank)

Issuer/Credit Card Bank

Internet

Internet

Cardholder

E-wallet

1

6,7

9

10

11

12

2,3

4

5

Merchant Server

Merchant

Payment Gateway

CA

P.S 2341Credit Card

Acquirer (Bank)

Issuer/Credit Card Bank

Internet

Internet

Cardholder

E-wallet

1

6,7

9

10

11

12

2,3

4

5

Figure 7-11 SET Operat ion Flow


20

7.2.4 Internet Security, IP Security (IPSec)

TCP/IP is the most popular protocol used in present networks. Since the TCP/IP

protocol is used in several areas, Internet becomes the largest network in the world.

Since Internet protocol does not define any secure mechanism, the transmission data

in Internet is easy to capture and decode. How to provide a secure transmission in the

largest network in this world is very important in real applications and research issues.

In order to ensure the network secure, where TCP/IP is provided. The IETF

establishes an open standard of network security protocol, i.e., Internet Protocol

security (IPSec), and expect to apply the security technology in the network layer for

providing both the transceiver and the receiver in security communication service.

Meanwhile, it also allows the upper application or protocol use these safe services.

Therefore, in section 7.2.4.1, we first introduce the concept of IPSec, and then

describe the mechanism of IPSec, which defines the IP Authentication Header, IP

Encapsulation Security Payload, and the key management, to achieve the request data

integrity, authentication, and privacy in security communication.

As developing of electronic commerce (E-commerce) for transacting secured data

between enterprise and customer, the Virtual Private Network (VPN) is promoted for

the purpose.

Due to the VPN has the advantage of inexpensive and easy setup, it has been

adopted by several enterprises. Therefore, in section 7.2.4.2, we will make thoroughly

discussion in the VPN concept and various VPNs design.

7.2.4.1 IP security (IPSec)

Because Internet Technology becomes mature in recent years, more users use the

Internet convenient in such a public network. Many commercial services were

constructed based on Internet; therefore, the private communication is concerned with

the users as they often need transfer the secret data. If there is not any trusted secure

network, it causes that the network user lacks of confident for using the network

commercial services.

For overcoming such an issue, several network security standards are proposed in


21

succession to Session Layer and Application Layer. As mentioned before, SET and

SSL can achieve secure HTTP, the PEM standard can achieve secure E-mail, which is

proposed by the PSRG group of IETF, and General Security Service Application

Program Interface (GSSAPI) supports secure transmission in Telnet, FTP and HTTP,

which is referred as RFC1508 and 1509.

In fact, these applications or protocols are based on the Internet Protocol (IP).

Therefore, a secure mechanism for IP is necessary to integrate different secure

mechanisms of various applications of upper layer.

In such situation, IETF established IP Security (IP Sec) for IPv4/v6 to achieve the

following goals, including Authentication, Integrity, Confidentiality, and Access

control, etc.

The first version of IPSec (RFC1825 to RFC1829) was proposed in 1995. There

are two primary modes of it, including IP Authentication Header (AH) and IP

Encapsulation Security Payload (ESP). The former mainly provides the integrity and

authentication of data, but the latter provides the secure data transfer. For using IPSec

in the IPv6 environment, it designs in two option headers that include the

Authentication Header and the Encapsulation Security Payload Header.

In the first version of IPSec, there is no description about key exchange and

management. The first version mainly defines the transformation of the format of a

packet. In 1998, the second version of IPSec (RFC2401, RFC2402, RFC2406) was

proposed, and Security Association (SA) and the key management- IKE (Internet Key

Management) are included. Consequently, the IPSec becomes completely after

including SA and IKE.

Security Association

For the purpose of private communication in IPSec, a secure environment is

required to transfer data between transmitter end and receiver. Security Association is

designed for building such a secure environment. Meanwhile, Security Association is

also the most important concept in the framework of IPSec.

For the transmitter end the receiver, SA provides a unidirectional connection of

secure transfer. In SA, several important parameters are defined, for instance, the

authentication algorithm and the key which is used in the authentication algorithm,


22

the encryption/decryption algorithm and the key which is used in the

encryption/decryption algorithm, and a valid period of keys, etc. Therefore, a private

and secure communication can be achieved with the same SA.

A 32-bit Security Parameter Index (SPI) can define a security association.

Moreover, a unique SA can be defined by IP address of a host, a security

identification code (represents AH or ESP), and SPI.

Since SA is a unidirectional, it requires two SAs to build

bidirectional point-to-point secure transfer. Furthermore, a SA uses

either AH or ESP as the security protocol only. Two SAs are required,

if both security protocols of AH and ESP are used at the same time.

Authentication In RFC1828, it suggests that IPSec uses MD5 algorithm to

authenticate. The main function is that the sender computes a message

from the sending IP packet and a secret key with MD5 algorithm, and

then adds the message into the sending packet. After receiving the IP

packet, receiver performs the same MD5 calculation with the IP packet

and the secret key to obtain message value. Then receiver compares the

message value with the added one in IP packet. If these two are the

same, the authentication is success; otherwise, it rejects. Because the

MD5 calculation computes with the whole IP packet, this method not

only performs authentication, but also certify for the data integrity.

In the aspect of supplying authentication services, IPsec defines

two modes of authentications including End-to-End mode and

End-to-Intermediate mode. The main difference is shown in Fig. 7-12.

In the End-to-End mode, both parties of the communication perform the

authentication. This mode was used when both parties of the

communication do not have confidence in the security of network

facilities but still expect to ensure the security of the transmission

themselves. In the End-to-Intermediate mode, the authentication

performed at one party and the router or firewall of the local area

network of the other party of the communication. In this way, the

router or firewall plays the role as a “Security Gateway”. In other


23

words, the security of the local area network that the security gateway

located is guaranteed by the security gateway.

Router/FirewallIntranet

Internet

End-to-end authentication

End-to-intermediateRouter/FirewallIntranet

Internet


End-to-intermediateRouter/FirewallIntranet

Internet


End-to-intermediate

Figure 7-12 Authent ication Types

Figure 7-13 shows the content format of the authentication header.

The first field, Next Header, represents the payload type. Following is the

8-bit Length field. The 16-bit Reserved field is reserved for future using.

In present, the value of Reserved field is set to 0. The SPI field represents a

unique SA. The Sequence Number Field represents the sequence

number of packets to prevent the replay attack.

Security Parameter Index (SPI)

Authentication Data (variable)

Sequence Number Field

ReservedLengthNext Header

Security Parameter Index (SPI)

Authentication Data (variable)

Sequence Number Field

ReservedLengthNext Header

0 8 16 31

Figure 7-13 Authent ication Header

Encapsulation Security Payload Encapsulation Security Payload provides secure IP packet

transmission. In present, the IP ESP adopts DES or Triple-DES as the

encryption standard. ESP does not only guarantee data secure, but also

achieve authentication, data integrity and prevent to the attack of

retransmission. There are two modes within ESP including Transport


24

Mode and Tunnel Mode. Transport mode is used to encrypt the block of

transport layer, and the tunnel mode is aims entire IP packet to

encrypt.

Transport mode ESP and tunnel mode ESP are shown in Fig. 7-14

and 7-15, respectively. In Transport mode ESP, ESP header locates

before the data block of transport layer. The advantage of this mode is

that the encrypted part is less than that of tunnel mode ESP. Since it is

not necessary additional IP header, the required bandwidth is less than

tunnel mode. Moreover, encrypt and decryption are done at both hosts

in the transport mode ESP. The transport mode ESP is preferable, if the

communication from end to end do not trust in the transmission route

various networks equipment security.

In the tunnel mode ESP, ESP header locates before the encrypted IP packet

and it produces a new IP header. This mode is suitable for the Internet

environment that uses security gateway to protect. During transferring,

sender or gateway performs the encryption procedure of IP packet, and

then the encrypted packet is sent to the receiver’s gateway. After the

receiver’s gateway receives it , i t decrypts the IP packet and sends the

original plaintext data to the receiver.

IP Header Ext. Header ESP Header Transport layer segment

Unencrypted Encrypted

IP Header Ext. Header ESP Header Transport layer segment


Figure 7-14 Transpor t Mode ESP

IP Header Ext. Header ESP Header IP header + Transport layer segment


IP Header Ext. Header ESP Header IP header + Transport layer segment


Figure 7-15 Tunnel Mode ESP

The AH in combination with the ESP:

Through the combination using with AH and ESP, it can achieve encryption

and authentication at the same time. The procedure of encryption can be done before

authentication or versus. In the case of encryption before authentication in


25

transmission mode as shown in Fig. 7-16, Encapsulation Security Payload is firstly

encrypted by ESP, which will be encapsulated by AH. In the tunnel mode, the entire

IP packet is encapsulated by ESP and encapsulated by AH. In the case of

authentication before encryption, this is suitable for the case of transmission mode, in

which the payload is encapsulated by AH and then encrypted by ESP as shown in Fig.

7-17.

IP Header Auth. Header ESP Header Transport layer segment

Scope of authentication

E-T

E-T : Encapsulating Security Payload trailing fields

Figure 7-16 Encrypt ion before authent icat ion

IP-H IP-H Transport layer segment

Scope of authentication

E-TESP-H A-H

Figure 7-17 Authent ication before encrypt ion

Key Management Because of AH authentication and ESP encryption need both encryption and

decryption keys. Therefore, key management and the exchange become important

in the IPSec standard. At present, the main key management protocol includes SKIP

(Simple Key-management for IP) and ISAKMP/Oakley (Internet Key Exchange, IKE).

SKIP is proposed by Sun Microsystem, which adopts the Diffie Hellman’s

key exchange algorithm to transmit the secret key in the network. In order to

guarantee it secure, the Public key must apply for the certificate via Certificate

Authority (CA). Therefore, it needs the public key infrastructure to fulfill the purpose.

In the case of IKE, Oakley defines the ways to distinguish and confirm key.

Nevertheless, in the case of ISAKMP, there are two steps in the operation of ISAKMP.

In the first step, both ends of ISAKMP communication should be setup a secure and

authenticated channel, ISAKMP SA, via negotiation, which is the first built SA. In the

second step, it uses the ISAKMP SA to build AH or ESP’s SA. The primary

difference between ISAKMP SA and IPSec SA is that ISAKMP SA is a bidirectional

SA, but IPSec SA is a unidirectional SA.


26

7.2.4.2 Virtual Private Network (VPN)

After introducing the standards and operation of IPSec, the most popular

application of IPSec in commercial, Virtual Private Network (VPN), is described

in this section. VPN is to build a private network via public networks such as

Internet. In previous, the private network communication between two enterprises

always used Lease line from ISP including Lease Line, ISDN, ATM, Frame Relay.

The disadvantage of it is the leased fee costs too much. The advantage is keeping

transfer in a secure network. On the other hand, the most popular public network,

Internet, is cheap, convenient, and scalable. Therefore, the goal of VPN is to build

a virtual private network via the public Internet to achieve the cheap, secure and

efficient transmission among enterprises. VPN are based on the following

technologies:

Tunneling

Encryption and Decryption

Key management

Authentication

The technologies of Encryption and Decryption, and Key

management have described in previous sections, Cryptology and IPSec.

Hence, we make a detail explanation of Tunneling and Authentication in this section.

Tunneling technologies

Tunneling is based on the technology of encapsulating packet. It builds private

communication tunnel via public networks. Currently, two tunnel technologies are

adopted in IPSec One is layer 2 tunneling and the other is layer 3 tunneling. In general,

if a company wants to use it's corporate network to provide PPP service, users must

connect to company's PPP server when they want to dial directly to the corporate

network. The advantage of the layer 2 tunneling is that users just dials to local

Network Access Server (NAS) and uses the NAS to establish tunnels. This could


27

reduce by a large amount the phone bill for directly connection. In other words, this

uses the Internet to transfer PPP frames. PPP can support many protocols; therefore,

layer 2 tunneling can support IP, IPX, NetBEUI, and AppleTalk at the same time.

Layer 3 tunneling technology has been explained in the section of IPSec.

Microsoft proposed PPTP, which is developed for using on VPN. Based on PPP,

PPP frames transmit IP packets in the tunnel. There are two types of PPTP tunnels:

client-initiated mode and ISP-initiated mode. In the client-initiated mode, client

initiates direct connection to the PPP server. In the ISP-initiated mode, client

establishes a PPP session with the ISP access server, and the ISP access server

establishes tunnels with remote PPTP server. The established tunnel can be shared by

several connections by the means of call ID.

L2TP mainly combines Layer 2 Forwarding (L2F), which is proposed by Cisco,

and PPTP protocol. Each end of the L2TP tunnel acts as a L2TP Access

Concentrator (LAC) and L2TP Network Server (LNS). Usually LAC acts as

the client site and the LNS acts as the server site. The L2TP has two types of message,

control and data. The control message mainly establishes and manages tunnels, and

can utilized reliable transfer mode such as ATM. Data message is transmitted by

packet frames and utilize unreliable transfer modes such as UDP. The tunnel

establishing, is the same as PPTP, can be shared by many connections by the means of

call ID.

Authentication

Two types of authentication are available, user authentication and device

authentication. There are several techniques in user authentication. The most use

is user account and password, or card authentication. Device authentication uses

X.509 electronic certificates delivered by Certificate Authority. Before exchanging

data between both ends, certificate should be sent to establish trust relationship. Both

ends compare the certificate. If it matched, then the authentication is successful.

Data exchange then would follow. Otherwise, rejects the exchange of data.

Classification of VPN

The standard document of VPN (RFC2764) classifies VPN into four types:

Virtual Lease Line, VLL

Virtual Private Routed Network, VPRN


28

Virtual Private Dial Network, VPDN

Virtual Private LAN Segment, VPLS

In the four types of VPN, VLL is the simplest one. User establishes point-to-point

link through Customer Premises Equipment (CPE). The connection between the host

and the ISP, so called stub link, can be any link level style connection, such as ATM

VCC or Frame Relay. As shown in Fig. 7-18, two ISP ends are connected to IP

backbone network, and establish connection through IP tunneling. VLL also sets the

stub link on the ISP end to work with the IP tunnel. An example of this is that the data

sending ISP edge node can pack the ATM AAL 5 payload and send it to the IP tunnel,

and the data receiving ISP edge node would unpack the data received and send the

original ATM AAL 5 payload to data receiving end. To the end user, the VLL structure

seems to establish a route that connections two ATM VCC or Frame Relay CPE ends.

CPE

10.2.3.5ISP edge node

IPBackbone CPE

ISP edge node

FrameRelay

Circuit

IP tunnel

subnet = 10.2.3.4/30

10.2.3.6

FrameRelay

Circuit

Figure 7-18 Example of Vir tual Lease Line

VPRN is designed to emulate an environment of multi-node wide area routed

network. This kind of VPN differs from other VPN kinds are that the packets are

transferred in the network layer. The whole VPRN ISP edge nodes form a full mesh

network. Every ISP edge node can transfer packets to the destination server by routing

mechanism. Therefore, every ISP routed network all has a VPRN packets forwarding

table. Figure 7-19 is an example of three ISP edge nodes forming a full mesh network.

The three ISP edge nodes are all connected to the other two by an IP tunnel. The

backup link in the figure is for CPE to establish another ISP edge node link when the

stub link failed. This enhances the overall strength of the whole network. A backdoor


29

link is a connection created by two CPE without going through ISP network.

The advantage of VPRN is that complex works is mostly done by the ISP edge

nodes. CPE's setup and work are reduced to minimum. CPE just needs to pass data to

ISP edge node through a stub link, and ISP edge node would transfer the data. There

is no need for CPE to establish links. Other than that, firewall and quality assurance

type of services are provided by the ISP edge nodes as well. But VPRN also has its

disadvantage. Due to the need to establish a full meshed network, this VPN structure

is not suitable to networks with large number of SIP edge node router.

IPBackbone

ISPedge

router10.5.5.0/30

ISPedge

router

ISPedge

router

CPE CPE

CPE CPE

IP tunnelstub link

backdoor link

10.11.11.1/3010.6.6.0/30

10.8.8.0/3010.7.7.0/30

10.11.11.4/30 10.11.11.7/30

backup link

Figure 7-19 Example of Vir tual Pr ivate Routed Network

VPDN is a user-connected tunnel by ad hoc tunnel linked to remote network.

In other words, user dials up or uses ISDN to connect to public IP network. The

VPDN uses the L2TP protocol. Through L2TP, an end of the user's PPP session can

extend through LAC to remote LNS server. Because L2TP is a combination of L2F

and PPTP, it also provides PPTP client-initiated and ISP-initiated mode. In the L2TP,


30

these are also called Compulsory Tunneling mode and Voluntary Tunneling mode. In

Fig. 7-20, in the Compulsory Tunneling mode, host side edge router acts as LAC and

corporate network acts as LNS. In this mode, LAC uses L2TP to extend the original

PPP session on the LAC to the remote LNS side. The Voluntary Tunneling is shown in

Fig. 7-21. The host itself acts as LAC, and establishes a tunnel with corporate network.

This way, the corporate router is not involved in the tunnel establishing and network

devices are not involved either.

HOST

10.2.3.5NAS (LAC)

IPBackbone Corp. Network

GW (LNS)

dialconnection

10.2.3.6

L2TP Tunnel

PPP session

Figure

7-20 Compulsory Tunnel ing Example of VPDN

HOST (LAC)

10.2.3.5NAS

IPBackbone Corp. Network

GW (LNS)

dialconnection

10.2.3.6L2TP Tunnel with PPP session

orIPSec Tunnel

Figure 7-21 Voluntary Tunnel ing Example of VPDN

VPLS uses Internet devices to emulate a local area network. Its structure is

very similar to VPRN. The main difference is that ISP router of VPLS is used to

execute network layer transfers, but ISP host executes link bridging. Figure 7-22 is an

example of VPLS.


31

IPBackbone

ISPedgenode

10.5.5.1/24

ISPedgenode

ISPedgenode

CPE

IP tunnelstub link

10.5.5.2/24

10.5.5.3/24

CPE CPE

Figure 7-22 Example of Vir tual Pr ivate LAN Segment

7.2.4.3 Open Source Implementation：FreeS/WAN

FreeS/WAN is a Linux software that processes IPSec and IKE. It works as a

module in a Linux kernel. The whole FreeS/WAN can be divided into three parts:

KLIPS: it is an abbreviation of Kernel IP Security. It is designed to process

packet handling with AH, ESP, and the packet processed part in kernel.

Pluto Daemon: It is responsible for processing IKE Daemon, and handling the

exchange and management of encrypted/decrypted keys.

Administrator interface

The flow chart of KLIPS is shown in Fig. 7-23. When the administrator uses

insmod command to activate KLIPS module, it executes init_module(). This function

will register KLIPS as a module to the Linux kernel, and then it uses ipsec_init() to

initialize some parameters. This part can be set manually by command, or can be done

by saving the setting in a file of ipsec.conf. When the administrator uses rmmod

command to remove the KLIPS module, it would then execute clean_module() to

clean up the data and release the memory.


32

START

init_module()

ipsec_init()

cleanup_module()

Figure 7-23 Flowchar t of KLIPS

Because IPSec needs to use tunnel technology to establish a private passage

between these two communication ends, it would use a data structure called Tunnel

Descriptor Block (TDB) to record information about the private tunnel, Information

such as source IP address, destination IP address, error code, tunnel status, etc are

saved. KLIPS also uses another data structure called radix tree to save the path

information needed for packet transferring mechanism when communication end acts

as a communication gateway.

Figs. 7-24 and 7-25 shows the work flow when ipsec_init() function is called. At

the beginning, ipsec_init() calls ipsec_tdbinit() to initialize the values in the TDB,

then it calls ipsec_radijinit() to initialize radix tree. Then it uses pfkey_init() to decide

the exchange and management method. In KLIPS, four virtual network devices ipsec0,

ipsec1, ipsec2, and ipsec3 are defined. The commands by the administrator are

directly placed to the virtual devices. The register_netdevice_notifier() function

provided by the kernel is to register these virtual network devices, like plug-in

network interface cards. Then through administrator's commands or kernel function

inet_add_protocol(), registration of security protocol such as AH, ESP, or IPCOMP

(IP Compression) to the inetd is completed. After this process, ipsec_init() calls

ipsec_tunnel_init_device() to initialize the four virtual network devices. Finally,

depending whether user uses sysctl command to control the ipsec, it decides to call

ipsec_sysctl_register() or not.


33

ipsec_tdbinit()

ipsec_radijinit()

pfkey_init()

register_netdevice_notifier()

ESP

AH

IPCOMP

NO

NO

YES

inet_add_protocol(&esp_protocol)

YES

YESinet_add_protocol(&comp_protocol)

inet_add_protocol(&ah_protocol)

Figure 7-24 Flowchar t of ipsec_init () (PART I)

ipsec_tunnel_init_device()

SYSCTL YES

ipsec_sysctl_register( )NO

RETURN

Figure 7-25 Flowchar t of ipsec_init () (PART II)

Figure 7-26 is the execution flow chart of Pluto Daemon. At the beginning, the

Pluto Daemon executes some initialization including the random number generator at

the beginning of the key exchange, private keys, and the chart that records the

connection status. It also loads some modules for decryption. Then the Daemon waits

for events. If events occurred are related to timer time out, it calls functions that are

designed to handle this issue. If not, then the event must be a packet from the other


34

side. At this point Pluto Daemon would call packet handling function to handle this

event.

START

initialization

wait for event

NO

timer event?

invoke packet handler

invoke timer handler

YES

Figure 7-26 Flowchar t of Pluto Daemon


35

7.3 Firewall 7.3.1 Introduction

Due to the development of electronic commerce is very high-speed, the business

transaction becoms very frequently. For this great mass fervor, the global enterprises

are proceeding as the E-commerce job to increase the performances of the internal

and external procedures at the enterprises that promoting competition and profit. The

digitization of information lets many secure data can be acquired from the enterprise

intranet but it also becomes the object hackers want to attack. Hence for supplying a

function to protect the secure data, there are many vendors research、develop and sell

various security products. The common product to protect the enterprise network is

firewall.

In a word, firewall provides an access control between two networks. The

transmission packets between private and public network must be checked from

firewall, the packets will deliver when they match the access rules otherwise will be

blocked or recorded to inform the system administrator. Firewall has the following

major characteristics:

Sevice request transmission:

Any servers or hosts in the private network will not be direct access from the

public network, any service request to servers or hosts must be transferred

through firewall, hence it can avoid direct attack from the public network.

Hidden private network:

Due to the private network will be isolated with the public network used firewall,

for users in the publice network who knows there is a firewall but is not aware

that has other servers or hosts in the private network. Firewall can hide the

private network outside of the public network and avoids direct accessing from

the public network.

Abnormal status record:

Firewall can check the transmission packets between the private and public

network, hence it can make records which accessing the private network. If

there are any abnormal statuses, it informs the system administrator to reduce

probability of attack or violence in the private network.


36

As above mention, firewall provides three major protections. First, it prevents

login to the hosts of the private network from user in public network that has not an

access permission, the login request will be blocked when it through the firewall.

Second, it monitors and records the using status to prohibit any abnormal data access.

Finally, it monitors any irregular commands in the private network. It informs the

system administrator to process the job of defences and remedies when detecting the

haker make an attack.

In accordance with the difference of check field of the packet, there are two kinds

of firewall, Network Layer Firewall and Application Layer Firewall. We will have a

detail description at section 7.3.2 and 7.3.3. Additional, we will introduce separately

that two Linux softwares --- Network Filter and Trusted Information System (TIS) for

understanding two kinds of firewall how to work and using flow chart to present the

different operation methods.

7.3.2 Network Layer Firewall

Network layer firewall also terms as packet filter, that is, this firewall system

filters the packets based on the network layer. Network layer firewall processes

packets based on the header of IP Packet and the rules which administrators gave

definition. The filtering field of packet can be protocol ID, source IP address,

destination IP address, source port number of TCP or UDP, destination port number of

TCP or UDP, etc.

Based on different frameworks, network layer firewall can be separated to

Screened Host Firewall and Screened Subnet Firewall. Figure 7-23 shows the

framework of Screened Host Firewall.

Internet

Baston HostIP filtering router

Private Network

allow

disallow

Figure 7-23 Screened Host Firewall

In the Screened Host Firewall, the incoming and outgoing packets must pass


37

through the Bastion Host and do not access directly between public network and

private network. In another word, it must be setting in the IP filtering router that only

permits the destination IP address is the Bastion Host IP Address that can get across

when packets from public network to private network. Another, it prohibits the packet

from private network to public network except the source ip address is the Bastion

Host.

Using this framework, we can understand that Bastion Host is the only network

node, which can go to the private network from public network. The security of the

entire private network is base on Bastion Host. Hence, Bastion Host is just the outside

gateway of the private network in this framework and must resist any attack. The

advantage of Screened Host Firewall is that the setting of filtering packet in the IP

filtering router is very simple, because packets of incoming or outgoing private

network must pass through the Bastion Host and just make access rules for it. The

drawback is if let some particular services go to private network that do not pass

through Bastion Host when administrator permits these situation for convenience, the

entire private network will be exposed on the public network and the security will

decrease dramatically if the packets go to private network via these services.

InternetIP filtering router IP filtering router

PrivateNetwork

Baston Host

DMZ

Figure 7-24 Screened Subnet Firewall

Figure 7-24 shows the framework of Screened Subnet Firewall. Utilizing two IP

filtering routers form private network and DMZ (Demilitarized Zone). In this

framework, because an IP filtering router has been built near the private network, the

hosts in the private network do not have any dangers to expose them on the public

network even though the IP filtering router near Internet is opening some services

which can go to private network that do not pass through Bastion Host. This can

resolve the drawback of Screened Host Firewall. The setting of the IP filtering router

is similar to Screened Host Firewall. The IP filtering router next to the public network


38

sets the access rules to confirm with destination IP address of incoming private

network must be the Bastion Host and source IP address from private network to

public network must be the bastion host too. The IP filtering router next to the private

network sets the access rules to confirm with destination IP address of outgoing

private network must be the Bastion Host and source IP address of incoming private

network must be the Bastion Host.

In Screened Host Firewall, Bastion Host is the monitored host but DMZ will be

the monitored subnetwork in Screened Subnet Firewall. DMZ is an area between

external firewall and internal firewall. In general, external firewall is the internet

access router in private network and internal firewall, which is used to connect DMZ

and private network. Using the framework that has DMZ allows private network to

own multi-layer protects via firewalls, hence can improve security.

7.3.3 Open Source Implementation: Netfilter

Netfilter is architecture of packet mangling. From a viewpoint of system kernel,

Netfilter is a group of checkpoints of packets that system kernel registers in the

packets when processing them pass through individual communication protocol.

These checkpoints are called Hook. In Netfilter, each Hook has a unique Hook

number. Hence, Netfilter will check the current communication protocol whether it

has a registered Hook when packets are being processed via Netfilter. If there is a

registered Hook, these packets must be checked and follow the definitive rules to

process. Processing packets have the following five actions: (The action of Netfilter is

defined in brackets) Pass acceptance, proceed with next communicat ion protocol.

(NF_ACCEPT )

Drop packets , fol lowed communication protocol does not need to process .

(NF_DROP)

Netf i l ter processes packet , fo l lowed communicat ion protocol does not

need to process . (NF_STOLEN )

Save packet in to the queue. (NF_QUEUE )

Call th is Hook to process packet again. (NF_REPEAT )

In Netfilter, executing the packet check is primary the program of IP tables. There

are five registered Hooks in Netfilter:


39

A. NF_IP_PRE_ROUTING

B. NF_IP_LOCAL_IN

C. NF_IP_FORWARD

D. NF_IP_POST_ROUTING

E. NF_IP_LOCAL_OUT

Figure 7-25 shows the description of five Hooks.

ROUTEA C D

Local Process

B E

ROUTE

Figure 7-25 Hooks regis tered with Netf i l ter

NF_PRE_ROUTING represents the Hook before the host receives the packet but

does not process the routing function yet. NF_LOCAL_IN is the Hook found which

destination address is the host after processed the routing function. NF_FORWARD is

the Hook found that must been transferred to another host after processed the routing

function. NF_POST_ROUTING is the Hook after completed the routing function.

NF_LOCAL_OUT represents the Hook the host sends the packet before does not

process the routing function.

When every Hook is proceeding to examine packets, the defined rules must be

applied. In Netfilter, there are three data structures as following to present the rules:

struct ipt_entry, includes the fields below:

strcut ipt_ip : IP header.

nf_cache: Using bit stream method represents which fields in the IP

header must be checked.

target_offset: Represents the initial location of stract ipt_entry_target.

next_offset: Records the size of content of whole rules that includes


40

ipt_entry_match and struct ipt_entry_target structures.

comefrom: The field is used to trace the transmission of packet in the

protocol stack.

struct ipt_counters: Records the packet amount of comparison with this

rule.

Struct ipt_entry_match: Records the content of compared packet.

Struct ipt_entry_traget: Records actions after comparing.

7.3.4 Application Layer Firewall

Application Layer Firewall executes the filtering jobs in the application layer of

ISO Reference Model. Because the carrier content of filtered packet must be checked

in application layer, Application Layer Firewall can provide more precise, intelligent

security function. As shown in Fig. 7-26, a common style of Application Layer

Firewall of Dual-Homed Gateway. Due to packets must be filtered in the application

layer, it will be blocked when direct using IP forwarding or routing to private

network.

Internet

Dual-Homed Gateway

Private Network

IP routing andforwarding disabled

Figure 7-26 Dual-Homed Gateway

The proxy server is a very common in current Application Layer Firewall. The

proxy server is an application program that transfers packets between private network

and Internet. It usually switches packets based on application layer services (e.g.

HTTP, FTP, Gopher,..., etc.), source IP or destination IP and other rules of

administrator setting in the content of packet carries. In addition, the proxy server still

has advantages for saving network bandwidth and ensuring security of private

network, e.g. HTTP proxy server. When the request of HTTP service of the host in

private network wants to connect the remote HTTP server occurred, HTTP proxy


41

server receives the requirement and checks whether it has the data in its cache

memory. If there is a hit, it will send the required data in the cache memory to the host

and does not connect directly to the remote HTTP server. This can reduce access time

and save network bandwidth. If there is a miss in the cache memory, proxy server

connects directly to the remote HTTP server and sends the HTTP request to get data

then forwards the data to the host which sending the requirement. This method can

avoid the hosts in private network to direct expose on the Internet and enhances the

security of private network.

7.3.5 Open Source Implementation：Trusted Information

System (TIS)

Trusted Information System (TIS) is a set of tools for application layer firewall

that is consist of many application programs. A set of tools of application program can

work alone or cooperate with other application programs to provide the services of

firewall. Entire set of tools provides the following major elements in according to the

supplying services.

Smap: SMTP service.

Netacl: TELNET, Finger and Access Control List.

Ftp-Gw: FTP proxy server.

Telnet-Gw: TELENT proxy server.

Rlogin-Gw: Rlogin proxy server.

Http-Gw: HTTP proxy server

Plug-Gw: News proxy server

When any programs executing in the TIS, netperm-table will be loaded to read

corresponding settings and rules of packet filtering. In another word, netperm-table is

a common setting file for all applications in a set of tools of TIS. The primary content

of netperm-table has tree fields: application name, parameter name, and parameter

content. The check of rules in netperm-table is from up to down, then from left to

right. Figure 7-27 is an example of http-gw part of netperm-table.


42

http-gw: userid roothttp-gw: directory /www_datahttp-gw: timeout 60http-gw: permit-hosts 177.3.4.* http-gw: deny-hosts *

http-gw: userid roothttp-gw: directory /www_datahttp-gw: timeout 60http-gw: permit-hosts 177.3.4.* http-gw: deny-hosts *

Figure 7-27 Example of h t tp-gw part of netperm-table

When http-gw starts, it reads the first field of netperm-table which rule setting is

http-gw. In Figure 7-27, the first setting informs http-gw to use root as its user ID

when it starts that is convenient to access the files or folders which can be accessed

owning the permission of system administrator. The second setting is the location of

directory which getting data from remote http server. The third setting is the longest

establishing time of connection between assigned proxy server and remote HTTP

server. The fourth setting only permits the users of subnet of 177.3.4.* to access this

proxy server. The last setting blocks any user to access the proxy server, using this

setting is usually for preventing holes in security due to the setting error. The rules of

setting is from up to down, so from the fourth column to the fifth column we can

confirm that only the subnet of 177.3.4.* can use this proxy server.


43

START

- DAEMON

ReadConfiguration

Get user's httprequest

Forward httprequest

Receive httpresponse

text/html

Content filterwith FSM

END

Block transferbetween

connections

bind listen

accept

fork=0

NO(inetd)

YES

YES

YES(child)

NO(parent)

NO

Figure 7-28 f low char t of h t tp-gw

Figure 7-28 is the flow chart of http-gw program. Http-gw provides two

execution models. One is processing the job of http proxy server from inetd. Another

is http-gw running a daemon to implement the job of http proxy server by itself. The

first, http proxy server executes the job of loading the setting file from netperm-table

and importing program from rules. When accepting the http request of host in private

network, the comparison of rules are processed. If the result is correct then the host

can use this http proxy server, the request of http forwards to the remote http server.

When receiving the reply of http from remote server, the first is to check the content

whether it is HTML format or not, if it is then filters it with its content, otherwise, the

data will be blocked into the private network.

The above status is for http-gw filtering html. In fact, http-gw also can filter

gopher and FTP via Internet explorer. Simultaneously, the rules of filtering is not only

for host in private network but also for host of external network or an assigned URL

to process the rules setting whether it can access or not. Http-gw usually


44

cooperates with squid to achieve an objective of proxy server who has a cache.

Http-gw there is not a cache service and it will be provided from squid. Squid will

prior check cache memory whether have a data for the host needs when the host of

private network brings the http request. Provides the host if it is existent, otherwise

forward this http request to http-gw. Then http-gw forwards http request to remote

http server then waits the response of http.


45

7.4 Intrusion Detection System

7.4.1 Introduction

Due to vast development of Information Technology (IT) and Internet, the

number of Internet-involving user is getting larger, component of Internet is getting

complicated, data transition between PC and Internet is becoming more and more

important, and services provided on Internet are getting critical. However,

development of Network Security somehow doesn’t keep up with above items, and

it’s often ignored because of efficiency and convenience.

Network Security consists of three components including, information protection,

resource protection, and privacy protection. Information protection tends to prevent

unauthorized user from obtaining or changing any sort of information. Resource

protection tends to keep the resources away from unauthorized user, and resource here

may be Internet online service or bandwidth. Privacy protection tends to prevent

unauthorized user from reading personal data or personal behaviors, such as

consuming or tracks of surfing Internet.

As far as Internet and Network Security are concerned, we focus on ways of

enterprise protecting themselves in messy situations. The following section will

introduce ways of attack and defense methods. This Chapter will also illustrate and

classify typical attacking models, including monitoring, password cracking, exploit,

scanning, malicious code, denial of service social engineering and any other defensive

ways, where encryption, authentication, access control, auditing, monitoring and

scanning are concluded as shown in Fig. 7-29 double framed blocks. And then have

the conclusion of still un-solved problems, including unknown exploit, denial of

service and social engineering.


46

IDS

Attack

Protect

GatherInformation

Intrusion Crack Target

Monitoring ScanningSocial

Engineering Direct Indirect

Passwordcracking

Securityholes

Maliciouscode, Virus

Malicious code,Backdoor

DoS

Prevent Control Detect Record

Encryption AuthenticationAccessControl

Monitoring Scanning Auditing

Figure 7-29 Type Tree of Intrusion and Protection

7.4.2 Intrusion

To ensure Network Security in inner enterprise, first we need to understand

nowadays attacking methods on Internet, so that proper adjustment can be made while

under attack, or even prevent it from happening. In this section, we will illustrate

attacking methods enterprise might encounter and classify those methods. Seven

methods of attacking have been selected, monitoring, password cracking, exploits,

scanning, malicious code, denial of service, and social engineering.

Generally speaking, attacking methods may come in three types, gather

information, intrude and destroy. Gathering information means obtaining critical or

private information, including monitoring, scanning and social engineering. Directly

intruding means easy access to and then enters the whole system, such as password

cracking. On the other hand, indirectly intruding means to get authorized by using

other methods, such as malicious code and backdoor programs. Destroying means to

cause damage or deny of an online service, such as virus or denial of service.

Also, intruding can be regarded as following three steps, gathering information,

intruding and after-intruding process. As shown in Fig. 7-30, gathering information

suggests getting all related information of object as much as possible, such as host IP

address/port, service it provides, user ID inside or even user password or


47

administrator’s password. Then intruding, enter the host directly with user’s password

or by any exploits. After succeeding in entry, it follows afterwards operation,

including clear any existing record of breaking in to prevent breaking-in evidence and

perform the backdoor program for next time entry.

GatherInformation

GatherInformation

Crack targetCrack target

IntrudeIntrude

Get InformationGet InformationEmbed backdoorfor next coming

Embed backdoorfor next coming

Clear logClear log

Figure 7-30 Intrusion Procedure

7.4.2.1 Monitoring

It means gathering information by monitoring computer system or packets.

Monitoring not actually involves truly destructive attacking, but often is done for

preparation. Hacker will obtain rights, password or even user password by monitoring.

Two types of monitoring will be described including, sniffing and snooping.

Sniffing

Sniffing suggests intercepting packets to access the information via local area

networks. Normally, host only accepts packets, which is destined to the host, but

through the changeable Network Adapter modes, the host will be able to accept all

the packets through it, such as Ethernet ”promiscuous mode”.

Sniffer is named for this sort of attack. Sniffer is one of the programs, and it is a

program that works under UNIX; the latest version is 0.3.7.beta. It can monitor


48

packets by different locations, ports, destination addresses, and be able to choose

whether it records the result of just simply directs the result to other terminals.

Meanwhile, CERT has received a new feedback from latest attacking program,

named “Distributed Network Sniffer”, and it contains server and client ends. Attacker

invades the host on Internet and installs “client” program. Then use “client” to

monitor all the packets, analyzing user ID and password, lastly sends those data to

“server”. Figure 7-31 describes the situation. Recently “client” program is under

Linux OS and it submits user ID and password through port 21845/udp. It’s extremely

powerful, as only as one of the hosts is intruded and the host installs “client”. Host of

all areas can be entirely intruded.

ServerServer

LAN LAN

LAN LAN

ClientClient

ClientClient

Figure 7-31 Distributed Network Sniffer

Snooper

System monitoring means monitoring memory, disks, or other stored data in

order to gain information inside the host. For instance, monitoring system’s memory

to observe or record which buttons user has used. Attacker may use this method to get

users’ or other hosts’ communicating behavior or data to intrude other hosts later.

Snooper usually uses a pack of backdoor programs. We will describe backdoor

program in malicious code as well as functions of system monitoring.


49

7.4.2.2 Password Cracking Password cracking means crack the password by performing programs or other

methods. It has two ways to achieve its goal, by guessing or using brute force to

figure out every possible password. By guessing, it might require a dictionary file.

This password could be UNIX user’s password, or a decoding password. This kind of

attack will focus on UNIX password to ensure user ID. If password of root is cracked,

attacker will take control of host, and, UNIX password often provides remote-access

function. Therefore attacker might take control host form anywhere.

Programs of this kind vary. It requires a system file where user’s ID and encoded

password are stored to crack the password. Just like password in UNIX and SAM of

Windows NT. Cracking program means with the use of system file, attacker tries to

guess about the password. If accessing to host for password without using the system

file, it is quiet possible the host will record attacker’s position, and normally system

only allows a certain amount of errors. Time of cracking depends on speed of system

and complication of password. It will take less time if the system is very fast and the

password is easy to guess.

L0phtCrack is a program of this type. It can crack the password under Windows

NT; it’s a program performing under Windows system, with its latest version of 2.5.2.

Not only it will crack passwords in Windows NT by using SAM to access to encoded

user password, but it will have the access to user password by other two ways;

registry and interception SAM packets in network. Registry system stores encoded

user password, while L0phtCrack allows access to encode password of user from

registry. If the user doesn’t register from PDC to NT domain, L0phtCrack will send

out SAM packets for identification in PDC. L0phtCrack could intercept SAM packets

passing through L0phtCrack host, and distilled the encoded password form SAM

packets.

7.4.2.3 Exploits

Exploits are designed, practiced or operated errors in programs or software.

Attacker may use them to obtain information, system administrator authority or crack

the system. Numerous programs or software exists on the world, and each may cause

errors; even no error occurs in designing or practicing. User’s operation error is still

possible. Therefore, number of exploits may be extremely huge.


50

Buffer overflow is the most common error and it is the reason why that

frequently happened. The cause of buffer overflow is to put data to buffer, in which

the size of data is larger than the capacity of buffer. If user puts 101 bytes data to a

claim-100-byte array, it will result in extra data overwrite other variables. Normally it

ends due to program error. Within appropriate put data, user may use the exploit of

buffer overflow to perform his own program. As Fig.7-32 shown is an example of the

exploit of buffer overflow.

stack pointer

return address

buffer (200 bytes)

stack pointer

cracked file address

buffer (200 bytes)

. . . . . .

. . .

. . .. . .

. . .

Put more data to bufferthen cause buffer overflowand point to the crackedfile address

void called(){ . . . char buffer[200]; . . .

}

Figure 7-32 Distributed Network Sniffer

When the called() function is called, operating system will set up a stack for the

function. In the above example, user just needs to put in appropriate data, which

includes necessary codes, size of data needs to cover the returning address or leads to

the address the execution program located. When it finishes performing, under a

normal situation, it will return to the calling function. Due to buffer overflow and

in-put data, the program attacker put will be performed.

There are two main types of exploit including, Remote Exploits and Local

Exploits.

Remote Exploits

Hacker may intrude remote systems to get unauthorized data, user’s ID and

password or system administrator authority by remote exploits, even though hacker

does not have authorized user ID of the remote system. Since target is the remote

system, such exploits usually take place in on-line service providing program or


51

software. For example sendmail, it is the most commonly used mail server in UNIX,

and is the most famous example of remote exploit. Latest version of “sendmail” is

8.11.0, and it has been updated for many times. Former version has some kinds of

exploits and most of that are buffer overflows, in which hacker performs his program

with the right of system administrator.

As latest remote exploits, those exploits caused by Redhat 6.2 Linux operating

errors and wu-ftpd buffer exploits will come to mind. In Redhat 6.2, a pack named

Piranha, it mainly works on web clustering, and at the same time, it includes

web-based GUI to manage the web clustering. The software will come out with a

default user ID piranha with password q while after installing. If the system operator

installed such a system without changing the default account, hacker may apply this

user ID to any program. So far, users of Redhat have chosen the option of “full

install” without knowing changing default password, which result in the remote

exploits. The software of wu-ftpd is also the most commonly used FTP server under

UNIX systems, in which have discovered an exploit of likely buffer overflow. It

occurs in the function of *printf() in the command of site exec. Hacker may use

formatted string to overwrite the return address to get the effect of likely buffer

overflow. Table 7.1 shows several remote exploits can access to the operator’s

password. (Reference: Security Focus )。

Table 7-1 可 Some remote exploits to obtain the administrator’s rights

Exploits Application Version Reason phf Remote Command Execution Vulnerability

Apache Group Apache 1.0.3 Input Validation Error

Multiple Vendor BIND (NXT Oveflow) Vulnerabilities

ISC BIND 8.2.1 Buffer Overflow

MS IIS FrontPage 98 Extensions Buffer Overflow Vulnerability

Microsoft IIS 4.0 Buffer Overflow

Univ. Of Washington imapd Buffer Overflow Vulnerability

University of Washington imapd

12.264 Buffer Overflow

ProFTPD Remote Buffer Overflow Professional FTP proftpd 1.2pre5 Buffer Overflow Berkeley Sendmail Daemon Mode Vulnerability

Eric Allman Sendmail 8.8.2 Input Validation Error

RedHat Piranha Virtual Server Package Default Account and Password Vulnerability

RedHat Linux 6.2 Configuration Error

Wu-Ftpd Remote Format String Stack Overwrite Vulnerability

Washington University wu-ftpd 2.6 Input Validation Error


52

Moreover, another example of remote exploit is the protocol-based attack. TCP/IP

is the primary protocols for Internet, so hosts in Internet need to use the TCP/IP

protocol to communicate with other hosts. The protocol-based attack tries to attack

remote host by TCP/IP errors, poor design of TCP/IP or unclear definition of TCP/IP.

Such as IP spoofing might be used to attack Address-based authentication system, in

which hacker intrudes system by spoofing the destination IP address as acceptable

address by the system. Most of such destructive attacks will describe in the denial

of service in the following section.

Local Exploits

In the attack of local exploits, hacker acquires unauthorized data or higher

priority authority such as administrator’s password, while attacker already has user ID

on this system. This kind of exploits usually occurs on the design of privileged

program or implementing errors.

Xterm is a Terminal Emulator in the X Window system. In early version, local

exploit had been found some local exploits of buffer overflow. If the system replaces

Xterm with SUID root, attacker might get the administrator authority with the

exploits.

7.4.2.4 Scanning The attack of scanning is to scan a target system to gather some information of it.

In fact, scanning is just like monitoring. Both of them do not attack and intrude the

target system, but prepare for the afterward attacks. Attacker gains wanted

information by scanning, such as service-providing programs, opened ports or even

finding exploits by comparing scanning information with existing exploits. Two types

of scanning include Remote Scanning and Local Scanning.

Remote Scanning

Remote scanning is to scan a remote target system to gather some information

including, host name, open-service, service-providing program, and possible remote

exploits. Its representative is Security Administrator’s Tool for Analyzing Networks

(SATAN), which is running under UNIX system. The latest version is 1.1.1 in 1995

and lasts until now. Latter remote scanning program is SAINT, which is the updated


53

and strengthened version of SATAN; latest version at this moment is 2.1.2, another

remote scanning program under UNIX, using client/server framework and adopting

the www as the client’s interface.

Local Scanning

Local scanning is to scan a local target system to gather some information

including, significant system files with questioned authorization, questions privileged

program and possible exploits within host itself. Its representative is COPS, a

program running under UNIX; however, it never really has is latest version. TIGER is

another program of local scanning and works under UNIX. The latest version is

2.2.4p1, still under constructing.

7.4.2.5 Malicious Code The attack of malicious code is that the hacker attacks a target system via some

external device or networks. External device might be floppy, CD-ROM, hot plug-in

hard disc or other possible media. This kind of attack usually happens after invading

successfully. Two types of malicious code include virus and backdoor programs.

Virus

Virus characteristics are self-replicating and destructive. This attack means

putting the virus into a target system, then attacks the target and infects other systems.

Internet Worm is the most famous virus, which is developed by Robert T. Morris.

The attack of Internet Worm is quiet easy, i.e., replicating itself to achieve the goals

of infecting and destructive. It all begins with a host. First, it checks the local target

system for whether it has outside connections. If yes, virus replicates itself and sends

it to the outside host. Second, the virus replicates itself to increase the number of virus.

Once the amount of Internet Wrom is too large to control, the system will stop

working. Recently, Code Red and Nimda are also taking much of public attention. It

is a new virus, which adopts so-called attack of Distributed Denial-of-service (DDoS).

It attacks the un-infected Microsoft IIS system with infected Microsoft IIS. Since vast

infection, it causes a waste of large bandwidth in Internet and results in that the

servers cannot accept normal request. Therefore, the attack of DDoS only takes a few

days to spread all over the world and results in serious traffic jam in networks.


54

Backdoor codes

The attack of backdoor codes usually takes after invading successfully. For

convenient intruding the same target next time, hacker adopts backdoor codes for this

purpose. Early backdoor codes are easy access for hacker in next time intrusion,

which usually set up under UNIX systems. Nowadays backdoor program shows up in

Window systems and has the fully control of operating system. Take Back Orifice

2000 (BO2K) for example, it is a backdoor program under Window environment. It

could take full control of system, which has already installed BO2K, via TCP or UDP

connection. It also supports functions of file transfer, monitoring, and recording the

user operation. Furthermore, it can be added with additional plug-in program to widen

its function, such as sending an e-mail to attacker while the hacker host has connected

to Internet.

7.4.2.6 Denial of Service The attack of denial of service is not to invade nor gain information, but blocking

out of service, which is provided by normal operation server. Hence, user will not be

accessible to the provided service. Most of this attack is separated from system

exploits, especially from remote exploits and its’ another protocol-based attack.

Exhausted limited resources are the primary function of such a denial of service

attack, so the service will be impossible to carry on. Such as the TCP SYN flood

attack is to fill all the waiting queues of attacker host, and the ICMP echo reply flood

attack is to exhaust all the bandwidth of the target host. In the case of TCP SYN flood

attack, since TCP adopts three-way handshaking to setup a connection, attacker calls

out continuous SYN packets to fill in non-existing or incorrect address, the victim

target system will not receive ACK packets of requiring. That results in full waiting

queues, which cannot accept other connections again. In the case of ICMP echo reply

flood attack, hacker simultaneously produces very large amount of ICMP echo

request to the target system. Since the target system will reply the same amount reply

back to ICMP echo requesters, the very large amount of ICMP packets will block the

network bandwidth completely.

There is some new mode of distributed attack of DoS, which is extended from

DoS. As Fig. 7-33 shown is an example of DDoS attack. Hacker controls some

handler from client end and each handler control several agents. Once hacker sends


55

attack command to all agents via all handlers, a large amount of attacks will take at

the same time. Moreover, communications between hacker and handler is encrypted.

command command

attack attack

commandport 27665/TCP

request: port 27444/UDPreply: port 31335/UDP

1. UDP flood attack2. TCP SYN flood attack3. ICMP echo request flood attack4. M attack5. Targa3 attack

Attacker

MasterMaster

AgentAgentAgent

Target Target Target Target

Figure 7-33 Distributed Denial of Service (DdoS)

Trinoo is a client/server denial-of -service attacking program, which is based on

the UDP flood attack. Attacker sends out large amount of UDP packets (which is

probably spoof address to avoid tracking) to victim system, which will result in traffic

jam or even stop the service. A Trinoo program includes several masters and more

numerous daemons. Attacker firstly connects to the master and orders an attacking

command with several important parameters, such as IP addresses of targets, when to

take the attack, and other attack parameters. After receiving an order at master, master

will connect to all the daemons. Then all daemons take attack to all predefined victim

systems. Attacking steps are as follows.

1. Attacker connects to master: using port27665/TCP.

2. Master connects to daemons; using port 27444/UDP.

3. Daemons responds to master: using 31335/UDP.

4. Attack of daemon program towards victim systems: using UDP flood

attacks.


56

Other DDoS programs examples are TFN and TFN2K that are very much alike

the same hierarchy. They differ in amount of attacking types.

7.4.2.7 Social Engineering The attack of social engineering is not by system or Internet. An example of that

is the attacker sends an e-mail or calls to the user and claims he is the system operator

for testing or other reasons to ask the user replying his authorized information. Social

engineering also includes peeking for password while hacker is behind user’s back.

7.4.3 Typical Defense

After describing the attack methods, we introduce several defense methods in this

section. More defenses more secure. We concludes six models of defense, including

Data Encryption, Authentication, Access Control, Auditing, Monitoring, and

Scanning.

The six models can be extracted into four types, i.e., prevention, control,

detection, and record. Prevention means keeping away form attacker, e.g., data

encryption. Control adopts authentication and access control to take control of

unauthorized user getting unauthorized password/ID. Detection means detecting any

attacks, such as monitoring and scanning. Record means recording after-attack

messages to track attackers, such as auditing. Tablet 7-2 includes most common used

protection applications and software. Data encryption has been described in section

7.2.1, authentication in section 7.2.2, and access control in section 7.3. Therefore,

auditing, monitoring and scanning will be described as follows.

Table 7-2 Protection application and software

Types of Defense Software URL

Data Encryption PGP http://web.mit.edu/network/pgp.html

SSH http://www.ssh.org Access Control Firewall-1 http://www.checkpoint.com

Ipchains http://netfilter.filewatcher.org/ipchains TCP Wrappers ftp://ftp.porcupine.org/pub/security/index.html Portmap ftp://ftp.porcupine.org/pub/security/index.html Xinetd http://synack.net/xinetd


57

Monitoring Tripwire http://www.tripwiresecurity.com RealSecure http://www.iss.net

Scanning Pc-cillin http://www.trend.com.tw

7.4.3.1 Auditing

Auditing records security-related events that will be saved in some files or log

files. The audited events include record of logging in, number of failure login, or

some important activities. Such log files are useful to track and analyze who or which

system takes the attack while this system is under attacking. Hence, the administrator

can protect system to avoid the same attack in the future.

Present operation systems usually provide auditing functions, such as the system

file of wtmp of UNIX. The wtmp file records all login and logout states of all users.

In Microsoft Windows systems, Event Viewer performs the same function of

auditing.

7.4.3.2 Monitoring Monitoring defense monitors system or Internet if any abnormal activities take

place. Such as monitoring by some user’s continuous logging failure then detect

attacker tries to intrude the system. While detecting attack, system will respond by the

following processes:

1. Call the system operator by sending an e-mail, pager or alarm.

2. Stop system or related services to reduce possible damage.

3. Try to track attacker. System may be using attack signature to have a clue of

attacker’s type, in order to track him.

There are two types of monitoring including Network-based monitor and

Host-based monitor. Network-based monitor can be monitoring if any abnormal

Internet activities in network hosts. It intercepts packets by enable the promiscuous

mode of network interface card (NIC), then analyzes any weird influence on host and

reacts appropriately. Network-based monitor could detect part of denial of service

attacks, such as TCP SYN flood attack. Network-based monitor could monitor SYN

packets. Once finds out the source of SYN is illegal, it will send a RST packets to

under-attack host and stops it from waiting impossible feedback.

Host-based monitor could monitor any abnormal behavior, such as outside host

connection request, user logging situation, activities of system operator and file

systems. If abnormal activities are detected, host-attack monitor will respond properly.


58

RealSecure and Tripwire are defensive programs of this type. Tripwire will have

important files functioned and have the result saved in database. Tripwire will exam

significant files regularly, compare these important files to database, if those files

have been modified, results will vary. Therefore, Tripwire can be used to monitor

significant file within systems.

7.4.3.3 Scanning Scanning here differs from scanning in defense model. Scanning means by using

know patterns to scan if any malicious code in system, i.e., virus or backdoor

programs. Normally anti-virus software is of this defense type. Scanning program

detects malicious codes by well-known patterns, so user might regularly update virus

patterns to detect malicious code.

7.4.3.4 Non-solution problems Figure 7-3 lists some typical defense methods to against typical attack methods.

Encryption can prevent monitoring. Authentication can prevent attack of spoof source

address in remote exploits. Access control can prevent scanning of attacker and part

of exploit attack, and it reduces partial denial of service. Auditing may record exploits

attack, scanning of attacker, malicious code, and denial of service. Scanning is used

for detecting there exists some malicious code in systems. From Figure7-3, we

understand that some security holes, denial of service and social engineering are

un-solved problems still now.

Table 7-3 Typical attack methods against typical defense methods

Encryption Authentication Access Control Auditing Monitoring Scanning Monitoring Prevent Password cracking

Security holes Prevent Decrease Record Detect Scanning Prevent Record Detect Malicious code Record Detect Detect DoS Decrease Record Detect Social Engineering

Security holes mean un-disclosure holes, so there are not any patching programs


59

released. Since there is so many software or programs, it is impossible to prevent

attacks via security holes. Nevertheless, an experienced programmer is able to reduce

the number of security holes.

In present, defense methods of using firewall can reduce possibility of denial of

service, recording by auditing, or detecting attack by scanning program. Most of the

addresses of attacking are spoof, so it is not much useful to keep the recorded data.

For instance, a web server in Internet should accept any connection from any host in

the world. It is difficult to distinguish that a connection is from normal user or hacker.

Therefore, the attack of denial of service still cannot be resolved today. Finally, the

attack of Social Engineering is also not able to obtain solutions, since the protection

concept of everyone are different.

7.4.4 Open Source Implementation

Snort is designed for the purpose of defense. It is a small detecting tool for

Internet in order to monitor smaller TCP/IP network and to provide sufficient data for

possible intrusion. More important, it is completely free and supports for every

popular environment. Compared to tcdump, Snort has two advantages, i.e., detecting

payloads of packet and providing friendly interface of packet analysis. Snort decodes

application layer packets, which allows Snort to detect buffer overflow and some

other forms of attack. Figure7-34 shows the operation result of Snort.

Figure 7-34 Display of Snort


60

Another significant function of Snort is that supports gathering specified packets by

using Berkeley Packet Filter, e.g., only processing the traffic of TCP with several

specified rules of filtering to make Snort more efficient.

Furthermore, Snort consists of three components including, packet decoder,

logging and alerting, and detection engine. First, the component of packet decoder

decodes from Data-Link layer to Application layers. Second, the component of

Logging and alerting includes three recording and alerting modes including, recording

by the way packets are encoded, IP address-based catalogue structure, and record in a

single file in binary tcpdump. Each of them has individual advantage. Recording by

the way packets are decoded, which will be allowed to quickly analyze gathered data

and recording with tcpdump will be fast relatively.

Moreover, alerting can send to syslog or record in a text file with two modes

including, full mode and quick mode. Full mode records complete and alerting

messages of packets by protocol, while quick mode just records header information of

packet in a file. The latter performs better while having heavier capacity. Finally, the

component of detection engine follows predefined rules to alter the packets, which

conforms these rules. Rules in Snort are a set of easy and powerful language. There

are three basic commands: pass, log and alert, in Snort. Command pass deletes

qualified packet, command log records specified packets, and command alert

produces events information. The simplest rule of only includes protocols, direction,

and interested ports. Some examples are listed as below.

log tcp any any -> 10.1.1.0/24 79

Snort will record all packets which are destined to the class C IP of 10.1.1.0.

Expressions of rules might include optional columns.

alert tcp any any -> 10.1.1.0/24 80 (content: "/cgi-bin/phf"; msg: "PHF

probe!";)

Snort will detect any actions of accessing PHF service in local server. If any qualified

packets are detected, Snort will produce an altering event, and record the complete

event. In addition, Snort provides several useful options, which are shown in Table

7-4.


61

Table 7-4 Options in Snort

Option Description

Content Looking for same content in packets, which is specified

Msg Setting up default message while certain event happens

Seq Recording special TCP sequence number

Ack Looking for a specified TCP ack number

Snort is an open source software. The description of its flow chart is explained

as follows. Firstly, Snort calls the function of netmask() and protocol name to

initialize all environment variables, packet counter, and recording files, etc. It sets full

mode as default mode and opens default checksum. Snort then performs looping

comparison with specified parameters and executes corresponding function based on

comparison result. Afterwards, it checks if any configuration data in command line,

then attempts to figure the default configuration file. Configurations file will be used

if user does not specify. The network interface card will be initiated to let the

operation of tcpdump similar function working normally. Steps of reading

configuration file are as looking for its location, initiating all the plug-in modes, and

using default rules and corresponding actions. After that, initiation of log file will be

performed, such as set-up of log file (saves or overwrites), using default directory of

log file or user specified directory. At the same time, it will exit if the user has not

specified three operating modes including detective rules, packet analysis and

recording rules. Later it decides if it is necessary running in daemon modes. If no, the

system will ask user for configuration. Then it will decide whether received packets

from files or network interface. If it reads packets from files, procedure of opening

file will be performed as well as calls function to do “capture frame size”. Afterwards,

examine opened sockets are all rights or not and call on functions to get local network

and netmask. It followed by calling Berkeley Packet Filter (BPF), and pcap filter will

be set up.

After finishing above procedures, the main program begins to set up packet

processor（Ethernet, Slip, t/r,…, etc.）, which is conformed by Data-Link layer. Later

program checks user’s rule system. If no, it initiates to plug-in modes, sets up default

actions, and security UID and GID. Then it checks if user adopts command line (such

as syslog) to record alerting messages. If yes, it will set up appropriate function index,


62

such as syslog alerting function. At the same time, it checks open-alert modes as

ALERT_FAST, ALERT_FULL, ALERT_STDOUT, etc. If there is not any specified

record type of alerting, it will set up alerting functions as full mode, then setting up

the index of alerting functions to call all output plug-ins. Otherwise, it sets up

recording function index. Finally, it creates thread for each network interface card, i.e.,

calling pthread_create() function and returning 0 as normally ending of main().

7.5 Pitfalls and Misleading

1. Private key vs. public key

2. Why RSA works?

3. Security of DES and Triple DES

4. SSL vs. SET

5. High-level firewall vs. Low-level firewall

7.6 Further readings [1] Dorothy E. Denning, Peter J. Denning, "Internet Besieged", Addison Wesley, Oct

1997 [2] SecurityFocus, "SecurityFocus.com", http://www.securityfocus.com [3] Cryptographic Algorithms, "DES",

http://www.ssh.fi/tech/crypto/algorithms.html#DES [4] Cryptographic Algorithms, "IDEA",

http://www.ssh.fi/tech/crypto/algorithms.html#IDEA [5] Cryptographic Algorithms, "RSA",

http://www.ssh.fi/tech/crypto/algorithms.html#RSA [6] Cryptographic Algorithms, "Diffie-Hellman",

http://www.ssh.fi/tech/crypto/algorithms.html#Diffie-Hellman [7] MIT distribution site for PGP, "Welcome to the MIT Distribution Center for PGP

(Pretty Good Privacy)", http://web.mit.edu/network/pgp.html [8] The Secure Shell Community Site, "The Secure Shell Community Site",

http://www.ssh.org [9] R. Rivest, "The MD5 Message-Digest Algorithm", Apr 1992,

http://sunsite.auc.dk/RFC/rfc/rfc1321.html

[10] S. Kent and R. Atkinson, “Security Architecture for the Internet Protocol,” IETF RFC 2401,November 1998


63

[11] B. Gleeson, A. Lin, J. Heinanen, G. Armitage and A. Malis, “ A Framework for IP Based Virtual Private Networks,” IETF RFC 2764, February 2000

[12] M. Curtin and M.J Ranum, ”Internet Firewalls: Frequently Asked Questions,” http://www.interhack.net/pubs/fwfaq/


64

7.7 Exercises

1. What’s the primary encryption function of each iteration of DES system?

2. Figure out the breaking time of key size 32, 56, 128, and 168 bits, if single decryption time is 1 us and 10-6 us, respectively.

3. In a public key system using RSA with public key is e=5, n=35. The trudy intercepts the ciphertext C=10. What’s the plaintext M?

4. The encryption scheme used for UNIX passwords is one way; it is not possible to reverse it. Therefore, would it be accurate to say that this is, in fact, a hash code rather than an encryption of the password?

5. What are the requirements of digital signature?

6. Based on what you have learned in this chapter, is it possible in SSL for the receiver to record SSL record locks that arrive out of order? If so, explain how it can be done. If not, why not?

7. What’s the different between network and application layer firewall?

8. What’s the procedure of DDoS attack? What’s the attack procedure of “Nimda” virus in October 2001?

9. What’s the difference between network and application layer firewall?

10. What’s the differences between virtual lease line,virtual private routed network, virtual private dial network, and virtual private LAN segment?

11. How to achieve authentication and privacy simultaneously by using authentication header and encapsulation security payload in IPSec?

Documents

Modern Computer Networks: An Open Source Approach …speed.cis.nctu.edu.tw/~ydlin/course/cn/mcn_writeup/1-in-1/chapter7.pdf · Modern Computer Networks: An Open Source Approach Chapter