Demystifying IPSec VPNs

  • View

  • Download

Embed Size (px)

Text of Demystifying IPSec VPNs

  • 8/6/2019 Demystifying IPSec VPNs


    Demystifying IPSec VPNs

    1 (21)


    By Abhishek Singh, CISSP# 82044

    Demystifying -IPSec VPNs

  • 8/6/2019 Demystifying IPSec VPNs


    Demystifying IPSec VPNs

    2 (21)


    In this article I will cover the basics of IPSec and will try to provide a window intothe mystical world of the IPSec VPNs. The intended audience is anyone who wantsto have a quick go through of the IPSec VPNs. This article will suite to readers of

    range Beginners to Intermediate.

    Gen t l e I n t r o to the I PSec

    The need to converse privately, securely and cost-effectively gave rise to thetechnology of VPNs. IPSec is one such technology (a framework to be precise)allowing people to achieve the former. IPSec is a framework comprising of the

    following RFCs:

    IP Authentication Using Keyed MD5 (RFC 1828)

    The ESP DES-CBC Transform (RFC 1829) HMAC: Keyed-Hashing for Message Authentication (RFC 2104) HMAC-MD5 IP Authentication with Replay Prevention (RFC 2085) Security Architecture for the Internet Protocol (RFC 2401) The NULL Encryption Algorithm and Its Use with IPsec (RFC 2410)

    IP Security Document Roadmap (RFC 2411) IP Authentication Header (RFC 2402) The OAKLEY Key Determination Protocol (RFC 2412) The ESP CBC-Mode Cipher Algorithms (RFC 2451) The Use of HMAC-MD5-96 Within ESP and AH (RFC 2403)

    The Use of HMAC-SHA-1-96 Within ESP and AH (RFC 2404)

    The ESP DES-CBC Cipher Algorithm with Explicit IV (RFC 2405) IP Encapsulating Security Payload (ESP) (RFC 2406) The Internet IP Security Domain of Interpretation for ISAKMP (RFC 2407)

    Internet Security Association and Key Management Protocol (ISAKMP) (RFC

    2408) The Internet Key Exchange (IKE) (RFC 2409)

    IPsec is designed to provide interoperable, high quality, cryptographically-based security forIPv4 and IPv6. The set of security services offered includes access control, connectionlessintegrity, data origin authentication, protection against replays (a form of partial sequenceintegrity), confidentiality (encryption), and limited traffic flow confidentiality. These services areprovided at the IP layer, offering protection for IP and/or upper layer protocols.

    These objectives are met through the use of two traffic security protocols, the AuthenticationHeader (AH) and the Encapsulating Security Payload (ESP), and through the use ofcryptographic key management procedures and protocols. The set of IPsec protocolsemployed in any context, and the ways in which they are employed, will be determined by thesecurity and system requirements of users, applications, and/or sites/organizations.

  • 8/6/2019 Demystifying IPSec VPNs


    Demystifying IPSec VPNs

    3 (21)


    So, we can say that the major components of the IPSec are:

    Security P rotocols Encapsulation Security Protocol (ESP) and AuthenticationHeader (AH).

    Key Management Protocols ISAKMP, IKE, OAKLEY

    Encryption Algorithms

    I PSec Modes

    IPSec operates in two modes:

    Tunnel mode: Compelete packet is ecrypted and a new header (Delivery Header)

    is attached to the encrypted packet.

    Transport Mode: Encryption done only to the payload of the packet and the

    original header is retained.

    ORG IP HDR andEncrypted Packet

    ESP Trailor ESP AUTHESP HDRDelivery HDR









  • 8/6/2019 Demystifying IPSec VPNs


    Demystifying IPSec VPNs

    4 (21)


    I PSec Phases

    IPSec Operates in two phases:

    Phase I :

    It lays the ground work for the actual transfer of data in the subsequent phase II

    and leads to a mutual agreement called Security Association. It contains ISAKMP(Internet Security Association Key Management Protocol) and Keymanagementprotocols like IKE (Internet Key Exchange), OAKLEY, SKEME. Though these termsare used as synonyms but infact there is a difference between them ISAKMP is the

    protocol which defines:

    How the peer is to be authenticated. Key material Exchange.

    Doesnt define how authenticated key exchange is done which is actually thejob of IKE.

    Traditionally the IKE messages are carried in the Payload of the ISAKMP Packets.

    The Internet Security Association and Key Management Protocol (ISAKMP) defineprocedures and packet formats to establish, negotiate, modify and delete SecurityAssociations (SA). SAs contain all the information required for execution of various

    network security services, such as the IP layer services (such as headerauthentication and payload encapsulation), transport or application layer services,or self-protection of negotiation traffic. ISAKMP defines payloads for exchangingkey generation and authentication data. These formats provide a consistentframework for transferring key and authentication data which is independent of thekey generation technique, encryption algorithm and authentication mechanism.

    ISAKMP is distinct from key exchange protocols in order to cleanly separate thedetails of security association management (and key management) from the details

    of key exchange. There may be many different key exchange protocols, each withdifferent security properties. However, a common framework is required for

    agreeing to the format of SA attributes, and for negotiating, modifying, anddeleting SAs. ISAKMP serves as this common framework.

    ISAKMP is intended to support the negotiation of SAs for security protocols at alllayers of the network stack (e.g., IPSEC, TLS, TLSP, OSPF, etc.). By centralizing

    the management of the security associations, ISAKMP reduces the amount ofduplicated functionality within each security protocol. ISAKMP can also reduce

    connection setup time, by negotiating a whole stack of services at once.

  • 8/6/2019 Demystifying IPSec VPNs


    Demystifying IPSec VPNs

    5 (21)


    ISAKMP is not bound to any specific cryptographic algorithm, key generationtechnique, or security mechanism. This flexibility provides independence fromspecific security mechanisms and algorithms provides a forward migration path to

    better mechanisms and algorithms. When improved security mechanisms aredeveloped or new attacks against current encryption algorithms, authenticationmechanisms and key exchanges are iscovered, ISAKMP will allow the updating of

    the algorithms and mechanisms without having to develop a completely new KMPor patch the current one.

    Fig. ISAKMP Header

    A brief discussion of the fields:

    Initiator Cookie (8 octets) - Cookie of entity that initiated SA establishment, SAnotification, or SA deletion.

    Responder Cookie (8 octets) - Cookie of entity that is responding to an SAestablishment request, SA notification, or SA deletion.






    Exchange Type


    Message ID


  • 8/6/2019 Demystifying IPSec VPNs


    Demystifying IPSec VPNs

    6 (21)


    Next Payload (1 octet) - Indicates the type of the first payload in the message.Common ones are:


    Security Association (SA)

    Key Exchange (KE)

    Identification (ID)

    Certificate (CERT)

    Certificate Request (CR)

    Hash (HASH)

    Signature (SIG)

    Nonce (NONCE)

    Delete (D)

    Vendor ID (VID)

    Major Version (4 bits) - indicates the major version of the ISAKMP protocol inuse (its fixed and is 1).

    Exchange Type (1 octet) - indicates the type of exchange being used. This dictatesthe message and payload orderings in the ISAKMP exchanges. Some commontypes:



    Identity Protection

    Authentication Only

    Flags (1 octet) - indicatesspecific options that are set for the ISAKMP exchange.

    There are 3 flags:

    Bit 0 (Encryption Bit): if set, signifies that all the subsequent payloadsare encrypted by the algorthim chosen in SA.

    Bit 1 (Commit Bit): Used for Key exachange synchronization to signalthat the encrypted material is not received before the completion of KeyEstablishement.

    Bit 2 (Authentication Only Bit): This bit is intended for use with the

    Informational Exchange with a Notify payload and will allow thetransmission of information with integrity checking, but no encryption.

  • 8/6/2019 Demystifying IPSec VPNs


    Demystifying IPSec VPNs

    7 (21)


    So far for ISAKMP, lets move on to the beast of transfer Internet Key exchange,

    IKE; IKE is the actual work horse traditionally used to establish the ground work forthe Phase II by setting up the Security Associations (SA) thereby defining thesecurity policies keys, encryption, authentication that are to be applied to the

    data (not the user but to the control data) IKE results in establishment of a controlconnection between two peers which will lay the framework of actual data transfer.

    The following attributes are used by IKE and are negotiated as part of the ISAKMPSecurity Association. (These attributes pertain only to the ISAKMP SecurityAssociation and not to any Security Associations that ISAKMP may be negotiating

    on behalf of other services.)

    Following are the attributes which will be negotiated during the IKE

    Encryption algorithm

    Hash algorithm

    Authentication method

    Information about a group over which to do Diffie-Hellman.

    Deffie-Hellman Algorithm:

    Postulated in 1976 by two mathematicians Bailey W. Diffiefrom Berkeley and Martin E. Hellman, defined the Diffie-Hellman Agre