7/26/2019 Nist Ipsec VPN
1/126
Special Publication 800-77
uide to IPsec VPNs
Recommendations of the National Instituteof Standards and Technology
Sheila FrankelKaren KentRyan Lewkowski
Angela D. OrebaughRonald W. RitcheySteven R. Sharma
7/26/2019 Nist Ipsec VPN
2/126
7/26/2019 Nist Ipsec VPN
3/126
NIST Special Publication 800-77 Guide to IPsec VPNs
Recommendations of the NationalInstitute of Standards and Technology
Sheila FrankelKaren KentRyan LewkowskiAngela D. OrebaughRonald W. RitcheySteven R. Sharma
C O M P U T E R S E C U R I T Y
Computer Security Division
Information Technology LaboratoryNational Institute of Standards and TechnologyGaithersburg, MD 20899-8930
December 2005
U.S. Department of Commerce
Carlos M. Gutierrez, Secretary
Technology Administration
Michelle O'Neill, Acting Under Secretary of Commerce
for Technology
National Institute of Standards and Technology
William A. Jeffrey, Director
7/26/2019 Nist Ipsec VPN
4/126
GUIDE TO IPSEC VPNS
Reports on Computer Systems Technology
The Information Technology Laboratory (ITL) at the National Institute of Standards and Technology
(NIST) promotes the U.S. economy and public welfare by providing technical leadership for the nationsmeasurement and standards infrastructure. ITL develops tests, test methods, reference data, proof of
concept implementations, and technical analysis to advance the development and productive use of
information technology. ITLs responsibilities include the development of technical, physical,administrative, and management standards and guidelines for the cost-effective security and privacy ofsensitive unclassified information in Federal computer systems. This Special Publication 800-series
reports on ITLs research, guidance, and outreach efforts in computer security and its collaborative
activities with industry, government, and academic organizations.
Certain commercial entities, equipment, or materials may be identified in thisdocument in order to describe an experimental procedure or concept adequately.
Such identification is not intended to imply recommendation or endorsement by the
National Institute of Standards and Technology, nor is it intended to imply that the
entities, materials, or equipment are necessarily the best available for the purpose.
National Institute of Standards and Technology Special Publication 800-77Natl. Inst. Stand. Technol. Spec. Publ. 800-77, 126 pages (December 2005)
ii
7/26/2019 Nist Ipsec VPN
5/126
GUIDE TO IPSEC VPNS
Acknowledgements
The authors, Sheila Frankel of the National Institute of Standards and Technology (NIST), and KarenKent, Ryan Lewkowski, Angela D. Orebaugh, Ronald W. Ritchey, and Steven R. Sharma of Booz Allen
Hamilton, wish to thank their colleagues who reviewed drafts of this document, including Bill Burr, Tim
Grance, Okhee Kim, Peter Mell, and Murugiah Souppaya from NIST. The authors would also like toexpress their thanks to Darren Hartman and Mark Zimmerman of ICSA Labs; Paul Hoffman of the VPN
Consortium; and representatives from the Department of Energy, the Department of State, the
Environmental Protection Agency, and the U.S. Nuclear Regulatory Commission for their particularlyvaluable comments and suggestions.
Trademark Information
Microsoft, Windows, Windows 2000, and Windows XP are either registered trademarks or trademarks of
Microsoft Corporation in the United States and other countries.
PGP is a trademark or registered trademark of PGP Corporation in the United States and other countries.
Cisco and Cisco IOS are registered trademarks of Cisco Systems, Inc. in the United States and certain
other countries.
Lucent Technologies is a trademark or service mark of Lucent Technologies Inc.
All other names are registered trademarks or trademarks of their respective companies.
iii
7/26/2019 Nist Ipsec VPN
6/126
GUIDE TO IPSEC VPNS
Table of Contents
Executive Summary............................................................................................................ES-1
1. Introduction ................................................................................................................... 1-1
1.1 Authority................................................................................................................ 1-11.2 Purpose and Scope............................................................................................... 1-11.3 Audience............................................................................................................... 1-11.4 Document Structure .............................................................................................. 1-1
2. Network Layer Security................................................................................................. 2-1
2.1 The Need for Network Layer Security.................................................................... 2-12.2 Virtual Private Networking (VPN)........................................................................... 2-4
2.2.1 Gateway-to-Gateway Architecture.............................................................. 2-52.2.2 Host-to-Gateway Architecture .................................................................... 2-62.2.3 Host-to-Host Architecture ........................................................................... 2-72.2.4 Model Comparison..................................................................................... 2-8
2.3 Summary............................................................................................................... 2-8
3. IPsec Fundamentals...................................................................................................... 3-1
3.1 Authentication Header (AH)................................................................................... 3-13.1.1 AH Modes.................................................................................................. 3-13.1.2 Integrity Protection Process........................................................................ 3-23.1.3 AH Header................................................................................................. 3-23.1.4 How AH Works........................................................................................... 3-33.1.5 AH Version 3.............................................................................................. 3-43.1.6 AH Summary.............................................................................................. 3-5
3.2 Encapsulating Security Payload (ESP).................................................................. 3-53.2.1 ESP Modes................................................................................................ 3-5
3.2.2 Encryption Process .................................................................................... 3-63.2.3 ESP Packet Fields ..................................................................................... 3-73.2.4 How ESP Works......................................................................................... 3-83.2.5 ESP Version 3............................................................................................ 3-93.2.6 ESP Summary............................................................................................ 3-9
3.3 Internet Key Exchange (IKE) ............................................................................... 3-103.3.1 Phase One Exchange .............................................................................. 3-103.3.2 Phase Two Exchange .............................................................................. 3-153.3.3 Informational Exchange............................................................................ 3-173.3.4 Group Exchange ...................................................................................... 3-173.3.5 IKE Version 2 ........................................................................................... 3-183.3.6 IKE Summary........................................................................................... 3-18
3.4 IP Payload Compression Protocol (IPComp)....................................................... 3-193.5 Putting It All Together.......................................................................................... 3-20
3.5.1 ESP in a Gateway-to-Gateway Architecture............................................. 3-203.5.2 ESP and IPComp in a Host-to-Gateway Architecture............................... 3-213.5.3 ESP and AH in a Host-to-Host Architecture.............................................. 3-22
3.6 Summary............................................................................................................. 3-23
4. IPsec Planning and Implementation ............................................................................ 4-1
4.1 Identify Needs ....................................................................................................... 4-1
iv
7/26/2019 Nist Ipsec VPN
7/126
GUIDE TO IPSEC VPNS
4.2 Design the Solution ............................................................................................... 4-24.2.1 Architecture................................................................................................ 4-34.2.2 Authentication ............................................................................................ 4-84.2.3 Cryptography ....................................................................
