© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Benjamin Feldon, Solutions Architect, AWS
Sidhartha Chauhan, Solutions Architect, AWS
November 30, 2016
Extending Data Centers to the Cloud
Connectivity Options and Considerations for Hybrid
Environments
NET305
Hybrid environments
Intro to VPN & AWS Direct Connect
Connectivity architectures
What are we connecting to?
What to expect from this session
What are the options for connecting into AWS?
What is appropriate for my workloads?
How can I start small and grow with time?
What is the meaning of life? (optional)
Key takeaways
Intro to VPN
• VPC != VPN
• IPsec authentication & encryption
• AWS options
• AWS Managed VPN
• Software VPN (EC2)
Intro to AWS Direct Connect (DX)
• Offered since 2011
• Private connection, separate from Internet
• Consistent network experience
• Connect through one of 40 locations
• Each connection connects into 1 AWS Region
• Multiple options for each AWS Region
Oregon
N. California
AWS Direct Connect (DX) in the United States
SuperNAP
Equinix SE
CoreSite LA
N. Virginia
CoreSite NY
Equinix DC
CoreSite SV
OhioEquinix CH
QTS Chicago
Equinix DA
CoreSite VA
Equinix LA
Equinix SV
TierPoint
EdgeConneX
Pittock Block
Frankfurt
AWS Direct Connect (DX) in Europe and Asia Pacific
Digital RealtyEircom Interxion Frankfurt
Sydney
Ireland
Tokyo
Singapore
Equinix OS
Beijing
Equinix TY
Equinix FR
Equinix SY
Global Switch
Equinix SG
CIDS
Sinnet
Eqinix LDInterxion
Interxion Madrid
Interxion Stockholm
Equinix AM
Global Switch
Mumbai
GPXSify Rabale
Seoul
KINX
Telehouse
VPC
VPC
VPC
AWS managed VPN
CORP
Internet
VGW
CGW
• AES-256
• SHA-2
• Phase 1 DH groups - 2, 14-18, 22, 23, and 24.
• Phase 2 DH groups - 1, 2, 5, 14-18, 22, 23, and
24.
• NAT-T
VPC
VPC
VPC
AWS managed VPN
CORP
Internet
VGW
CGW
1 VPN Connection = 2 VPN tunnels
23.22.66.xx
50.16.172.yy
VPC
VPC
VPC
AWS managed VPN, multiple VPCs
CORP
Internet
VGW
CGW
CGW
VGW
2 VPCs X 2 CGWs = 8 VPN tunnels
VPC
VPC
VPC
AWS managed VPN, multiple VPCs
CORP
Internet
VGW
CGW
CGW
VGW
2 VPCs X 2 CGWs = 8 VPN tunnels
AWS managed VPN
Cost
Performance
Flexibility
Resiliency
• Easy install, minutes to setup
• NAT-T, AES-256, SHA-2 and latest
DH groups
• Static (1 prefix) or BGP (<100
prefixes)
• Repeat for every VPC
• $0.05 per VPN connection hour
• Data transfer
• Leverage both VGW endpoints (2
tunnels per VPC)
• Think about CGW redundancy (4
tunnels per VPC)
• Multi Gbps can be achieved per
VPC (limited at VGW)
London DX
Seattle DX
Branch
Remote
workforce
Global HQ
Regional HQ
us-west-2 region
Transit VPCVPC
VPC
eu-west-1 region
Transit VPC VPC
VPC
ap-northeast-1
region
Transit VPC VPC
VPC
VPN
VPN
Transit VPC Global VPN
backbone
https://aws.amazon.com/answers/networking/transit-vpc/
Software VPN (EC2)
Cost
Performance
Flexibility
Resiliency
• Any open-source or commercial vendor
• Opens up proprietary feature sets
• Customer responsible for HA and scaling
• Advanced solutions can be built using
automation
• Vendor licensing
• EC2 hourly cost
• High availability cost
• Data transfer
• VPC endpoint HA achieved by
additional EC2 instance in 2nd AZ
• Customer-side HA also recommended
• Defined by EC2 instance size & type
• Multi Gbps can be achieved per
VPN instance (for all tunnels)
• Multiple instances for the same VPC
are possible
Direct Connect – physical connectivity
1) Customer presence in the same DX location
2) Circuit between customer data center and DX location
Direct Connect – physical connectivity
1) Customer presence in the same DX location
2) Circuit between customer data center and DX location
3) Service provider network extending to DX location
1) Customer router in colo
CORP
Internet
AWS Direct
Connect
Routers
Customer
Router
Colocation
DX Location
`
VPC
VPC
VPC
AWS Direct Connect
Letter of Authorization and Connecting Facility Assignment
Please consider this letter as notification for connecting facility assignment for the purpose of
establishing or augmenting connectivity between the parties identified above. This document authorizes
a connection to the ports indicated above. All charges for the physical connection are the sole
responsibility of company.
For location specific information on requesting a cross-connect, visit the "Requesting Cross-Connects"
section of the user guide:
http://docs.aws.amazon.com/DirectConnect/latest/UserGuide/Colocation.html
The requester(s) use of AWS services will be governed by the terms of the AWS Customer Agreement
(available at http://aws.amazon.com/agreement), or a separate agreement between the requester(s)
and AWS.
EXPIRATION NOTICE The authorized connectivity must be completed within 90 days of this LOA-CFA's
issue date or this LOA-CFA will expire.
* Amazon Corporate LLC is a subsidiary of Amazon.com, Inc.
Issue Date .
Oct 13, 2016
Issued By* .
Amazon Web Services Spain S.L.
Facility - Meet Me Room .
Interxion MAD2 – MAD2.211
Customer Demarcation/ZSide .
Rack: R77B1.R99B09 Patch Panel: PP2:SOUTH Strands: 40818
Requested By .
Company requesting name
Issued To .
Interxion, Madrid, ESP
Connection ID ..
MAD50_Test
Optic and Connector Types ..
1000BASE-LX Single Mode Fiber (SMF) Lucent Connector (LC)
Letter of Authorization
and Connecting
Facility Assignment
1) Customer router in colo
CORP
Internet
AWS Direct
Connect
Routers
Customer
Router
Colocation
DX Location
`
VPC
VPC
VPC
3) Service provider network
CORP
Internet
AWS Direct
Connect
Routers
DX Location
Service Provider
Network
VPC
VPC
VPC
3) Service provider network
CORP
Internet
AWS Direct
Connect
Routers
DX Location
Service Provider
Network
VPC
VPC
VPC
DX physical connectivity considerations
AWS account that owns the DX port?
Adding/removing virtual interfaces?
Routing ownership?
End-to-end costs?
Direct Connect – physical connectivity
1) Customer presence in the same DX location
2) Circuit between customer data center and DX location
3) Service provider network extending to DX location
Direct Connect – physical connectivity
1) Customer presence in the same DX location
2) Circuit between customer data center and DX location
3) Service provider network extending to DX location
Customer’s AWS account, interface control, routing. Cost: port + data transfer
Direct Connect – physical connectivity
1) Customer presence in the same DX location
2) Circuit between customer data center and DX location
3) Service provider network extending to DX location
Customer’s AWS account, interface control, routing. Cost: port + data transfer
Same as #1; add circuit cost. Sub 1-Gig can create only 1 virtual interface
Direct Connect – physical connectivity
1) Customer presence in the same DX location
2) Circuit between customer data center and DX location
3) Service provider network extending to DX location
Customer’s AWS account, interface control, routing. Cost: port + data transfer
Same as #1; add circuit cost. Sub 1-Gig can create only 1 virtual interface
Depends on provider’s offering
Direct Connect – physical connectivity
1) Customer presence in the same DX location
2) Circuit between customer data center and DX location
3) Service provider network extending to DX location
Customer’s AWS account, interface control, routing. Cost: port + data transfer
Same as #1; add circuit cost. Sub 1-Gig can create only 1 virtual interface
Depends on provider’s offering
Direct Connect cost considerations
Port hour + data transfer
Data in $0; data out differs by region
Factor in circuit costs
Calculate data center Internet costs (VPN)
Direct Connect
CORP
Internet
AWS Direct
Connect
Routers
Customer
Router
Colocation
DX Location
`
VPC
VPC
VPC
Direct Connect
CORP
Internet
AWS Direct
Connect
Routers
Customer
Router
Colocation
DX Location
`
VPC
VPC
VPC
Direct Connect + VPN
CORP
Internet
AWS Direct
Connect
Routers
Customer
Router
Colocation
DX Location
`
VPC
VPC
VPC
2 X DX ports
CORP
Internet
Customer
Router
Colocation
DX Location
`
AWS Direct
Connect
Routers
VPC
VPC
VPC
Internet
2 X DX ports, 2 X customer routers
CORP
Colocation
DX Location
`
AWS Direct
Connect
Routers
Customer
Routers
`
VPC
VPC
VPC
CORP
Internet
DX Location
AWS Direct
Connect
Routers
2 X DX ports, 2 X circuits into 2 data centers
VPC
VPC
VPC
2 X DX, active/active
CORP
Internet
DX Location
AWS Direct
Connect
Routers
10 Gbps active
10 Gbps active
20 Gbps
VPC
VPC
VPC
2 X DX, active/standby
CORP
Internet
DX Location
AWS Direct
Connect
Routers
10 Gbps standby
10 Gbps
10 Gbps active
VPC
VPC
VPC
2 X DX, active/active
CORP
Internet
DX Location
AWS Direct
Connect
Routers
10Gbps Active
10 Gbps active
10 Gbps
VPC
VPC
VPC
2 X DX, 2 X DX locations
CORP
Internet
Customer
Routers
Colocation
DX Location 1
`
Customer
Routers
Colocation
DX Location 2
`
AWS Direct
Connect Routers
AWS Direct
Connect Routers
VPC
VPC
VPC
2 X DX, 2 X DX locations
CORP
Internet
Customer
Routers
Colocation
DX Location 1
`
Customer
Routers
Colocation
DX Location 2
`
AWS Direct
Connect Routers
AWS Direct
Connect Routers
VPC
VPC
VPC
VPN backup
CORP
Internet
Customer
Routers
Colocation
DX Location 1
`
Customer
Routers
Colocation
DX Location 2
`
AWS Direct
Connect Routers
AWS Direct
Connect Routers
VPC
VPC
VPC
Direct Connect (DX)
Cost
Performance
Flexibility
Resiliency
• 14 AWS regions, 40 POPs worldwide
• LOA provided within up to 72 hours
• Lead time of circuit build-out could take
weeks
• Port hours
• Data out transfer
• Service provider circuit / MPLS
• Colo cage (if applicable)
2 x DX in 2 locations + VPN
2 x DX in 2 separate locations
2 x DX in 1 DX location
DX + VPN
DX
• 1 Gbps or 10 Gbps ports
• 100, 200, 300, 400 or 500 Mbps
ports available through partners
• Equal-cost multipath via BGP means
2x10 G = 20 Gbps
Adapting the architecture
• Start with 1 AWS managed VPN
• Use VPN while DX is being built out
• Port hour charges begin when DX is up or 90 days
• DX is favored over VPN when both exist
• Add additional DX ports for resiliency / bandwidth
• Plan for failure, including facility failure
• Control traffic flow both ways using BGP and routing
• Raise support cases with AWS with any questions
Multiple VPCs
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
Multiple VPCs – VPC Peering
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
VPC Peering
Multiple VPCs – VPC Peering
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
VPC Peering
Multiple VPCs – VPC Peering
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
VPC Peering
Connecting to VPC over DX
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
Private virtual interface
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
Private virtual interface
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
VLAN
400
Private virtual interface
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
VLAN
400
BGP
BGP
Private virtual interface
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
VLAN
500
VLAN
400
Private virtual interface
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
VLAN
500
VLAN
400
VLAN
600
Private virtual interface
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
BGP
BGP
BGP
BGP
BGP
Access to VPC resources
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
Access to VPC resources
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
Access to VPC resources
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
VPC Peering
Access to VPC resources
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
VPC Peering
Access to VPC resources
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
VPC Peering
Hairpinning
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
Hairpinning
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
Prod
Test
Dev
VPC
VPC
VPC
Non
-Pro
dP
rod
Access to S3 using VPC Endpoints
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
VPC
VPC
VPC
Non
-Pro
dP
rod
VPC Endpoints
Access to S3 using VPC Endpoints
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
VPC
VPC
VPC
Non
-Pro
dP
rod
VPC Endpoints
Access to S3 using VPC Endpoints
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
VPC
VPC
VPC
Non
-Pro
dP
rod
VPC Endpoints
Public Virtual Interface
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
VPC
VPC
VPC
Public Virtual Interface
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
VPC
VPC
VPC
VLAN
800
BGP
BGP
Public Virtual Interface – Filtering prefixes
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
VPC
VPC
VPC
VLAN
800
BGP
BGP
PROMPT> ec2-describe-prefix-lists
PREFIXLIST pl-12345678 com.amazonaws.us-east-1.s3
CIDR 54.123.456.7/19
Public Virtual Interface
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
VPC
VPC
VPC
VLAN
800
BGP
BGP
Public Virtual Interface + VPN
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
VPC
VPC
VPC
Public Virtual Interface – US Regions
AWS Direct
Connect
Routers
DX Location
VPC
VPC
VPC
Customer
Router
VPC
VPC
VPC
AWS Direct
Connect
Routers
DX Location
VPC
VPC
Customer
Router
VPC
VPC
VPC VPC
us-east-1
us-west-1 us-west-2
AWS Direct
Connect
Routers
DX Location
VPC
VPC
Customer
Router
VPC
VPC
VPC VPC
us-east-1
us-west-1 us-west-2
AWS Direct
Connect
Routers
DX Location
VPC
VPC
Customer
Router
VPC
VPC
VPC VPC
us-east-1
us-west-1 us-west-2
AWS Direct Connect in the United States
Equinix SV
us-west-1
us-west-2
us-east-1
AWS Private Network
VPN to VGW
What are the options for connecting into AWS?
What is appropriate for my workloads?
How can I start small and grow with time?
What is the meaning of life? (optional)
Review
What are the options for connecting into AWS?
Review
• AWS-managed VPN
• Software VPN (EC2)
• Private virtual interface
• Public virtual interface
VPN Direct Connect
How can I start small and grow with time?
Review
• Connect using VPN in parallel to DX build out
• More DX locations = more resiliency
• Plan and test for resiliency, and repeat
periodically
• Talk to your AWS team
Related Sessions
• NET402 Deep Dive - AWS Direct Connect and VPNs
• NET301 - Cloud Agility and Faster Connectivity with
AT&T NetBond and AWS
• ARC401 - From One to Many: Evolving VPC Design