Monitoring a virtual network infrastructuresummary of the paper in Computer Communication Review
Nov. 2010DOI:10.1145/1880153.1880161
Augusto Ciuffoletti
24 gennaio 2012
Provisioning networking for an IaaS infrastructure
• The IaaS approach has been initially aimed mainly atstorage/computing resources
• Technology is ready to support the provision of networkresources as well
• There is demand for composite networking infrastructures(like Grids)
• However IaaS providers still offer flat LANs• We claim that the problem is in the interface between user
and provider
Provisioning networking for an IaaS infrastructure
• The IaaS approach has been initially aimed mainly atstorage/computing resources
• Technology is ready to support the provision of networkresources as well
• There is demand for composite networking infrastructures(like Grids)
• However IaaS providers still offer flat LANs• We claim that the problem is in the interface between user
and provider
Provisioning networking for an IaaS infrastructure
• The IaaS approach has been initially aimed mainly atstorage/computing resources
• Technology is ready to support the provision of networkresources as well
• There is demand for composite networking infrastructures(like Grids)
• However IaaS providers still offer flat LANs• We claim that the problem is in the interface between user
and provider
Provisioning networking for an IaaS infrastructure
• The IaaS approach has been initially aimed mainly atstorage/computing resources
• Technology is ready to support the provision of networkresources as well
• There is demand for composite networking infrastructures(like Grids)
• However IaaS providers still offer flat LANs
• We claim that the problem is in the interface between userand provider
Provisioning networking for an IaaS infrastructure
• The IaaS approach has been initially aimed mainly atstorage/computing resources
• Technology is ready to support the provision of networkresources as well
• There is demand for composite networking infrastructures(like Grids)
• However IaaS providers still offer flat LANs• We claim that the problem is in the interface between user
and provider
Why IaaS networking is difficult
• It is more difficult to describe a network than a host
• Quite evident: (topology, technology, throughput) versussingle option (small, medium, large)
• Monitoring and controlling the network is more complex• Less evident: explaining why and how to cope with is the
topic of the paper
Why IaaS networking is difficult
• It is more difficult to describe a network than a host• Quite evident: (topology, technology, throughput) versus
single option (small, medium, large)
• Monitoring and controlling the network is more complex• Less evident: explaining why and how to cope with is the
topic of the paper
Why IaaS networking is difficult
• It is more difficult to describe a network than a host• Quite evident: (topology, technology, throughput) versus
single option (small, medium, large)• Monitoring and controlling the network is more complex
• Less evident: explaining why and how to cope with is thetopic of the paper
Why IaaS networking is difficult
• It is more difficult to describe a network than a host• Quite evident: (topology, technology, throughput) versus
single option (small, medium, large)• Monitoring and controlling the network is more complex• Less evident: explaining why and how to cope with is the
topic of the paper
How to implement a network in the cloud
• The key technology is the virtual bridge (IEEE802.1Q)
• This technology is mature and available OTS• Implements an ethernet LAN over an arbitrary (bridged)
topology• Frames (layer 2) labeled with a virtual network identifier (VNI)• Bridges route labeled frames across ports based on a VNI• The same port can be associated with multiple VNI
How to implement a network in the cloud
• The key technology is the virtual bridge (IEEE802.1Q)• This technology is mature and available OTS
• Implements an ethernet LAN over an arbitrary (bridged)topology
• Frames (layer 2) labeled with a virtual network identifier (VNI)• Bridges route labeled frames across ports based on a VNI• The same port can be associated with multiple VNI
How to implement a network in the cloud
• The key technology is the virtual bridge (IEEE802.1Q)• This technology is mature and available OTS• Implements an ethernet LAN over an arbitrary (bridged)
topology
• Frames (layer 2) labeled with a virtual network identifier (VNI)• Bridges route labeled frames across ports based on a VNI• The same port can be associated with multiple VNI
How to implement a network in the cloud
• The key technology is the virtual bridge (IEEE802.1Q)• This technology is mature and available OTS• Implements an ethernet LAN over an arbitrary (bridged)
topology• Frames (layer 2) labeled with a virtual network identifier (VNI)
• Bridges route labeled frames across ports based on a VNI• The same port can be associated with multiple VNI
How to implement a network in the cloud
• The key technology is the virtual bridge (IEEE802.1Q)• This technology is mature and available OTS• Implements an ethernet LAN over an arbitrary (bridged)
topology• Frames (layer 2) labeled with a virtual network identifier (VNI)• Bridges route labeled frames across ports based on a VNI
• The same port can be associated with multiple VNI
How to implement a network in the cloud
• The key technology is the virtual bridge (IEEE802.1Q)• This technology is mature and available OTS• Implements an ethernet LAN over an arbitrary (bridged)
topology• Frames (layer 2) labeled with a virtual network identifier (VNI)• Bridges route labeled frames across ports based on a VNI• The same port can be associated with multiple VNI
Example
• An enterprise in a 3 floor building
• Logistics motivate a per floor cabling (left)
Example
• An enterprise in a 3 floor building• Logistics motivate a per floor cabling (left)
Example
• Usual bridges force network topology to follow logistics
• Instead enterprise organization might prefer a differentarrangement...
Example
• Usual bridges force network topology to follow logistics• Instead enterprise organization might prefer a different
arrangement...
Example
• Enterprise might prefer separate networks for desktops(VLAN1), printers (VLAN2) and archive (VLAN3)
• Each having an interface on each floor
Example
• Enterprise might prefer separate networks for desktops(VLAN1), printers (VLAN2) and archive (VLAN3)
• Each having an interface on each floor
Example
• The introduction of IEEE802.1Q bridges solves theproblem (right)
• Virtual LANs span across floors• Frames do not leak outside each VLAN (security)
Example
• The introduction of IEEE802.1Q bridges solves theproblem (right)
• Virtual LANs span across floors
• Frames do not leak outside each VLAN (security)
Example
• The introduction of IEEE802.1Q bridges solves theproblem (right)
• Virtual LANs span across floors• Frames do not leak outside each VLAN (security)
Bridging virtual hosts
• Inside a IaaS cloud, a real host in a data-centre implementsseveral virtual hosts
• An internal software bridge splits communication acrossvirtual TAP interfaces
• Virtual hosts appear as sharing a LAN• Routing through the provider infrastructure is transparent to
the user• Frames are confined within user’s virtual infrastructure• Frame routing is efficiently managed at layer 2• Virtual networks can be dynamically configured using a
spanning tree protocol
Bridging virtual hosts
• Inside a IaaS cloud, a real host in a data-centre implementsseveral virtual hosts
• An internal software bridge splits communication acrossvirtual TAP interfaces
• Virtual hosts appear as sharing a LAN• Routing through the provider infrastructure is transparent to
the user• Frames are confined within user’s virtual infrastructure• Frame routing is efficiently managed at layer 2• Virtual networks can be dynamically configured using a
spanning tree protocol
Bridging virtual hosts
• Inside a IaaS cloud, a real host in a data-centre implementsseveral virtual hosts
• An internal software bridge splits communication acrossvirtual TAP interfaces
• Virtual hosts appear as sharing a LAN
• Routing through the provider infrastructure is transparent tothe user
• Frames are confined within user’s virtual infrastructure• Frame routing is efficiently managed at layer 2• Virtual networks can be dynamically configured using a
spanning tree protocol
Bridging virtual hosts
• Inside a IaaS cloud, a real host in a data-centre implementsseveral virtual hosts
• An internal software bridge splits communication acrossvirtual TAP interfaces
• Virtual hosts appear as sharing a LAN• Routing through the provider infrastructure is transparent to
the user
• Frames are confined within user’s virtual infrastructure• Frame routing is efficiently managed at layer 2• Virtual networks can be dynamically configured using a
spanning tree protocol
Bridging virtual hosts
• Inside a IaaS cloud, a real host in a data-centre implementsseveral virtual hosts
• An internal software bridge splits communication acrossvirtual TAP interfaces
• Virtual hosts appear as sharing a LAN• Routing through the provider infrastructure is transparent to
the user• Frames are confined within user’s virtual infrastructure
• Frame routing is efficiently managed at layer 2• Virtual networks can be dynamically configured using a
spanning tree protocol
Bridging virtual hosts
• Inside a IaaS cloud, a real host in a data-centre implementsseveral virtual hosts
• An internal software bridge splits communication acrossvirtual TAP interfaces
• Virtual hosts appear as sharing a LAN• Routing through the provider infrastructure is transparent to
the user• Frames are confined within user’s virtual infrastructure• Frame routing is efficiently managed at layer 2
• Virtual networks can be dynamically configured using aspanning tree protocol
Bridging virtual hosts
• Inside a IaaS cloud, a real host in a data-centre implementsseveral virtual hosts
• An internal software bridge splits communication acrossvirtual TAP interfaces
• Virtual hosts appear as sharing a LAN• Routing through the provider infrastructure is transparent to
the user• Frames are confined within user’s virtual infrastructure• Frame routing is efficiently managed at layer 2• Virtual networks can be dynamically configured using a
spanning tree protocol
Network monitoring (in the age of the cloud)
• The user needs to monitor the performance of the network
• One basic need is load balancing• Network load is not covered by ordinary load balancing
services• In a composite network (not a 2-tiers WS), the provider
cannot anticipate what part of the network is under stress at acertain time
Network monitoring (in the age of the cloud)
• The user needs to monitor the performance of the network• One basic need is load balancing
• Network load is not covered by ordinary load balancingservices
• In a composite network (not a 2-tiers WS), the providercannot anticipate what part of the network is under stress at acertain time
Network monitoring (in the age of the cloud)
• The user needs to monitor the performance of the network• One basic need is load balancing• Network load is not covered by ordinary load balancing
services
• In a composite network (not a 2-tiers WS), the providercannot anticipate what part of the network is under stress at acertain time
Network monitoring (in the age of the cloud)
• The user needs to monitor the performance of the network• One basic need is load balancing• Network load is not covered by ordinary load balancing
services• In a composite network (not a 2-tiers WS), the provider
cannot anticipate what part of the network is under stress at acertain time
Example: Grid in the Cloud
• Depending on processing phase, different links areunder stress
Example: Grid in the Cloud
• Collection of experimental data: Data acquisition toStorage
Example: Grid in the Cloud
• Data retrieval: Storage to Web Server
Example: Grid in the Cloud
• Data processing: Storage to Computing resources
User configurable monitoring
• Under such circumstances, network monitoring requestscannot be anticipated by the provider
• The network monitoring activity should be dynamicallyconfigured by the user
• Configuration requires access to real and virtual networkinterfaces and to switching devices
• The provider wants to maintain control over these devices
User configurable monitoring
• Under such circumstances, network monitoring requestscannot be anticipated by the provider
• The network monitoring activity should be dynamicallyconfigured by the user
• Configuration requires access to real and virtual networkinterfaces and to switching devices
• The provider wants to maintain control over these devices
User configurable monitoring
• Under such circumstances, network monitoring requestscannot be anticipated by the provider
• The network monitoring activity should be dynamicallyconfigured by the user
• Configuration requires access to real and virtual networkinterfaces and to switching devices
• The provider wants to maintain control over these devices
User configurable monitoring
• Under such circumstances, network monitoring requestscannot be anticipated by the provider
• The network monitoring activity should be dynamicallyconfigured by the user
• Configuration requires access to real and virtual networkinterfaces and to switching devices
• The provider wants to maintain control over these devices
The network monitoring proxy
• A proxy interacts with the user willing to implement networkmonitoring
• A network of proxies covers the complex networkinginfrastructure of the provider
• The proxy configures network probes so that they are ready torespond to user requests in the expected way
• A proxy has an authority that may be limited to a part of theprovider infrastructure
The network monitoring proxy
• A proxy interacts with the user willing to implement networkmonitoring
• A network of proxies covers the complex networkinginfrastructure of the provider
• The proxy configures network probes so that they are ready torespond to user requests in the expected way
• A proxy has an authority that may be limited to a part of theprovider infrastructure
The network monitoring proxy
• A proxy interacts with the user willing to implement networkmonitoring
• A network of proxies covers the complex networkinginfrastructure of the provider
• The proxy configures network probes so that they are ready torespond to user requests in the expected way
• A proxy has an authority that may be limited to a part of theprovider infrastructure
The network monitoring proxy
• A proxy interacts with the user willing to implement networkmonitoring
• A network of proxies covers the complex networkinginfrastructure of the provider
• The proxy configures network probes so that they are ready torespond to user requests in the expected way
• A proxy has an authority that may be limited to a part of theprovider infrastructure
The network monitoring interface
• An interface must be provided between the user applicationand the proxy
• The interface should inform the user about relevant changesin the virtual topology
• The interface has the effect of restricting the authority of auser controlled sensor to owned traffic trunks
• The interface may be implemted using tools like SOAP,possibly embedded in a Enterprise System Bus
• For performance reasons the same interface is not used totransfer streams of sensor results
The network monitoring interface
• An interface must be provided between the user applicationand the proxy
• The interface should inform the user about relevant changesin the virtual topology
• The interface has the effect of restricting the authority of auser controlled sensor to owned traffic trunks
• The interface may be implemted using tools like SOAP,possibly embedded in a Enterprise System Bus
• For performance reasons the same interface is not used totransfer streams of sensor results
The network monitoring interface
• An interface must be provided between the user applicationand the proxy
• The interface should inform the user about relevant changesin the virtual topology
• The interface has the effect of restricting the authority of auser controlled sensor to owned traffic trunks
• The interface may be implemted using tools like SOAP,possibly embedded in a Enterprise System Bus
• For performance reasons the same interface is not used totransfer streams of sensor results
The network monitoring interface
• An interface must be provided between the user applicationand the proxy
• The interface should inform the user about relevant changesin the virtual topology
• The interface has the effect of restricting the authority of auser controlled sensor to owned traffic trunks
• The interface may be implemted using tools like SOAP,possibly embedded in a Enterprise System Bus
• For performance reasons the same interface is not used totransfer streams of sensor results
The network monitoring interface
• An interface must be provided between the user applicationand the proxy
• The interface should inform the user about relevant changesin the virtual topology
• The interface has the effect of restricting the authority of auser controlled sensor to owned traffic trunks
• The interface may be implemted using tools like SOAP,possibly embedded in a Enterprise System Bus
• For performance reasons the same interface is not used totransfer streams of sensor results
Data streaming
• Data are treated as a stream flowing from the sensor to theuser
• The stream may be encoded for security purposes• The stream may travel using owned traffic trunks
Data streaming
• Data are treated as a stream flowing from the sensor to theuser
• The stream may be encoded for security purposes
• The stream may travel using owned traffic trunks
Data streaming
• Data are treated as a stream flowing from the sensor to theuser
• The stream may be encoded for security purposes• The stream may travel using owned traffic trunks
Grid legacy: Monalisa
• Distributed infrastructure management
• Glues together diverse monitoring tools• Dynamic control of sensors• Fully deployed• BUT• Monitoring controlled by Grid Admin• Data stored in SQL repositories and retrieved to users
Grid legacy: Monalisa
• Distributed infrastructure management• Glues together diverse monitoring tools
• Dynamic control of sensors• Fully deployed• BUT• Monitoring controlled by Grid Admin• Data stored in SQL repositories and retrieved to users
Grid legacy: Monalisa
• Distributed infrastructure management• Glues together diverse monitoring tools• Dynamic control of sensors
• Fully deployed• BUT• Monitoring controlled by Grid Admin• Data stored in SQL repositories and retrieved to users
Grid legacy: Monalisa
• Distributed infrastructure management• Glues together diverse monitoring tools• Dynamic control of sensors• Fully deployed
• BUT• Monitoring controlled by Grid Admin• Data stored in SQL repositories and retrieved to users
Grid legacy: Monalisa
• Distributed infrastructure management• Glues together diverse monitoring tools• Dynamic control of sensors• Fully deployed• BUT
• Monitoring controlled by Grid Admin• Data stored in SQL repositories and retrieved to users
Grid legacy: Monalisa
• Distributed infrastructure management• Glues together diverse monitoring tools• Dynamic control of sensors• Fully deployed• BUT• Monitoring controlled by Grid Admin
• Data stored in SQL repositories and retrieved to users
Grid legacy: Monalisa
• Distributed infrastructure management• Glues together diverse monitoring tools• Dynamic control of sensors• Fully deployed• BUT• Monitoring controlled by Grid Admin• Data stored in SQL repositories and retrieved to users
Grid legacy: Monalisa
• Distributed infrastructure management• Glues together diverse monitoring tools• Dynamic control of sensors• Fully deployed• BUT• Monitoring controlled by Grid Admin• Data stored in SQL repositories and retrieved to users
Rigid, hardly scalable
Grid legacy: GD2
• Distributed infrastructure management
• Glues together diverse monitoring tools• Sensors are controlled by user application• Monitoring sessions activated on demand• Data returned to the user with an UDP stream• BUT• only a proof of concept implementation
Grid legacy: GD2
• Distributed infrastructure management• Glues together diverse monitoring tools
• Sensors are controlled by user application• Monitoring sessions activated on demand• Data returned to the user with an UDP stream• BUT• only a proof of concept implementation
Grid legacy: GD2
• Distributed infrastructure management• Glues together diverse monitoring tools• Sensors are controlled by user application
• Monitoring sessions activated on demand• Data returned to the user with an UDP stream• BUT• only a proof of concept implementation
Grid legacy: GD2
• Distributed infrastructure management• Glues together diverse monitoring tools• Sensors are controlled by user application• Monitoring sessions activated on demand
• Data returned to the user with an UDP stream• BUT• only a proof of concept implementation
Grid legacy: GD2
• Distributed infrastructure management• Glues together diverse monitoring tools• Sensors are controlled by user application• Monitoring sessions activated on demand• Data returned to the user with an UDP stream
• BUT• only a proof of concept implementation
Grid legacy: GD2
• Distributed infrastructure management• Glues together diverse monitoring tools• Sensors are controlled by user application• Monitoring sessions activated on demand• Data returned to the user with an UDP stream• BUT
• only a proof of concept implementation
Grid legacy: GD2
• Distributed infrastructure management• Glues together diverse monitoring tools• Sensors are controlled by user application• Monitoring sessions activated on demand• Data returned to the user with an UDP stream• BUT• only a proof of concept implementation
Grid legacy: GD2
• Host A issues a network monitoring request (e.g.,bandwidth from X to Y)...
• that is taken in charge by the proxy in A’s domain...
Grid legacy: GD2
• Host A issues a network monitoring request (e.g.,bandwidth from X to Y)...
• that is taken in charge by the proxy in A’s domain...
Grid legacy: GD2
• that has authority to control a relevant sensor (agent ingd2 terms)...
• and data are returned to the originating host as astream.
Grid legacy: GD2
• that has authority to control a relevant sensor (agent ingd2 terms)...
• and data are returned to the originating host as astream.
Grid legacy: GD2
In principle this might be implemented in a virtualinfrastructure, using virtual components
Conclusions
• the techniques for complex networks in the cloud are availableOTS
• the hard issue is the control of the virtual network• network monitoring must be translated from virtual to real
(and back)• an effective infrastructure takes into accout multi-domain use
cases for scalability
Conclusions
• the techniques for complex networks in the cloud are availableOTS
• the hard issue is the control of the virtual network
• network monitoring must be translated from virtual to real(and back)
• an effective infrastructure takes into accout multi-domain usecases for scalability
Conclusions
• the techniques for complex networks in the cloud are availableOTS
• the hard issue is the control of the virtual network• network monitoring must be translated from virtual to real
(and back)
• an effective infrastructure takes into accout multi-domain usecases for scalability
Conclusions
• the techniques for complex networks in the cloud are availableOTS
• the hard issue is the control of the virtual network• network monitoring must be translated from virtual to real
(and back)• an effective infrastructure takes into accout multi-domain use
cases for scalability
Thank you!
Full reference:Augusto Ciuffoletti. Monitoring a virtual network infrastructure: an IaaSperspective. SIGCOMM Comput. Commun. Rev. 40, 5 47-52.DOI=10.1145/1880153.1880161http://doi.acm.org/10.1145/1880153.1880161
Download published version
Download preliminary version