Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
UserGroupTechnical Seminars
Our presentation will begin at 10:00 am Central
Resilient EtherNet/IP Networking:Device Level Ring (DLR)
April 2020
2020 Online Events
https://trcautomation.solutions/events/
May 21st
CIP Security
June 18th
ControlLogix Redundancy
April 22nd
Remote Assistance with Augmented Reality (AR)
April 29th
Electronic Circuit Protection –Features and Benefits
April 23rd
Legacy Migration Series Part 1: SLC-500 to CompactLogix
April 27th
Legacy Migration Series Part 2: ControlLogix/CompactLogix to ControlLogix 5580/CompactLogix 5380
Presenters
Brandon SinghNetwork Specialist
The Reynolds CompanyDallas/Fort Worth
Luis RamosSolution Architect
Rockwell AutomationHouston
Wayne WelkAutomation Specialist
The Reynolds CompanyNew Orleans
David NuteAutomation Specialist
The Reynolds CompanyHouston
Resilient Network Protocols for High-
Availability Systems
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 5
High-availability process control systems require networks that match them in reliability. With multiple choices available, this session will help you understand the commonly used options, with a focus on PRP (Parallel Redundancy Protocol).
Resilient network protocols for high-availability systems
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 6
Overview
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 7
High-availability networks help prevent the loss of production caused by media breaks an infrastructure failures.
Why do you care?
• Media breaks and infrastructure failures
• Can be intermittent
• Can be difficult to physically locate
• Can be difficult to repair
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 8
High-availability networks help prevent the loss of production caused by media breaks an infrastructure failures.
Why do you care?
• Media breaks and infrastructure failures
• Can be intermittent
• Can be difficult to physically locate
• Can be difficult to repair
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 9
Network resiliency topics to be covered
• Redundant media
• DLR (Device Level Ring)
• PRP (Parallel Redundancy Protocol)
• Provides both media redundancy and a degree of infrastructure redundancy
• Redundant infrastructure
• 1715 redundant adapters
• 1756 redundant adapters
• REP
• Flexlinks
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 10
Redundant media and infrastructure
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 11
Network design considerationsTopology options
Redundant star Ring Linear
Cabling requirementsEase of configurationImplementation costsBandwidthRedundancy and convergenceDisruption during network upgradeReadiness for network convergenceOverall in network TCO and performance Best OK Worst
Redundantstar
HMI
CiscoCatalyst 2955
Cell/Area Zone
Switch Stack
Controllers,Drives, and Distributed I/O
Cell/Area Zone
Ring
HMI
Cell/Area Zone
Controllers
Switch Stack
Linear
Cell/Area Zone
Distributed I/O
Drives
Controllers
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 12
Application and technical drivers
Device-level topology
• Linear
• Comfort level with traditional field bus topology
• Help minimize cost of additional switches
• Simplify network cabling
• Applicable for certain applications that physically have a linear layout
• Conveyor applications
• Material handling application
• Ring
Making the linear topology into a ring provides single fault tolerance
Network still functions if there is a (single) break
Better fault tolerance over normal star topology
A resiliency protocol is needed in order to:
Keep packets from circling the ring forever
Reconfigure to linear topology in event of a fault
Detect ring restoration and reconfigure to ring mode
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 13
Networking design considerations
Network convergence
• Network convergence (healing, recovery, etc.) time – is a measure of how long it takes to detect a fault, find an alternate path, then start forwarding network traffic across that alternate path
• MAC tables must be relearned
• Multicast on uplinks must be relearned
• During the network convergence time, some portion of the traffic is dropped by the network because interconnectivity does not exist
• If the convergence time is longer than the Logix controller connection timeout, the IACS devices on the affected portion of the network may stop operating and may affect the industrial automation application
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 14
Device Level Ring protocol
Device Level Ring topology
• Layer 2 protocol
• Meaning that it’s transparent to higher-level protocols
• Single-ring topology
• Fast fault detection and ring recovery
• Ring supervisor
• Supervises the ring, one or more
• Normally a scanner or a dedicated supervisor
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 15
Device Level Ring protocol
Device Level Ring topology
• ODVA - open standard enabling suppliers to develop compatible products
• Support for ring and linear topologies, fiber and copper implementations
• Network traffic is managed to helps deliver timely delivery of critical data (Quality of Service, IEEE-1588 Precision Time Protocol, Multicast Management)
• Ring is a single fault tolerant network
• Designed for 1-3 ms convergence for simple device networks
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 16
Device Level Ring protocol
Device Level Ring topology
• Supervisor blocks traffic on one port
• Sends Beacon frames on both ports to detect break in the ring
• Once ring is restored, supervisor hears beacon on both ports, and transitions to normal ring mode, blocking one port
Beacon Beacon
Announce AnnounceBlocking
Forwarding
I/O I/O
Active Ring Supervisor
Controller
VFDI/O
IES
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 17
Device Level Ring protocol
Device Level Ring topology
• All faults that are detectable at physical layer
• Physical layer failure detected by protocol-aware node
• Status message sent by ring node and received by ring supervisor
Link Status Link Status
Link Failure
Active Ring Supervisor
I/O I/O
Controller
VFDI/O
IES
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 18
Device Level Ring protocol
Device Level Ring topology
• After failure detection, ring supervisor unblocks blocked port
• Network configuration is now a linear topology
• Fault location is readily available via diagnostics
Link Failure
Forwarding ForwardingActive Ring Supervisor
I/O I/O
Controller
VFDI/O
IES
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 19
Device Level Ring protocol
Device Level Ring topology
• Once ring is restored, supervisor hears beacon on both ports, and transitions to normal ring mode, blocking one port
Beacon
Active Ring Supervisor Beacon
Announce AnnounceBlocking
Forwarding
I/O I/O
Controller
VFDI/O
IES
PRP (Parallel Redundancy Protocol)
Network design considerations
Redundant Star Ring Linear
Cabling RequirementsEase of ConfigurationImplementation CostsBandwidthRedundancy and ConvergenceDisruption During Network UpgradeReadiness for Network ConvergenceOverall in Network TCO and Performance Best OK Worst
RedundantStar
HMI
CiscoCatalyst 2955
Cell/Area Zone
Switch Stack
Controllers,Drives, and Distributed I/O
Cell/Area Zone
Ring
HMI
Cell/Area Zone
Controllers
Switch Stack
Linear
Cell/Area Zone
Distributed I/O
Drives
Controllers
• What is PRP?• PRP, parallel redundancy protocol, IEC standard
62439-3• Fault tolerant, fully redundant Ethernet infrastructure
at Layer 1 and 2• Same packet is sent on both LANs • Separate and different protocol than DLR or
EtherChannel• Typical applications for PRP
• Where redundant network infrastructure is desired• Process applications (eg. 24x7x365 operational
requirements)• ControlNet redundant media migration opportunities,
such as transportation tunnels
PRP (Parallel Redundancy Protocol)
LAN A LAN B
Overview
• PRP terminology• DAN, Dually Attached Node, has PRP protocol built in and attaches to both LANs• SAN, Singly Attached Node, is a node that does not have PRP built, attaches to one LAN• RedBox, facilitates PRP connectivity for standard devices• VDAN, Virtually attached Node would be the device connected through the RedBox.
PRP (Parallel Redundancy Protocol)
LAN ALAN B
SAN
DANs in I/O chassis
VDANs
DAN in controller chassis
RedBox – Stratix® 5400
Infrastructure Switches
IEC standard terminology
PRP (Parallel Redundancy Protocol)Plant network connectivity
• Network switches • The infrastructure switch passes the PRP-marked packets just like
any other packet. • Stratix® managed switches are recommended for Diagnostic support• Managed switches required due to larger frame sizes (1506 bytes)
• Network convergence times• Zero. No Switchover required• A transmitting DAN duplicates the packet and sends it out on both
LANs, the receiving DAN discards the duplicate• PRP end devices (e.g. 1756-EN2TP, 5094-AENTR)
• One IP address, one MAC ID per device• The same identity is used on Both LAN A and LAN B• Makes PRP seamless to the application
• Network monitoring• Network faults will be seamless to the application and therefore
detection via monitoring is critical• To achieve Monitoring accuracy, it is recommended that different
network addressing is used for Infrastructure devices on LAN A vs. LAN B
PRP (Parallel Redundancy Protocol)Details
• Stratix® 5410 • Support for up to 512 VDANs• Connectivity to 2 PRP channels or pairs
• Stratix® 5400• Support for DLR up to 3 rings for VDANS• Support for up to 512 VDANs• Connectivity to 1 PRP Channel or pair
PRP (Parallel Redundancy Protocol)Redbox details
VDANs in DLR ring (up to 3 rings) RedBox – Stratix®
5400
DAN in controller chassis
PRP (Parallel Redundancy Protocol)ControlLogix® redundancy support available as of V31.050
LAN ALAN B
SAN
DANs in I/O chassis
VDANs
RedBox –Stratix® 5400
PRP Application TechniqueENET-AT006https://literature.rockwellautomation.com/idc/groups/literature/documents/at/enet-at006_-en-p.pdf
PRP White PaperENET-WP041 https://literature.rockwellautomation.com/idc/groups/literature/documents/wp/enet-wp041_-en-p.pdf
PRP Design and Implementation GuideENET-TD021https://literature.rockwellautomation.com/idc/groups/literature/documents/td/enet-td021_-en-p.pdf
PRP Design and Implementation GuideENET-TD021https://literature.rockwellautomation.com/idc/groups/literature/documents/td/enet-td021_-en-p.pdf
Redundant Infrastructure
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 33
1715 redundant I/O featuresRedundant power
supply
Two slot adapter backplane
DLR ports
Redundant input modules
Redundant output
modules
Redundant termination assemblies
• 24VDC discrete input module• 24VDC discrete output module• 4 to 20 ma analog input module• 4 to 20 ma analog output module• Redundant 24VDC power supply
connections
Three slot I/O backplanes
Redundant ethernet
adapters when on a DLR ring
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 34
1715 redundant I/O benefits
• Redundant Ethernet adapters
• Features:
• Fault-tolerant I/O
• Ability to operate on device level ring topologies
• Benefit: integrates all levels of a system on a common fault-tolerant network
• Advantages:
• Automatic switch over in the event of any fault in a module pair
• Requires no additional hardware to implement and no programming required
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 35
• Overview
• Firmware target: April 2020• Provides primary and backup
adapters for 1756 I/O chassis• Key features
• Redundant adapter functionality is a firmware update to the existing 1756-EN4TR, no hardware update required
• Customer benefits
• Allows customers to extend network redundancy through to the 1756 adapter level
• Provides additional resiliency compared to a single adapter
• Existing 1756 I/O chassis can be modified for redundant adapter -slots 0 and 1 are used for redundant adapter
1756-EN2T
1756-EN4TR pair
1756-EN4TR redundant adapter firmware enhancement
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 36
• When used with ControlLogix®
Redundancy and redundant switches, it provides for no single point of failure
1756-EN2T
1756-EN4TR pair
1756-EN4TR redundant adapter firmware enhancement
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 37
• DLR architecture available at release
1756-EN2TR
1756-EN4TR pair
1756-EN4TR redundant adapter firmware enhancement
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 38
• Single controller architecture supported at release
• Star or DLR1756-EN4TR or 1756-EN2T
1756-EN4TR pair
1756-EN4TR redundant adapter firmware enhancement
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 39
Details and rules • I/O modules only, no controllers in the redundant adapter chassis• Motion in redundant adapter chassis not supported• Supported in future releases of 1756-EN4TR redundant adapter:
• CIP Safety modules• PRP• CIP Security
• Redundant adapters reside in slots 0 and 1 only• Rotary switch on the redundant 1756-EN4TR modules need to be set to #7 for redundant adapter with
linear/DLR
1756-EN4TR redundant adapter firmware enhancement
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 40
Topology optionsNetwork design considerations
Redundant Star Ring Linear
Cabling RequirementsEase of ConfigurationImplementation CostsBandwidthRedundancy and ConvergenceDisruption During Network UpgradeReadiness for Network ConvergenceOverall in Network TCO and Performance Best OK Worst
RedundantStar
HMI
CiscoCatalyst 2955
Cell/Area Zone
Switch Stack
Controllers,Drives, and Distributed I/O
Cell/Area Zone
Ring
HMI
Cell/Area Zone
Controllers
Switch Stack
Linear
Cell/Area Zone
Distributed I/O
Drives
Controllers
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. | #ROKLive | 41
PRP Application TechniqueENET-AT006https://literature.rockwellautomation.com/idc/groups/literature/documents/at/enet-at006_-en-p.pdf
Other Resillient Protocols
Link Aggregation Control Protocol (LACP) port aggregation – IEEE 802.3ad
Redundant Star Topology Built into Stratix 5410/5400/5700/8000/8300 Aggregates multiple physical links into one logical
link Provides resiliency between connected switches
if a connection is broken
StratixAccess
Switches
F - Forwarding
F F F F
DistributionSwitch
Catalyst 3850 Switch Stack
Spanning Tree Protocol(MSTP)
MSTP is an IEEE standard Ring and redundant star topology Built into Stratix 5410/5400/5700/8000/8300 Provides:
Loop-free network Redundancy in case of failure
Distribution is the root bridge Operates in a plug-and-play fashion Coordinate with IT before implementing
FB
F - Forwarding
F
DistributionSwitch
Catalyst 3850 Switch Stack
StratixAccess
Switches
B
B - Blocking
REP- Resilient Ethernet Protocol
The Resilient Ethernet Protocol (REP) is a Cisco proprietary protocol that provides an alternative to the Spanning Tree Protocol (STP)
A REP segment is a chain of ports connected to each other and configured with a segment ID. Each segment consists of standard (nonedge) segment ports and two user-configured edge ports. A router can have no more than two ports that belong to the same segment, and each segment port can have only one external neighbor
Because REP runs on a physical-link basis and not on a per-VLAN basis, only one hello message is required for all VLANs, thus reducing the load on the protocol.
Although REP and DLR are not compatible protocols they are frequently used together.
DLR Reference Architectures
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. 47
Deploying DLR within CPwE Design Guide – Phases Review
Phase II: Mixed device/switch-Level Ring (2019)
Phase III: DLR Redundant Gateway (2020)
Deploying DLR within a CPwE Design Guide publication ENET-TD015
Phase I: Switch-Level Ring (2018)
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. 48
Phase 1 – DLR Reference ArchitectureSwitch-Level Ring
Up to 24 switches Mixed Stratix 5400
and 5700 switches 100 Mbps or 1 Gbps
but not mixed Single VLAN
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. 49
Single Mixed Device/Switch-Level DLR Ring at 100 Mbps (50 ring nodes)Phase 2 – DLR Reference Architecture
Results
• Traffic disruption localized to the DLR ring were 3 msor less.
• Local CIP Safety control and messaging applications reported no I/O or produce/consume connection loss.
• Local high-speed motion application with a 2 msCUR reported no motion faults.
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. 50
Single Mixed Device/Switch-Level DLR Ring at 1 GbpsPhase 2 – DLR Reference Architecture
Results
• Traffic disruption localized to the DLR ring were 3 msor less.
• Local CIP Safety control and messaging applications reported no I/O or produce/consume connection loss.
• Local high-speed motion application with a 2 msCUR reported no motion faults.
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. 51
Multiple Mixed Device/Switch-Level DLR Ring at 100 Mbps (All Rings at same speed)Phase 2 – DLR Reference Architecture
Results
• Traffic disruption localized to the DLR ring were 3 msor less.
• Local CIP Safety control and messaging applications reported no I/O or produce/consume connection loss.
• Local high-speed motion application with a 2 msCUR reported no motion faults.
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. 52
Multiple Mixed Device/Switch-Level Ring at Mixed Ring Speeds (Ring 1 at 1 Gbps and Ring 2 and 3 at 100 Mbps)Phase 2 – DLR Reference Architecture
Results
• Traffic disruption localized to the DLR ring were 3 msor less.
• Local CIP Safety control and messaging applications reported no I/O or produce/consume connection loss.
• Local high-speed motion application with a 2 msCUR reported no motion faults.
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. 53
Switch-Level DLR with Redundant GatewayPhase 3 – DLR Reference Architecture
Recommendation
Results• Traffic disruption localized to the DLR ring were 3 ms or less.
• Local CIP Safety, Standard CIP control and messaging applications reported no I/O or produce/consume connection loss.
• Local high-speed motion application with a 2 ms CUR reported no motion faults.
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. 54
Single Mixed Device/Switch-Level DLR (100 Mbps) with Redundant GatewayPhase 3 – DLR Reference Architecture
Recommendation
Results• Traffic disruption localized to the DLR ring were 3 ms or less.
• Local CIP Safety, Standard CIP control and messaging applications reported no I/O or produce/consume connection loss.
• Local high-speed motion application with a 2 ms CUR reported no motion faults.
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. 55
Multiple Mixed Device/Switch-Level DLR with Redundant Gateway at Mixed Ring Speeds (Single VLAN)Phase 3 – DLR Reference Architecture
Recommendation
Results• Traffic disruption localized to the DLR ring were 3 ms or less.
• Local CIP Safety, Standard CIP control and messaging applications reported no I/O or produce/consume connection loss.
• Local high-speed motion application with a 2 ms CUR reported no motion faults.
• It was observed and recommended not to apply safety and high-speed motion beyond the local DLR.
PUBLIC | Copyright ©2019 Rockwell Automation, Inc. 56
Multiple Mixed Device/Switch-Level DLR with Redundant Gateway at Mixed Ring Speeds (Multiple VLAN)Phase 3 – DLR Reference Architecture
Recommendation
Results• Traffic disruption localized to the DLR ring were 3 ms or less.
• Local CIP Safety, Standard CIP control and messaging applications reported no I/O or produce/consume connection loss.
• Local high-speed motion application with a 2 ms CUR reported no motion faults.
• It was observed and recommended not to apply safety and high-speed motion beyond the local DLR.
Networking Resources
CPwE websites CPwE Solution Graphic White Papers and Design Guides Overview Documents Cisco / Rockwell Automation
Alliance Profile Prepare industrial operations for
the future with Cisco and Rockwell Automation
Top 10 Recommendations for Plant-wide EtherNet/IP Deployments
Networking ResourcesCPwE Architectures - Collection of Architected, Tested & Validated Designs
Networking ResourcesCPwE Architectures - Collection of Architected, Tested & Validated Designs
Topic Design Guide WhitepaperDesign Considerations for Securing IACS Networks N/A ENET-WP031A-EN-P
Converged Plantwide Ethernet – Baseline Document ENET-TD001E-EN-P N/A
Deploying 802.11 Wireless LAN Technology within a Converged Plantwide Ethernet Architecture ENET-TD006A-EN-P ENET-WP034A-EN-P
Deploying Identity and Mobility Services within a Converged Plantwide Ethernet Architecture ENET-TD008B-EN-P ENET-WP037C-EN-P
Securely Traversing IACS Data Across the Industrial Demilitarized Zone (IDMZ) ENET-TD009B-EN-P ENET-WP038B-EN-P
Deploying Network Address Translation within a Converged Plantwide Ethernet Architecture ENET-TD007A-EN-P ENET-WP036A-EN-P
Migrating Legacy IACS Networks to a Converged Plantwide Ethernet Architecture ENET-TD011A-EN-P ENET-WP040A-EN-P
Deploying A Resilient Converged Plantwide Ethernet Architecture ENET-TD010B-EN-P ENET-WP039D-EN-P
Deploying Industrial Firewalls within a Converged Plantwide Ethernet Architecture ENET-TD002A-EN-P ENET-WP011B-EN-P
Deploying Device Level Ring within a Converged Plantwide Ethernet Architecture ENET-TD015C-EN-P ENET-WP016D-EN-P
OEM Networking within a Converged Plantwide Ethernet Architecture ENET-TD018A-EN-P ENET-WP018A-EN-P
Cloud Connectivity to a Converged Plantwide Ethernet Architecture ENET-TD017A-EN-P ENET-WP019B-EN-P
Deploying Industrial Data Center within a Converged Plantwide Ethernet Architecture ENET-TD014A-EN-P ENET-WP013A-EN-P
Deploying Scalable Time Distribution within a Converged Plantwide Ethernet Architecture ENET-TD016A-EN-P ENET-WP017B-EN-P
Deploying Network Security within a Converged Plantwide Ethernet Architecture ENET-TD019A-EN-P ENET-WP023B-EN-P
Deploying Parallel Redundancy Protocol within a Converged Plantwide Ethernet Architecture ENET-TD021A-EN-P ENET-WP041A-EN-P
Integrated Architecture® Builder (IAB) Updates and additions to better-reflect CPwE
structure, hierarchy and best practices Improved Switch Wizard for distribution (e.g.
Stratix® 5410) and access (e.g. Stratix® 5700 switches)
Implemented VLANs in the EtherNet/IP™
network editor Parallel Redundancy Protocol (PRP) Support CIP™ traffic is measured per segment, not just
controller scanner and adapter centric
System Configuration Drawings Updates and additions to better reflect CPwE
recent enhancements
Networking ResourcesRockwell Automation® Tools
UserGroupTechnical Seminars
Thank you for attending