Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
IP/Ethernet in the TSO Utilinet Europe 2013, Brussels, 3rd July 2013
Rui Franco, RENTELECOM, REN Group
1
Curriculum Vitae Rui Franco
Currently: RENTELECOM Director, REN Group
Rui Franco (1967) is graduated in Electrical Engineering and Computer Science by
the Technical University of Lisbon. Since May 2011 he is the Director of
RENTELECOM the telecommunications provider of REN, the Portuguese energy
transport group (electricity and gas). Previously he was the partner in charge for the launch of Nae,
an ICT consultancy, in the Portuguese market.
His previous job assignments included management positions during 7 years at Marconi (former
Portuguese international telecommunications carrier) and 10 years as head of Wholesale Business
Unit at OniCommunications the Portuguese utility based fixed line alternative telecommunications
operator. Later he was the head of T-systems Portugal (Deutsche Telekom group) in charge for
launch in the Portuguese market.
2
Agenda
1. REN Group and RENTELECOM
2. Current situation
3. Future trends and Evolution
4. Conclusions
1. REN Group and RENTELECOM
REN Group
REN - Redes Energéticas Nacionais operates in
two major business areas:
Electricity - The transmission in very high
voltage electricity and overall technical
management of the National Electricity System;
Gas - The transport of high-pressure natural
gas and overall technical management of the
National Natural Gas System, guaranteeing the
reception, storage and regasification of LNG and
underground storage of natural gas.
4
1. REN Group and RENTELECOM
RENTELECOM started in 2002, with providing internal services and with leveraging the existing
surplus of network capacity in the electricty and gas backbones, but is nowadays a true alternative
in the wholesale and corporate market, with the following business areas:
RENTELECOM, leveraging in the telecommunications market
Dark Fiber
• National
• International
Colocation
• Towers
• Technical areas
Housing
• Racks
• Rooms
Leased Lines
• SDH / DWDM
• Ethernet / IP-MPLS
ICT Services
• Special Projects
• Voice VPNs,…
5
Agenda
1. REN Group and RENTELECOM
2. Current situation
3. Future trends and Evolution
4. Conclusions
2. Current situation
Technical Evolution
Telecom technologies at REN, a short history:
Before 1980 – only Power line communication was used to transmit information 1980’s – Setup of analog and digital Radio links 1990 – Setup of Optical Fibers (Radio links still used as main and redundant routes) 1992 – PDH on Optical fiber and radio links 2003 – SDH setup on Optical fiber 2006 – DWDM setup on Optical Fibers 2006 – IP/MPLS & Ethernet setup
7
2. Current situation
Drivers for reaching current situation
Along the period of 2000-2010, REN made intensive
investments in its network and more than doubled its
electrical grid (gas grid grew slower):
Demand side – clients:
Dispatch and Automation services (SCADA, Command and Control, …), required scalable, flexible and reliable telecoms network to cope with grid growth. Telecoms are mission critical and outsourcing was not an option. RENTELECOM – A business opportunity to leverage on technical know how and physical assets.
Offer side – telecoms network: The intensive building of new lines allowed installation of OPGW cables at incremental cost. Nowadays REN manages a backbone network that is +90% covered with our own OF. OPGW cables are extremely reliable. Technical know how is an asset.
A perfect match!
8
2. Current situation
Services carried by the telecoms network
Security network services:
RTU/SCADA ICCP (Inter Control Center Protocol) – connection to Spanish TSO REE and to local DSO EDP. Dispatch centers interconnection – REN manages 2 dispatch centers (primary / secondary) Telemetering Power quality monitoring Remote management and supervision: Protection, control and automation, low voltage power supply. Video surveillance and physical intrusion Voice in industrial facilities NTP – Network time protocol - time tags Wide Area Monitoring / Phasor Measurement Units Line differential protection Teleprotection
Other services:
Datacenters interconnection and data storage replication Corporate & LAN interconnections Telecommunications networks (RENTELECOM)
Almost all services use the IP/MPLS network except the Line differential
protection and the Teleprotection services
REN already has a very “smart” and strongly automated grid with
IP/MPLS as a significant enabler
9
2. Current situation
Current telecommunications network architecture
DWDM SDH
IP / MPLS
SDH
SCADA Teleprotection Line Differential Protection TDM voice Command and Control Long distance high bitrate L 2
services Alarms Corporate network RENTELECOM services ...
SCADA Command and control VOIP TDM voice Telemetering Corporate network RENTELECOM services ...
SCADA interconnection
Corporate network
SCADA TDM Voice
PDH
PDH
PLC
Analog Voice Protection services
REN Gas
REN Group
REN Electricity
REN must manage IP/MPLS and legacy technologies back to PDH and even PLC
with significant O&M costs
Electric Grid protocols:
RTU/SCADA: IEC 60870-5-104 widely used with 101 residual and to be abandoned
Differential protection: G.703 64kbps (to be abandoned) E1 2Mbps and IEEE C37.94
Remote Mngmt & Spv: FTP, SSH, Telnet & others (proprietary)
10
2. Current situation
Current telecommunications network dimension
Optical Fiber cables (mostly OPGW): 7.000km 1.100km radio links supported by 20 telecom towers still used for redundancy 2 datacenters managed by REN (Lisbon and Oporto) – also used by IT 285 Points of presence (at electric substations, gas stations and DCs):
PDH: 360 nodes / 1 supplier SDH: 390 nodes / 2 suppliers DWDM: 22 nodes IP/MPLS: 750 routing & switching nodes / 2 suppliers (not including Station Bus which is not managed by the telecommunications team) Protection Signalling Equipment: 450 nodes / 2 suppliers TDM Voice: 124 PABXs / 4 suppliers VOIP: 3 communication servers and multiple gateways / 2 suppliers
11
2. Current situation
Current IP/MPLS network architecture (electric grid only)
The IP/MPLS network was designed to ensure full redundancy and optimized topology according to the traffic pattern (Control centers <=> Substations):
Substation
LAN
Substation
LANSubstation
LAN
Substation
LAN
Substation
LAN
IP/MPLS Core
IP/MPLS Edge
Ethernet Access
REN
Ele
ctr
ic
REN
Gro
up
Consists of 7 core nodes (5+2 main ones) all connected at 1Gbps (oversized trunks, hub & spoke)
Each substation is connected to 2 of the 7 core nodes
At each substation there are aggregators and access nodes (Ethernet) for each service (100Mbps)
The current architecture ensures: Minimum delay (best paths & minimum hops) Minimum jitter (oversized trunks)
The current architecture does not ensure: Guaranteed symmetric paths (hard to implement and not required by current client services)
12
2. Current situation
IP/MPLS adoption by security network services (1/2)
Migration was smooth and nowadays, most of the security network services use IP/MPLS
with the exception of protection services (Teleprotection and Line Differential
Protection)
SCADA services were early adopters of IP due to the fact that TDM based interfaces could
no longer accommodate grid growth (new substations)
So adoption was fast but massive usage was slower due to IEDs* lifecycle - Other services
adoption of IP followed along 4 / 5 years and is still ongoing (e.g., remote access and
supervision 55%, video surveillance 78%, …)
* - Intelligent Electronic Devices 13
2. Current situation
IP/MPLS adoption by security network services
Protection equipment is not IP native - so far, protection Services are based on equipment
which requirements are specified specifically for the usage of TDM networks
Protection equipment has requirements concerning latency, jitter, symmetry that are hard
to meet by IP/MPLS services
Protection services are supported by TDM / SDH – but even the usage of SDH requires a
fine tuning of the telecoms network (e.g., Line Differential Protection with redundant
paths but no self healing)
14
2. Current situation
Current telecoms network management (electricity & gas)
REN has chosen to manage such diversified set of technologies and assets with its own
dedicated telecoms team
Telecoms team has a total of 20 professionals with Engineering (25%) and O&M (75%) skills
Most of the field services are increasingly outasked
The team manages the following O&M support facilities (24x7):
A first line Helpdesk for first interaction with internal and RENTELECOM customers
A Supervision Center with monitoring and remote control
capabilities for most of the telecoms network
Telecommunication costs (capex & opex) are recognized by the
energy regulator as part of system management costs
REN Supervision Center
15
2. Current situation
Achievments (1/2)
Internal clients are enthusiastic about IP adoption
IP adoption allowed overall improvements in business processes:
REN has a Remote Access Center dedicated to SAS* and auxiliary power
supply systems, which allows the remote monitoring and control of most
recent technologies and equipment:
Activities like commissioning can be largely followed remotely
Real time monitoring of key SAS* equipment already in place
IP usage allowed new applications and a more efficient fulfillment of REN
obligations:
Support to onsite maintenance teams
Support to power system fault analysis (better data retrieval)
Implementing new settings on IEDs and corresponding backups
All electric substations and gas stations are not manned * - Substation Automation System
REN Remote Access Center
REN Dispatch at Sacavém
16
2. Current situation
Achievments (2/2)
In 2012 REN Electric achieved, for the first time an
Average Interruption Time of zero seconds (16,2
seconds in 2011)
Telecommunications know how associated to utilities is
already leveraged for commercial telecoms
(RENTELECOM) and will be further leveraged in the
scope of the group activities diversification (coupled to
dispatch and automation & control services):
An example are the services currently provided by
RENTELECOM to EDP in accessing their substations
using our IP/MPLS core network
17
2. Current situation
Issues (1/2)
Operating legacy technologies along with IP/MPLS is too costly in terms of O&M Legacy telecommunication technologies are at commercial end of life so its replacement must be faced as well as impact to telecom network customers Commercial off the shelf telecom technologies have shorter life cycles than utility equipment (e.g. SCADA/RTU IEDs):
Particularly true for younger technologies such as IP/MPLS Depreciation allowed by regulator is longer than commercial life cycle Utility security network equipment (IEDs) is still a “step behind” in comparison to commercial telecom technologies – both will have to converge
18
2. Current situation
Issues (2/2)
Operating IP/MPLS is much more complex and demanding than operating TDM technologies. This requires workforce adaptation and criterious outasking both for telecommunications and Automation Teams Migration to IPv6 – not an issue since in a TSO like REN the number of sites and nodes fit in the IPv4 addressing space (few 1000’s) Security issues – has been a concern and has to be more and more assessed Due to the foreseen future need of real time inter-substation services, the current IP architecture will have to be assessed.
19
Agenda
1. REN Group and RENTELECOM
2. Current situation
3. Future trends and Evolution
4. Conclusions
3. Future trends and Evolution
Smart grid for a TSO (1/4)
Smart grid – a very broad definition… A smart grid is a digitally enabled electrical grid that gathers, distributes, and acts
on information about the behavior of all participants (suppliers and consumers) in
order to improve the efficiency, importance, reliability, economics, and
sustainability of electricity services (source: wikipedia)
TSOs are required to ensure the highest uptime in their backbones.
TSOs grids are already very “smart” (automated) but there are trends that will impose an even
“smarter” grid:
Renewable Energy Sources (RES) and Distributed Energy Resources (DER)
New businesses and Ancillary services
European SuperGrid
21
In the near future TSOs will have to ensure a wider monitoring and control scope as
energy sources are more dispersed and intermittent, energy flows in the grid change,
new businesses come into place and interconections will have a more relevant role
3. Future trends and Evolution
Smart grid for a TSO (2/4)
Renewable Energy Sources (RES) and Distributed Energy Resources (DER)
Climate Change concerns are imposing reduction of GHG (Green House Gas) emissions implying decabornisation and more RES which means:
Increasing intermittent sources (wind, sun) and distributed micro generation changing energy flows in the network (consumer becomes “prosumer”)
Significant increase in RES will create issues with power quality, voltage stability, harmonics, protection and control schemes and may affect grid functions such as frequency control. Currently RES are managed by TSOs in a limited manner (VHV and HV). As they increase and move into MV and LV, they will have to be manageable by TSOs – this implies more monitoring and control capabilities extending into DSOs networks and more information to analyze – a smarter grid!
European Union 2020 growth
strategy concerning climate / energy,
includes:
Reduce GHG emissions by at least
20% Energy from renewables at least
20% Energy efficiency to increase 20%
22
3. Future trends and Evolution
Smart grid for a TSO (3/4)
New businesses and Ancillary services
As RES and DER become significant, new services will have to become available for TSOs:
New businesses – it is foreseen that DER Aggregators will be a key player to be used by TSOs to manage the grid New businesses – it is foreseen that Demand Aggregators will aggregate local demand from consumers providing this service to TSOs as a one entity Balancing market – Already used by TSOs at national level and limited to classical production - will have to be expanded at European level and include also RES and DER Storage – besides hydro, it is still a subject of intensive R&D but may become a new service in the medium term
REN Dispatch at Vermoim
23
In the scope of the SmartGrid, new interfaces between the DSOs, TSOs,
Aggregators, Consumers and Generators will have to be developed
3. Future trends and Evolution
Smart grid for a TSO (4/4)
European SuperGrid
Geographic specialization in different production technologies calls for stronger and smarter Transmission Infrastructures
EU Energy roadmap 2050 foresees a European Super Smart Grid, integrating all European and neighbor grids (Desertec, Medgrid and Mediterranean Solar Plan initiatives) The European SuperGrid will be an overlay HVDC grid, centrally managed across different borders and TSOs, thus requiring “Smarter grid” technologies (Med-TSO)
Super Grid connecting Mediterranean ring
24
3. Future trends and Evolution
Drivers for telecommunication network evolution
Policy framework evolution will put more pressure on TSOs and push SmartGrid:
To manage more complex grids with more interconnections, more players and diverse flows of
energy. This means more points to control and more information to collect
To ensure quality and be more efficient – the regulatory framework will likely continue evolving
favoring quality & efficiency:
Ensuring quality in more complex grids, requires better knowledge of the grid which implies
massive data analysis.
More efficiency can be achieved by direct cost reduction (less technologies) and better O&M
End of life of legacy technologies (e.g.,SDH) will force evolution and full migration to IP/MPLS
with no SDH underlying is most likely
Technical standards are aligned with above trends and already provide support for both
suppliers of SAS equipment and telecoms equipment (e.g., IEC 61850)
Massive data collection and analysis will only be feasible using a full IP integrated network. 25
3. Future trends and Evolution
Intelligent substation – Incident analysis, an example of IP usage at REN
The project ongoing consists in the implementation of an Automatic Data Acquisition & Processing
System for data related to power system faults
Data acquired will be made available in near real time to key users such as:
System operator and fault analysis personnel
Protection and Control engineering teams
Substation apparatus maintenance teams
Line maintenance teams
Results to be obtained include relevant information for the operation and maintenance of the
bulk power system such as:
Fault location applied to power lines
Identification of fault type i.e. phase, magnitude, duration, cause, etc…
Information about the reaction of the SAS to the faults in the grid (relevant for the SAS
maintenance policy)
Automatic analysis of the performance of the protection system.
26
3. Future trends and Evolution
Near future evolution of telecoms the network architecture
SDH
IP / MPLS
SDH
Teleprotection
Differential Protection
TDM Voice
Long distance , high
bitrate L 2 services
Rentelecom
...
SCADA
Corporate network
VOIP
TDM Voice
Command & Control
Telemetering
Rentelecom
...
SCADA
Alarms
VoIP
Management &
remote monitoring
Corporate network
...
DWDM
REN Gas
REN Group
REN Electric
In the very near future all Gas security services will migrate to IP/MPLS (1 / 2 years)
In the very near future PDH will be discontinued (1 / 2 years)
SDH / DWM will be in place for a long time (4 / 6 years more)
27
3. Future trends and Evolution
Near future IP/MPLS network architecture (electric and gas)
IP/MPLS Core
IP/MPLS Edge
Ethernet Access
Substation
LAN
Substation
LANSubstation
LAN
Substation
LAN
Substation
LAN
Substation
LAN
Substation
LAN
Substation
LAN
Substation
LAN
Substation
LANSubstation
LAN
Substation
LAN
Substation
LAN
REN
Gas
REN
Ele
ctr
ic
REN
Gro
up
2 more main nodes will be added to the existing 7, all connected at 1Gbps; Each gas station will be connected on a regional ring;
The IP/MPLS network will keep the same architecture for the electric grid. Gas grid will be included
in the same network but will not have the same demanding requirements
28
Agenda
1. REN Group and RENTELECOM
2. Current situation
3. Future trends and Evolution
4. Conclusions
4. Conclusions Conclusions
REN has been an early adopter of IP/MPLS
REN has successfully conducted a smooth migration of security network services to IP/MPLS (still
ongoing)
REN expects to continue migrating services to IP/MPLS in the near future (full migration of gas
services in the short term)
Policy, climate change, regulatory and business trends will impose pressure on TSOs for running
smarter grids. IP/MPLs is a fundamental enabler to correspond to such demands
REN understands that a full IP network is the long term future, but in the short to medium term,
some services like Protection services will still require TDM / SDH technology (trials ongoing)
Despite standardization efforts, convergence to IP of telecoms network and every device in the
electric substation is still required
30