1

Abstract--This paper explores and fortifies the need for a robust communications infrastructure for the upcoming smart grid and computes the bandwidth requirement for a hypothetical grid infrastructure. It presents the architecture of the current distribution system and shows that even for a medium-sized grid, the latency requirements of messages on the smart grid will require optical fibers as the transmission medium.

Index Terms--Communication systems, Optical fibers, Power

distribution, Power system communication.

I. INTRODUCTION: The United States electricity grid is an extremely large and

a very complex system. This grid was designed many years ago and is a mixture of old and new technology. Many of the grid components are near the end of their normal life spans. There have been five massive blackouts over the past 40 years, three of which have occurred in the past nine years [1]. More blackouts and brownouts are occurring due to the slow response times of mechanical switches, a lack of automated analytics, and poor visibility, i.e., a lack of situational awareness on the part of grid operators [1].

There is a fast evolving need for long transmission lines because of all the energy trading that has accelerated during the last several years. The grid is now being used extensively for long distance transportation of electricity and for switching among providers so as to obtain electricity at the lowest cost. The grid was never designed for these uses, so such uses present an enormous stress on its capabilities. One result of this demand is the increased congestion that occurs when the low cost energy cannot be delivered to all the loads because of insufficient transmission capabilities. One particular area of concern is the Eastern Interconnection [2]. The Eastern Interconnection is one of two major alternating current (AC) power grids in North America. There is great deal of objection to the placement of new transmission lines in general. This objection makes it difficult to obtain the approval for placement of new transmission lines, which leads to more line congestion due fewer transmission lines than are needed [2].

A critical component of the modernization effort of the US power system is the smart grid (SG). The SG has various names such as power grid, intelligent grid, grid wise, modern grid, perfect grid, or future grid [3]. Efforts of modernization

Amit Aggarwal is Ph.D. student in The School of Electrical and computer Engineering, University of Oklahoma-Tulsa, OK 74135 USA(Email: amit.aggarwal@ou.edu) Swathi Kunta is Master’s student in the Telecommunications Engineering Program, The School of Electrical and computer Engineering, University of Oklahoma-Tulsa, OK 74135 USA (Email: swathik@ou.edu) Pramode K. Verma is Director of Telecommunication Engineering Program in The School of Electrical and computer Engineering, University of Oklahoma-Tulsa, Tulsa, OK 74135 USA (Email: pverma@ou.edu)

of grids have so far led to characterization of the functions of the SG. According to the United States Department of Energy's Modern Grid Initiative report [4], a SG must be able to heal itself, allow consumer participation and, in particular, motivate them to participate in the operations of the grid, resist physical and cyber attacks, and provide high quality power to have more stable power with lower frequency and voltage fluctuations which are provided by the SG. Power outages and power quality issues cost the U.S. businesses more than $100 billion on average each year [3]. So, assuring more stable power provided by the SG technologies will reduce downtime and prevent such high losses, accommodate generation options, and enable electricity markets to flourish and optimize the use of assets.

The SG must provide a two-way communication, along with sensing and control, among the points of generation and the end user. Therefore, it is a combination of energy-related communication, software, and hardware. The combination is only possible with the creation of a new communication systems architecture, integration, and protocols.

Because of the growing environmental concerns, the current grid must be updated and must accommodate power generated from renewable sources like wind, rain, sunlight, tides and geothermal heat. In 2009, President Barack Obama asked the United Congress to act without delay to pass legislation that included doubling energy production from alternate sources in the next three years and the building of the SG [5].

Advanced Metering Infrastructure (AMI) is a key factor in the SG. AMI is an architecture for automated, two-way communication between a smart utility meter and a utility company. Smart meter is an advanced meter that identifies power consumption in much more detail than a conventional meter and communicates the information back to the utility for monitoring and billing purposes [6]. Consumers can be informed of how much power they are using so that they could control their power consumption. By managing the peak load through consumer participation, the utility will likely provide electricity at lower rates for all.

The world's largest smart meter deployment was undertaken by Enel SPA in Italy [7]. Austin, Texas took the lead to build SGs in order to improve the energy infrastructure [7]. At times of peak demand, the electric utilities could turn off the load themselves. To put it in other words, the complete SG will reduce customer’s expenses and their carbon footprint.

II. CHARACTERISTICS OF THE PRESENT GRID: Our current grid was built 50 years ago and it is aging

under increasing stress so this problem makes the distribution system vulnerable or unsafe to the American Industry and the

A Proposed Communications Infrastructure for the Smart Grid

Amit Aggarwal, Graduate Student Member, IEEE, Swathi Kunta, Nonmember and Pramode K. Verma, Senior Member, IEEE

978-1-4244-6266-7/10/$26.00 ©2010 IEEE

2

environment. While there have been upgrades in or invention of other products, the grid has remained substantively the same.

The present grid allows one-way communication only from the generating system to the downstream points of distribution. This is clearly insufficient in a system where a point of consumption could also become a point of generation.

Features of the present Grid do not offer the capabilities that are provided by the SG. The present grid is not equipped to deal with congestion caused by electricity traffic being stimulated by the long-distance demands power trading caused by deregulation of the U.S. electricity markets. Slow response times of mechanical switches, lack of automated analysis of problems, inability to see the whole grid, are contributing to an increase in failures of the grid [8]. In many areas of the United States, the only way the utility companies know there is an outage is when a customer reports it.

The current distribution system has over ten thousand transmission substations in the United States, upward of two thousand distribution stations, more than 130 million customers, and around 5600 distributed energy facilities [6]. The North American grids involve almost 3500 utility organizations.

III. STANDARDS AND INTEROPERABILITY: The SG requires many standards. To make the SG effective

a set of standards must be in place. The Institute of Electrical and Electronics Engineers (IEEE) has recently taken the initiative to define these standards and write guidelines on how the grid should operate using the latest in power engineering, communications, and information technology. The standards group that was created is known as the IEEE P2030 group [9]. Its first meeting was held in June 2009 at Intel headquarters in Santa Clara in California. Three task forces were formed to tackle distribution systems including the integration of different energy sources, transmission sub-stations, load side requirements, and cyber security. These task forces will focus on power engineering technology, information technology, and communications technology.

The power engineering technology group will work on the functional requirements of interoperability, drawing on various existing and ongoing effort by groups such as International Society of Automation and International Electro- technical Commission. The information technology group will look at the issues of privacy, security, data integrity, interfaces, and interoperability. And the communications technology group will define the communication requirements between devices on the SG and establish boundaries for generation, transmission, and distribution in conjunction with the customer [9].

Interoperability of an SG is the ability of diverse systems to work together, use the parts, exchange information or equipment from one another, and work cooperatively to perform a task. It enables integration, effective cooperation, and two-way communication among the many interconnected elements of the electric power grid [10], [9]. On April 13th, 2009 the National Institute for Standards and Technology (NIST) names George W. Arnold as the first International coordinator for SG interoperability [10].

The Energy Independence and Security Act (EISA) assigned some responsibilities to the National Institute for Standards and Technology (NIST) to develop a framework that includes protocols and standards for information management to achieve interoperability of SG devices and systems [11]. The NIST has developed a three phase approach to identify SG standards. Phase 1 addresses the engagement of stakeholders in a participatory public process to identify applicable standards and gaps in currently available standards and priorities for new standardization activities, ending with the final publication of the Framework report after public comments have been incorporated. Phase 2 will establish a private-public partnership and form an SG Interoperability Panel to drive longer-term progress. Phase 3 will develop and implement a framework for testing and certification of how standards are implemented in SG devices, systems, and processes [3], [11].

The National Institute for Standards and Technology has created a document called SG Interoperability Standards Roadmap. This document identifies the short and long term plans for architecture development, associated standards and infrastructure development for the SG [9].

The NIST conceptual reference model identifies seven domains like bulk generation, transmission, distribution, markets, operations, service provider, and customer and major actors and applications within each. The reference model also identifies interfaces among domains and actors and applications over which information must be exchanged and for which interoperability standards are needed [11].

The following section discusses the architecture of the existing distribution system.

IV. THE DISTRIBUTION NETWORK ARCHITECTURE: This section explains the North American electric

distribution network architecture. The architecture explained here is an adaptation of the topology of the distribution network architecture explained in [12] and [13].

A. Present Distribution Architecture The electricity grid delivers power from points of

generation to consumers and this delivery network functions through two primary systems: The transmission system and the distribution system. The transmission system delivers electricity from power plants to distribution substations while the distribution substation delivers electricity from distribution substations to consumers [12].

The electricity distribution network consists of Transmission Substations (TS) (located near power generation plants), the distribution substations (DS), and the feeders near consumers. Fig. 1 shows the basic distribution network (DN) architecture. The TS supply the generated power on High Voltage Transmissions lines (>230KV) to the DS. Usually, “handoff” from electric transmission to electric distribution occurs at the DS [6]. This fleet of DSs takes power from transmission level voltages and distributes it to lower voltage distribution lines. The DS first converts it to medium voltage levels and then feeders close to users convert that voltage to lower voltage levels for use by end-users. Thus, distribution starts at the DS and ends at the customer’s meter.

3

B. Some facts and figure about current Distribution System The following figures characterize the current distribution

system [6], [12].

• Number of transmission substation, i.e., power plants in U.S. is 10,287.

• Number of Distribution Substations is 2179. • Number of Electricity customers is about 131 million. • Number of distributed energy facilities is roughly

5600. • There are more that 3,100 electric utilities [6].

o 213 stockholder-owned systems provide power to 73% of customers.

o 2,000 Public utilities run by states and local government agencies providing power to 15% of the customers.

o 930 electric cooperatives provide power to 12% customers.

The SG system will have sensors and points of communication at almost each node of the grid including meters at customer’s premises, feeders, distributions lines, DS, TS, control centers, and distribution energy resources. It is apparent from the data above that the SG will have a communication infrastructure which is far richer than that in the current grid.

V. COMMUNICATION NEEDS OF SMART GRID: Computers and data networks are becoming more and more

important part of our life. The next generation of devices is going to incorporate more and more of these technologies into them. So, the need of high quality power with lower frequency and voltage fluctuations is increasing. To provide good quality power to consumers is one of the goals of the SG technology. To achieve this, there will be a need to implement sensors at various points in the distribution network and these sensors will need a communication network to communicate with different distribution elements and the control center.

As mentioned earlier, sensing, communication, and control are the three fundamental building blocks that will convert a distribution system into a SG. Sensing will have the ability to

detect malfunctions or deviations from normal operational ranges that would warrant action. Further, since in a SG, a point of electricity consumption can also become a point of generation, the sensing process will be closely linked with the metering process. Communication will allow inputs from sensors to be conveyed to the control centers of the grid which will generate control messages for transmission to various points on the grid resulting in appropriate action. The communication infrastructure has to be robust enough to accept inputs from the user and make him or her an integral part of the process. By the same token, the user must be capable of getting the appropriate level of information from the grid on demand or as programmed.

The SG needs the flexibility of adding more and more devices and services into it and more end-user interaction like real time monitoring of energy meters. As discussed by Francisco Lobo et al. [13], an IP-based network will provide an effective solution for the communication needs of the SG. An IP-based network as the backbone makes use of new technologies independent of the service implemented by the distributed network operator. The cost of deployment and maintenance can be reduced significantly with use of IP-based technologies [13]. Although the IP network is a good choice for the future, this network must meet some stringent requirements as mentioned by C.H. Hauser et al. [14]. The two most important requirements are mentioned here:

• Latency - It is one of the most stringent requirements for the grid. If the control center misses any input then it might substitute the missing input with inputs from other sensors which can produce different actions leading to erroneous results. The latency is in the order of a few milliseconds (~ 10ms) [14].

• Large numbers of messages - As new elements are added to the network with the evolution of the grid system, the new network should be able to transport more messages simultaneously without any major effect on latency. The numbers of messages will likely increase must faster than the number of elements on the network.

It’s important that this network be decoupled from the global Internet because the Internet will not be able to meet the latency requirements of the SG, and it will potentially result in security vulnerability for the SG.

Previous studies by C.H. Hauser et al. [14] have concluded that a 10 millisecond average latency for a 400 bit message using a T1 line will result in a utilization of just 6% of the T1 line. In the following subsection, we compute the communication bandwidth requirements for an assumed distribution network.

Fig. 2 shows a model where a DS serves 100,000 customers. In this model, a DS is connected to approximately 10,000 feeders and each feeder connects to approximately 10 customers. DS is also connected to the control center and other distributed energy resources (DERs). The DS may be connected to one or multiple power generating stations through the TS.

Feeders/ Transformer

Transmission Substations

Distribution Substation

Distribution Substation

Distribution Substation

Control Center

Control Center

Distributed Energy

Resources

High Voltage Distribution Network

Feeders/ Transformer

Feeders/ Transformer

Feeders/ Transformer

Feeders/ Transformer

Feeders/ Transformer

Feeders/ Transformer

Feeders/ Transformer

Medium Voltage Distribution Network

Meters Meters Meters Meters Meters Meters

Low Voltage Distribution Network

Fig. 1. Electricity Distribution Network Architecture

Meters Meters Meters Meters Meters Meters

4

A. Bandwidth requirements for Smart Grid Communication:

In the following, we are going to calculate the bandwidth required for the communication network that will support the SG. Assuming that every electric meter generates a message every second to the DS, the total number of messages is 100,000 messages per second. The feeders themselves will be generating messages to each other and to the DS. We can easily postulate the existence of 1 million messages per second during a busy period. If we further assume that the average length of each message is 100 bits and latency requirements dictate that the mean delay is bounded to 10 milliseconds, the transmission speed can be computed using queuing theory. Using conventional terminology [15], we have mean length of the message 1/ 100μ = bits, mean latency

T=10 msec and number of messages 610λ = per sec. We model the packets in the SG arriving at servers located at the control center as M/M/1 traffic. We have assumed that the packets continue to follow the Poisson discipline at each node. Also, the inter-arrival time ( 1/t λ= ) and service time ( 1 /x μ= ) are both exponentially distributed.

Using results in [15],

1 (1)Tcμ λ

=−

and plugging the values of μ , λ and T we can evaluate the transmission line bandwidth c as 100.01 Mbps.

It is well understood that the mean value of the latency evaluated from (1) is not very meaningful if the delay varies within wide limits. If we require that the delay is limited to 10 msec for 99% of the messages, then we will need a larger bandwidth. We have the probability of delay exceeding the defined threshold t (=10 msec) given by [15] and [16],

(1 )1 1{ } 1 (2)c tp P W t e μ ρρ − −= ≤ = −

For our analysis, we have, 1 0.99p = , 10t = msec,

1/ 100μ = bits and 610λ = messages per second. Using these figures in (2), we can evaluate c as 100.056 Mbps. It can be easily observed that both these situations result in a very poor bandwidth utilization of the transmission facilities. Unfortunately, a higher level of utilization will not permit meeting the assumed latency constraint.

Let us take one more case where average message length (1/ μ ) is 400 bits. By using (1), the bandwidth can be calculated as 400.04 Mbps. If the delay is limited to 10t =

msec for 99% of the messages, then from (2) we will need a bandwidth c as 400.086 Mbps. We can easily observe that the bandwidth requirement in this case has gone up substantially.

This requirement of high bandwidth will require optical fibers for the communication medium along the electric transmission and distribution lines to meet the latency requirements.

The illustrative example shown above assumes only a single server. In the SG of multiple levels of hierarchy, several queuing systems in tandem will likely arise. In such a situation, the mean latencies encountered at each system will simply be additive.

It is obviously possible that the messages in the SG are structured as fixed size messages. The exponentially distributed message lengths as assumed in the analysis presented earlier will no longer be valid. However, using existing queuing theory results, such analyses can be easily carried out. The most compelling part of the analysis is the very large amount of bandwidth that will be needed even in a moderately sized distribution system.

B. Optical fibers for smart grid communication Utilities have different ways to install optical fibers. It will

be easier for the utilities to install optical fibers as they have the right of way along their transmission facilities using existing transmission poles and underground conduits. Thus, the cost of installing optical fiber cables for the SG will be inexpensive for a utility company.

The scale, size and number of messages will likely increase geometrically as the scale of the distribution network increases. However, speeds of transmission in optical fibers have increased tremendously over the past several years. Thus, the problem of scaling can be easily handled with fiber. Since, optical fibers can easily support speeds of several hundred gigabits per second, optical fiber-based networks will be the long term solution for the evolving needs of SGs.

VI. CONCLUSION: This paper has presented the need for a Smart Grid if we

were to deliver on the requirements of electric generation, distribution, and usage in the future. We have shown that Smart Grids build on the technologies of sensing, communication, and control. We have postulated a medium size distribution network and computed the bandwidth requirements of the communication facilities in the grid. Based on the assumptions we have used, we can already foresee needs for communicating at 100Mbps and above even for a moderate size distribution system.

VII. REFERENCES: [1] U.S. Department of Energy, [online] Available: www.oe.energy.gov. [2] Gail E.Tverberg, “The U. S. electric grid: will it be our undoing?”, May

2008 [Online] Available: http://www.energybulletin.net/node/43823. [3] Smart Grid [online] Available: http://en.wikipedia.org/wiki/Smart_grid. [4] “A vision for the modern Grid”, National Energy Technology

Laboratory , United States department of energy, March 2007 [online] Available: http://www.netl.doe.gov/moderngrid/docs/A Vision for the Modern Grid_Final_v1_0.pdf. Retrieved 2008-11-27.

[5] Seeing the Forecast: Obama Archives, [online] Available: www.seeingtheforest.com/ archives/obama/.

Transmission Substations

A Distribution Substation and Control Center

Feeders (approximately 10,000)

Meters (100,000)

Communication lines

DER

Fig. 2. Quantification of Distribution Network from one DS

5

[6] U.S. Departent of Energy, Office of Electricity Delivery and Energy Reliablity. GridWorks: Overview of the Electric Grid. [Online] Available: http://sites.energetics.com/gridworks/grid.html.

[7] Smart Grids Climate Lab, [online] Available: http://climatelab.org/ Smart_Grids.

[8] The Distributed Storage-Generation “Smart” Electric Grid of the Future, by Roger N. Anderson, White paper, Columbia University.

[9] Kathy kowalienko,“Smart Grid projects pick up speed”, IEEE, The Institute, Standards, Article 06 Aug., 2009.

[10] Smart Grids Interoperability Standards Project,[online] Available : http://www.nist.gov/smartgrid/.

[11] NIST framework and Roadmamp for Smart Grid interoperabilty standards release 1.0, [online] Available: www.nist.gov/public_affairs/ releases/smartgrid_interoperability.pdf.

[12] U.S. Department of Energy, Office of Electricity Delivery and Energy Reliablity: The Electricity Delivery System. February 2006 [Online] Available: http://sites.energetics.com/gridworks/pdfs/factsheet.pdf.

[13] Francisco Lobo, Ana Cabello, Alberto Lopez, David Mora, Rosa Mora, 2008, “Distribution Network as Communication system”, Frankfurt : CIRED Seminar 2008: Smart Grids for distribution paper no 0022, June 2008.

[14] Carl H. Hauser, David E. Bakken, Ioanna Dionysiou, K. Herald Gjermundrød, Venkata S. Irava, Joel Halkey and Anjan Bose, 2008,”Security, trust, and QoS in Next generation control and comunication for large ppower systems”,: Int. J. Critical Infrastructures, Vol. 4.

[15] Leonard Kleinrock, Queueing Systems, Vol 1:Theory, New York: Wiley, 1975 p. 94-109.

[16] Ling Wang and Pramode K. Verma, “The Notion of Cost and Quality in Packet Switched Networks: An Abstract Approach”, Proc. IEEE International Workshop on Communications Quality and Reliability, May 12-14, 2007, Naples, FL.

[17] U.S. Department of Energy, Office of Electricity Delivery and Energy [online] Available : www.electricity.doe.gov, February 2006.

[18] NIST Unveils Plan For Smart Grid Interoperability, September2009, [online] Available: http://www.facilitiesnet.com/powercommunication/ article/NIST-Unveils-Plan-for-Smart-Grid-Interoperability--11196.

VIII. BIOGRAPHIES Amit Aggarwal received the B.Tech. degree in Electronics and Communications Engineering from Punjab Technical University in 2005, and M.S. degree in Telecommunication Engineering from University of Oklahoma-Tulsa in 2008. Since 2007- till date he has worked in Telecommunications Systems Laboratory, to study VoIP networks, network security and related Issues. He is a Ph.D. student now in University of Oklahoma-Tulsa.

Swathi Kunta earned her Bachelor’s degree in Electronics and Communications Engineering from Jawaharlal Nehru Technological University in 2008 and is pursuing her M.S. in Telecommunications Engineering at University Of Oklahoma-Tulsa. She is doing her research on Wireless Sensor Networks in Telecommunications Systems Laboratory.

Pramode K. Verma is Williams Chair in Telecommunications Networking and Director Telecommunications Engineering at The University of Oklahoma – Tulsa. He has more than 20 years of leadership experience in the telecommunications industry. In his last position with Lucent Technologies as Managing Director – Business Development, Global Service Providers Business and Business Communications System, his responsibilities included creating strategic alliances

and partnerships with leading organizations, and managing the associated profit and loss. He also held professional and management positions with Lucent Technologies – Bell Laboratories for fifteen years.

Dr. Verma obtained his doctorate in Electrical Engineering from Concordia University in Montreal, Canada in 1970 and an MBA from the Wharton School of the University of Pennsylvania in 1984. He is the author/co-author of over 50 articles and several books in telecommunications, computer communications and related fields.

He is a past president of the International Council for Computer Communication, a Washington D.C.-based global organization; a senior member of the Institute of Electrical and Electronics Engineers, New York and registered as a Professional Engineer, Province of Ontario, Canada.