Software Testing of a Simple Network

Jack H. Arabian Comparative Management Associates, Milton, MA 02186, USA

E-mail: Jack Arabian <jarabian@gmail.com> Key Words – Software, Test, Networks, Nodes, Model, Simulation, Packet, Byte, Overload, Queue, Pipe, Wireless, Hard-Wired. Acknowledgements

The author gratefully acknowledges the support of

Scalable-Network Technologies of Los Angeles, CA, USA. 1. Introduction

Testing of Software for Networks presents a challenge to the collection of data to resolve cost issues. As described in reference [1], it has been shown that Process simulation can be modeled and run with respect to optimizing the output of the model for typical parameters of cost, time, and resources. It remains, however to demonstrate that the same can be accomplished in testing network designs. This paper describes the test of a simple network with nodes hard-wired and wireless, identifying delays, and bytes sent and received.

2. Requirements

Given the model shown in Figure 1, the following steps were taken:

1. Determine the number and types of nodes to be analyzed.

2. Initialize the metrics of each node and its connection to the network

3. Run the simulator. 4. Analyze the statistics of the simulator run and

make changes where necessary. 3. Procedure

1. For mapping the simple network in design mode (Figure 1 below), we used a commercial network mapping application called Exata/QualNet™. In the network model, we wanted to flood the receiving node (#4) in order to show the loss of bytes from the four input nodes of #1, 2, 5, and 6. This is sometimes characterized as “fire-hosing” or “having too small a pipe.” (All figures are reduced in scale to fit on the page, but can be expanded electronically.)

Figure 1 Model of the Simple Network

978-1-4577-1958-5/11/$26.00 ©2011 IEEE

2. See Figure 2. Nodes #2, 5, and 6 are hard-wired using a variable bit rate (VBR), while Node #1 is wireless using a Multicast Constant Bit Rate (MCBR), all of which try to transmit bytes through Node #3 to Node #4. Node #1 transmits wirelessly, but cannot

reach Node #3 because of the set distance. Note that the radiation patterns (shown as green spheres around Nodes #1 and #3, do not extend far enough to connect the two nodes.

Figure 2 Simple Network Overloaded (in Red) After Simulation

3. Costing analysis was accomplished with the statistics in the “stat” files of Figures 3 and 4. It can be seen that the difference between bytes sent and bytes received is significant. (This is a deliberate overload in order to demonstrate the costing procedure.)

4. The total of bytes sent by the VBR Client is 3.4 X 108. The total of bytes sent by the wireless MCBR is

0.000125 X 108 to give a total of bytes sent to be 3.4000125 X 108. The total of bytes received is 0.0029 X 108.

The percent of total bytes received is therefore 0.0853%.

The lost bytes total then is 3.397225 X 108.

Figure 3 Stat File for the VBR Client for Total Bytes Sent

Figure 4 Stat File for the VBR Server for Total Bytes Received (at Node #4)

5. Industry metrics on the dollar (USD) cost effect of lost bytes is presently difficult to find. Data is generated, but most likely proprietary at present. The search for such a metric led to Reference [2] which states, “Computer scientists have developed an inexpensive solution for diagnosing delays in data center networks as short as tens of millionths of seconds—delays that can lead to multi-million dollar losses. ” Thus, a 10-microsecond delay defect is “equated” to roughly a million dollars by the following reasoning, which is debatable, but the best we have. If a packet (or message) can be equated to approximately 10 bytes for a period of 100-microseconds, then one byte can be equated to 10 microseconds. If we can use it as an industry number of approximately $1/lost byte, we can estimate the effect of an overloaded network design. The lost bytes total as calculated above is 3.397225 X 108, which when converted, amounts to USD $339.7225 X 106 or roughly 339.7 Million dollars. Good background information can also be found at Reference [3], an excerpt of which follows: “Different communications protocols use different conventions for distinguishing between the elements and for formatting the data. In Binary Synchronous Transmission, the packet is formatted in 8-bit bytes, and special characters are used to delimit the different elements. Other protocols, like Ethernet, establish the start of the header and data elements by their location relative to the start of the packet. Some protocols

format the information at a bit level instead of a byte level.” 4. Conclusion

It is costly to have defective networks and nodes. There are many factors involved in the cost of defective design of networks. The size of development team, stage of development when the defect occurs, routing protocols and subtlety of the defect are only a few of the possibilities. Testing software, therefore has to be designed to detect the defect, and as early as possible in the design cycle. Otherwise the costs can be overwhelming.

This is yet another compelling argument for QA engineers to justify up-front test costs similar to the electronics design programs of JTAG (Joint Test Action Group for boundary scan) or BIST (Built-in Self Test) circuitry. 5. Live Demo

For this presentation, a 1-minute live demonstration of the QualNet™ simulation shows dynamic, graphic animation of the Simple Network Test and Stat files with compelling metrics.

6. Future Work

This model is generic to many software test processes in which delays and byte/packet loss can occur. Many other aspects need to be shown, such as the variation in dollar cost of various routing and protocol routines. Process mapping and dynamic time simulation will also lead to future work in creating a true System of Systems (SoS) model through the ability to connect multiple processes such as other M&S applications which can be ported to and enhance the quality of the model.

7. References [1] Selected Cost Factors in Modeling and Testing Hardware and Semiconductor Defects by Dynamic Discrete Event Simulation // IEEE Proceedings East-West Design and Test Symposium 2008, Lvov, Ukraine, September 2008. By Jack H. Arabian, Engineering Specialist, Jacobs Engineering & Technology Division, Bedford, MA 01731 [2] Millionths of a Second Can Cost Millions of Dollars: A New Way to Track Network Delays, San Diego, CA, August 20, 2009 http://www.jacobsschool.ucsd.edu/news/news_releases/release.sfe?id=878 [3]http://en.wikipedia.org/wiki/Packet_(information_technology)