Configuring a Multi-homed EMC NetWorker Storage Node An

EMC Proven Professional Knowledge Sharing 2010

Configuring a Multi-homed EMC NetWorker Storage NodeAn EMC NetWorker Case Study Gururaj Kulkarni and Anand Subramanian

Gururaj KulkarniAssociate Principal Software Quality [email protected]

Anand SubramanianSenior Software Quality [email protected]

2010 EMC Proven Professional Knowledge Sharing 2

Table of Contents

1. Introduction ___________________________________________________________________ 3 1.1 About this article _________________________________________________________________3

1.2 Advantages of using multi-homed Storage Node configuration ______________________3

2. Multi-homed storage node configuration ____________________________________________ 4 2.1 Requirements for multi-homed setup ______________________________________________4

2.2 Considerations for multihomed configuration ______________________________________4

3. Typical NetWorker setup with a multihomed storage node ______________________________ 5

4. Best practices and recommendations _______________________________________________ 8 4.1 Domain Name Service ____________________________________________________________8

4.2 Link aggregation/NIC teaming _____________________________________________________8

4.3 Infrastructure and operational recommendations ___________________________________9

5. Conclusion ___________________________________________________________________ 10

6. Glossary _____________________________________________________________________ 10

Disclaimer: The views, processes or methodologies published in this compilation are those of the authors. They do not necessarily reflect EMC Corporation’s views, processes, or methodologies


1. Introduction EMC NetWorker™ is an enterprise class Backup and Recovery solution. It is three-tiered software:

1. NetWorker Client (that hosts the data to be backed up)

2. NetWorker Server (co-ordinates the entire backup/recover process and tracks the metadata)

3. NetWorker Storage Node (that connects to diverse storage devices and writes/reads data).

The NetWorker Storage Node is a key player during data backup and recovery. It must have

tremendous power to transfer data from / to storage devices, since the rate at which data is backed up /

retrieved determines the overall speed of the Backup-Recovery application. In most enterprise backup

infrastructures, the Storage Node is a powerful server with superior system configuration. Since most

high end machines have multiple NIC ports, it is worthwhile to utilize the power of more than one NIC to

improve overall system throughput.

I/O operations are the most expensive in any backup infrastructure. By having a multi-homed

NetWorker storage node, the system’s bus speed, CPU speed, and disk throughput will be utilized to

their maximum capabilities, thus providing improved performance throughput.

1.1 About this article This article details the steps to configure more than one NIC port on the Storage Node to transfer data.

We will also present recommendations and best practices when configuring a NetWorker Storage Node

with multiple NICs. By following these simple guidelines, you can achieve increased backup and

recovery throughput to help reduce the size of the backup window.

1.2 Advantages of using multi-homed Storage Node configuration Here are some of the components that play a major role during data transfer on storage nodes:

• CPU speed

• Number of CPUs

• Disk utilization

• System bus speed

• Network bandwidth

• Physical memory on the storage node

• Fibre Channel (FC) connectivity


The response time on storage nodes is impacted if any of these underlying components exceed their

manufacturer’s rated capacity. For example, if the incoming or outgoing data exceeds the rated Gigabit

Ethernet capacity, there will be contention for read or write operations. This will result in lower device

throughput. In these cases, you must have a load sharing mechanism to improve the data transfer rate.

One solution is to configure multiple NICs.

Industry standard benchmarks have demonstrated that with a GigE network, you can achieve up to

90% of NIC utilization, approximately 90-110 MB/s. If a dual NIC setup is used on the storage node,

you can expect up to 180% increased NIC utilization. This will increase the backup speed and reduce

the backup window.

Another important outcome is the reduced backup window realized by the improved net throughput,

thus meeting the Recovery Time Objective (RTO).

2. Multi-homed storage node configuration Consider this information when configuring multi-homed storage nodes.

2.1 Requirements for multi-homed setup Here are requirements to setup IP addresses and hostnames when using multiple NICs:

• Each IP on any host must always resolve to a unique primary hostname. According to TCP RFC

793, you cannot use the same primary hostname for multiple NICs.

• Each IP belonging to any specific host must reside in a separate subnet. According to TCP RFC

793, you cannot have multiple IPs belonging to the same subnet and break outgoing session

integrity.

• You should enter each hostname that belongs to any host (NetWorker server, storage node or

client) in the Alias attribute for the Client resource in NetWorker.

2.2 Considerations for multi-homed configuration

Client configuration issues • To connect from a NetWorker server to a NetWorker client over a specific NIC, configure the

client using the name that is only reachable over the desired NIC.

• To direct client backup data to a NetWorker storage node over a specific NIC, set the Storage

Nodes attribute in the Client resource to a desired hostname that is only reachable over the

desired NIC.


• To direct metadata from the client to the backup server over a specific NIC, set the Server

NetWorker Interface attribute in the Client resource to a server name that is only reachable over

the desired NIC.

• Note that all instances of a target client resource must have the same value for the Server

NetWorker Interface attribute. This metadata includes saveset control session information as

well as all Index database operations. The amount of data transferred over the network is

proportionate to the number of files being saved, and is much lower than the actual backup

data. This field is optional and has to be set correctly only if the client cannot communicate with

the backup server over the primary NIC.

Device configuration issues

• To direct metadata from devices on a storage node to a backup server over a specific NIC, set

the Server NetWorker Interface attribute for the Device resource to a server name that is only

reachable over the desired NIC. The amount of data transferred over the network is

proportionate to the size of savesets divided by the target block size plus some overhead, but

on average is very low.

• Note that all devices from the same storage node must have the same value for the Server

NetWorker Interface attribute. This metadata includes device control session information as well

as all Media database operations (connecting back to nsrmmdb). This field is optional and has

to be set correctly only if the storage node cannot communicate with the backup server over the

primary NIC.

3. Typical NetWorker setup with a multi-homed storage node Typically, NetWorker customer environments will have a separate LAN in a data center for better

management and security of data. In the following diagram, NetWorker clients use the management

LAN to send catalog information (NetWorker metadata) to the NetWorker server, and use the backup

LAN to send actual data to the NetWorker storage node.


The following diagram shows a typical customer setup using a multi-homed storage node.


The following diagram demonstrates the use of a dual-NIC NetWorker storage node with multiple

NetWorker clients from different subnets:


4. Best practices and recommendations

4.1 Domain Name Service If you are using Domain Name Service (DNS) for hostname resolution, ensure forward and reverse

lookups among the server, storage node, and clients are functioning properly. With multiple NIC

configurations, the hostname resolution should be very accurate. Incorrect DNS configurations will

cause backup or recover operations to fail with a message such as “Probe job had unrecoverable

errors.”

At times, there may be a partial DNS resolution (for example, forward lookup works fine, but reverse

lookup fails; or vice versa). These problems may cause a reduction in net throughput. Always ensure

both forward and reverse lookups are resolved properly.

Name resolution through a local DNS server will cause less traffic over the network when compared to

resolution through a remote DNS server. This will lead to faster backups.

4.2 Link aggregation / NIC teaming Link aggregation, or IEEE 802.1AX-2008, is a computer networking term that describes using multiple

network cables/ports in parallel to increase the link speed beyond the limits of any one single cable or

port, and to increase redundancy for higher availability. Other terms for link aggregation include

Ethernet trunk, NIC teaming, port channel, port teaming, port trunking, link bundling, EtherChannel,

Multi-Link Trunking (MLT), NIC bonding, and others.

Link aggregation at the TCP level, regardless of the protocol or algorithm used, has no effect on a

single TCP session. Thus, combining multiple links into a single link will not increase the backup

performance of a single session. Depending on the algorithm used, there may be a positive effect when

executing parallel backup jobs with multiple NICs, which will act in a load-balancing manner. To

properly accomplish this, use a link aggregation algorithm that is TCP-session-based and not host-

based. A best practice would be to use the IEEE 803.3ad/802.1ax Link Aggregation Control Protocol

(LACP).

The use of trunked interfaces is transparent from the NetWorker point of view, and their configuration

inside NetWorker does not differ from the configuration of standalone interfaces. Note that use of TCP

trunking can still be combined with multi-homing as some NICs on the system can be trunked and

some left working on separate subnets.


4.3 Infrastructure and operational recommendations Here are our recommendations for hardware and operational components of the backup system.

The underlying network infrastructure plays a major role during backup/recover operations because

NetWorker is a LAN backup solution. We recommend having a Gigabit Ethernet infrastructure to

achieve improved throughput.

Use high capacity and high speed disk drives (for example, SAS, FC disks), tape drives (for example,

LTO-4), and media (for example, LTO-4 tapes) during backup. Test the disk or tape subsystem using

native OS tools before the backup to check for optimum performance.

Make sure there are a sufficient number of CPUs and sufficient CPU speed on the storage node to

support the data transfer over multiple NICs. This is true for bus speed as well.

For Windows multi-homed storage nodes, make sure that all the NIC drivers are up to date.

The internal bus is an important component that plays a major role in storage node performance. The

bus performance depends on the type of bus, width of the bus, and clock speed.

PCI and PCIe buses are accepted industry-wide and have capacities of up to 400 MB/s. It is a good

practice to have multiple buses in a system to transfer the data.


The following example illustrates a calculation of bus speed: Frequency (1Hz) = 1/t -> 1/s Bus specification: 64bit/100MHz 64bit = 8Bytes 8Bytes X 100MHz = 800MB/s I/O is divided into 2 -> Input and Output 800/2 = 400MB/s Max. Throughput on the bus 400M/s = 4 X 1GigE ports or 2 X LTO-3 at 160MB/s

Note that high-speed network transfer also puts a lot of pressure on CPU; therefore, the above

numbers are true if the CPU is not a bottleneck. For the same reason, we do not recommend using

multiple high-speed NICs directly on the backup server; use a storage node instead.

The NetWorker performance benchmarking with a 10-client dual NIC storage node setup has shown

that having more than four backup devices yields better throughput. Having the device target sessions

equally distributed among the clients is also a best practice.

5. Conclusion By using multi-homed NetWorker storage nodes, you can maximize the advantages provided by a

storage node in the NetWorker infrastructure. Additionally, you can improve RTO due to increased net

throughput by using a multi-homed storage node.

6. Glossary DNS Domain Name Service FC Fibre Channel

GigE Gigabit Ethernet IEEE Institute of Electrical and Electronics Engineers LAN Local Area Network NIC Network Interface Card NMC NetWorker Management Console NW NetWorker PCI Peripheral Component Interconnect

PCI-e Peripheral Component Interconnect – express RFC Request for Comments SAS Serially Attached SCSI SCSI Small Computer System Interface TCP Transmission Control Protocol (RFC 793)

VLAN Virtual Local Area Network

Documents

Configuring a Multi-homed EMC NetWorker Storage Node An