Deploying COS · Before deploying the COS, you must complete the following preliminary tasks: Step 1 Prepare your COS VMware datacenter topologies and networks. The COS requires the

C

C H A P T E R 2
Deploying COS
To deploy the COS 3.0.1 software, perform the following tasks in order:

1. Pre-Deployment Tasks, page 2-1

2. COS Node Hardware Installation, page 2-1

3. Configuring a COS Node for Data Mover, page 2-18

4. Configuring COS, page 2-26

Pre-Deployment TasksBefore deploying the COS, you must complete the following preliminary tasks:

Step 1 Prepare your COS VMware datacenter topologies and networks. The COS requires the following networks:

• Management — This is the primary COS network that connects all of the components.

• Cache Fill — This is the data network. It connects the COS Appliances.

Step 2 Download the COS Service Manager Open Virtual Appliance (OVA) file and load the image into a repository (HTTP server) that is accessible by vCenter.

Step 3 Determine the blade and VM layout that you will use, and your management IP address allocation scheme for the network interfaces.

Step 4 Install the external DNS and NTP servers.

• Determine the origin service FQDN and prepare the downstream client (client-facing) DNS servers to point to the COS Appliance dataplane (primary interface) IP addresses.

COS Node Hardware InstallationCOS 3.0.1 software can currently be deployed on the Cisco UCS C3160 Rack Server. For information about installing the hardware, see the Cisco UCS C3160 Rack Server Installation and Service Guide.

The instructions in this section apply only to COS installation on the Cisco UCS C3160 Rack Server. Before you begin, be sure that you have the following:

• C3160 server installed per manufacturer instructions

• Cisco Integrated Management Controller (CIMC) Connection to the server

2-1isco Cloud Object Storage Release 3.0.1 User Guide

Chapter 2 Deploying COS COS Node Hardware Installation

• ISO image of the COS Software

C3160 Hardware OverviewThe C3160 is a 4RU server. When configured for COS, the C3160 includes:

• 2 solid state drives in top slots 55 and 56, configured as RAID 1, to receive the operating system and COS installation

• 54 hard disk drives in top slots 1-54, configured as JBOD with 216 TB total capacity (4 TB per drive)

• 2 SSD drives in rear-loading slots (RAM1)

• Four 10 GbE ports, two per SIOC

• RAM2

C3160 Network Requirements The C3160 provides four 10 GbE ports for video and management traffic. To separate the management traffic from video, port 1 on each storage I/O controller (SIOC) must be divided into two vLANs, with the management vLAN limited at 1 Gbps and the video vLAN limited at 9 Gbps. You must provide the appropriate vLAN IDs defined by the switch to which these ports connect.

Although not essential, the second 10 GbE port on each SIOC should be configured as a vLAN, limited at 10 Gbps, and assigned to the same video vLAN ID as the 9 Gbps vLAN. The vLAN are defined in hardware by virtue of the modular LAN on motherboard (mLOM) of the virtual interface card (VIC) attached to the SIOC. The vLANs appear to Linux as physical NIC ports.

Figure 2-1 provides a view of the network architecture of a COS cluster based on the UCS C3160.

2-2Cisco Cloud Object Storage Release 3.0.1 User Guide


Figure 2-1 C3160 COS Cluster Architecture

Configuring VLANs

You must configure the first downstream switch to receive the traffic from these two vLAN IDs and route the traffic appropriately. A Bash script named setup_’UCSC-C3160_vLAN.sh’ is provided to help configure the vLANs. You can download this script from the same Cisco download page that contains the COS product ISO package.

This script is executed from a remote Linux node with network access to the C3160 CIMC. Execution of the script occurs after the C3160 CIMC IP address is defined but before COS is installed.

This script requires the following input parameters:

• CIMC IP address

• CIMC username and password



• Management vLAN ID

• Video/data vLAN ID

When it runs, the script prompts for any parameters missing from the command line. Password values entered in response to such a prompt are not echoed to the screen.

Example./setup_UCSC-C3160_vLAN.sh -c 10.10.10.10 -u admin -m 42 -d 2001Password:

Following COS installation, this script can be used to reset the C3160 VICs to the COS configuration, provided that the operating system network is set up properly and the CIMC has an IP address.

Installation ProceduresInstallation of COS on the C3160 involves the following procedures, which must be performed in the prescribed sequence:

1. Prepare the C3160 for COS installation

2. Install the COS ISO image on the C3160

3. Configure the COS node network on the C3160

4. Configure the DNS resolver

5. Restart the network service

6. Configure the NTP service

7. Configure Cassandra for COS

8. Configure the COS daemon

9. Configure Cserver

10. Verify that all services are running

11. Verify log files creation

12. Create user accounts and verify node/cluster health

This section describes these steps in detail.

Prepare the C3160 for COS Installation

Step 1 Remove any hard drives (up to four possible) inserted from the rear of the C3160.

Step 2 Connect the CIMC RJ45 port on SIOC1 (on the left from the rear) to your management network.

Step 3 Connect the 10 Gbps fiber connections to your video switch.

Note Your video network switch should already be configured to support the video and management vLAN IDs that will be created on port 1 of SIOC1.

Step 4 Connect the power cords to the C3160.

Step 5 Temporarily attach a monitor and keyboard to the C3160.



Step 6 Apply power to the C3160 and press F8 early in the power-up sequence to enter Cisco CIMC Configuration.

Step 7 Under NIC Properties, set the CIMC IP, mask, and gateway as appropriate for your installation.

Step 8 Press F10 to save your changes, then press F5 to refresh the screen and confirm that the IP address is set properly.

Step 9 Press ESC to continue with the CIMC configuration (the system will not yet reboot).

Step 10 Configure the vLANs using the setup_UCSC-C3160_vLAN.sh script as described in Configuring VLANs, page 2-3.

Install the COS ISO Image on the C3160

Step 1 Attach the COS installation ISO file using one of the following methods:

• Connect an external DVD drive at any USB port on the C3160.

Note When using an external DVD drive, installation may fail with missing files reported. If this occurs, the likely cause is that the DVD drive is drawing excessive current from the USB port of the C3160. To work around this issue, use a DVD drive that has its own AC power adapter.

• From the CIMC web page, navigate to Server > Remote Presence > Virtual Media > Add New Mapping and create a virtual media device.

Note Experience has shown that using the HTTP protocol produces the best results.

After the virtual media mapping is defined, select the CIMC BIOS > Config Boot Order option, define a CIMC Mapped DVD subtype mapping, and place it in boot position 1.

• Use the CIMC Java KVM Concole Virtual Media menu option to create a virtual media device associated with a COS ISO found on your desktop computer. After the virtual media mapping is defined, select the CIMC BIOS > Config Boot Order option, define a KVM Mapped DVD mapping, and place it in boot position 1.

Step 2 Reboot the C3160 and confirm that it boots from the COS ISO image.

Note If the boot order was not properly set before rebooting, press F6 early in the C3160 power-up sequence to select the boot device.

Installation is automatic, and typically requires 10-15 minutes to complete from DVD (installation from virtual media is typically slower).

Note The default system baud rate is 9600. To define a different baud rate, enter auto ks_baud_rate=<rate> at the installation boot: prompt before the 10 second auto-installation timeout occurs.

Step 3 When the Reboot message appears, disconnect the installation DVD drive or disable the virtual installation media, and then press the Enter key to reboot the C3160.



Configure the COS Network on the C3160

Step 1 When the C3160 finishes rebooting, log in to the console using the following credentials:

• Username — root

• Password — rootroot

The COS initialization (cosinit) script launches automatically to set up the management network that will be located on eth0.

Executing cosinit for platform configurations only

ATTENTION!!!

cosinit script should be run only to configure the device after an image installation.This script modifies the network and other critical configurations based on the deployment type.

Improper use of this script may result in mis-configuring the device or making it inaccessible.

If a new image is installed on this server, a reboot is required before running cosinit.If a reboot is already performed, please continue. Otherwise, please exit and executei cosinit after rebooting the server

Do you want to continue ? (yes/no) [y]: y

Note If you exit the cosinit script, you can relaunch it manually from the /opt/cisco/cos/config folder.

Step 2 When asked if you want to continue, enter y and configure the management interface as follows (IP addresses shown below are examples only):

Enter management interface [eth0]:Please ensure an IP address and netmask are configured formanagement interface eth0:

Select option 1 to configure Management InterfaceSelect the option for 'Done' to exit this menu:1. Configure management interface (eth0)2. DoneChoice [2]: 1Do you want to disable interface eth0? (yes/no) [n]: nEnter an IP address for eth0: 10.10.10.20Enter a netmask for eth0: 255.255.255.0Enter a broadcast address for eth0 [10.10.10.255]:

Select option 1 to configure Management InterfaceSelect the option for 'Done' to exit this menu:1. eth0 ip:10.10.10.20 mask:255.255.255.0 bcast:10.10.10.255 2. DoneChoice [2]: 2Backing up old scripts in /etc/sysconfig/network-scriptsWriting new ifcfg-ethX scripts

Note If you exit the cosinit script, you can relaunch it manually from the /opt/cisco/cos/config folder.

Step 3 Continue the script by entering the hostname of the node and the details of the default gateway as follows (IP addresses shown below are examples only):



Enter a hostname [localhost.localdomain]: COS-Node-1Enter the number of the eth interface that connects to the gateway: 0Enter the default gateway IP address [10.10.10.1]: 10.10.10.1Backing up /etc/sysconfig/networkWriting new /etc/sysconfig/networkBacking up /etc/hostsWriting new /etc/hostsRestarting network services, this may take a minute:Shutting down loopback interface: [ OK ]Bringing up loopback interface: lo: Disabled Privacy Extensions[ OK ]Bringing up interface eth0: ADDRCONF(NETDEV_UP): eth0: link is not readyigb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RXADDRCONF(NETDEV_CHANGE): eth0: link becomes ready[ OK ]Network services restarted; may take a few seconds to establish connectivityReboot for hostname changes to take effectNetwork configuration complete

Step 4 Ping the gateway to validate the configuration:

[root@localhost ~]# ping 10.10.10.164 bytes from 10.10.10.1: icmp_seq=1 ttl=255 time=2.58 ms64 bytes from 10.10.10.1: icmp_seq=1 ttl=255 time=2.58 ms64 bytes from 10.10.10.1: icmp_seq=1 ttl=255 time=2.58 ms64 bytes from 10.10.10.1: icmp_seq=1 ttl=255 time=2.58 ms

Step 5 Edit /etc/cods.conf, uncomment the db host field (about two pages down), and specify the IP address of the local management interface as the DB Host.

db host <mgmt ip>

Step 6 Reboot the COS node.

[root@localhost -]# reboot

Step 7 Configure IP Pools as described in Configuring IP Pools, page 2-27.

Initialize the COS Node - PAM Configuration

Step 1 Provided that cosinit ran in an earlier procedure, you can now execute cosinit to skip the network configuration and begin PAM configuration, as shown in the following example:

[root@Colusa-69 config]# ./cosinit -skipnw

ATTENTION!!!cosinit script should be run only to configure the device after an image installation.This script modifies the network and other critical configurations based on the deployment type. Improper use of this script may result in mis-configuring the device or making it inaccessible.

If a new image is installed on this server, a reboot is required before running cosinit.If a reboot is already performed, please continue. Otherwise, please exit and execute cosinit after rebooting the server

Do you want to continue ? (yes/no) [y]:Use detected platform: C3160-R1 ? (yes/no) [y]:Configuring the BAUD rate to 9600 ...=====================================================IP Pool Configuration



=====================================================Enter IP Pool name (Hit 'Enter' to skip IP Pool configuration): colusa-69

Please select the interfaces you want to add to colusa-69:[ ] 1. eth1[ ] 2. eth2[ ] 3. eth4[ ] 4. eth55. Done

Choice [5]: 1

Note The IP pool name should be the same as the one created in the COS Service Manager GUI.

Step 2 Enter the appropriate IP address of the COS Service Manager PAM, as follows:

• For a single PAM, enter the IP address as the docServer address.

• For multiple PAMs in an HA environment, enter the fully qualified domain name (FQDN) of the docServer; for example, mgmt-docserver.cos88.cisco.com.

Note The docserver port is 5087.

Please select the interfaces you want to add to colusa-69:[*] 1. eth1[ ] 2. eth2[ ] 3. eth4[ ] 4. eth55. Done

Choice [5]: 2

Please select the interfaces you want to add to colusa-69:[*] 1. eth1[*] 2. eth2[ ] 3. eth4[ ] 4. eth55. Done

Choice [5]: 3

Please select the interfaces you want to add to colusa-69:[*] 1. eth1[*] 2. eth2[*] 3. eth4[ ] 4. eth55. Done

Choice [5]: 4

Please select the interfaces you want to add to colusa-69:[*] 1. eth1[*] 2. eth2[*] 3. eth4[*] 4. eth55. Done

Choice [5]: 5Enter IP Pool name (Hit 'Enter' to complete IP Pool configuration):Enter Cluster Name:=====================================================



DocServer Configurations=====================================================Enter DocServer IP or FQDN: 172.22.116.35Enter DocServer Port [5087]:Backing up exisitng /tmp/.cosnodeinit.../tmp/.cosnodeinit generated successfullyShutting down monit: [ OK ]Starting monit: Monit start delay set -- pause for 120s [ OK ]Shutting down cos-aic-client: [ OK ]monit: Cannot connect to the monit daemon. Did you start it with http support?Starting cos-aic-client: [ OK ]monit: Cannot connect to the monit daemon. Did you start it with http support?Stopping syslog-ng: [ OK ]Starting syslog-ng: [ OK ]

cosinit finished successfully. Please reboot the box!!!

Execution of cosinit results in the following:

• The file /etc/sysconfig/network-scripts/ifcfg-eth0 is generated, containing the management interface configuration.

• The file /etc/sysconfig/network is generated, containing the hostname and default gateway.

• The file /etc/hosts is generated, containing the host information.

• The network is restarted.

• The file /tmp/.cosnodeinit is created, reflecting the configuration specified in cosinit.

• The file /boot/grub/grub.conf is updated, reflecting the current baud rate.

• The COS AIC client service (cos-aic-client) is started.

Step 3 Confirm that the COS AIC client service is started as follows:

[root@Colusa-69 config]# service cos_aicc statuscos-aic-client is running

The COS node is now registered with COS Service Manager.

Configure the DNS Resolver

The DNS Resolver file, resolv.conf, is added automatically by cos-aid-client after cosinit is run. Do not edit the file resolve.conf manually.

Restart the Network Service

Step 1 Restart the network service as follows:

[root@Colusa-69 config]# service network restartShutting down interface eth0: [ OK ]Shutting down loopback interface: [ OK ]Bringing up loopback interface: [ OK ]Bringing up interface eth0: [ OK ]

Step 2 Confirm that you can now ping the router and other servers in your network.



Configure the NTP Service

Step 1 Do not edit the NTP Server file manually. This file is updated automatically by cos-aic-client after cosinit is run.

Step 2 Enable and start the NTP service as follows:

[cos-node@ root]# chkconfig ntpd onStarting ntpd: [ OK ]

Configure Cassandra for COS

Step 1 Enable the syslog service to be started upon reboot:

[cos-node@ root]# chkconfig syslog-ng on

Step 2 Ensure that the host name is set properly (IP addresses shown are examples only):

[cos-node@ root]# cat /etc/hosts

# Lines below this one modified by cosnetconfig127.0.0.1 localhost.localdomain localhost10.168.5.18 cos02.cisco.com::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

Step 3 Create the directory for the Cassandra database:

[cos-node@ root]# mkdir /arroyo/db/cassandra[cos-node@ root]# chown cassandra:cassandra /arroyo/db/cassandra

Step 4 Validate the proper ownership:

[cos-node@ root]# ls –latotal 36drwxr-xr-x. 4 root root 4096 Nov 20 08:13 .drwxr-xr-x. 6 root root 4096 Nov 20 07:38 ..drwxr-xr-x 5 cassandra cassandra 4096 Nov 20 08:13 cassandradrwx------. 2 root root 16384 Nov 20 07:33 lost+found

Step 5 Do not edit the cassandra configuration file (/etc/cassandra/conf/cassandra.yaml) manually. This file is updated automatically by cos-aic-client after the COS node is added to a cluster.

Step 6 Stop the cosd service, then temporarily stop cserver and cosd from starting automatically.

Note Cserver must be started after Cassandra.

[cos-node@ root]# service cosd stop[cos-node@ root]# chkconfig cserver off[cos-node@ root]# chkconfig cosd off

Step 7 Start the Cassandra service:

[cos-node@ root]# service cassandra start

Step 8 Verify that the Cassandra service was successfully started:

[cos-node@ root]# service cassandra status



Step 9 Confirm that the Cassandra is running as follows:

Note The number of the lines in the output is directly related to the number of nodes in the cluster (single node – one line, two nodes – two lines, etc…)

[cos-node@ root]# nodetool status

Datacenter: datacenter1=======================Status=Up/Down|/ State=Normal/Leaving/Joining/Moving-- Address Load Tokens Owns (effective) Host ID RackUN COS_NODE_1_IP 117.35 KB 256 100.0% bf862009-40eb-474a-a86b-1700c724755a rack1UN COS_NODE_2_IP 768.4 KB 256 100.0% 39e62fbf-d0fb-4a92-9eb9-869b488ed5af rack1

Step 10 Apply the cos scheme in the Cassandra database as follows (IP addresses shown are examples only):

[cos-node@ root]# cqlsh 10.168.5.18 -f /opt/cisco/cos/config/cos.cql

Note • Before running this command, ensure that the scheme is correctly configured for the number of nodes in the cluster. Edit the cos.cql file and update the value of DC according to number of nodes in the cluster (one node – DC: 1, two nodes – DC: 2, three nodes and above – DC: 3), save the file, and then execute the command.

• When creating a cluster, it is necessary to run this command only on one node.

Step 11 After verifying that the command completed successfully, enable Cassandra on startup:

[cos-node@ root]# chkconfig cassandra on

Configure the COS Daemon

Step 1 Edit the file /opt/cisco/cos/etc/cosd.conf by uncommenting and setting desired values for the following:

• auth prefix AUTH_

• auth admin user .super_admin

• auth admin key rootroot

• cluster name

• cluster url

• auth token duration

Note The cluster url is the storage URL, and must be configured per local settings.

Sample Configuration File (All IP addresses are examples only)

[cos-node@ root]# cat /etc/cosd.conf

# cosd.conf.default#



# Copyright (c) 2013-2014 by Cisco Systems, Inc.# All Right Reserved.# The “auth prefix” distinguishes the built-in COS authentication service# from other authentication service providers.auth prefix AUTH_# The “auth admin user” is the user accoutn that has all administrative# privileges for the built-in COS authentication service.auth admin user .super_admin# The “auth admin key” is the authentication key (password) for the# auth admin user.auth admin key rootroot# The “cluster name” is the name of this cluster.cluster name cluster.ca.01# The “cluster url” is the URL for the built-in COS authentication service# for this cluster. (The default value is suitable only for single-server# clusters.)cluster url http://auth01.domain.net/v1# The “auth token duration” specifies the length of time in seconds that# an authentication token will be valid. The validity period begins# at the time the token was initially created. The default duration is# 24 hours.auth token duration 86400# The “log facility” specifies the syslog “faility” that will be# associated with COS daemon entries. The default value is that used for# system daemons.#log facility 3# The “log priority” specifies the maximum log-entry priority level that# will be logged. (Lower priority values are associated with more urgent# log entries.) The default value specifies that “notice”-priority and# lower will be logged. (Debug- and info-priority entries are excluded# by default.)#log priority 5# The “db host” value specifies a list of hosts:ports separated by “;”# to the database administration port of the cluster. (ex:# “host1;host2:9000;host3” would try to connect to host1 on port 8091,# if that fails it’ll connect to host2 on port 9000 etc). The hostname# may also be specified as a URI looking like: http://localhost:8091/poolsdb host 10.168.5.18# The “db user” specifies the database user name for COS daemon# transactions.#db user default# The “db password” specifies the database password for COS daemon# transactions.#db password <none># The “db bucket” specifies the database “bucket” name used to store# COS metadata#db bucket default# The “db object view” specifies the database view used to list containers# or objects, and to count the objects and content bytes subordinate to# accounts and containers.#db name view_design/cos/_view/object

Step 2 Start the cosd service:

[cos-node@ root]# service cosd start



Configure Cserver

Step 1 Create and configure the CServer setup file:

[cos-node@ root]# vi /arroyo/test/setupfile

When configuring Cserver, keep the following in mind:

• When building a cluster, the serverid value changes from server to server.

• The enic numbering is the numbering by which the Linux kernel recognizes the vNICs.

• Remember to update the number of disks.

Sample Setup File

In the sample file below:

• RAID is enabled in the node. To move from RAID to replication, set allow vault raid to 0. For local replication, add the line vault local copy count 1, where the number 1 represents the number of copies COS holds for the objects. For mirroring between nodes in a cluster, add the line vault mirror copies 1, where the number 1 represents the number of copies COS holds for the objects.

• For the enics that the management VLAN was derived from, it is also necessary to configure rate limiting on the CServer side. In the line that configures enic 2 below, erl 1 is added to indicate that this enic is rate limited.

# CServer core configuration. Changes to this file require a server reboot.serverid 1

groupid 3333 arrayid 6666 streamer 0 vault 0 cache 0 bypass_isacheck 1 cos 1 #e1000e adapters: maxrate 975 #igb adapters: maxrate 975 #ixgbe adapters: maxrate 9850 enic adapters: egress rate limited maxrate 8850 enic 2: streaming 1 fill 1 ip 18.60.0.101 tport 0 cport 0 tgid 0 erl 1 enic adapters: maxrate 9850 enic 3: streaming 1 fill 1 ip 18.60.0.102 tport 0 cport 0 tgid 0 enic 4: streaming 1 fill 1 ip 18.60.0.103 tport 0 cport 0 tgid 0 enic 5: streaming 1 fill 1 ip 18.60.0.104 tport 0 cport 0 tgid 0 allow vault raid 1 allow tcp traffic 1 disks 58 model test 4 er_enable 0 rtp_enable 0

Step 2 Configure the CServer subnet table, /arroyo/test/SubnetTable.

Note Skip this step if the server is not part of a cluster.

On every COS node, create /arroyo/test/RemoteServers and add the cache-fill IP addresses of the other COS nodes.

Syntax

network COS_NODE_NETWORK netmask COS_NODE_NETMASK gateway COS_NODE_GATEWAY



Sample SubnetTable For Adding 2 Remote Servers

remote server id <remote server id x> ip <cache-fill ip1> ip <cache-fill ip2> ip <cache-fill ip3> ip <cache-fill ip4> end remote server id <remote server id y> ip <cache-fill ip1> ip <cache-fill ip2> ip <cache-fill ip3> ip <cache-fill ip4> end remote server

Step 3 Start CServer for the first time, using the -C option to initialize cache drives for the first time:

[cos-node@ root]# /arroyo/test/run -C

Note The log rotate scripts are not installed, so keep an eye on the size of the /arroyo/log partition and clean up as needed.

If CServer was successfully started, re-enable cosd and CServer on startup:

[cos-node@ root] chkconfig cosd on[cos-node@ root] chkconfig cserver on

Step 4 Reboot the server.

Once the machine is up, continue to the next step.

Verify That All Services are Running

Step 1 Verify that cassandra is running:

[cos-node@ root] nodetool statusDatacenter: datacenter1=======================Status=Up/Down|/ State=Normal/Leaving/Joining/Moving-- Address Load Tokens Owns (effective) Host ID RackUN 10.168.5.2 117.35 KB 256 100.0% bf862009-40eb-474a-a86b-1700c724755a rack1UN 10.168.5.18 768.4 KB 256 100.0% 39e62fbf-d0fb-4a92-9eb9-869b488ed5af rack1

Step 2 Verify that cosd is functional:

[cos-node@ root] curl –v http://fqdn/info

{"cluster":{"config_ver":2,"fqdn":"cos-utah50.lindon.lab.cisco.com","name":"local","enable_wos":true},"swauth":{"reseller_prefix":"AUTH_","path_prefix":"auth/","max_key_len":256,"max_user_len":256,"token_life":86400,"max_token_life":86400,"max_account_len":256},"swift":{"version":"2.2.0","max_account_len":256,"max_container_len":256,"max_object_len":1024,"max_container_list":10000,"max_object_list":10000},"log":{"default":"notice"},"rio":{"path_prefix":"rio/“}}



Verify Log Files Creation

Verify that log files are being written to /arroyo/log (for example, cosd.log, http.log, protocoltiming.log) as follows:

[cos-node@ root] ls /arroyo/log/cosd.*/arroyo/log/cosd.log.20141120

[cos-node@ root] ls /arroyo/log/http.*/arroyo/log/http.log.20141120

[cos-node@ root] ls /arroyo/log/protocoltiming.*/arroyo/log/protocoltiming.log.20141120

Create User Accounts and Verify Node/Cluster Health

To create users and accounts for a COS node, install the swauth package on any Linux server and use the commands described below.

Note For additional details on the CLIs, see COS Command Line Utilities, page D-1 and the Cisco Cloud Object Storage Release 3.0.1 API Guide.

COS Username Scripts

COS stores the usernames and passwords for the GUI in the file etc/opt/cisco/cos/public/users.properties stored on the PAM VM. COS provides the following scripts to help you manage usernames and passwords:

• To list current users, enter the following command:

listusers-gui

• To add a new user, enter the following command:

adduser-gui username password

• To change an existing user, enter the following command:

chpass-gui username password

• To delete a user, enter the following command:

deluser-gui username

You can enter multiple commands when adding, changing, or deleting multiple users or passwords. For example, the following sequence of commands adds two users, changes two users, and deletes one user:

adduser-gui username1 password1

adduser-gui username 2 password2

chpass-gui username3 password3

chpass-gui username4 password4

deluser-gui username5

Note Usernames and passwords for the COS GUI can be strings of any length, are not case-sensitive, and can include uppercase or lowercase letters, numbers, and any special characters. However, the characters <, >, |, !, and & must be preceded by a backslash.



COS Swift CLI

The cos-swift command line utility is based on the Swift API, and can be used to manage storage accounts, storage containers, and storage objects in a COS cluster.

Syntax

cos-swift [–t <auth-token>] [–a <storage-url>] [–v/– –verbose][–h/– –help <subcommand>] [subcommand <options>]

COS Swauth CLI

The cos-swauth command line utility is based on the Swauth API, and can be used to manage authentication accounts and users in a COS cluster.

COS Cluster CLI

The cos-cluster command line utility can be used to manage the COS cluster membership of individual nodes.

• join – add a COS node to a COS cluster

• <cluster> – specify the domain name or IP address of a node in the cluster

• leave – temporarily remove a COS node from a COS cluster

Note When a node is temporarily removed, it is failed over, and the cluster may become non-resilient if only one copy of the objects on the node remains. After temporarily removing a COS node from a COS cluster, the cluster must be rebalanced. Only one node can be removed from the cluster at a time using this subcommand.

• –w/ – –wait <minutes> – specify the time in minutes to allow for the conclusion of TCP sessions on the node

• rejoin – add a COS node back to the COS cluster from which it was temporarily removed using the leave command

• remove – permanently remove a COS node from a COS cluster

Note When a node is permanently removed, the cluster is rebalanced.

• view – view a list of all the COS nodes in a COS cluster and the status of each node

PAM Installation and ConfigurationTo deploy the PAM from the vCenter, do the following:

Step 1 Log into the vCenter.

Step 2 Click File > Deploy OVF Template. The Source dialog appears.

Step 3 Select the location of the OVF template (for example, mendocino-shenandoah-301.16144) and click Next. The OVA Template Details dialog appears.



Step 4 Verify the OVF template details and click Next. The End User License Agreement dialog appears.

Step 5 Accept the End User License Agreement and click Next. The Name and Location dialog appears.

Step 6 Specify a name and location for the deployed template and click Next. The Deployment Configuration dialog appears.

Step 7 Select the MOS PAM deployment configuration from the drop-down list and click Next. The Storage dialog appears.

Step 8 Select a destination storage file for the VM files and click Next. The Disk Format dialog appears.

Step 9 Confirm that Thin Provision disk format for virtual disk storage is selected and click Next. The Network Mapping dialog appears.

Step 10 For each Source Network listed, choose the appropriate Destination Network from the drop-down list and click Next. The Properties dialog appears.

Step 11 Enter or choose the following information to customize the software solution for the deployment:

• IP address(es) of the NTP server(s)

• IP address of the DNS server

• External DNS transaction signature key (enter twice to confirm)

• Transaction signature algorithm

• IP address or hostname of two HA peers, if enabled

• (Optional) machine hostname

• Machine domain name

• Network adapter IP address

• Network adapter subnet mask

• Network adapter gateway IP address

Step 12 Click Next, then click Finish to deploy the OVA and create the PAM VM. The progress bar shows the status of the deployment.

Step 13 Verify connectivity and open an SSH session into the PAM VM.

Upgrading PAM

Perform these steps as needed to upgrade the current PAM installation.

Step 1 Disable the COS service by placing it in Maintenance status.

Step 2 Run a backup procedure on one of the PAMs in HA mode as follows:

ssh admin@nodesudo /opt/cisco/mos/bin/backup.py backup

Note Before running the backup.py script, be sure that the LANG or LC_* environment variables are set correctly.

Step 3 Confirm that the backup script generates a db folder and three files:

• node.txt

• state.json

• version.txt


Chapter 2 Deploying COS Configuring a COS Node for Data Mover

Step 4 Upgrade the COS nodes to the 3.0.1 image, and then shut them down.

Step 5 Shut down the three older PAMs.

Step 6 Rename the PAM VMs so they can be recovered in the event of a problem with the upgrade.

Step 7 Launch the new (Version 2.4) PAMs with the same names as the older ones.

Step 8 Restore the database backup on one of the new PAMs as follows:

ssh admin@nodeIPsudo /opt/cisco/mos/bin/restore.py backupsudo /opt/cisco/mos/bin/nodeRestore.py backup"

Step 9 Reboot the three new PAMs.

During reboot, cos-aic runs migration scripts, migrating all objects and references to new schemas and creating new aic_* objects as required. The COS Service remains in Maintenance status during this time.

Step 10 Restart each of the COS nodes.

Step 11 Return the COS Service to active (inService) status.

Configuring a COS Node for Data MoverCOS Release 3.0.1 includes components for a new data-mover architecture. This architecture includes the nginx software package, which is installed automatically either when installing COS 3.0.1 or when upgrading from a previous release. Other procedures are needed to configure any non-data-mover COS nodes to use the data-mover components.

After installing COS 3.0.1 or updating from a previous release, first verify that the nginx RPM was installed automatically as follows:

[root@utah10 ~]# cos_pkgs | grep nginx| nginx | 1.6.0 | 2.1.0.cos0.45 | b91 |

After confirming that nginx is installed, proceed with the configuration steps described in this section.

Configuration OverviewThe main areas for configuration updates are the network adapters. In earlier COS releases, the COS network configuration is generally divided into two functions: management and data. The Swift and Swauth functions are serviced on the data network, with the cserver network stack owning the data network interfaces.

With the data-mover architecture, the data network is further divided into service and internal interfaces. The service interfaces are externally facing, and handle the servicing of Swift and Swauth functions. Further, the Linux network stack owns the service network interfaces. The internal network interfaces are used for the cserver cluster communication internal to the cluster, and remain owned by cserver.

The necessary network configuration changes focus on dividing the data network interfaces into the service and internal interfaces. This simply means releasing interfaces from cserver and configuring them solely for use by the Linux network stack. Our general recommendation is to divide half of the data interfaces to service interfaces, and the other half to internal interfaces.

The configuration files that need to be changed on a COS node to convert to a data-mover configuration are as follows:



• /arroyo/test/setupfile

• /arroyo/test/RemoteServers

• /etc/sysconfig/network-scripts/route-eth*

• /etc/modprobe.d/blacklist.conf

After updating these files as described below, you can configure the nginx service and verify its functionality.

Updating /arroyo/test/setupfile and /arroyo/test/RemoteServersThe /arroyo/test/setupfile and /arroyo/test/RemoteServer configuration files must be updated to remove any configurations that apply to interfaces being converted to service interfaces. These changes release the interfaces from use by cserver.

Below are some examples of changes to configuration of a CDE250 configured with four 1 Gb data interfaces. The changes release eth2 and eth3 for use as service interfaces, and leave eth4 and eth5 for cserver internal use.

Prior to the configuration change:

[root@utah10 ~]# cat /arroyo/test/setupfile ...e1000e 0: streaming 1 fill 1 ip 10.0.10.0 tport 0 cport 0 tgid 0 e1000e 1: streaming 1 fill 1 ip 10.0.10.1 tport 0 cport 0 tgid 0 e1000e 2: streaming 1 fill 1 ip 10.0.10.2 tport 0 cport 0 tgid 0 e1000e 3: streaming 1 fill 1 ip 10.0.10.3 tport 0 cport 0 tgid 0 ... [root@utah10 ~]# cat /arroyo/test/RemoteServers remote serverid 1ip 10.0.10.0ip 10.0.10.1ip 10.0.10.2ip 10.0.10.3end remote server

After the configuration change:

[root@utah10 ~]# cat /arroyo/test/setupfile ...e1000e 2: streaming 1 fill 1 ip 10.0.10.2 tport 0 cport 0 tgid 0 e1000e 3: streaming 1 fill 1 ip 10.0.10.3 tport 0 cport 0 tgid 0 ... [root@utah10 ~]# cat /arroyo/test/RemoteServers remote serverid 1ip 10.0.10.2ip 10.0.10.3end remote server

Updating /etc/sysconfig/network-scripts/route-eth*The route-eth* files contain configuration for source-based routing in the Linux network stack. A small but important change must be made to this configuration. The following is an example of the necessary change, with areas of change highlighted.

Before the change:



[root@utah10 bin]# cat /etc/sysconfig/network-scripts/route-eth210.0.0.0/16 dev eth2 10.0.10.0 table 102default via 10.0.0.1 dev eth2 table 102

After the change:

[root@utah10 bin]# cat /etc/sysconfig/network-scripts/route-eth210.0.0.0/16 dev eth2 src 10.0.10.0 table 102default via 10.0.0.1 dev eth2 src 10.0.10.0 table 102

Updating /etc/modprobe.d/blacklist.confThe e1000e and ixgbe directives included in /etc/modprobe.d/blacklist.conf must be removed. This change allows the service interfaces to start automatically on boot.

Before the change:

[root@utah10 bin]# cat /etc/modprobe.d/blacklist.conf | tailblacklist snd-pcsp

# I/O dynamic configuration support for s390x (bz #563228)blacklist chsc_schblacklist e1000blacklist e1000eblacklist ixgbeblacklist csdblacklist cdd_sysblacklist avs_cserver

After the change:

[root@utah10 bin]# cat /etc/modprobe.d/blacklist.conf | tailblacklist snd-pcsp

# I/O dynamic configuration support for s390x (bz #563228)blacklist chsc_schblacklist e1000blacklist csdblacklist cdd_sysblacklist avs_cserver

Configuring the nginx ServiceThe nginx software component service is managed with a SYS V init script. That is included in the nginx RPM. The service needs to be turned on for the service to automatically start upon boot. No special configuration file changes are needed for the nginx component at this time.

[root@utah10 ~]# chkconfig nginx on[root@utah10 ~]# chkconfig ...cassandra 0:off1:off2:on3:on4:on5:on6:offcos_aicc 0:off1:off2:on3:on4:on5:on6:offcosd 0:off1:off2:on3:on4:on5:on6:off...cserver 0:off1:off2:on3:on4:on5:on6:off...nginx 0:off1:off2:on3:on4:on5:on6:off...


Chapter 2 Deploying COS Setting Up the External DNS Server

Note For previous COS systems, all data interfaces were published to DNS as part of the COS cluster hostname. For the data-mover architecture, only the service interfaces are published to DNS. Therefore, the internal interface IP addresses must be removed.

Verify nginx FunctionalityAfter applying the configuration changes, reboot the system and allow the services to start. You can then verify proper functionality of the nginx component using a simple curl command executed directly on the COS node. The following example exercises the Swauth list-accounts API:

[root@utah10 bin]# curl http://10.0.10.0/auth/v2 -H "X-Auth-Admin-User: .super_admin" -H "X-Auth-Admin-Key: rootroot" -v* About to connect() to 10.0.10.0 port 80 (#0)* Trying 10.0.10.0... connected* Connected to 10.0.10.0 (10.0.10.0) port 80 (#0)> GET /auth/v2 HTTP/1.1> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.6.0

zlib/1.2.3 libidn/1.18 libssh2/1.4.2> Host: 10.0.10.0> Accept: */*> X-Auth-Admin-User: .super_admin> X-Auth-Admin-Key: rootroot> < HTTP/1.1 200 OK< Server: nginx/1.6.0; cos/2.1.0-b91< Date: Fri, 30 Jan 2015 19:31:39 GMT< Content-Type: application/json; charset=utf-8< Content-Length: 35< Connection: keep-alive< {"accounts": [ {"name": "ryan"}]}* Connection #0 to host 10.0.10.0 left intact* Closing connection #0

Setting Up the External DNS ServerThe administrator is responsible for creating the transaction signature (TSIG) key for the TSIG algorithm on the external DNS server. Valid TSIG algorithms are hmac-md5, hmac-sha1, hmac-sha224, hmac-sha256, hmac-sha384, and hmac-sha512. The following procedure shows how to create the TSIG key for the hmac-md5 algorithm.

Step 1 Generate the TSIG key by entering the following command:

dnssec-keygen -a HMAC-MD5 -b 128 -n HOST testdns.com.

where:

• HMAC-MD5 is the TSIG algorithm.

• 128 is the number of bits in the key.



• testdns.com. is the name of the key. The name of the key, the domain name of the PAM, and the DNS zone in the external DNS server should all be the same (in this example, testdns.com).

• The command must end with a period (.), which is required when generating the key.

Sample output:

dnssec-keygen -a HMAC-MD5 -b 128 -n HOST testdns.com.

Ktestdns.com.+157+05519

This command creates a key file and a private key file.

• Sample .key file:

Ktestdns.com.+157+05519.key

testdns.com. IN KEY 512 3 157 ujLdXfCZenQZQKZlFy42fw==

• Sample .private key file:

Ktestdns.com.+157+05519.private

\Private-key-format: v1.3

Algorithm: 157 (HMAC_MD5)

Key: ujLdXfCZenQZQKZlFy42fw==

Bits: AAA=

Created: 20140325141250

Publish: 20140325141250

Activate: 20140325141250

Step 2 Create the key file (in this example, testdns.com.key) in the /etc/ directory.

Sample key file:

testdns.com.key

key testdns.com. {

algorithm hmac-md5;

secret “ujLdXfCZenQZQKZlFy42fw==“;

};

where:

• hmac-md5 is the TSIG algorithm.

• ujLdXfCZenQZQKZlFy42fw== is the TSIG key.

Step 3 Add the key file path to the /etc/named.conf file by inserting the following line:

include “/etc/testdns.com.key“;

Step 4 Configure the DNS zones (used when deploying the PAM) on the external DNS server as shown in the following examples.

a. Update the /etc/named.conf file with the details of the DNS zone (in this example, testdns.com) and reverse zone, for all interfaces.

In the following sample information:

– The DNS zone is related to the 172.20.216.xx subnet, which can be the Management interface.

– For the Data In and Data Out interfaces, similar information related to the reverse zone must be added.



– Data In is related to the 15.1.1.x subnet.

– Data Out is related to the 25.1.1.x subnet.

– testdns.com. is the key name.

DNS Zone Detailszone testdns.com IN {type master;file “slaves/db.testdns.com“;allow-update { key “testdns.com.“; };notify yes;};

Reverse Zone Details (Management Interface)zone 216.20.172.IN-ADDR.ARPA IN {type master;file “slaves/db.216.20.172“;allow-update { key “testdns.com.“; };notify yes;};

Reverse Zone Details (Data In Interface)zone 1.1.15.IN-ADDR.ARPA IN {type master;file “slaves/db.1.1.15“;allow-update { key “testdns.com.“; };notify yes;};

Reverse Zone Details (Data Out Interface)zone 1.1.25.IN-ADDR.ARPA IN {type master;file “slaves/db.1.1.25“;allow-update { key “testdns.com.“; };notify yes;};

b. Create db.* files in the /var/named/slaves directory in the external DNS server for the DNS zone and reverse zone, for all interfaces.

Sample db File for DNS Zone (db.testdns.com)@ 86400 IN SOA dns pam 2014031001 3600 1800 604800 86400

@ 86400 IN NS dns

dhcp 86400 IN CNAME dns

dns 86400 IN A pam_ip_address

ntp 86400 IN CNAME dns

Sample db File for Reverse Zone of Management Interface (db.216.20.172)$ORIGIN .

$TTL 86400 ; 1 day

216.20.172.IN-ADDR.ARPA IN SOA dns.testdns.com. pam.testdns.com. (



2012071021 ; serial

3600 ; refresh (1 hour)

1800 ; retry (30 minutes)

604800 ; expire (1 week)

86400 ; minimum (1 day)

)

NS dns.testdns.com.

$ORIGIN 216.20.172.IN-ADDR.ARPA.

$TTL 7200 ; 2 hours

xx PTR dns.testdns.com. //xx = last two digits of the PAM IP

Sample db File for Reverse Zone for Data In Interface (db.1.1.15)

$ORIGIN .

$TTL 86400 ; 1 day


2012071021 ; serial





)

NS dns.testdns.com.


$TTL 7200 ; 2 hours

1 PTR dns.testdns.com.

Sample db File for Reverse Zone for Data Out Interface (db.1.1.15)

$ORIGIN .

$TTL 86400 ; 1 day


2012071021 ; serial





)

NS dns.testdns.com.



Chapter 2 Deploying COS Editing NTP Server Information

$TTL 7200 ; 2 hours

1 PTR dns.testdns.com.

Step 5 Make sure that the PAM and the external DNS server are in sync for the time and date.

Editing NTP Server Information

Note NTP servers are configured when the PAM is deployed. You cannot add or delete NTP servers using this procedure.

To edit NTP server information using the COS Service Manager GUI, follow these steps:

Step 1 Log in to the COS Service Manager GUI as described in Using the COS Service Manager GUI, page A-2.

Step 2 Go to Infrastructure > Platform Services.

In the NTP Servers table, the following information is displayed for each NTP server:

• Name (required)

– The name of the NTP server is a string of up to 30 characters. Acceptable characters include uppercase and lowercase letters, numbers, periods (.), dashes (-), and underscores (_).

– The name must not begin with a period (.), and it is not case-sensitive.

• Description — The description is a string of up to 100 characters, and can include uppercase or lowercase letters, numbers, and any special characters.

• Region — The region in which the NTP server resides.

• Servers — The hostname or IPv4 address of the NTP server.

Step 3 Edit the Description, Region, and Servers fields of an NTP server as required.

Viewing DNS Server Information

Note DNS servers are configured when the PAM is deployed.

To view DNS server information using the COS Service Manager GUI, follow these steps:


Step 2 Go to Infrastructure > Platform Services.

In the DNS Servers table, the following information is displayed for each DNS server:

• Name (required)


Chapter 2 Deploying COS Configuring COS

– The name of the DNS server is a string of up to 35 characters. Acceptable characters include uppercase and lowercase letters, numbers, periods (.), dashes (-), and underscores (_).

– The name cannot begin with a period (.), and it is not case-sensitive.

• Description — The description is a string of up to 100 characters, and can include uppercase or lowercase letters, numbers, and any special characters.

• Region — Region in which the DNS server resides.

• IPv4 Address — IPv4 address of the DNS server.

• TSIG — Transaction Signature (TSIG) of the DNS server.

• TSIG Algorithm — Name of the TSIG algorithm used by the DNS server.

Configuring COSTo configure COS using the COS Service Manager GUI, follow these steps:


Step 2 Go to Infrastructure > Regions & Zones.

A zone is a set of Cloud platform components (compute, network, storage, and security) that are fate-shared. The zone can be mapped to the underlying Cloud platform provider, such as a datacenter in vCenter, an Availability Zone, or any other combination of fate-shared Cloud resource topologies. Each zone is associated with one Cloud Controller.

A region is made up of one or more zones. It is an abstract representation of the underlying Cloud platform. A region can be associated with a geographical region, one or more datacenters, or a service area. Release 3.0.1 of the COS software supports only one region.

Step 3 Expand a region to display the zones associated with that region.

Step 4 Click Add Row to add zones to the region, as needed.

Step 5 Go to Infrastructure > IP Address Pools and Networks to define IP pools for the COS node.

Step 6 Go to Infrastructure > COS Cluster and create a COS cluster.

Step 7 Go to Services > Cloud Object Stores, click the COS service instance, and create one COS endpoint.

Step 8 On the COS node, execute /opt/cisco/cos/config/cosinit and configure the IP pool name, DocServer IP and Port (which is the COS Service Manager management IP). This will register the COS node with the COS Service Manager.

Step 9 Go to Infrastructure > COS Nodes and configure Cluster Name by choosing the created cluster name from the drop-down list.

Step 10 Ensure that the Admin State for the node is set to Inservice. Click Save.

Step 11 Go to Services > Overview and confirm that you can see the registered COS Node.



Configuring the Service Instance TemplateTo configure the COS service instance template using the COS Service Manager GUI, do the following:


Step 2 Go to Services > Cloud Object Storage. The Service Summary page opens, displaying a list of COS service instances.

Step 3 Click the COS service instance to be configured. The Service Definition tab for that instance opens, containing the General and Service Endpoints areas.

Step 4 The fields of the General area are listed below. You can edit them appropriately.

• Title — The title of the COS service. The default title is An unused COS Service.

• Description — A brief description of the COS service. The default description is COS Service.

• Admin State — The administrative state of the COS service. From the drop-down list, you can choose to Enable or Disable the service.

• Service Template — Currently, this read-only field identifies the default service template, Cisco Object Store (COS) Service.

Step 5 There is no default COS endpoint when the PAM is brought up, so you must create one manually. The following parameters are displayed for the service endpoint and can be edited appropriately:

• Name — The name of the service endpoint is set to the default value ce1 and cannot be changed.

• Description — A brief description of the service endpoint.

• Region — The region to which the service endpoint belongs.

• Min Nodes — The minimum number of nodes that must be associated the service endpoint.

• Desired Nodes — The desired number of nodes for the service endpoint.

• Max Nodes — The maximum number of nodes that can be associated with the service endpoint.

• Max Storage — This field is to be ignored for Release 2.0.1.

• Cluster Name — No default cluster is loaded when the PAM is brought up, so you must create one manually.

• Auth Profile — The default value is auth-1.

• State — The state of the service endpoint. From the drop-down list, choose Disabled or Enabled.

Step 6 For your changes to take effect, click Save. To discard your changes, click Cancel.

Configuring IP PoolsBefore installing COS nodes, you must create a network and configure IP pools using the COS Service Manager GUI.


Step 2 Go to Infrastructure > IP Address Pools and Networks and create a suitable network for the IP pools.



Step 3 To add a new IP pool, click IP Address Pools > Add Row.

Step 4 In the Name field, enter a unique name for the IP pool. In the Description field, enter a brief description for the IP pool.

Step 5 To associate an IP address range with the IP pool, click IP Address Ranges > Add Row.

Step 6 Enter appropriate values in the fields described below:

Note If you attempt to initialize a COS node using an IP pool with an insufficient number of available IP addresses, the initialization fails and an Event is generated. For more information on this class of events, see COS AIC Server Events, page 3-6.


Note After you have associated an IP pool with a COS node, do not edit or delete the pool before dissociating the pool from the node.

Editing or Deleting IP PoolsTo edit or delete configured IP pools using the COS Service Manager GUI, follow these steps:


Step 2 Go to Infrastructure > IP Pools and Networks.

Step 3 Before editing or deleting an IP pool, ensure that it is not serving any COS nodes.

a. Dissociate any COS nodes associated with the IP pool to be edited or deleted, and link these nodes to another IP pool.

Step 4 Check the box against the name of the IP pool to be edited or deleted.

Step 5 Click Edit or Delete.

Table 2-1 IP Address Ranges Fields

Field Description

Range Start The first IP address of the range.

Range End The last IP address of the range.

Netmask The netmask for the range.

Gateway The Gateway for the range.



Configuring a COS ClusterAs part of the COS installation, a single default node cluster is created. The cluster can be configured using the COS Service Manager GUI.

To view or edit the settings of a COS node cluster, follow these steps:


Step 2 Go to Infrastructure > COS Clusters.The default COS node cluster is displayed.

Step 3 Edit the fields described below based on your deployment.


COS Node Initial ConfigurationCOS nodes ship with a pre-installed image. When provisioning a COS node on a network, you can upgrade the node to a different image. After installing the image and rebooting the node, you are prompted to start configuration by running the cosinit script.

• Enter yes to run cosinit immediately.

• If you wish to configure the node later, enter no. You can run the cosinit later using the following command: run /opt/cisco/cos/config/cosinit

When it runs, the cosinit script performs the initial setup and adds the node to the COS management system.

The cosinit script can be run with the options shown below:

cosinit [–skipnw] [–input <configFile>] [–help]

• [–skipnw] — Skip network configurations

• [–input <configFile>] — Specify an input file for configuration

• [–help] — Displays the usage

Example 1 /opt/cisco/cos/config/cosinit -skipnw

This is used to configure IP pool and DocServer, and register the cos node into COS Service Manager.

Example 2/opt/cisco/cos/config/cosinit -input configFile

Table 2-2 COS Node Cluster Fields

Field Description

Authentication FQDN The FQDN for COS authentication requests.

Storage FQDN Currently, this must be the same as the Authentication FQDN.



This enables cosinit to run with a config file. All the parameters will be read from the config file.

Sample configFileInterfaceConfig : skip

(Optional; if the line above is added, management network configuration is skipped.)

IpAddress : 172.22.99.218NetMask : 255.255.254.0Broadcast : 172.22.99.255DefaultGateway : 172.22.98.1HostName : CDE460-218PartNum : CDE460-4R1DocServerHost : 172.22.116.35DocServerPort : 5087BaudRate : 9600eth4 : 4eth5 : 4eth6 : 4eth7 : 4

Executing the cosinit script completes the following tasks:

• Configuring the management interface — ethX, IP, netmask, broadcast.

• Configuring the default gateway.

• Configuring the hostname.

• Mapping the network interfaces to IP addresses in the network.

• Configuring Doc Server IP address and Port.

• Recording appliance model name.

• Configuring the baud rate by reading the value in install_baud_rate. If the install_baud_rate file is not found under the /root directory, the operator is prompted to enter a baud rate during cosinit execution or specify the value in the configFile.

When the cosinit script is successfully executed:

• /etc/sysconfig/network-scripts/ifcfg-eth0 is generated and the management interface configuration is saved in it.

• /etc/sysconfig/network is generated, and the hostname and default gateway info are saved in it.

• /etc/hosts is generated and the host info is saved in it.

• Network is restarted.

• /tmp/.cosnodeinit file is created with the configuration specified in cosinit.

• The baud rate is updated in the /boot/grub/grub.conf file.

• The COS AIC client and other applications are started.

• The COS AIC client reads the cosnodeinit generated by cosinit.

• Using the Doc Server IP address and port mentioned in cosnodeinit, the AIC client connects to the Doc Server.

• The AIC client sends the cosannounce to the Doc Server. cosannounce is a partially populated smcosnode document.

The COS node is now registered with the COS management system. You can configure the node using the COS Manager GUI, add it to a COS cluster, and make it operational.



Automated Node Configuration (Optional)

Beginning with COS 3.0.1, you can use automated configuration by providing a file to cosinit that contains a ClusterName and an IP Pool reference for at least one service interface. This enables the system to configure the COS node without further intervention through the Service Manager GUI and API.

Example

Run cosinit with the input file:

/opt/cisco/cos/config/cosinit -input cosinit_input_file

[root@cos470-1 ~]# cat cosinit_autoInterfaceConfig : skipPartNum : C3160-R1ClusterName :cluster-1DocServerHost : 172.22.98.59DocServerPort : 5087eth1 : colusa-57eth2 : colusa-57eth4 : colusa-57eth5 : colusa-57

This results in one of the following:

• If ClusterName has a value, automated configuration is triggered. Then, if at least one service interface has an IP Pool configured, the AIC Client sets the adminstate to inService in the smcosnode document before sending cosannounce to DocServer. Otherwise, adminstate is set to Maintenance.

The AIC Server handles the rest of the configuration automatically by assigning an IP address from the specified IP Pool and adding the node to the cluster. The AIC Client then writes the necessary configuration files to the COS node as usual. When configuration is complete, the AIC Client automatically starts the cassandra, cosd and cserver cos node services.

• If ClusterName has no value, manual configuration is needed. You must manually set the COS node adminstate from Maintenance to inService, assign the node to a cluster, and enable the service interfaces from the GUI.

Note When using automated configuration of multiple COS nodes, configure the first node, and then wait before configuring the second node until the Cassandra database service to appear as Running in the GUI of the first node configured. Otherwise, there may be unexpected behavior in the seed list configuration for the Cassandra database of the nodes added after the first node.

Adding a COS Node to a COS ClusterAfter a COS node is installed and initialized, it must be added to a COS cluster to begin servicing COS requests.

To add a node to the COS cluster using the COS Service Manager GUI, do the following:


Step 2 Go to Infrastructure > COS Nodes.



Step 3 Check the box against the name of the node you want to edit and click Edit.

Step 4 From the Cluster Name drop-down list, choose the created COS cluster.


When a COS node is added to a COS cluster, the following files are written on to the node:

• setupfile — This file holds the primary configuration data for the COS node and is required by CServer.

• RemoteServers — This file holds the service interface IP addresses for all of the COS nodes in the cluster.

• SubnetTable — This file holds the IP, Netmask, Gateway and Network data for each service interface of a COS node.

• cosd.conf — This file holds the configuration for the cosd service.

• cassandra.yaml — This file, located in /etc/cassandra/conf/, is responsible for the configuration of the Cassandra service.

When the node Admin State is set to Inservice, all enabled service interfaces of that node are written to the DNS (internal or external).

Note To avoid possible bootstrapping issues with the Cassandra service, be sure to provide (or use a script to provide) a delay of at least two minutes between adding two nodes in sequence to the cluster.

If the replication factor is too high, use the following script on any one node in the cluster to adjust it:

sh /opt/cisco/cos-aic-client/cassandra/cassandra-adjust-replication.sh { 1 | 2 | 3 }

For the final argument, use 1 for one node, 2 for two nodes, and 3 for three or more nodes.

Configuring a COS NodeAfter a COS Node is installed and initialized, you can modify its configuration parameters using the COS Service Manager GUI as follows:




Step 4 Edit the fields described below based on your requirements.

Table 2-3 COS Node Fields

Field Description

Description A brief description of the node.

Model The part number of the node.

Zone The zone to which the node belongs.



Note The Admin State of a COS node must be set to Maintenance before removing the node from a COS cluster.

Step 5 To modify the service interfaces of the node, follow the procedure Configuring the Service Interface of a COS Node, page 2-33.


Configuring the Service Interface of a COS NodeTo modify the settings of a service interface of a functioning COS node using the COS Service Manager GUI, follow these steps:




Step 4 Check the box against the name of service interface you wish to edit and click Edit.

Step 5 Edit the fields described below based on your requirements.


Configuring Cassandra Database MaintenanceThe Cassandra database requires periodic maintenance with an anti-entropy repair that must be manually configured to execute every two days on each COS node. For more information on the Cassandra repair process, see the Cassandra 2.1 documentation available at:

Cluster Name The COS cluster to which the node belongs.

Admin State Indicates whether the node is Inservice or under Maintenance.

Table 2-3 COS Node Fields

Field Description

Table 2-4 Service Interface Fields

Field Description

IP Pool The IP pool from which this interface will be assigned an IP address. You may associate a different IP pool with this interface only when the node Admin State is set to Maintenance.

Enabled From the drop-down list, choose True or False to enable or disable the service instance, respectively.



www.datastax.com/documentation/cassandra/2.1/

To execute the periodic repair, we recommend configuring a CRON job on each node, and scheduling the CRON jobs on the nodes to begin at different times so as to avoid the repair running concurrently on more than one node at a time. As a reference, we recommend providing 15 minutes between scheduling of the repair operation on each node.

The command syntax for the repair is:

/usr/bin/nodetool repair -par -inc


Documents

Deploying COS · Before deploying the COS, you must complete the following preliminary tasks: Step 1 Prepare your COS VMware datacenter topologies and networks. The COS requires the