Server Performance

Home

Purchase PDFs

Forums

About

Quick HOWTO : Ch22 : Monitoring Server Performance

Contents

[hide]

1 Introduction

2 Debian / Ubuntu Differences

3 SNMP

o 3.1 OIDs And MIBs

3.1.1 Figure 22-1 SNMP OID Structure

3.1.2 Table 22-1 OIDs And Their Equivalent MIBs

o 3.2 SNMP Community Strings

o 3.3 SNMP Versions

o 3.4 Doing SNMP Queries

3.4.1 Installing SNMP Utilities on a Linux Server

3.4.2 Managing the snmpd Server

3.4.3 Allowing SNMPd to Listen on Multiple Interfaces

3.4.4 SNMP Utilities Command Syntax

3.4.5 Configuring Simple SNMP on a Linux Server

3.4.6 SNMP On Other Devices

o 3.5 Basic SNMP Security

3.5.1 SNMP Versions 1 and 2

3.5.2 SNMP Version 3

o 3.6 Simple SNMP Troubleshooting

Share on facebookShare on twitter More Sharing Services21

Search

Other Linux Home Networking Topics

Introduction to Networking

Linux Networking

Simple Network Troubleshooting

Troubleshooting Linux with Syslog

Installing Linux Software

The Linux Boot Process

Configuring the DHCP Server

Linux Users and sudo

Windows, Linux and Samba

Sharing Resources with Samba

Samba Security and Troubleshooting

Linux Wireless Networking

Linux Firewalls Using iptables

Linux FTP Server Setup

Telnet, TFTP and xinetd

Secure Remote Logins and File Copying

Configuring DNS

Dynamic DNS

4 MRTG

o 4.1 Figure 22-2 A Typical MRTG Web Page

o 4.2 MRTG Download and Installation

o 4.3 MRTG Configuration Files

4.3.1 SNMPv1

4.3.2 SNMPv2

4.3.3 SNMPv3

o 4.4 Setting MRTG Polling Intervals and Web Page Locations

o 4.5 Getting MRTG To Poll Multiple Devices

o 4.6 Configuring Apache To Work With MRTG

4.6.1 Basic Security

o 4.7 How To View The MRTG Graphs In Your Web Browser

o 4.8 Using MRTG To Monitor Other Subsystems

o 4.9 Troubleshooting MRTG

4.9.1 Basic Steps

4.9.2 Setting The Correct Character Set

4.9.3 Incorrect SNMPv3 Engine ID

4.9.4 Fedora Core 1 MRTG Errors With Net-SNMP

5 Webalizer

o 5.1 How To View Your Webalizer Statistics

6 The top Command

7 The vmstat Command

8 The free Utility

9 Conclusion

Introduction

You can monitor your system's Web performance quite easily

with graphical Linux tools. You'll learn how to use several in this

chapter, including MRTG, which is based on SNMP and

monitors raw network traffic, and Webalizer, which tracks Web

site hits.

Debian / Ubuntu Differences

The Apache Web Server

Configuring Linux Mail Servers

Monitoring Server Performance

Advanced MRTG For Linux

The NTP Server

Network-Based Linux Installation

Linux Software RAID

Expanding Disk Capacity

Managing Disk Usage with Quotas

Remote Disk Access with NFS

Configuring NIS

Centralized Logins Using LDAP and RADIUS

Controlling Web Access with Squid

Modifying the Kernel to Improve Performance

Basic MySQL Configuration

LHN Linux Forums - Latest Threads

JOsh Martin Lebanon! newbie

(General Chat) JOsh Martin Jew. Hi,

im new to this site regards JOsh

Martin Israel

Problem in NIC configuration

(Redhat / Fedora) I have centos 5.4 to

be used as proxy and I have 2 NIC I

need to configure eth0 as apuplic ip

Example: 188.177.166.155 GW

188.177.166.1 eth1...

James W Horne Jr New Jersey

Greetings (General Chat)

James W Horne jr new jersey!

newbie (General Chat)

network card stop working (Linux -

Hardware, Networking & Security)

Printing a topic to a printer.

This chapter focuses on Fedora / CentOS / RedHat for simplicity

of explanation. Whenever there is a difference in the required

commands for Debian / Ubuntu variations of Linux it will be

noted.

The universal difference is that the commands shown are done

by the Fedora / CentOS / RedHat root user. With Debian /

Ubuntu you will either have to become root using the "sudo su

" command or you can temporarily increase your privilege level

to root using the "sudo " command.

Here is an example of how to permanently become root:

user@ubuntu:~$ sudo su -

[sudo] password for peter:

root@ubuntu:~#

Here is an example of how to temporarily become root to run a

specific command. The first attempt to get a directory listing fails

due to insufficient privileges. The second attempt succeeds

when the sudo keyword is inserted before the command.

user@ubuntu:~$ ls -l /var/lib/mysql/mysql

ls: cannot access /var/lib/mysql/mysql:

Permission denied

user@ubuntu:~$ sudo ls -l /var/lib/mysql/mysql

[sudo] password for peter:

total 964

-rw-rw---- 1 mysql mysql 8820 2010-12-19

23:09 columns_priv.frm

-rw-rw---- 1 mysql mysql 0 2010-12-19

23:09 columns_priv.MYD

-rw-rw---- 1 mysql mysql 4096 2010-12-19

23:09 columns_priv.MYI

-rw-rw---- 1 mysql mysql 9582 2010-12-19

23:09 db.frm

...

...

...

user@ubuntu:~$

(Announcements and Suggestions) A

lot of my time is spent away from my

own computer and I need to be able

to print out the articles from LHN.

For instance, I have not been able

to...

iptables, forwarding between

interfaces (Linux - Hardware,

Networking & Security) Hi all, I'm

trying to setup a router by hand using

iptables but after a lot of examples

i've tried from google i cant get

anything to work, my...

thomas polano leasing newbie here.

(General Chat)

Boshoff Beyers Greetings (General

Chat)

Boshoff Beyers Newbie.. (General

Chat)

Emiola Tobi Samson! newbie

(General Chat) Emiola Tobi Samson.

Hi, im new to this site regards Emiola

Tobi Samson

Now that you have got this straight, lets continue with the

discussion.

SNMP

Most servers, routers and firewalls keep their operational

statistics in object identifiers (OIDs) that you can remotely

retrieve via the Simple Network Management Protocol (SNMP).

For ease of use, equipment vendors provide Management

Information Base (MIB) files for their devices that define the

functions of the OIDs they contain. That's a lot of new terms to

digest in two sentences, so take a moment to look more closely.

OIDs And MIBs

OIDs are arranged in a structure of management information

(SMI) tree defined by the SNMP standard. The tree starts from a

root node, which then descends through branches and leaves

that each add their own reference value to the path separated

by a period.. Figure 22-1 shows an OID structure in which the

path to the enterprises OID branch passes through the org, dod,

internet, and private branches first. The OID path for enterprises

is, therefore, 1.3.6.1.4.1.

Figure 22-1 SNMP OID Structure

Management Information Bases (MIBs) are text definitions of

each of the OID branches. Table 22-1 shows how some

commonly used OIDs map to their MIB definitions. For example,

the SMI org MIB defines all the topmost OIDs found at the next

layer, which is named dod; the internet MIB under dod defines

the function of the topmost OIDs in the directory, mgmt,

experimental, and private branches. This MIB information is very

useful for SNMP management programs, enabling you to click

on an OID and see its value, type, and description.

Table 22-1 OIDs And Their Equivalent MIBs

OID MIB

1.3 org

1.3.6 dod

1.3.6.1 internet

1.3.6.1.1 directory

1.3.6.1.2 mgmt

1.3.6.1.3 experimental

1.3.6.1.4 private

1.3.6.1.4.1 enterprises

You can refer to an OID by substituting the values in a branch

with one of these more readable MIB aliases. For example, you

can reference the OID 1.3.6.1.4.1.9.9.109.1.1.1.1.5 as

enterprises.9.9.109.1.1.1.1.5.1 by substituting the branch name

(enterprises) for its OID numbers (1.3.6.1.4.1).

Remember, only the OID value at the very tip of a branch, the

leaf, actually has a readable value. Think of OIDs like the

directory structure on a hard disk. Each branch is equivalent to a

subdirectory, and the very last value at the tip (the leaf)

correlates to a file containing data.

The Linux snmpget command outputs the value of a single leaf,

and the snmpwalk command provides the values of all leaves

under a branch. I'll discuss these commands later; for now, all

you need to know is that the command output frequently doesn't

list the entire OID, just the MIB file in which it was found and the

alias within the MIB. For example

SNMPv2-MIB::sysUpTime.0

Here the OID value was found in the SNMPv2-MIB file and

occupies position zero in the sysUpTime alias.

Equipment manufacturers are usually assigned their own

dedicated OID branch under the enterprises MIB, and they must

also provide information in universally accepted OIDs for ease of

manageability. For example, NIC interface data throughput

values must always be placed in a predefined location in the

general tree, but a memory use value on a customized

processor card may be defined in a MIB under the

manufacturer's own OID branch.

SNMP Community Strings

As a security measure, you need to know the SNMP password,

or community string, to query OIDs. There are a number of

types of community strings, the most commonly used ones are

the Read Only or "get" community string that only provides

access for viewing statistics and system parameters. In many

cases the Read Only community string or password is set to the

word "public;" you should change it from this easy-to-guess

value whenever possible. The Read/Write or "set" community

string is for not only viewing statistics and system parameters

but also for updating the parameters.

SNMP Versions

There are currently three versions of SNMP.

1. SNMP Version 1: The first version of SNMP to be

implemented, version 1 was designed to be a protocol

to provide device statistics and error reporting without

consuming a lot of system resources. Security was

limited to community strings and access controls based

on the IP address of the querying server. Data

communication wasn't encrypted.

2. SNMP Version 2: The second version of SNMP, often

referred to as v2c, expanded the number of supported

error codes, increased the size of counters used to

track data, and had the ability to do bulk queries that

more efficiently loaded response packets with data.

SNMP v2c is backward compatible with version 1.

3. SNMP Version 3: This version provides greater security

and remote configuration capabilities than its

predecessors. Access isn't limited to a single

community string for read-only and read/write access,

as usernames and passwords have been introduced.

Views of OIDs in a MIB can also be limited on a per-

user basis. Support for encrypted SNMP data transfer

and transfer error detection is also provided.

Remember their differences, because you will need to specify

the version number when doing SNMP queries.

Doing SNMP Queries

Configuring SNMP on a server isn't hard, but it does require a

number of detailed steps.

Installing SNMP Utilities on a Linux Server

If you intend to use your Linux box to query your network

devices, other servers or even itself using SNMP commands,

MRTG or any other tool, you need to have the SNMP client

programs installed. The devices you query need to have a

constantly running SNMP server application running to provide

the clients with the data they are requesting.

Youll need to install both the client and server software on your

Linux server to maximize your understanding of all the examples

provided.

Note: With Fedora / Redhat the packages to install would be

net-snmp-utils (client) and net-snmp (server) and with Debian /

Ubuntu the packages are snmp (client) and snmpd (server).

Most RedHat and Fedora Linux software product packages are

available in the RPM format, whereas Debian and Ubuntu Linux

use DEB format installation files. When searching for these

packages, remember that the filename usually starts with the

software package name and is followed by a version number, as

in net-snmp-utils-5.1.1-2.i386.rpm. (If you need an installation

refresher, see Chapter 6, "Installing Linux Software").

Managing the snmpd Server

Managing the snmpd daemon is easy to do, but the procedure

differs between Linux distributions. Here are some things to

keep in mind.

1. Firstly, different Linux distributions use different daemon

management systems. Each system has its own set of

commands to do similar operations. The most

commonly used daemon management systems are

SysV and Systemd.

2. Secondly, the daemon name needs to be known. In this

case the name of the daemon is snmpd.

Armed with this information you can know how to:

1. Start your daemons automatically on booting

2. Stop, start and restart them later on during

troubleshooting or when a configuration file change

needs to be applied.

For more details on this, please take a look at the "Managing

Daemons" section of Chapter 6 "Installing Linux Software"

Note: Remember to configure your daemon to start

automatically upon your next reboot.

Allowing SNMPd to Listen on Multiple Interfaces

The /etc/sysconfig/snmpd file (Redhat / CentOS /Fedora) and

the /etc/default/snmpd file (Debian / Ubuntu) define some

universal operating parameters for your SNMP server to work.

One of its functions is to define which network interfaces on

which SNMP will listen for queries. In many distributions the

default behavior is to only listen on localhost (127.0.0.1). The

SNMPDOPTS line in the snmpd file defines the IP addresses on

which to listen. In this example, only localhost is defined.

#

# File: snmpd

#

SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -g snmp -

I -smux -p /var/run/snmpd.pid 127.0.0.1'

This can be verified using the netstat command which shows

snmpd listening only on localhost.

root@ubuntu:/tmp# netstat -au | grep snmp

udp 0 0 localhost:snmp *:*

root@ubuntu:/tmp#

You can activate all interfaces by removing the reference to

127.0.0.1, like this:

#

# File: snmpd

#

SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -g snmp -

I -smux -p /var/run/snmpd.pid'

This can be verified using the netstat command which shows

snmpd listening on all addresses (*).

root@ubuntu:/tmp# netstat -au | grep snmp

udp 0 0 *:snmp *:*

root@ubuntu:/tmp#

Note: Remember to restart snmpd for this to take effect.

SNMP Utilities Command Syntax

The SNMP utility tools package installs a number of new

commands on your system for doing SNMP queries, most

notably snmpget for individual OIDs and snmpwalk for obtaining

the contents of an entire MIB. Both commands require you to

specify the community string with a -c operator. They also

require you to specify the version of the SNMP query to be used

with a -v 1, -v 2c, or -v 3 operator for versions 1, 2, and 3,

respectively. The first argument is the name or IP address of the

target device and all other arguments list the MIBs to be

queried.

This example gets all the values in the interface MIB of the local

server using SNMP version 1 and the community string of

craz33guy.

[root@bigboy tmp]# snmpwalk -v 1 -c craz33guy

localhost interface

...

...

IF-MIB::ifDescr.1 = STRING: lo

IF-MIB::ifDescr.2 = STRING: eth0


...

...

IF-MIB::ifPhysAddress.1 = STRING:


0:9:5b:2f:9e:d5


0:b0:d0:46:32:71

...

...

[root@bigboy tmp]#

Upon inspecting the output of the snmpwalk command, you can

see that the second interface seems to have the name eth0 and

the MAC address 0:9:5b:2f:9e:d5. You can now retrieve the

individual MAC address using the snmpget command.

[root@bigboy tmp]# snmpget -v 1 -c const1payted

localhost ifPhysAddress.2


0:9:5b:2f:9e:d5

[root@bigboy tmp]#

You can confirm this information using the ifconfig command for

interface eth0; the very first line shows a matching MAC

address.

[root@bigboy tmp]# ifconfig -a eth0

eth0 Link encap:Ethernet HWaddr

00:09:5B:2F:9E:D5

inet addr:216.10.119.244

Bcast:216.10.119.255 Mask:255.255.255.240

...

...

[root@bigboy tmp]#

You'll now see how you can configure SNMP on your Linux

server to achieve these results.

Configuring Simple SNMP on a Linux Server

By default Fedora, installs the net-snmp package as its SNMP

server product. This package uses a configuration file named

/etc/snmp/snmpd.conf in which the community strings and other

parameters may be set. The version of the configuration file that

comes with net-snmp is quite complicated. I suggest archiving it

and using a much simpler version with only a single line

containing the keyword rocommunity followed by the community

string. Here is an example.

1) Save the old configuration file and make sure the new

configuration file has the correct permissions

[root@bigboy tmp]# cd /etc/snmp/

[root@bigboy snmp]# mv snmpd.conf

snmpd.conf.old

[root@bigboy snmp]# touch snmpd.conf

[root@bigboy snmp]# chmod 600 snmpd.conf

[root@bigboy snmp]# vi snmpd.conf

2) Enter the following line in the new configuration file to set the

Read Only community string to craz33guy.

rocommunity craz33guy

3) Configure Linux to start SNMP services on each reboot with

the chkconfig or sysv-rc-conf command depending on your linux

variation:

For Fedora / RedHat / CentOS it would be:

[root@bigboy root]# chkconfig snmpd on

For Ubuntu / Debian it would be:

user@ubuntu:~$ sudo sysv-rc-conf snmpd on

4) Start SNMP to load the current configuration file.

[root@bigboy root]# systemctl start

snmpd.service

Starting snmpd: [ OK ]

[root@bigboy root]#

5) Test whether SNMP can read the system and interface MIBs

using the snmpwalk command.

[root@bigboy snmp]# snmpwalk -v 1 -c craz33guy

localhost system

SNMPv2-MIB::sysDescr.0 = STRING: Linux bigboy

2.4.18-14 #1 Wed Sep 4 11:57:57 EDT 2002 i586

SNMPv2-MIB::sysObjectID.0 = OID: NET-SNMP-

MIB::netSnmpAgentOIDs.10

SNMPv2-MIB::sysUpTime.0 = Timeticks: (425)

0:00:04.25

SNMPv2-MIB::sysContact.0 = STRING:

root@localhost

SNMPv2-MIB::sysName.0 = STRING: bigboy

...

...

...

[root@bigboy snmp]# snmpwalk -v 1 -c craz33guy

localhost interface

IF-MIB::ifNumber.0 = INTEGER: 3

IF-MIB::ifIndex.1 = INTEGER: 1



IF-MIB::ifDescr.1 = STRING: lo

IF-MIB::ifDescr.2 = STRING: wlan0


...

...

...

[root@bigboy snmp]#

Now that you know SNMP is working correctly on your Linux

server, you can configure SNMP statistics gathering software,

such as MRTG, to create online graphs of your traffic flows.

SNMP On Other Devices

In the example, you were polling localhost. You can poll any

SNMP-aware network device that has SNMP enabled. All you

need is the IP address and SNMP Read Only string and you'll

be able to get similar results. Here is an example of a query of a

device with an IP address of 192.168.1.1.

[root@bigboy snmp]# snmpwalk -v 1 -c chir1qui

192.168.1.1 interface

Note: When issuing snmpwalk and snmpget commands,

remember to use the proper version switch (-v 1, -v 2c, or -v 3)

for the version of SNMP you're using.

Basic SNMP Security

The security precautions that need to be taken with SNMP vary

depending on the version you are using. This section outlines

the basic steps for protecting your MIB data.

SNMP Versions 1 and 2

The most commonly supported versions of SNMP don't encrypt

your community string password so you shouldn't do queries

over insecure networks, such as the Internet. You should also

make sure that you use all reasonable security measures to

allow queries only from trusted IP addresses either via a firewall

or the SNMP security features available in the snmp.conf file.

You can also configure your server to use the TCP wrappers

feature outlined in Appendix I, "Miscellaneous Linux Topics," to

limit access to specific servers without the need of a firewall.

In case you need it, the snmpd.conf file can support limiting MIB

access to trusted hosts and networks.

The snmpd.conf file has two security sections; a section with

very restrictive access sits at the top of the file and is

immediately followed by a less restrictive section. The example

that follows is a modification of the less restrictive section. You

will have to comment out the more restrictive statements at the

top of the file for it to work correctly.

## sec.name source

community

## ======== ======

=========

com2sec local localhost craz33guy

com2sec network_1 172.16.1.0/24 craz33guy

com2sec network_2 192.168.2.0/24 craz33guy

## Access.group.name sec.model

sec.name

## ================= =========

========

group MyROGroup_1 v1

local

group MyROGroup_1 v1

network_1

group MyROGroup_2 v2c

network_2

## MIB.view.name incl/excl MIB.subtree

mask

## ============== ========= ===========

====

view all-mibs included .1

80

## MIB

## group.name context sec.model

sec.level prefix read write notif

## ========== ======= =========

========= ====== ==== ===== =====

access MyROGroup_1 "" v1 noauth

exact all-mibs none none

access MyROGroup_2 "" v2c noauth

exact all-mibs none none

In our example:

Only three networks (localhost, 172.16.1.0/24, and

192.168.1.0/24) are allowed to access the server with the

craz33guy community string.

Each network is matched to a either a group called

MyROGroup_1 using SNMP version 1, or group called

MyROGroup_2 using SNMP version 2.

All the MIBs on the server are defined by the view named

all-mibs.

An access statement ensures that only the defined

networks have read only access to all the MIBs.

MyROGroup_1 only has version 1 access with

MyROGroup_2 only having version 2 access.

Modification of the MIBs via SNMP is denied because the

word "none" is in the write section of the access statement.

These precautions are probably unnecessary in a home

environment where access is generally limited to devices on the

home network by a NAT firewall.

After editing the snmpd.conf file and restarting the snmpd

daemon on server bigboy, the remote client smallfry can poll the

interface MIB using v2, but bigboy cannot do so on localhost.

However bigboy can poll itself on localhost using v1. This is

expected.

root@bigboy:/tmp# snmpwalk -v 2c -c craz33guy

localhost system

Timeout: No Response from localhost

root@bigboy:/tmp# snmpwalk -v 1 -c craz33guy

localhost system


2.6.32-26-generic-pae #48-Ubuntu SMP Wed Nov 24

10:31:20 UTC 2010 i686



DISMAN-EVENT-MIB::sysUpTimeInstance =

Timeticks: (38537) 0:06:25.37

SNMPv2-MIB::sysContact.0 = STRING: root


...

...

...

root@bigboy:/tmp#

[root@smallfry ~]# snmpwalk -v 2c -c craz33guy

192.168.2.111 system


2.6.32-26-generic-pae #48-Ubuntu SMP Wed Nov 24

10:31:20 UTC 2010 i686



DISMAN-EVENT-MIB::sysUpTimeInstance =

Timeticks: (1022) 0:00:10.22

SNMPv2-MIB::sysContact.0 = STRING: root


SNMPv2-MIB::sysLocation.0 = STRING: Unknown

SNMPv2-MIB::sysORLastChange.0 = Timeticks: (2)

0:00:00.02

SNMPv2-MIB::sysORID.1 = OID: SNMP-FRAMEWORK-

MIB::snmpFrameworkMIBCompliance

...

...

...

[root@smallfry ~]#

SNMP Version 3

SNMPv3 is a much more secure alternative to earlier versions

as it encrypts all its data and uses a username / password

combination for client authentication. The username should be

located in the /etc/snmp/snmpd.conf file with a corresponding

automatically generated password located in the /var/net-

snmp/snmpd.conf file. Here is how it's done.

1. Install the net-snmp-devel (RedHat / Fedora / CentOS) or

libsnmp-base (Ubuntu / Debian) package as it contains the utility

that will allow you to generate the password. If you need an

installation refresher, see Chapter 6, "Installing Linux Software")

2. Stop the snmpd process.

[root@bigboy tmp]# systemctl stop snmpd.service

Stopping snmpd: [ OK ]

[root@bigboy tmp]#

3. Automatically create username and password entries using

the net-snmp-config command. In this example the

authentication password of "rootsrockreggae" for the read only (-

ro) username "username4snmpv3" is encrypted using the MD5

algorithm. The data received will not be encrypted.

[root@bigboy tmp]# net-snmp-config --create-

snmpv3-user -ro -a MD5 -A rootsrockreggae

username4snmpv3

adding the following line to /var/net-

snmp/snmpd.conf:

createUser username4snmpv3 MD5

"rootsrockreggae" DES

adding the following line to

/etc/snmp/snmpd.conf:

rouser username4snmpv3

[root@bigboy tmp]# systemctl start

snmpd.service


[root@bigboy tmp]#

Note: In Fedora / RedHat / Centos, the SNMPv3 username /

password pair will be stored in the file /var/net-

snmp/snmpd.conf. The type of user, in this case read-only, will

be stored in the file /etc/snmp/snmpd.conf.

Note: In Debian / Ubuntu, the SNMPv3 username / password

pair will be stored in the file /var/lib/snmp/snmpd.conf. The type

of user, in this case read-only, will be stored in the file

/usr/share/snmp/snmpd.conf.

4. To encrypt the data received we use the net-snmp-config

command again to create a username just for this purpose. A

new read only username "securev3user" and authentication

password "mandeville" are used, but this time the data will be

encrypted using the DES algorithm with the privacy password of

"savlamar".

[root@bigboy tmp]# net-snmp-config --create-

snmpv3-user -ro -a MD5 -A mandeville -x DES -X

savlamar securev3user

adding the following line to /var/net-

snmp/snmpd.conf:

createUser securev3user MD5 "mandeville" DES

savlamar

adding the following line to

/etc/snmp/snmpd.conf:

rouser securev3user

[root@bigboy tmp]#

5. Start the snmpd process.


snmpd.service


[root@bigboy tmp]#

6. First we'll do a query from remote host smallfry. We specify

the authentication password and authentication encryption

method, and we also use the -l flag to indicate that

authentication will be used, but that data privacy will be disabled

(the authNoPriv option).

[root@smallfry ~]# snmpget -v 3 -u

username4snmpv3 -l authNoPriv -a MD5 -A

rootsrockreggae 192.168.1.100 SNMPv2-

MIB::sysORDescr.8

SNMPv2-MIB::sysORDescr.8 = STRING: The

management information definitions for the SNMP

User-based Security Model.

[root@smallfry ~]#

The query returns an easy to read string, "The management

information definitions for the SNMP User-based Security

Model". This unencrypted string can also be seen in the tshark

packet capture of the server's interface.

[root@bigboy tmp]# tshark -n -i eth1 -x port

161

Capturing on eth1

...

...

...

0.005889 192.168.1.100 -> 192.168.1.50 SNMP

get-response

0000 00 c0 4f 46 0c 2e 00 b0 d0 46 32 71 08 00

45 00 ..OF.....F2q..E.

0010 00 f0 00 00 40 00 40 11 b3 b2 c0 a8 02 c8

c0 a8 ....@.@.........

0020 02 32 00 a1 80 0a 00 dc 87 38 30 81 d1 02

01 03 .2.......80.....

0030 30 11 02 04 45 a2 23 54 02 03 00 ff e3 04

01 01 0...E.#T........

0040 02 01 03 04 38 30 36 04 0d 80 00 1f 88 80

71 11 ....806.......q.

0050 68 72 0e b1 e7 45 02 01 12 02 01 39 04 0f

75 73 hr...E.....9..us

0060 65 72 6e 61 6d 65 34 73 6e 6d 70 76 33 04

0c 46 ername4snmpv3..F

0070 6c 74 26 51 4d aa 65 61 59 06 1a 04 00 30

7f 04 lt&QM.eaY....0..

0080 0d 80 00 1f 88 80 71 11 68 72 0e b1 e7 45

04 00 ......q.hr...E..

0090 a2 6c 02 04 43 4e da d7 02 01 00 02 01 00

30 5e .l..CN........0^

00a0 30 5c 06 0a 2b 06 01 02 01 01 09 01 03 08

04 4e 0\..+..........N

00b0 54 68 65 20 6d 61 6e 61 67 65 6d 65 6e 74

20 69 The management i

00c0 6e 66 6f 72 6d 61 74 69 6f 6e 20 64 65 66

69 6e nformation defin

00d0 69 74 69 6f 6e 73 20 66 6f 72 20 74 68 65

20 53 itions for the S

00e0 4e 4d 50 20 55 73 65 72 2d 62 61 73 65 64

20 53 NMP User-based S

00f0 65 63 75 72 69 74 79 20 4d 6f 64 65 6c 2e

ecurity Model.

4 packets captured

[root@bigboy tmp]#

7. Next we'll do a query that will return a response over an

encrypted data channel while crossing the network (the authPriv

option).

[root@smallfry ~]# snmpget -v 3 -u securev3user

-l authPriv -a MD5 -A mandeville -x DES -X

savlamar 192.168.1.100 SNMPv2-MIB::sysORDescr.8

SNMPv2-MIB::sysORDescr.8 = STRING: The

management information definitions for the SNMP

User-based Security Model.

[root@smallfry ~]#

The query returns the same string, but the tshark packet capture

only sees encrypted data, with only the username being visible.

[root@bigboy tmp] # tshark -n -i eth1 -x port

161

Capturing on eth1

...

...

...

0.003675 192.168.1.200 -> 192.168.1.50 SNMP

Source port: 161 Destination port: 32778 [UDP

CHECKSUM INCORRECT]

0000 00 c0 4f 46 0c 2e 00 b0 d0 46 32 71 08 00

45 00 ..OF.....F2q..E.

0010 01 00 00 00 40 00 40 11 b3 a2 c0 a8 02 c8

c0 a8 ....@.@.........

0020 02 32 00 a1 80 0a 00 ec 87 48 30 81 e1 02

01 03 .2.......H0.....

0030 30 11 02 04 17 52 82 96 02 03 00 ff e3 04

01 03 0....R..........

0040 02 01 03 04 3e 30 3c 04 0d 80 00 1f 88 80

71 11 ....>0

00e0 37 50 3c 8e 65 dd 8f 3c 49 71 96 59 f9 d3

a8 23 7P

ac1d95033f4afgf31243eb6907df

.1.3.6.1.6.3.10.1.2.2

0xf1f4bb00452211d27b50c273c09031ac 0x00

usmUser 1 3 0x80001f8880711168720eb1e745

0x757365726e61657534736e6d70763300

0x757365726e616d6534736e6d70763300

NULL .1.3.6.1.6.3.10.

1.1.2 0x5e35c9f5352519aa4f53eded09bbdddd

.1.3.6.1.6.3.10.1.2.2

0x5e35c9f5122519aa4f53eded09bbdddd ""

setserialno 1464593474

Practice using the net-snmp-config command so that you can

become familiar with the syntax it uses to edit the SNMP

configuration files. When in doubt, you can get a full syntax

listing of the command if you use it without any arguments like

this:

[root@bigboy tmp]# net-snmp-config

...

...

SNMP Setup commands:

--create-snmpv3-user [-ro] [-A authpass] [-X

privpass]

[-a MD5|SHA] [-x

DES|AES] [username]

...

...

[root@bigboy tmp]#

With experience, you should become confident enough to edit

the configuration files by yourself.

As you can see, SNMPv3 is more secure than previous versions

and should be your first SNMP choice whenever possible.

Simple SNMP Troubleshooting

If your SNMP queries fail, then verify that:

You restarted your snmp.conf file so the configuration

settings become active. Remember, the snmpd.conf file is

only read by the snmpd daemon when it starts up.

You are using the correct community string.

Firewalls aren't preventing SNMP queries from the SNMP

client to the SNMP target.

Your SNMP security policy allows the query from your

network.

Any TCP wrappers configuration on your SNMP target

machine allows SNMP queries from your SNMP client.

Generally speaking in a home environment protected by

NAT your TCP wrappers files (/etc/hosts.allow) and

(/etc/hosts.deny) should be blank.

Network routing between the client and target devices is

correct. A simple ping or traceroute test should be sufficient.

The snmpd daemon is running on the SNMP client.

You are querying using the correct SNMP version.

Your /var/log/messages file does not contain errors that may

have occurred while starting snmpd.

Troubleshooting to get functioning SNMP queries is important as

many other supporting applications, such as MRTG which I'll

discuss next, rely on them in order to work correctly.

MRTG

MRTG (Multi-Router Traffic Grapher) is a public domain

package for producing graphs of various router statistics via a

Web page. You can easily create graphs of traffic flow statistics

through your home network's firewall/router or even your Linux

box's NIC cards using MRTG. The product is available from the

MRTG Web site (www.mrtg.org) and also on your distribution

CDs. Figure 22-2 shows a sample MRTG graph.

Figure 22-2 A Typical MRTG Web Page

MRTG Download and Installation

You need to install MRTG before proceeding. Most RedHat and

Fedora Linux software products are available in the RPM format.

When searching for the file, remember that the MRTG RPM's

filename usually starts with mrtg and a version number, as in

mrtg-2.10.5-3.i386.rpm.

In addition to MRTG, you need to install the SNMP utility tools

as explained earlier and you need to have a Web server

package installed for MRTG to work. RedHat Linux usually

comes with the Apache Web server software preinstalled. The

easiest way to tell if Apache is installed is to run the rpm -q httpd

command. If you don't get a positive response, you can refer to

Chapter 20, "The Apache Web Server", for installation details.

By default Apache expects the HTML files for your Web site to

be located in /var/www/html. MRTG places its HTML files in

/var/www/mrtg.

MRTG Configuration Files

By default, MRTG maps the inbound and outbound data

throughput rates on the device it is polling. Methods for

specifying other OIDs, such as CPU and memory usage, are

discussed in Chapter 23, "Advanced MRTG for Linux." For now,

I'll stick with configurations that monitor network interfaces.

Various Linux distributions place their default MRTG

configuration files in different locations. For the purposes of this

tutorial well create the /etc/mrtg directory for this purpose.

root@smallfry-u:/tmp# mkdir -p /etc/mrtg

Now lets see how we use this directory in configuring MRTG to

work with the different versions of the SNMP protocol

SNMPv1

Use MRTG's cfgmaker command to create a configuration file

named mrtg.cfg for the server (bigboy) using a Read Only

community string of craz33guy. Place all data files in the

directory /var/www/mrtg.

[root@bigboy tmp]# cfgmaker --

output=/etc/mrtg/mrtg.cfg \

--global "workdir: /var/www/mrtg" -ifref=ip \

--global 'options[_]: growright,bits' \

craz33guy@localhost

--base: Get Device Info on craz33guy@localhost:

--base: Vendor Id:

--base: Populating confcache

--snpo: confcache craz33guy@localhost: Descr lo

--> 1

--snpo: confcache craz33guy@localhost: Descr

wlan0 --> 2

...

...

...

--base: Walking ifAdminStatus

--base: Walking ifOperStatus

--base: Writing /etc/mrtg/mrtg.cfg

[root@bigboy tmp]#

SNMPv2

As explained in the SNMP section, there are different versions

of SNMP. If your query doesn't work, check to make sure you

are using the required version and then check other SNMP

configuration parameters on the target device. You can specify

MRTG's SNMP query version with the --snmp-options cfgmaker

option. Here is an example of cfgmaker using an SNMP version

2 query of a router with an IP address of 192.168.1.3. The --

snmp-options option's five colons before the 2 are important.

[root@bigboy tmp]# cfgmaker --

output=/etc/mrtg/192.168.1.3.cfg \

-ifref=ip --global "workdir: /var/www/mrtg" \

--snmp-options=:::::2 [email protected]

SNMPv3

The cfgmaker command can also be used to poll SNMPv3

enabled devices. The process to set this up varies with each

Linux Distribution.

Fedora / RedHat / CentOS - Additional Packages

You need to first install the Net::SNMP PERL module using

these commands:

[root@bigboy tmp]# yum -y install perl-Net-SNMP

or

[root@bigboy tmp]# perl -MCPAN -e "install

Net::SNMP"

If you fail to install the module, you will get an error looking like

this:

Undefined subroutine &main::snmpmapOID called

at ./cfgmaker line 1480.

Ubuntu / Debian - Additional Packages

You need to first install the libnet-snmp-perl, libcrypt-hcesha-

perl, libcrypt-des-perl, and libdigest-hmac-perl libraries.

root@ubuntu:/tmp# apt-get -y install libnet-

snmp-perl libcrypt-hcesha-perl libcrypt-des-

perl libdigest-hmac-perl

If you fail to install the libraries, you will get an error looking like

this:

WARNING: SNMP V3 libraries not found, SNMP V3

disabled. Falling back to V2c.

SNMPopen failed: SNMPv3 support unavailable

(Required module Digest/SHA1.pm not found)


(Required module Crypt/DES.pm not found)


(Required module Digest/HMAC.pm not found)

Next Steps

Next you need to know the SNMPv3 ID of the host you intend to

poll. In fedora, this is value is located in the /var/net-

snmp/snmpd.conf file. You can use the grep command to obtain

it.

[root@bigboy tmp]# grep oldEngineID /var/net-

snmp/snmpd.conf

oldEngineID 0x80001f8880711168720eb1e745

[root@bigboy tmp]#

You can then repeat the cfgmaker command with options

specific to the privacy and authentication schemes configured

on the SNMP target device. Using the configurations of our

previous SNMPv3 example, our configuration for encrypted

authentication only would look like this:

[root@bigboy tmp]# cfgmaker --global 'WorkDir:

/var/www/mrtg' \

--global 'Options[_]: growright, bits' \

--output=/etc/mrtg/192.168.1.100.cfg \

--enablesnmpv3 --username=username4snmpv3 \

--authpassword=rootsrockreggae --authproto=md5

\

--snmp-options=:::::3 \

--contextengineid=0x80001f8880711168720eb1e745

\

[email protected]

Our configuration for encrypted authentication and data privacy

only would look like this:

[root@bigboy tmp]# cfgmaker --global 'WorkDir:

/var/www/mrtg' \

--global 'Options[_]: growright, bits' \

--output=/etc/mrtg/192.168.1.100-secure.cfg \

--enablesnmpv3 --username=securev3user --

authpassword=mandeville \

--authproto=md5 --privpassword=savlamar --

privprotocol=des \

--snmp-options=:::::3 \

--contextengineid=0x80001f8880711168720eb1e745

\

[email protected]

Note: The MRTG cfgmaker command reliably supports

SNMPv3 as of MRTG version 2.15. Prior to this version you

would commonly see this error when attempting to do SNMPv3

queries.

SNMP V3 requires a --username parameter as part

of the User Security Model for router

[email protected]:::::3 at ./cfgmaker

line 121.

Setting MRTG Polling Intervals and Web Page Locations

Now we need to configure Linux to use the newly created MRTG

configuration files as part of a regular polling cycle and then

deposit the results in a web directory. Here is how its done.

1. Create the /var/www/mrtg directory in which MRTG will place

its files

root@smallfry-u:/tmp# mkdir -p /var/www/mrtg

2. Edit /etc/mrtg/mrtg.cfg, (or whatever the name of your

configuration file may be) and remove the sections related to

interfaces you don't need to monitor. A certain candidate would

be the virtual loopback interface Lo: (with the IP address of

127.0.0.1), which doesn't pass any external network traffic at all.

3. Run MRTG using /etc/mrtg/mrtg.cfg as your argument three

times. You'll get an error the two times as MRTG tries to move

old data files, and naturally, the first time it is run, MRTG has no

data files to move.

[root@bigboy tmp]# env LANG=C /usr/bin/mrtg

/etc/mrtg/mrtg.cfg

Rateup WARNING: /usr/bin/rateup could not read

the primary log file for

localhost_192.168.1.100

Rateup WARNING: /usr/bin/rateup The backup log

file for localhost_192.168.1.100 was invalid as

well

Rateup WARNING: /usr/bin/rateup Can't remove

localhost_192.168.1.100.old updating log file

Rateup WARNING: /usr/bin/rateup Can't rename

localhost_192.168.1.100.log to



/etc/mrtg/mrtg.cfg

Rateup WARNING: /usr/bin/rateup Can't remove



/etc/mrtg/mrtg.cfg

[root@bigboy tmp]#

4. Use MRTG's indexmaker command to create a Web index

page using your new mrtg.cfg file as a guide. The MRTG Web

GUI expects to find the index file in the default MRTG Web

directory of /var/www/mrtg/, so the format of the command

would be.

[root@bigboy tmp]# indexmaker --

output=/var/www/mrtg/index.html \

/etc/mrtg/mrtg.cfg

5. MRTG is run every five minutes by default, and the file that

governs this is /etc/cron.d/mrtg. For MRTG to work correctly,

edit this file, replacing all occurrences of /usr/bin/mrtg with env

LANG=C /usr/bin/mrtg. The explanation for changing the

language character set for MRTG is given in the

"Troubleshooting MRTG" section.

This isn't all. You need to view the graphs too. This will be

covered later, but first I'll show you how to poll multiple devices.

Getting MRTG To Poll Multiple Devices

The MRTG installation process creates a cron file named

/etc/cron.d/mrtg. This file tells the cron daemon to run MRTG

using the mrtg.cfg file every five minutes to poll your network

devices. (This file may be located in either /etc or /etc/mrtg). You

can configure MRTG to poll multiple devices, each with a

separate configuration file. Here's how:

1. Create a new configuration file using the steps from the

previous section; choose a filename that is not mrtg.cfg.

2. Add a new MRTG line in /etc/cron.d/mrtg for each new

configuration file you create.

0-59/5 * * * * root env LANG=C /usr/bin/mrtg

/etc/mrtg/mrtg.cfg


/etc/mrtg/device1.cfg


/etc/mrtg/device2.cfg

3. Run the indexmaker command, and include all of your

/etc/mrtg configuration files, to regenerate your Web index page.


output=/var/www/mrtg/index.html \

/etc/mrtg/mrtg.cfg /etc/mrtg/device1.cfg /etc/mrtg/device2.cfg

4. Other versions of Linux keep their MRTG cron entries inside

the /etc/crontab file. Edit this file using the same syntax as the

Fedora /etc/cron.d/mrtg file, and then restart the cron daemon to

re-read the configuration:


crond.service

You could also create a script with the /usr/bin/mrtg

/etc/mrtg/device.cfg entries in it and make cron run it every five

minutes. This way you can just edit the script each time you add

a device without having to restart cron.

Configuring Apache To Work With MRTG

MRTG is useful because it can provide a graphical

representation of your server's performance statistics via a Web

browser. This is configured slightly differently depending on your

Linux variation.

Fedora / RedHat / CentOS:

With these distributions MRTG creates an add-on configuration

file named /etc/httpd/conf.d/mrtg.conf that includes all the

necessary Apache commands for MRTG to work.

Debian / Ubuntu:

With these distributions the mrtg.conf file may not exist and

youll need to create one in the /etc/apache2/conf.d directory.

Some additional configuration may need to be done, because by

default MRTG accepts Web requests from the Linux console

only. You can add your home network to the file by inserting the

network on the Allow from line, or you can allow universal

access by commenting out that line along with the Deny from

line. This example adds access from the 192.168.1.0 network.

# File: mrtg.conf in /etc/httpd/conf.d or

/etc/apache2/conf.d

Alias /mrtg /var/www/mrtg

Order deny,allow

Deny from all

Allow from localhost 192.168.1.0/24

If you want to access MRTG from the Internet, then you'll have

to comment out the Deny statement and allow from all IP

addresses:

# File: mrtg.conf in /etc/httpd/conf.d or

/etc/apache2/conf.d

Alias /mrtg /var/www/mrtg

Order deny,allow

Allow from all

Remember to restart Apache once you have made these

modifications in order for these changes to take effect.

Basic Security

If you are accessing MRTG graphs from the Internet, you may

want to add password protection to the directory by using a

.htaccess file as described in Chapter 20, "The Apache Web

Server".

How To View The MRTG Graphs In Your Web Browser

You can now access your MRTG graphs by pointing your

browser to the URL:

http://server-ip-address/mrtg/

Using MRTG To Monitor Other Subsystems

MRTG will generate HTML pages with daily, weekly, monthly,

and yearly statistics for your interfaces. By default, MRTG

provides only network interface statistics. Chapter 23,

"Advanced MRTG for Linux", has detailed examples and

explanations of how to monitor Linux disk, CPU, memory, and

Web connection data. The MRTG Web site, www.mrtg.org, also

has links to other sites that show you how to monitor many other

subsystems on a variety of devices and operating systems.

Troubleshooting MRTG

There are many simple steps you can use to troubleshoot

MRTG. Take a look at some of the most common ones.

Basic Steps

MRTG won't work if SNMP queries don't work. Make sure you

follow the SNMP troubleshooting steps if you have any

difficulties.

Setting The Correct Character Set

MRTG usually works only if your system uses an ASCII-based

(Western European) character set. If it isn't set, then you'll get

errors such as this every time you run MRTG from the command

line or as part of a cron job:

[root@bigboy tmp]# mrtg /etc/mrtg/mrtg.cfg

-----------------------------------------------

--------------------

ERROR: Mrtg will most likely not work propperly

when the environment

variable LANG is set to UTF-8. Please

run mrtg in an envir..

where this is not the case:

env LANG=C /usr/bin/mrtg ...

-----------------------------------------------

--------------------

[root@bigboy tmp]#

Your system's character set is defined in /etc/sysconfig/i18n,

and the current Fedora default of en_US.UTF-8 won't work, but

en_US will after a system reboot. This is not necessarily a good

idea, especially if the native language Linux uses on your

system is not ASCII based, other things may fail to work.

A better solution is to always run MRTG using this command

instead of using just plain /usr/bin/mrtg.

env LANG=C /usr/bin/mrtg

This will modify the character set used by MRTG alone and

shouldn't affect anything else.

Incorrect SNMPv3 Engine ID

The added security of SNMPv3 forces each client to create its

own serial number or engine ID. If you use an incorrect identifier

you may get noSuchInstance errors like this when polling with

MRTG.

2008-07-26 19:42:40: WARNING: Expected a number

but got 'noSuchInstance'

2008-07-26 19:42:40: WARNING: Expected a number

but got 'noSuchInstance'

2008-07-26 19:42:40: ERROR:

Target[localhost_3][_IN_] ' $target->[1]{$mode}

' did not eval into defined data

2008-07-26 19:42:40: ERROR:

Target[localhost_3][_OUT_] ' $target-

>[1]{$mode} ' did not eval into defined data

Always make sure you are using the correct ID and try again.

Fedora Core 1 MRTG Errors With Net-SNMP

A bug appears in the MRTG implementation for some Fedora

Core 1 MRTG versions when polling another Fedora Core 1

server.

When using a -ifref=ip statement with the cfgmaker command,

every line in the configuration file that is generated becomes

commented out. When it works, this statement is very

convenient, because it makes MRTG provide graphs sorted by

the IP addresses of the interfaces instead of the default, which is

the much harder to recognize interface MAC address.

Upgrading to the latest Core 1 version of MRTG will fix the

problem.

### Interface 6 >> Descr: | Name: | Ip:

'192.168.1.100'

###

### The following interface is commented out

because:

### * has a speed of which makes no sense

### * got 'Received SNMP response with error

code

### error status: noSuchName

### index 1 (OID: 1.3.6.1.2.1.2.2.1.10.6)

### SNMPv1_Session (remote host:

"localhost" [127.0.0.1].161)

### community:

"craz33guy"

### request ID: 824482716

### PDU bufsize: 8000 bytes

### timeout: 2s

### retries: 5

#

# Target[localhost_192.168.1.100]:

/192.168.1.100:craz33guy@localhost:

# SetEnv[localhost_192.168.1.100]:

MRTG_INT_IP="192.168.1.100" MRTG_INT_DES

# MaxBytes[localhost_192.168.1.100]: 0

# Title[localhost_192.168.1.100]: Traffic

Analysis for 192.168.1.100

# PageTop[localhost_192.168.1.100]: Traffic

Analysis for 192.168.1.100

As all the lines in the configuration file are commented out with a

# character, indexmaker fails to create an index.html file and

gives errors.


output=/var/www/mrtg/stats/index.html

/etc/mrtg/mrtg.cfg

Use of uninitialized value in hash element at

/usr/bin/indexmaker line 307.

[root@bigboy tmp]#

Webalizer

Webalizer is a Web server log file analysis tool that comes

installed by default on RedHat/Fedora Linux. Each night,

Webalizer reads your Apache log files and creates a set of Web

pages that enable you to view Web surfer statistics for your site.

The information provided includes a list of your Web site's most

popular pages sorted by hits along with traffic graphs showing

the times of day when your site is most popular. Configuring

Webalizer to Work with Apache The required configuration steps

to integrate Webalizer with the Apache web server is slightly

differently depending on your Linux variation.

Fedora / RedHat / CentOS

With these distributions MRTG creates an add-on configuration

file named /etc/httpd/conf.d/webalizer.conf that includes all the

necessary Apache commands for Webalizer to work.

The main Webalizer configuration file is named

/etc/webalizer.conf. Edit it to ensure that the OutputDir directive

is set to /var/www/usage like this. This will be important later.

OutputDir /var/www/usage

Debian / Ubuntu

With these distributions the webalizer.conf file may not exist and

youll need to create one in the /etc/apache2/conf.d directory.

The main Webalizer configuration file is named

/etc/webalizer/webalizer.conf. Edit it to ensure that the OutputDir

directive is set to /var/www/usage like this. This will be important

later.

OutputDir /var/www/usage

Some additional configuration may need to be done, because

the Webalizer configuration may only accept Web requests from

the Linux console. You can add your home network to the file by

inserting the network on the Allow from line, or you can allow

universal access by commenting out that line along with the

Deny from line. This example adds access from the 192.168.1.0

network.

# File: webalizer.conf in /etc/httpd/conf.d or

/etc/apache2/conf.d

Alias /usage /var/www/usage

Order deny,allow

Deny from all

Allow from localhost 192.168.1.0/24

If you want to access Webalizer from the Internet, then you'll

have to comment out the Deny statement and allow from all IP

addresses:

# File: webalizer.conf in /etc/httpd/conf.d or

/etc/apache2/conf.d

Alias /usage /var/www/usage

Order deny,allow

Deny from all

Allow from all

Remember to restart Apache once you have made these

modifications in order for these changes to take effect.

How To View Your Webalizer Statistics

After configuring Apache, you will be able to view your data by

visiting the URL http://server-ip-address/usage. The statistics

will be updated when the /etc/cron.daily/*webalizer* script is

automatically run each day.

The top Command

You can monitor the amount of memory and CPU resources

your system is using the top command.

[root@bigboy tmp]# top

3:04pm up 25 days, 23:23, 2 users, load

average: 0.00, 0.02, 0.00

78 processes: 76 sleeping, 2 running, 0 zombie,

0 stopped

CPU states: 0.9% user, 0.5% system, 0.0%

nice, 0.8% idle

Mem: 384716K av, 327180K used, 57536K

free, 0K shrd, 101544K buff

Swap: 779112K av, 0K used, 779112K free

130776K cached

PID USER PRI NI SIZE RSS SHARE

STAT %CPU %MEM TIME COMMAND

27191 root 15 0 1012 1012 780 R

5.6 0.2 0:00 top

4545 root 16 0 5892 5888 4956 S

0.9 1.5 169:26 magicdev

1 root 15 0 476 476 432 S

0.0 0.1 0:05 init

2 root 15 0 0 0 0 SW

0.0 0.0 0:00 keventd

5 root 15 0 0 0 0 SW

0.0 0.0 0:41 kswapd

6 root 25 0 0 0 0 SW

0.0 0.0 0:00 bdflush

[root@bigboy tmp]#

Here the CPU usage is under 1.0% and 14% of memory

(57536K) is free. The amount of free memory may appear low,

but in this case, the server doesn't seem to be swapping idle

processes from memory to the swap disk partition as it isn't

being used at all. Excessive swapping can cause your system to

slow down dramatically, the simplest ways to avoid this is to add

more RAM or reduce the number of processes or users that are

active on your system.

If your system seems slow but the CPU and memory usage is

low, then start looking at networking problems, such as poor

duplex negotiation, bad cables, and network congestion due to

excessive traffic.

The vmstat Command

You can also determine memory and swap usage with the

vmstat command, which provides a summary of what top

produces. In the example, memory is still 14% free (57,452MB

used from a total of 130,780) and swap isn't being used at all.

[root@bigboy tmp]# vmstat

procs memory swap

io system cpu

r b w swpd free buff cache si so

bi bo in cs us sy id

0 0 0 0 57452 101584 130780 0 0

0 4 18 1 3 1 1

[root@bigboy tmp]#

As your memory fills up, your system will temporarily store

programs and data on your hard disk's "swap" partition. Excess

swapping of programs and data between disk and memory can

cause your system to slow down significantly and memory

usage should be monitored to allow you to plan ways to either

increase RAM or tune the way your system operates. System

tuning is beyond the scope of this book, but there are many

reference guides which can show you how to do this.

The free Utility

The free utility can determine the amount of free RAM on your

system. The output is easier to understand than vmstat's. Here's

a sample.

[root@bigboy tmp]# free

total used free

shared buffers cached

Mem: 126060 119096 6964

0 58972 40028

-/+ buffers/cache: 20096 105964

Swap: 522072 15496 506576

[root@bigboy tmp]#

You should generally try to make your system run with at least

20% free memory on average, which should allow it to handle

moderate spikes in usage caused by running memory-intensive

cron batch jobs or tape backups. If you cannot achieve this,

consider running more efficient versions of programs, offloading

some applications to servers with less load, and, of course,

upgrading the capacity of your RAM.

Conclusion

Server monitoring is always a good practice, because it can help

you predict when things are going to go wrong or long term

trends in your Web traffic.

MRTG can be expanded not only to monitor traffic on your

server's NIC cards, but also to graph many of the statistics listed

in top, free, and vmstat. Chapter 23, "Advanced MRTG for

Linux", shows you how.

Documents

Server Performance