Marcus Rochelle - Landis+Gyr - Monitoring with Nagios Enterprise Edition

Monitoring with

NagiosXI Enterprise

Edition

By Marcus [email protected]

Table of Contents

o Bio

o Landis+Gyr

o NagiosXI Monitoring Project

o Landis+Gyr Monitoring Summary

o Nagios XI System Architecture

o Nagios XI System Diagram

o Nagios XI Key Benefits for Landis+Gyr

o Nagios Fusion

o Nagios XI Feature Component / Monitoring Add-ons� Thruk

� Active Directory Integration

� Automated Host Management in Nagios XI

� Interfacetable_v3t Plugin for Network Monitoring

� Nagios XI Email Monitoring via Exchange Web Services

� NSClient++ Monitoring Agent

� Backend APIs

Bio

Marcus Rochelle

o Nagios Certified Administrator

o Masters in Information Systems

o 30 + Years in the IT Industry

• 10 Years with Landis+Gyr

• 24 Years with IBM

� Is the global industry leader in energy measurement solutions for utilities.

� Offers a complete portfolio of energy meters and

integrated smart metering solutions, enabling utilities and end-users to make better use of scarce resources, save operating costs and protect the environment by managing energy better – and to build the smart grid.

Landis+Gyr

� Local presence in more than 30 countries on all five continents

� Over 3,500 utility relationships worldwide with the largest installed global base in

the industry (>300mm devices)

� 25 years of smart metering innovation and over 60 years of direct load

management expertise

� Over 25mm smart grid-enabled endpoints deployed or contracted globally

� Over 15 million endpoints actively managed in long-term contracts

� ISO certified for quality and environmental processes

� A worldwide team of over 900 engineers and research professionals

� Committed to improving energy efficiency and environmental conservation

Landis+Gyr

Monitoring Prior to NagiosXI

• Customized Scripts• Bash

• Borne

• Korne

• Perl

• etc

• Cacti

• ManageEngine OpManager

• SolarWinds

• Multiple Other Apps

NagiosXI Project initiated Oct 2011 – Sep 2012

• Objective to use NagiosXI as the preferred monitoring

platform

• First Nagios Conference attended 2013 which lead to a

number of significant improvements.

o Mod Gearman

o NSClient++

o Thruk

NagiosXI Project initiated Oct 2011 – Sep 2012

NAGIOS XI MONITORING IMPLEMENTATION - PROPOSED SCHEDULE

Date Activity Description

OCT 2011 Nagios Development Server complete.

OCT 2011 Nagios XI virtual training image configured for testing.

OCT 2011 Nagios XI image installed to test monitoring plug ins.

NOV 2011 High Level Design Draft.

DEC 2011 High Level Design Complete

Nagios XI development environment complete

Pilot Managed Service Program and Hosted Customers selected

MAR 2012 Nagios XI plugin testing initiated

MAY 2012 Nagios XI monitoring across pilot hosts

MAY 2012 Nagios XI pilot monitoring assessment

MAY 2012 Nagios XI signoff for full production implementation

JUN 2012 Nagios XI plugin testing complete

JUN 2012 Nagios XI initial production configuration initiated

JUL 2012 Nagios XI Revision to production configuration

SEP 2012 Nagios XI monitoring fully implemented across data center and network

Landis+Gyr Monitoring Summary

• Landis+Gyr Customer Operations uses Nagios-XI to monitor

over 105,000 hosts and services

• 80099 Concentrators / Routers

• 2700 Host / Servers

• 23135 Services

• 35 checks / sec

Nagios XI System Architecture

• Customer Operations uses 20 Nagios servers to monitor all systems and services.

14 Dedicated Nagios XI servers for Managed Services One to two servers per utility site depending on sizeConfiguration and change activity is automated

5 Nagios XI servers for Alpharetta and Lenexa Data Centers

1 Nagios Fusion server to aid in centralized management

Nagios XI System Diagram

Physical Hardware of Master Nagios Server

HP ProLiant BL460c Gen8 2 Processor Sockets with 8 cores per socket.

CPUs: Qty 8 Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz

Memory 24GB 250GB SSD Volume

Nagios XI Program Utility Networks

Nagios XI Program Utility Networks

Nagios-Fusion Tactical Summary

Nagios XI Feature Components / Monitoring

Add-onso Nagios XI Key Attributes for Landis+Gyr

o Thruk

o Active Directory Integration

o Automated Host Management in Nagios XI

o Interfacetable_v3t Plugin for Network Monitoring

o Nagios XI Email Monitoring via Exchange Web Services

o NSClient++ Monitoring Agent

o Backend APIs

Nagios XI Key Attributes for Landis+Gyr

o Enabled Maximum Use of Prior Monitoring Development

o User Friendly GUI

o Short Learning Curve

o Online Help

o Online Administrator Guide

o Notification Management o used to interface with Microsoft CRM (Problem Management System)

o Online Updates

o Monitoring Wizardo Initial Host Configuration / Template

o Bulk Host Cloning and Import Used Afterwards

o Nagios CCM (Core Config Manager)o Recent enhancements allowing multiple user updates a plus

o Renaming Tool

o Bulk Modifications










o Thruk






o Backend APIs

www.thruk.org



o Thruk






o Backend APIs

Nagios XI Feature ComponentsActive Directory Integration

All users are authenticated against the AD

domain controller using the same login

used for their Laptop or workstation.

Nagios XI Feature Components

https://assets.nagios.com/downloads/nagiosxi/docs/Authenticating_with_Active_Directory_in_Nagios_XI.pdf


All users are authenticated

against the AD domain controller

using the same login for their

Laptop or workstation.

Nagios XI Feature ComponentsThruk – Adding the Logout Button

1. Edit /etc/thruk_local.conf

show_logout_button = 1

2. Edit /etc/httpd/conf.d/thruk.conf

# put the RewriteLock outside your virtual hosts (only required for apache < 2.4)

<IfModule !mod_authz_core.c>

RewriteLock "/var/cache/thruk/apache_rewrite.lock"

</IfModule>

# put everything else inside your vhost

RewriteMap thruk_users prg:/usr/share/thruk/thruk_auth

RewriteCond %{REQUEST_URI} !^/thruk/cgi-bin/restricted.cgi

# limit thruk cookie auth to thruk itself in standalone mode

RewriteCond %{REQUEST_URI} ^/thruk

RewriteCond %{HTTP_COOKIE} (thruk_auth=[^;]+|$) [NC]

RewriteRule ^/(.*)$ /%1/%{REMOTE_ADDR}/____/$1/____/%{QUERY_STRING} [C,NS]

RewriteRule ^(.*)$ ${thruk_users:$1|/loginbad/} [C,NS]

RewriteRule ^/pass/(.*)$ /$1 [NS,PT,L,E=!REMOTE_USER]

RewriteRule ^/redirect/(.*)$ /$1 [NS,L,R=302]

RewriteRule ^/loginok/([^/]+)/(.*)$ /$2 [NS,PT,L,E=REMOTE_USER:$1]


Active Directory Authentication NagiosXI with Thruk

1. Edit /etc/httpd/conf.d/thruk.conf

2. Add the following entry:

<Location /thruk/>

Options ExecCGI FollowSymLinks

Order allow,deny

Allow from all

AuthBasicProvider file

AuthUserFile /etc/thruk/htpasswd

Require valid-user

AuthBasicProvider ldap

AuthType Basic

AuthzLDAPAuthoritative off

AuthName "Active Directory Login"

AuthLDAPURL "ldap://< ip address >:3268/dc=YOUR,dc=COMPANY,dc=NET?sAMAccountName?sub" NONE

AuthLDAPBindDN "< ldapuser >@your.company.net"

AuthLDAPBindPassword "< Password >"

Require valid-user

</Location>



o Thruk






o Backend APIs

Nagios XI Feature ComponentsAutomated Host Management

• Customer Operations uses 20 Nagios servers to monitor all systems

and services.• 14 Dedicated Nagios servers for Managed Services

One to two servers per utility site depending on size.

• Under the control of the Master Nagios XI server, the 14 dedicated servers for

Managed Services are automated for configuration updates.

• The Master Nagios XI server monitors for host device change activity. If any device

is moved or replaced, a configuration check is performed against the Utility

database, and using the “Automated Host Management in Nagios XI“ procedures,

synchronizes the Nagios XI server to match.

• 80099+ Host configurations are automated and managed from the Master nagios

server.


https://assets.nagios.com/downloads/nagiosxi/docs

/Automated_Host_Management.pdf

.•






o Thruk






o Backend APIs

Nagios XI Feature ComponentsEmail Monitoring via EWS

Nagios XI Feature ComponentsPresentation Addendum, Code, and Notes

• Thruk – Adding the Logout Button

• Active Directory Authentication NagiosXI with Thruk

• Perl code for check_mail.pl Office365

• Perl Code for delete_mail.pl Office365



o Thruk






o Backend APIs

Nagios XI Feature ComponentsInterfacetable_v3t

Interfacetable_v3t is a Nagios(R) add-on that allows you to monitor the network

interfaces of a node (e.g. router, switch, server) without knowing each interface in detail.

www.tontonitch.com

Nagios XI Feature Components



o Thruk






o Backend APIs

Nagios XI Feature Components – NSClient++

NSClient++ is a powerful and extremely flexible monitoring agent.

• Used to monitor 6700+ Message Queues, App-pools, Tasks, and Serviceso Queue Count

o Oldest Message in Queue

www.nsclient.org

By Michael Medin


NSClient++ used to execute check_msmq.vbs for monitoring of Microsoft

messaging queues.o Queue Count

o Oldest Message in Queue

https://exchange.nagios.org/directory/Plugins/Operating-Systems/Windows-NRPE/check_msmq-

2Evbs/details

� Modification made to check_msmq.vbs to allow indication for max age in queue.


message queue name

Server Name



https://exchange.nagios.org/directory/Plugins/Operating-Systems/Windows-NRPE/check_msmq-2Evbs/details

Modified check_msmq.vbs to check for aged message

Required Augments

• [QueueName]

• [warn]

• [crit]

• [Max Age in Mins]

• Both check_msmq.vbs and nsclient.ini included in presentation addendum


o Used for checking Windows Older Technologies

o Used when monitoring requires the remote execution of PowerShell, scripts,

and other host commands

o Example configurations (nsclient.ini) included in presentation addendum.


NSClient++ used to monitor Scheduled Tasks



o Thruk






o Backend APIs

Nagios XI Feature ComponentsBackend API URLs

The Nagios XI backend API can be used to access current and historical information on

monitored hosts and services for integration into third-party frontends.

Nagios XI Feature Components Backend API

URLs

Nagios XI Feature Components Backend API

URLs

Nagios XI Feature Component -

Backend API URLs

62

Thank you!

Any Questions?

Presentations & Public Speaking

Marcus Rochelle - Landis+Gyr - Monitoring with Nagios Enterprise Edition