Datagram SyslogServer manual.pdf

8/10/2019 Datagram SyslogServer manual.pdf

1/27

Datagram Consulting stermalmsgatan 21, 114 26 Stockholm, SwedenTel +46 8 544 952 00www.datagram.se

Datagram

Datagram SyslogServer manual

Version 2.3April 2011


2/27

Datagram

PAGE 2 (27)

Table of contents:

Introduction ............................................................................................................................................. 3 What is SyslogServer? ..................................................................................................................... 3

Installation ............................................................................................................................................... 4 Prerequisites .................................................................................................................................... 4 Trial Version installation ................................................................................................................. 4 Enterprise Edition installation ......................................................................................................... 5 Database configuration .................................................................................................................... 5 ODBC Configuration ...................................................................................................................... 7 Enterprise Edition installation and configuration .......................................................................... 10 Using other database products ....................................................................................................... 11

Configuration ........................................................................................................................................ 12 Tab General ................................................................................................................................... 12 Tab Database ................................................................................................................................. 13 Tab Log files ................................................................................................................................. 15 Tab Mail Support........................................................................................................................... 16

Tab Advanced................................................................................................................................ 18 Appendix 1 Database table information ............................................................................................. 20

Permissions: .................................................................................................................................. 24 MS SQL commands: ..................................................................................................................... 25

Troubleshooting .................................................................................................................................... 26


3/27

Datagram

PAGE 3 (27)

Introduction

Welcome to the Manual and Installation Guide for Datagram SyslogServer.

Here you will find a detailed installation procedure as well as full descriptions of the configurationalternatives. This manual covers both Enterprise and Small Business Edition versions.

If you miss any information in this file please inform us ([email protected]). You can also readthe FAQ at www.syslogserver.com.

Customers can always contact us at [email protected].

What is SyslogServer?

The Datagram SyslogServer is installed as a service on Microsoft Windows. By using a supplieddatabase, preferably any Microsoft SQL Server, syslog messages can be received and stored in thedatabase. SyslogServer also includes alarm logic based on received messages.

Other than the configuration program, the Datagram SyslogView application is used as interface to thereceived messages. Datagram SyslogAgent is a GNU licensed software to enable Microsoftinstallations to send local ecent logs to the SyslogServer using the Syslog Protocol standard.

Two different editions exist; Enterprise Edition and Trial Version. Earlier, a Small Business Editionsimilar to Trial Version was also available. These two types appear similar, but have very differentdatabase handling at a closer look.

The Trial Version is based on an Access file database. It is very easily installed and is suitable forsmaller companies. There are some limitations on the features and especially on the number of IP

address sources that will be parsed. The Trial Edition is not intended for high rates of Syslog entries(several hundred per second), or large volumes of entries (millions). Still, for most situations in thesmall company it is quite sufficient.

The Enterprise Edition is intended for large companies, or situations with many syslog entries. Theinstallation requires some configuration since an ODBC connection is used. The enterprise Editionworks with Microsoft SQL servers and to some extent others types of databases. True databaseperformance is a prerequisite for a true enterprise Syslog solution. No limitations in terms of alarmsand filters are imposed on the Enterprise Edition.


4/27

Datagram

PAGE 4 (27)

Installation

Prerequisites

Datagram SyslogServer runs on Windows 2000/XP/2003/Vista/2008 and Windows 7.

For the Trial Version, an Access file is used. This does not impose any special prerequisites on theplattform.

For the Enterprise edition, a database must be available. Since ODBC is used, it does not need toreside on the same server. Microsoft SQL has been verified to work well with Datagram SyslogServerand is therefore fully supported. Please note that Access isnot supported in the Enterprise Edition forperformance and data integrity reasons.

Trial Version installationThe installation procedure for Datagram SyslogServer Trial Version is quite easy. Execute theconfiguration program, and choose install service. The service can then be configured and started.


5/27

Datagram

PAGE 5 (27)

Enterprise Edition installation

The installation procedure for Datagram SyslogServer Enterprise Edition consists of three parts:

Create new databaseSyslog Configure an ODBC connection Install SyslogServer and configure for database use

These parts are described in order below.

Database configuration

Datagram SyslogServer needs a database called Syslog. This database and possibly a user, must bemanually created. SyslogServer does not perform these tasks as most db-admins do not want thirdparty software to get root privileges and commit to major changes in their database.

The table initiation is performed during service startup.

Create a new database called Syslog from the SQL Server Enterprise Manager:

Create a new SQL account that has full privileges for the Syslog database, and has Syslog asdefault database. If Windows Authentication is used, make sure to configure the SyslogServerservice to be run with an account that has access to the database.


6/27

Datagram

PAGE 6 (27)

Set permissions for user Syslog to databaseSyslog.

Create a system ODBC connection described later in this chapter


7/27

Datagram

PAGE 7 (27)

During the first start of SyslogServer from the SyslogConf application, the tables will beinitialized

Downgrade the newly created SQL account to suitable levels. A list of minimum permissionscan be found in the Appendix

Done!

ODBC Configuration

The following steps describe the ODBC configuration for Windows 2000/XP/2003/Vista/2008.

Start wizard from Administrative toolsData sources (ODBC). Note that for x64 systems analternative path to the ODBC configuration must be used, as the ODBC connection is still 32 bit inorder to continue support Access. You find the ODBC configuration for 32bit at%windows%/syswow64/odbcad32.exe.

Go to tab System DSN. A user DSN must not be used, since services cannot access userDSN:s.

Press button Add. After choosing SQL Server you should see the picture similar to the onebelow.

Ensure the DSN (data source name) is 'Syslog'. Enter Server name and pressnext.


8/27

Datagram

PAGE 8 (27)

ChooseWindows authentication or SQL Server authentication depending on database settings.The Connect to SQL Server is optional, but can be helpful to ensure that the configurationworks. Pressnext .

Set the default database toSyslog. No other changes are needed.


9/27

Datagram

PAGE 9 (27)

On the next page, make sure SQL Server system messages are in English Syslogserver acton these messages. Then pressFinish .

The SyslogServer service is now ready to be started from Syslogserver application.


10/27

Datagram

PAGE 10 (27)

Enterprise Edition installation and configuration

Execute the Configuration program from the Start menu(Start Programs Datagram SyslogServer SyslogServer).

Enter the license information to enable Enterprise Edition features. The text at the bottom ofthe main window indicates currently used license.

Depending on database authentication settings, enter a username and password in the ODBCconfiguration, or insert it into the SyslogServer application using the Db credentials button.

Start the service. Upon prompt is the tables needed shall be automatically created, choose yes.

Done!


11/27

Datagram

PAGE 11 (27)

Using other database products

Datagram SyslogServer needs a database called Syslog containing six tables. If the database product isMS SQL the configuration is partially automated. Please note that Microsoft Access is only supportedin the Trial Version.

Please contact us for questions regarding using other database products.

The following steps are needed.

Create a new database called Syslog

Create an account that has at least the privileges described in the Appendix for the databaseSyslog

Create a system ODBC connection with DSN name Syslog. It should point to the SyslogDatabase. Authentication is performed from the Syslog Server program

Create the six tables. They are described in the Appendix in MS SQL syntax

When starting the SyslogServer Service from the graphical interface, a test of table formatcompatability is performed. Any encountered database errors are reported. If you experienceproblems, please contact us at [email protected]


12/27

Datagram

PAGE 12 (27)

Configuration

Once the service has been installed the configuration options become available. Five separate tabsexist.

Almost all settings can be changed without restarting the service as of version 2.2. Certain parameters(such as port and log directory) will take effect within one minute. Database credentials and licenseinformation are the exception - the service must be stopped.

Before the service can be successfully started the database credentials need to be configured and anODBC source must have been configured. See the installation chapter for more information.

Tab General

Service

By pressing the button Install Syslog Server is added as a Windows Service. It is configuredto start at boot time. You can manually start/stop/uninstall the service from this interface too.When choosingStart Service a database check is performed to ensure everything is configuredcorrectly. Any encountered errors are reported.


13/27

Datagram

PAGE 13 (27)

The icon to the left changes according to the service status (Error, stopped, starting/stopping,running).

Port Information

Syslog Server listen port Choose which UDP port to listen on. Default is 514.

Getting Started section

This section describes the usual installation precedure.

Start by installing the Windows service.

Button Enter license enables the license window. Make sure you paste the exact textin order for correct calculation of your license. The license information is printed atthe bottom of the main SyslogServer window. An incorrect license will be displayedas 'Invalid license'. The license text is made by Company name, followed by numberof IP's, followed by which versions are covered, followed by ENT (enterprise) or SBE(Small business Edition). The license number is 13 characters long.

Button Db credentials enable the database credentials configuration window in theEnterprise edition. This information is saved in the registry. The credentials can be

password-less (i.e. user without password). However, a user name must be enteredbefore the service can be started. By default the user name is Syslog.

Tab Database

SyslogServer Enterprise Edition features a backup aid, which moves older syslog entries to a separatedatabase table, the Syslog_backup table at midnight according to settings. From there regular databasebackup can easily be done without affecting the main Syslog database. If the primary table, Syslog,becomes very large (i.e. many million entries) this procedure also greatly increases the speed of theDatagram SyslogView application.


14/27

Datagram

PAGE 14 (27)

Move entries to storage database...: Choose if the backup feature is to be active. Whenactivated, the Syslog Server performs backup operations shortly after midnight if any syslogentries quality for purge to the syslog_backup table.

Delete entries from main database when...: This field describes how many days logs are to bekept in the Syslog table before being permanently deleted. This option is only available ifentries are not moved to the backup database.

Delete entries from storage database when...: This field describes how many days logs are tobe kept in the Syslog_backup table before being permanently deleted. This option is onlyavailable if entries are moved to the backup database.

Maintenence settings

When the backup operation (if any) has completed after midnight, Syslogserver checks to see ifit's time for an optimization. For many of the database solutions, there exists various optimizationcommands. These commands are designed to keep the performance up by restructuring indices orcompacting the database.

Next optimization date indicates Syslogserver plan for the next optimization. A settingvery far into the future would effectively inactivate optimization.


15/27

Datagram

PAGE 15 (27)

Optimization interval indicates how often the optimizations are performed, by default 4days.

Tab Log files

Log File Directory

All received syslog messages are first saved to a regular file on the hard disk, and theninserted into the database. The primary reasons to first store it in a file are to mitigate databasebottlenecks and possible database unavailability. It also provides the user with an alternativeSyslog backup source. These backup files can later be loaded into the temporary table foranalysis of entries that has been removed from the main Syslog table. At service start time anew log file is always created. The file name is based on creation time(syslog_yymmdd_hhmmss.log).

Log file directory: Choose where the log files are to be stored. Click on the "..." buttonto choose directory. A directory on local disks should be used. For the Small BusinessEdition, this catalog is also used for the Access database file (called Syslog.mdb).

Log file rotation size

Log file rotation size: Configure at what size file rotation occurs, expressed inmegabytes. In case of database unavailability, a log rotation still occurs. The old data


16/27

Datagram

PAGE 16 (27)

file is still queued to be inserted as long as the service is running. Only one backlogfile at the time is supported. Therefore, larger file size supports longer databasedowntime. Maximum size is 2000Mb.

Log file backup setting

Delete log file after successful insertion: Choose if the log file is to be deleted aftercompleted successful database insertion. Unexpected events, such as unexpected

shutdown or major software errors, results in log files that will not be deleted by theSyslogServer. The reason for this is that log entries in that log file might not havebeen correctly entered into the database.

Tab Mail Support

Mail alarm settings

Datagram SyslogServer offers mail alarm support. Alarms can be presented via SyslogView'spop-ups and/or email. If email messages are to be sent, both fields must contain valid values.

Mail Gateway: The name or IP address of the mail gateway. The gateway will becontacted on port 25 using the SMTP protocol.

Mail Sender: All alarm emails will have this sender name. Depending on gatewayconfiguration, the email sender address must contain a valid domain in order for themail to be accepted by the gateway. Using your companys' regular email domainname should work fine (for instance [email protected])


17/27

Datagram

PAGE 17 (27)

Administrator address

As of version 2.2, Syslogserver can send notification to the administrator should errors occur.

Administrator mail adress: The adress to which the administrator alarms will be sent.

Reporting Level: How severe an error that is to be reported by mail. Three reportinglevels are available:

o Permanent errors only: Syslogserver terminating, or similar (Syslog severityalert).

o Critical and above: Normal operation halted, but may recover. Risk forpermanent errors. (Syslog severity critical or higher).

o Error and above: An error occured, but normal operations continues.Information may be lost. (Syslog severity error or higher).

A maximum of 6 mails per 30 minutes can be sent, in order to avoid spamming in case orreoccuring errors. The last allowed mail in the 30 minutes interval will contain text indicatingthat further errors in the near future will not be sent.


18/27

Datagram

PAGE 18 (27)

Tab Advanced

Alarm check interval

Syslogserver performes alarm rule parsing every two minutes, by default, to identify possiblenew alarms. The interval can be modified. A long interval means that some time might passbefore the alarm is identified. A too short interval means that Syslogserver might not completewithin the interval. If that does happen, Syslogserver generates an information messages. Forinstallations where this happens regularly, this value should be changed to a higher setting.

If the alarm process is not done within six times the alarm loop timer, it is terminated and theerror is reported.

Name lookup

The 'Syslog Host' column in syslogView always indicates from what IP the syslog entry camefrom. The 'Host' column contains what the host information in the actual syslog message said.From SyslogAgent, the host field is always the host name. For Unix, the IP address is often,but not always, used. There are also units that ignore the host field (not respecting RFC 3164).


19/27

Datagram

PAGE 19 (27)

Checking the box instruct the SyslogServer to resolve the 'Host' column information, therebytranslating IP addresses into names. There is a performance penelty for this setting, but it canbe mitigated by setting a high purge interval.

The DNS cache purge interval can for static environments be set to hours or even days. In adynamic environment, minutes may instead be a more reasonable settings.

Load backup files to temporary table

Datagram SyslogServer offers loading of old backup files into a temporary table for analysispurposes. The temporary table is not indexed, hence searches are slower than for the main,Syslog, table. Use SyslogView to analyze the contents by changing database table focus in themenu.

Load file(s): Choose one of more backup files. The counter in SyslogConf windowshow the progress.

Clear table: This button empties the Analysis table.


20/27

Datagram

PAGE 20 (27)

Appendix 1 Database table information

Tables and permissions in Microsoft SQL Server syntax.

Table 'syslog':

Make id the primary key. Set additional indicies on facility, severity, header_host and msg_tag.

Table 'syslog_analyze':

Identical in structure as 'Syslog', but without the indicis.


21/27

Datagram

PAGE 21 (27)

Table 'syslog_backup':

Do not set auto-increment on this table, as the Id is kept whentransfering from the Syslog table. No other indicies are used.


22/27

Datagram

PAGE 22 (27)

Table 'alarmdefinitions':

Alarmdefinitions: Set id to auto-increment. No other index is used.


23/27

Datagram

PAGE 23 (27)

Table 'triggered_alarms':

Triggered_alarms: Set id to auto-increment. No other index is used.


24/27


25/27

Datagram

PAGE 25 (27)

MS SQL commands:

These commands create suitable database, tables and indicies. They do not restrict permissions.

create database syslog;use syslog;

create table syslog (id BIGINT NOT NULL IDENTITY PRIMARY KEY, facility tinyint,severitytinyint,syslog_time datetime,syslog_host varchar(16),header_time varchar(20),header_hostvarchar(255),msg_tag varchar(32),msg_content varchar(1024));

create index fac on syslog(facility);create index sev on syslog(severity);create index hhost on syslog(header_host);create index tag on syslog(msg_tag);

create table syslog_backup (id BIGINT NOT NULL PRIMARY KEY, facility tinyint,severitytinyint,syslog_time datetime,syslog_host varchar(16), header_time varchar(20),header_hostvarchar(255),msg_tag varchar(32),msg_content varchar(1024));

create table triggered_alarms (id INT NOT NULL IDENTITY PRIMARY KEY, alarm_definition_idsmallint,Definition_name varchar(64),triggered_time datetime, Log_entries_list varchar(280));

create table alarmdefinitions (id INT NOT NULL IDENTITY PRIMARY KEY, active bit, deleted bit,definition_name varchar(64), SQL_query varchar(1024), border_value varchar(8), larm_intervalsmallint, next_run_time datetime, email_addresses varchar(255),popup_active bit);

create table license_information (id INT NOT NULL PRIMARY KEY, licensestring varchar(255),license varchar(32));

create table syslog_analyze (id BIGINT NOT NULL IDENTITY PRIMARY KEY, facilitytinyint,severity tinyint,syslog_time datetime,syslog_host varchar(16), header_timevarchar(20),header_host varchar(255),msg_tag varchar(32),msg_content varchar(1024));


26/27

Datagram

PAGE 26 (27)

Troubleshooting

In case of problems these tips might help you. Also check out the FAQ and troubleshooting guide atwww.syslogserver.com.

Problem: The service cannot be installed!

Verify that you are an Administrator on the server. Otherwise reboot and try again.Check the Event log for more information.

Problem: The service will not start when I press Start Service!

SyslogServer performs several checks at startup. The program will inform you if itfails to contact the database, fail to authenticate, or if the tables seem to be non-existant or of the wrong format. Also ensure you have proper user rights to startservices.

Problem: No syslog entries appear in SyslogView!

Start the SyslogConfig application. Stop and start the service. Any database problemswill be reported. SyslogServer Enterprise Edition users must also ensure that a systemDSN is used (rather than a user DSN) since services can not use user DSN:s. Alsoensure that no filters are active in the SyslogView application that might filter out allmessages! A good debugging source for further investigation is a file calledservice.log in the installation catalogue.

Problem: SyslogServer fails to parse a certain syslog message correctly.

Many OS:es actually fail to follow the Syslog protocol (RFC 3164). Therefore,

SyslogServer has a parsing library for different known styles. For a message that doesnot fall into any of the known types, a generic format is used. Any strange format isthe result of the generic type or a message parsed with an unsuitable style. Pleasereport this to [email protected]. Supply the line from the input log file, and alsowhat system the entry came from.

Problem: SyslogServer sends syslog messages that my license has been exceeded.

Datagram SyslogServer bases the license on the number of different IP addressess thathas been received since the service last started. If the service has been active for along time some changes in IP presence might have occured, causing SyslogServer tohave received a total of more IP addresses than is currently connected on the network.


27/27

Datagram

PAGE 27 (27)

The solution is to simply restart the service in order to reset the counter. Please notethat the IP address of the SyslogServer is excluded from the license count. ASyslogAgent installation for the SyslogServer is always recommended.

Problem: I'm experiencing other problems with SyslogServer.

We are sorry to hear you experience problems with our product, and we want to help

you.SyslogServer uses a problem log file called service.log which can be found in the installationdirectory. The contents in this file might help you in understanding the problems, and therefore maybesolving it. In any case, we are most interested in your service log file.

There is also debug flags that can be used for increased file logging. Specify -DALARM and/or-DBACKUP and/or -DSERVICE as an argument to the service.

Please contact us at [email protected] (customers at [email protected]) for moreassistance.

Documents

Datagram SyslogServer manual.pdf