State of the art logging

State of the art loggingSyslog-ng, journal, CEE/Lumberjack and ELSA

Péter Czanikcommunity manager

Topics

• No, it is not about cutting trees :-)• What is syslog? And syslog-ng?• Free-form messages against name-value pairs• The new buzzword: journal• Standardization efforts: CEE/Lumberjack• Name-value pairs at work: ELSA

What is syslog?

• Logging: recording events

• Syslog:- Application: collecting events- Protocol: forwarding events

What is syslog-ng?

• “Next Generation” syslog server• “Swiss army knife” of logging

• More input sources (files, sockets, and so on)• Better filtering (not only priority, facility)• Processing (rewrite, normalize, correlate, and so

on)• More destinations (databases, encrypted network,

and so on)

What is new since 2.0

• 2.0 is best known, but EOL• Most important new features since 2.0:

- PatternDB and CSV message parsing- Correlation- SQL and MongoDB destinations- JSON formatting- Modularization- Multi-threading

• Next: 3.4- JSON parsing- More flexible configuration

Free form log messages

• Most logs are in /var/log• Most are from syslog (but also wtmp, apache, and

so on)• Most are: date + hostname + text

Mar 11 13:37:56 linux-6965 sshd[4547]: Accepted keyboard-interactive/pam for root from 127.0.0.1 port 46048 ssh2

• Text = English sentence with some variable parts• Easy to read

Why it does not scale?

• Few logs (workstation) → easy to find information• Many logs (server) → difficult to find information• Relevant information is presented differently by

each application• Difficult to process them with scripts

• Answer: structured logging- Events represented as name value pairs

Solution from syslog-ng: PatternDB

• Most messages are static texts with some variable parts embedded

• PatternDB parser:- Can extract useful information into name-value pairs- Add status fields based on message text

• Example:- user=root- action=login- status=failure

• It requires patterns• syslog-ng: name-value pairs inside

Journal

• The logging component of systemd• Name-value pairs inside:

- Message- Trusted properties- Any additional name-value pairs

• Native support for name-value pair storage

Journal: the enemy?

• FAQ: Q: is journal the enemy? A: No!• Journal is limited to Linux/systemd (syslog-ng: all

Linux/BSD/UNIX)• Journal is local only (syslog-ng: client – server)• Journal does not filter or process log messages

• Journal + syslog-ng complement each other• Logs forwarded to syslog-ng through:

/run/systemd/journal/syslog

• syslog-ng can filter, process and forward logs to many different destinations (one day also to journal)

• Journal, syslog-ng, Windows eventlog, rsyslog, auditd, and so on are based on name-value pairs

• All use different field names• Standardization is a must: CEE → Common Event

Expression• Events: name-value pairs instead of free-form text

- Taxonomy: name-value pairs to describe events (example: status)- Dictionary: name-value pairs for event parameters (example: user)

• PatternDB can turn free-form messages into CEE

Lumberjack

• Make CEE happen → implementation• Coordinated by RedHat

- CEE (Mitre), syslog-ng, rsyslog, and so on- Open, with high traffic mailing list- https://fedorahosted.org/lumberjack/

• API(s) to make structured logging easier• Work on dictionary, taxonomy, transport issues

Name-value pairs in action: ELSA

• ELSA: Enterprise Log Search and Archive• Based on syslog-ng, PatternDB and MySQL• Simple and powerful web GUI• Extreme scalability• Patterns focused on network security (Cisco,

Snort, HTTP, Bro, and so on)

Some logs

Diagram

A few extras

Questions?

• Questions?

Thank You!Péter Czanik

community managerpeter.czanik@balabit.com

State of the art logging

Technology

26) The Hollywood Art State Of The Art

state of the art

The State of Art History Contemporary Art

The Art Of Application Logging PHPNW12

Northwest McGregor Field CO2 Huff ‘n’ Puff: A Case Study ......• The test provides the opportunity to investigate the use of state-of-the-art downhole logging techniques (reservoir

Rev (7-13) STATE GAME LANDS #: 184 SALE: 184-2019-01 ...€¦ · The PGC recommends that logging crews working on State Game Lands be trained in safe logging techniques and sound

State of the Art Logging. Kibana4Solr is Here!

Adaptive Logging: Optimizing logging and recovery …ooibc/alogging-sigmod16.pdf · Adaptive Logging: Optimizing Logging and Recovery ... of data logging versus command logging becomes

DOE Award No.: DE-FE0023919...will be accomplished through the planning and execution of a state-of-the-art drilling, coring, logging, testing and analytical program that assess the

Current state of art

ART..... State of art

State Rules Open Logging ‘Loophole’

PREDICTING HISTORICAL LOGGING CAPITOL STATE FOREST, WA

The State of the Logging Workforce in the Southern United

10 THE DECLINE IN ILLEGAL LOGGING IN KACHIN STATE · Part B: Global Witness Research and Investigations in Kachin State 2006-09 / 10 The Decline in Illegal Logging in Kachin State

The Art of Logging An exploration with Apache Log4j 2 Gary Gregory garydgregory@gmail.com

Scran for Art and Design Logging in Browsing and Searching Topics Scran Training PowerPoint 6

State of the art logging

Thru-Pipe Logging System (ThruLog) Wireline Logging System20191108… · Slam Logging System (SlamLog) Slim Hostile Logging System (HostileLog) Thru-Pipe Logging System (ThruLog)

Logging Configuration - Cisco€¦ · Logging Configuration Thefollowingdescribeshowtoenableauditandeventloggingonthecontroller. • LoggingConfigurationOverview,page1 Logging Configuration