58
SECURITY-RELATED MONITORING WITH ZABBIX

SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

Page 2: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

2

SECURITY MONITORING – WHY ?

Potential issues

Software vulnerabilities

Weak configurations

Unnecessary open ports

Physical intrusions

Page 3: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

3

WHY ZABBIX ?

Zabbix is not a dedicated security monitoring tool….

but you can monitor the following

configuration files

log files

SNMP traps

and much more…….

Page 4: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

CHECKSUM MONITORING

Page 5: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

5

WHY MONITOR CHECKSUMS ?

Because it is the simplest way to detect changes to important files

works out of box

very simple to setup

efficient

Page 6: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

6

HOW TO MONITOR CHECKSUMS ?

Two types of checksums supported:

vfs.file.cksum[file] - calculates a 32-bit ckecksum (CRC-32)

vfs.file.cksum[/etc/passwd] = 1222364044

vfs.file.md5sum[file] - calculates a 128-bit MD5 hash

vfs.file.md5sum[/etc/passwd] =7bae6950d85b2d6fc4620d2a783b93b5ea95280ab086e3ff1442d9ede113fd17

Page 7: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

CONFIG FILES

Page 8: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

8

CONFIGURATION ISSUES

Default configuration gives a lot of information

And while it is very useful for deployment and troubleshooting….

It can contain known weaknesses

It can also give very valuable information to potential hackers !!!

Page 9: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

9

HOW TO MONITOR CONFIGURATION ?

You can monitor the content of a configuration filevfs.file.contents[file] – returns back the content of a file

The most important parts of a configuration file can be monitored using dependent items.

Page 10: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

10

ZABBIX EXAMPLE

Page 11: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

11

RESULTS

Page 12: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

VULNERABILTY SCANS

Page 13: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

13

WHAT IF YOU ARE NOT A SECURITY EXPERT ?

External programs can be used to check vulnerabilities

Output can be parsed, and useful information extracted

Triggers can be created to send out alerts

Page 14: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

14

HOW IT WORKS

External check Report item

DependentItems

Triggers Alarm

Page 15: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

15

MONITORING USING SCRIPTS

Example of a security report

Page 16: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

16

HOW TO EXTRACT INFORMATION ?

Use Zabbix built – in preprocessing

Regular expressions

JSON PATH

XML PATH

CSV to JSON

JavaScript

Page 17: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

17

LLD PREPROCESSING POSSIBILITIES

Page 18: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

18

DEPENDENT ITEMS EXAMPLE

Page 19: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

ADVANCED VULNERABILTY SCANS

Page 20: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

20

CAN YOU DO SOMETHING WITH THIS REPORT ?

Page 21: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

21

PROCESS THE REPORT USING LLD

Any JSON format data can be processed by LLD

External check Report LLD RuleDependent items

Page 22: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

22

LLD RULE DESIGN

Page 23: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

23

LLD ITEM PROTOTYPES

Page 24: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

24

ITEMS CREATED FROM THE REPORT

Page 25: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

25

WHAT IF THE SCRIPT TAKES TOO LONG TO EXECUTE ?

Maximum execution time is 30s …..

In this case cron jobs or other scheduling mechanisms can be used

Page 26: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

SERVICES MONITORING

Page 27: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

27

CAN WE MONITOR SERVICES OUT OF BOX ?

Yes – using new Zabbix agent 2

Two new item keys supportedsystemd.unit.discovery[<type>] List of systemd units and their details.

type - all, automount, device, mount, path, service (default), socket, swap, target

systemd.unit.info[<unit name>,<property>,<interface>] Systemd unit information

unit name - unit nameproperty - unit property (e.g. ActiveState (default), LoadState, Description)interface - unit interface type (e.g. Unit (default), Socket, Service)

Page 28: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

28

SERVICES MONITORING EXAMPLE

Page 29: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

PORT MONITORING

Page 30: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

30

CAN WE MONITOR OPEN PORTS ?

Yes, of course !

Zabbix can do it out of box

check open ports using net.tcp.port[] simple check

use discovery to scan your entire network for open ports

Page 31: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

31

WHY WE NEED TO MONITOR OPEN PORTS ?

Why do you need this ?

Applications with weak security (telnet, ftp)

Unneeded applications with known vulnerabilities

Less open ports – more secure system

Page 32: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

32

SIMPLE NETWORK DISCOVERY RULE

Page 33: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

UNSECURE WEB PAGES

Page 34: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

34

HOW CAN WE FIND HTTP ENABLED PAGES ?

HTTPS is the recommended web protocol today

Open HTTP port does not mean the page is not redirected to HTTPS

How to check it ?

Use Zabbix built in web scenarios

check the response code

The HTTP response status code 301 Moved Permanently is used for permanent URL redirection

Page 35: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

35

WEB SCENARIO EXAMPLE

Page 36: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

36

CAN WE FIND UNSECURE HTTPS PAGES ?

You can use a webscenario to authenticate the certificate

verify that SSL certificate of the web server is valid

(trusted by a known certificate authority, not expired etc.)

verify that the Common Name field or the Subject Alternate Name field of the web server certificate matches the servername.

Page 37: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

37

HTTPS CERTIFICATE VALIDATION

Page 38: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

EXPIRED CERTIFICATES

Page 39: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

39

CAN WE FIND EXPIRED CERTIFICATES?

Community made externals cripts can be used to warn you

about yourcertificate expiration

You can monitor (for example)

time until expiration (if valid)

expired days ago (if expired)

Page 40: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

SNMP TRAPS

Page 41: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

41

CAN WE MONITOR SNMP TRAPS ?

Yes, using Zabbix SNMP trapper item

What to monitor ?

Administrative logins

Ports status up/down

New devices (MAC security)

Thresholds reached (Network attacks)

Any other security related checks

Page 42: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

42

HOW SNMP TRAPS WORK ?

Device SNMP trap SNMP trapper Zabbix Alarm

Page 43: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

SENSOR MONITORING

Page 44: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

44

MONITOR YOUR ENVIRONMENT WITH ZABBIX

Temperature sensors

Smoke sensors

Humidity sensors

Door sensors

Motion detection sensors

Page 45: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

LOG FILE MONITORING

Page 46: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

46

WHY DO YOU NEED TO MONITOR LOGS ?

A lot of security related information can be found in log files

For example

Unsuccessful logins

Successful logins !

Elevation of privileges

Page 47: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

47

LOG FILE MONITORINGLog files can be parsed to find important informationDependent items can be created from log itemsTriggers can be created to alert about serious security issuesInformation from log files can be extracted and used in trigger names and tags

Page 48: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

48

MASTER LOG ITEMMaster item contains all important log information

Page 49: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

49

DEPENDENT LOG ITEMSDependent items extract information from the main log

Page 50: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

50

DEPENDENT LOG ITEMS

Page 51: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

51

GATHERING USEFULL INFORMATION

Information can be extracted from the logs using function

regsub (<pattern>,<output>)

Extracted information can be used in

Trigger names

Trigger tags

Page 52: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

52

LOG TRIGGERS

Log line:

sudo: kaspars : user NOT in sudoers ; TTY=pts/3 ; PWD=/home/kaspars ; USER=zabbix ; COMMAND=/bin/ping

Examples to extract user and executed command

{{ITEM.VALUE}.regsub("sudo: (.+) :", user: \1)}

{{ITEM.VALUE}.regsub("COMMAND=(.+)", command: \1)}

Page 53: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

53

LOG BASED TRIGGER EXAMPLE

Page 54: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

54

USE TAGS TO FILTER INFORMATON!

Page 55: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

55

CAN WE MONITOR WINDOWS LOGS ?

Yes, a special key eventlog can be used

You can filter event logs by

Source ( Security, System etc…)

Severity ( “Warning”, “Error”, “Critical” etc…)

Eventid

4625 – Logon Failure

4740 - A user account was locked out)

Page 56: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

SECURITY-RELATED MONITORINGWITH ZABBIX

ZABBIX INTEGRATIONS

Page 57: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

57

ZABBIX INTEGRATION

Page 58: SECURITY-RELATED MONITORING WITH ZABBIX › files › zabbix_summit_2019 › Kaspars_Me… · SECURITY-RELATED MONITORING WITH ZABBIX UNSECURE WEB PAGES. 34 HOW CAN WE FIND HTTP ENABLED

THANK YOU!