The Linux Audit Framework

The Linux Audit FrameworkGary Smith, EMSL, Pacific Northwest National Laboratory

A Little Context

Cyber Security is all about managing risk.How do you think about managing risk?The Five Golden Principles of Security

Know your systemPrinciple of Least PrivilegeDefense in DepthProtection is key but detection is a must.Know your enemy.

2

Introduction

Linux audit helps make your system more secure by providing you with a means to analyze what is happening on your system in great detail. It does not, however, provide additional security itself—it does not protect your system from code malfunctions or any kind of exploits. Instead, Audit is useful for tracking these issues and helps you take additional security measures, like SELinux, to prevent them.Audit consists of several components, each contributing crucial functionality to the overall framework. The audit kernel module intercepts the system calls and records the relevant events.

3

Introduction (cont.)

The auditd daemon writes the audit reports to disk. Various command line utilities take care of displaying, querying, and archiving the audit trail.

4

Linux Audit Framework Capabilities

Audit enables you to do the following: Associate Users with Processes

Audit maps processes to the user ID that started them. This makes it possible for the administrator or security officer to exactly trace which user owns which process and is potentially doing malicious operations on the system.

Review the Audit TrailLinux audit provides tools that write the audit reports to disk and translate them into human readable format.

Review Particular Audit EventsAudit provides a utility that allows you to filter the audit reports for certain events of interest.

5

Linux Audit Framework Capabilities (1)

You can filter for: User Group Audit ID Remote Hostname Remote Host Address System Call System Call Arguments File File OperationsSessionSuccess or Failure

6

Linux Audit Framework Capabilities (2)

Apply a Selective AuditAudit provides the means to filter the audit reports for events of interest and also to tune audit to record only selected events. You can create your own set of rules and have the audit daemon record only those of interest to you.

Prevent Audit Data LossAudit provides several mechanisms to prevent the loss of audit data in the event of a loss of system resources.

7

The Components of Linux Audit

8

Configuring The Linux Audit Framework

Before you can actually start generating audit logs and processing them, you must configure the audit framework.Julius Caesar said, “Gallia est omnis divisa in tres partes”, and just like Gaul, the configuring the audit framework is divided into three parts:

The Audit Daemon ConfigurationThe Audit RulesThe Audispd Daemon Configuration

9

/etc/audit/auditd.conf

The /etc/audit/auditd.conf configuration file determines how the audit system functions once the daemon has been started. For most use cases, the default settings shipped with the package should suffice.Let’s take a look at a sample auditd configuration file.

10

A Sample auditd.conf

log_file = /var/log/audit/audit.loglog_format = RAWlog_group = rootpriority_boost = 4flush = INCREMENTALfreq = 20num_logs = 5dispatcher = /sbin/audispddisp_qos = lossyname_format = NONE##name = mydomainmax_log_file = 6max_log_file_action = ROTATEspace_left = 75space_left_action = SYSLOGaction_mail_acct = rootadmin_space_left = 50admin_space_left_action = SUSPENDdisk_full_action = SUSPEND

11

Setting Up Audit Rules

We’ve given auditd its marching orders; now we have to define what we are interested in auditing.Audit rules are used to specify which components of your system are audited. There are three basic types of audit rules:

Basic audit system parameters File and directory watches System call audits

Before creating an audit rule set and before rolling it out to your system, carefully determine which components to audit. Extensive auditing can cause a substantial logging load.Remember: First match wins!

12

Setting Up Audit Rules (1)

Make sure that your system provides enough disk space to store large audit logs and test your audit rule set extensively before rolling it out to production. Audit rules can either be passed to the audit system by the command line using auditctl or bundled into a rules file located under /etc/audit/audit.rules that is read during the start of the audit daemon.

13

A Sample audit.rules

# basic audit system parameters-D -b 8192 -f 1 -e 1 # some file and directory watches -w /etc/audit/auditd.conf -p rxwa -w /etc/audit/audit.rules -p rxwa -w /var/log/audit/ -w /etc/passwd -p rwxa -w /sbin/auditctl –p x# an example system call rule -a entry,always -S umask

14

File Watches Caveats

Directory watches produce less verbose logs than exact file watches. When in need of detailed file-related records, enable separate file watches for all files of interest. Pathname globbing of any kind is not supported by audit. Always use the exact pathnames. Auditing can only be performed on existing files. Any files added while the audit daemon is already running are ignored until the audit rule set is updated to watch the new files.

15

Assigning Keys to Rules

Assigning keys to your audit rules helps you to identify any records related to this rule in the logs. An example rule plus key:

-w /etc/selinux -k MAC-PolicyYou may use the same key on different rules in order to be able to group rules when searching for them. It is also possible to apply multiple keys to a rule

-w /sbin/auditctl -p x -k privileged -k ids-exec-infoUsing the ausearch log analyzer, you can easily filter for any events related to this particular rule.

16

Auditing the Execution of Setuid/Setgid Binaries

Let’s say that as a matter of compliance, you have to audit the execution of setuid/setgid binaries on your system.How do you do set that up?First, run a script like this at boot time from /etc/rc.local sending the output to a temp file, /tmp/snorf, for example.

17

Auditing the Execution of Setuid/Setgid Binaries (1)

#!/bin/bash# Find all the file systems that are locally mountedfor i in `/bin/egrep '(ext4|ext3|ext2)' /etc/fstab | /bin/awk '{print $2}'`do# Find all the files on the file system found above and print out# and audit rule for it/usr/bin/find $i -xdev -type f $ -perm -4000 -o -perm 2000 $ -print | \ /bin/sort | /bin/awk '{ print "-a always,exit -F path=" $1 " -F perm=x \-F auid>500 -F auid!=-1 -k privileged -k ids-exec-high" }'done

18


And you get something like this (YMMV depending on what’s installed).

-a always,exit -F path=/bin/fusermount -F perm=x -F auid>500 -F auid!=-1 -k privileged -k ids-exec-high-a always,exit -F path=/bin/ping -F perm=x -F auid>500 -F auid!=-1 -k privileged -k ids-exec-high-a always,exit -F path=/bin/ping6 -F perm=x -F auid>500 -F auid!=-1 -k privileged -k ids-exec-high-a always,exit -F path=/bin/su -F perm=x -F auid>500 -F auid!=-1 -k privileged -k ids-exec-high-a always,exit -F path=/usr/bin/chage -F perm=x -F auid>500 -F auid!=-1 -k privileged -k ids-exec-high-a always,exit -F path=/usr/bin/chfn -F perm=x -F auid>500 -F auid!=-1 -k privileged -k ids-exec-high-a always,exit -F path=/usr/bin/chsh -F perm=x -F auid>500 -F auid!=-1 -k privileged -k ids-exec-high-a always,exit -F path=/usr/bin/crontab -F perm=x -F auid>500 -F auid!=-1 -k privileged -k ids-exec-high

19


Then, point auditctl at the temp file to add the newly created audit rules. The auditctl program is used to control the behavior, get status, and add or delete rules into the kernel’s audit system.

/sbin/auditctl –R /tmp/snorfA couple of things about auditctl:

auditctl is not a filter, so output cannot be piped into it.Rules files for auditctl must be owned by root.

20

System Call Auditing

System call auditing lets you track your system's behavior on a level even below the application level. The audit subsystem supports an ample collection of events, to include the tracing of arbitrary system calls identified by system call name, or by system call number.The audit subsystem can also filter by PID, UID, system call success, system call argument, and many other possibilities. When designing these rules, consider that auditing a great many system calls may increase your system load and cause you to run out of disk space. Remember: First match wins!

21

System Call Audit Rules Examples

-a always,exit –S settimeofday-a always,exit -F arch=b64 -S chmod-a always,exit -F arch=b64 -S chown -F exit=-EACCES-a always,exit -F arch=b64 -S creat -F exit=-EACCES –F uid>=500-a always,exit -F arch=b64 -S unlink -F auid>=500 -F auid!=-1-a always,exit -F arch=b64 -S init_module -S delete_module –k modules –k ids-sys-info-a always,exit -S execve –F auid>500 -F uid=0 –k root-exe-a never,exit -F path=/somefile.dat -S unlink -S unlinkat -S rename -S renameat

For details on setting up audit rules, see auditctl(8).

22

Creating Audit Reports

The audit records are stored in /var/log/audit/audit.log.grep is your friend and you can pull stuff out of the audit log and get stuff like this:type=SYSCALL msg=audit(1365719016.212:333043): arch=c000003e syscall=171 success=yes exit=0 a0=7fff86310c37 a1=6 a2=d a3=7fff8630f3b0 items=0 ppid=22300 pid=22311 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=37491 comm="domainname" exe="/bin/hostname" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=73797374656D2D6C6F63616C65016964732D7379732D6C6F77

Maybe grep isn’t your friend, after all.The raw audit data auditd stores in the /var/log/audit directory is quite complex.To find what you want, you might have to sift through bazillions of other events before you locate the one that you want.

23

Creating Audit Reports (1)

Use aureport to create concise, human-readable reports.Some of the useful options are:

--summary--failed--start and --end (aureport understands today, yesterday, now, recent, this-week, this-month, and this-year)--auth, --avc, --login, --user, --executable, --syscall

To get started, do aureport –summary and you get something like this:

24


Summary Report======================Range of time in logs: 03/01/2013 08:17:01.765 - 03/11/2013 16:07:49.252Selected time for report: 03/01/2013 08:17:01 - 03/11/2013 16:07:49.252Number of changes in configuration: 669Number of changes to accounts, groups, or roles: 10Number of logins: 11Number of failed logins: 2941Number of authentications: 38Number of failed authentications: 5Number of users: 3Number of terminals: 10Number of host names: 6Number of executables: 20Number of files: 597Number of AVC's: 347Number of MAC events: 11Number of failed syscalls: 76Number of anomaly events: 0Number of responses to anomaly events: 0Number of crypto events: 11882Number of keys: 20Number of process IDs: 17444Number of events: 120152

25


Lets look at some of the failed logins with aureport –l –failed:

Login Report============================================# date time auid host term exe success event============================================1. 03/07/2013 15:00:37 root 192.168.1.101 ssh /usr/sbin/sshd no 1781162. 03/07/2013 15:01:52 root 192.168.1.101 ssh /usr/sbin/sshd no 1781353. 03/07/2013 15:01:54 (unknown user) 192.168.1.101 ssh /usr/sbin/sshd no 1781464. 03/07/2013 15:01:54 (unknown user) 192.168.1.101 ssh /usr/sbin/sshd no 1781475. 03/07/2013 15:02:01 root 192.168.1.101 ssh /usr/sbin/sshd no 1781696. 03/07/2013 15:02:02 (invalid user) 192.168.1.101 ssh /usr/sbin/sshd no 1781847. 03/07/2013 15:02:02 (invalid user) 192.168.1.101 ssh /usr/sbin/sshd no 1781878. 03/07/2013 15:02:02 (unknown user) 192.168.1.101 ssh /usr/sbin/sshd no 1781999. 03/07/2013 15:02:04 (unknown user) 192.168.1.101 ssh /usr/sbin/sshd no 17820310. 03/07/2013 15:02:04 (unknown user) 192.168.1.101 ssh /usr/sbin/sshd no 17820511. 03/07/2013 15:02:04 (invalid user) 192.168.1.101 ssh /usr/sbin/sshd no 17821012. 03/07/2013 15:02:06 (unknown user) 192.168.1.101 ssh /usr/sbin/sshd no 17822013. 03/07/2013 15:02:06 (invalid user) 192.168.1.101 ssh /usr/sbin/sshd no 17823014. 03/07/2013 15:02:33 ftp 192.168.1.101 ssh /usr/sbin/sshd no 178332

26

Drilling Deeper with ausearch

Using aureport lets you to create overall summaries of what is happening on the system, but if you want to drill deeper into the details of a particular event, ausearch is the tool to use. ausearch allows you to search the audit logs using special keys and search phrases that relate to most of the flags that appear in event messages in /var/log/audit/audit.logA methodology to use is find an event class of interest with aureport and then drill down into the nitty-gritty with ausearch.For instance, you use aureport –syscall –failed to see the failed system calls. Use ausearch and one of the event ids to get more information.

27

Drilling Deeper with ausearch (1)

From aureport –syscall –fail we get:Syscall Report=======================================# date time syscall pid comm auid event=======================================1. 04/09/2013 16:19:15 87 4006 semodule 25016 53039

From ausearch –i –a 53039 we get:type=PATH msg=audit(04/09/2013 16:19:15.564:53039) : item=0 name=/etc/selinux/targeted/modules/tmp/disable_dontaudit inode=394090 dev=fd:00 mode=dir,700 ouid=root ogid=root rdev=00:00 obj=unconfined_u:object_r:semanage_store_t:s0type=CWD msg=audit(04/09/2013 16:19:15.564:53039) : cwd=/roottype=SYSCALL msg=audit(04/09/2013 16:19:15.564:53039) : arch=x86_64 syscall=unlink success=no exit=-2(No such file or directory) a0=0x7f7e0a429c60 a1=0x0 a2=0x2 a3=0x7fff08fa5450 items=1 ppid=20218 pid=4006 auid=blotto uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts0 ses=6353 comm=semodule exe=/usr/sbin/semodule subj=unconfined_u:unconfined_r:semanage_t:s0-s0:c0.c1023 key=MAC-policy key=ids-sys-low key=ids-file-info

28

Drilling Deeper with ausearch (2)

A useful feature, if you tagged your audit rules with keys, is to search for events based on those keys.For instance, ausearch -i -k identity –start recent and we get:

type=PATH msg=audit(03/10/2013 17:28:14.821:326340) : item=0 name=/etc/shadow inode=786095 dev=fd:00 mode=file,000 ouid=root ogid=root rdev=00:00type=CWD msg=audit(04/11/2013 17:28:14.821:326340) : cwd=/home/gorgotype=SYSCALL msg=audit(04/11/2013 17:28:14.821:326340) : arch=x86_64 syscall=open success=no exit=-13(Permission denied) a0=0x7fffe185972e a1=O_WRONLY|O_TRUNC a2=0x0 a3=0x7fffe18571a0 items=1 ppid=22814 pid=25045 auid=gorgo uid=gorgo gid=users euid=gorgo suid=gorgo fsuid=gorgo egid=users sgid=users fsgid=users tty=pts0 ses=46582 comm=cp exe=/bin/cp key=identity key=ids-file-info

29

Visualizing Audit Data

Neither the data trail in /var/log/audit/audit.log nor the different report types generated by aureport, provide an intuitive reading experience to the user. The aureport output is formatted in columns and thus easily available to any sed, perl, or awk scripts that users might connect to the audit framework to visualize the audit data.A solution: mkbar and mkgraph were created by Steve Grubb at Red Hat. They are available from http://people.redhat.com/sgrubb/audit/visualize/.Note: These scripts need gnuplot and graphviz to create their visualizations.

30

http://people.redhat.com/sgrubb/audit/visualize/

Visualizing Audit Data (1)

Create a plot of events with aureport -e -i --summary | mkbar events

31


Create a summary of syscall events with aureport -s -i --summary | mkbar syscall

32


To create a summary chart of successful or failed events of any of the above event types, just add the --success or --failed option to the respective aureport command. To cover a certain period of time only, use the -ts and -te options on aureport. Any of these commands can be tweaked further by narrowing down its scope using grep or egrep and regular expressions.

33

Visualizing Audit Data Relationships

To illustrate the relationship between different kinds of audit objects, such as users and system calls, use the script mkgraph.Graphs can also be combined to illustrate complex relationships. See the comments in the mkgraph script for further information and an example.

34

Visualizing Audit Data Relationships (1)

35

To graph the syscalls to programs, do aureport -s -i | awk '/^[0-9]/ { printf "%s %s\n", $6, $4 }' | sort | uniq | mkgraph syscall-vs-program

Visualizing Audit Data Relationships (2)

To graph to successful programs to files, do LC_ALL=C aureport -f -i --success | awk '/^[0-9]/ { print $7" "$4 }' | sort | uniq | mkgraph program-vs-file

36

Resources

The Audit Manual PagesThere are several man pages installed along with the audit tools that provide valuable and very detailed information:

auditd(8) The Linux Audit daemon auditd.conf(5) The Linux Audit daemon configuration file auditctl(8) A utility to assist controlling the kernel's audit system autrace(8) A program similar to strace ausearch(8) A tool to query audit daemon logs aureport(8) A tool that produces summary reports of audit daemon logs audispd.conf(5) The audit event dispatcher configuration file audispd(8) The audit event dispatcher daemon talking to plugin programs.

37

Resources (1)

http://people.redhat.com/sgrubb/audit/index.html The home page of the Linux audit project. This site contains several specifications relating to different aspects of Linux audit, as well as a short FAQ. /usr/share/doc/audit The audit package itself contains a README with basic design information and sample .rules files for different scenarios:

capp.rules: Controlled Access Protection Profile (CAPP)lspp.rules: Labeled Security Protection Profile (LSPP)nispom.rules: National Industrial Security Program Operating Manual Chapter 8(NISPOM)stig.rules: Secure Technical Implementation Guide (STIG)

38

http://people.redhat.com/sgrubb/audit/index.html

T-t-t-t-that’s all, folks!

39

Gary SmithInformation System Security Officer, Molecular

Science Computing, EMSL, Pacific Northwest National Laboratory

Richland, [email protected]

mailto:[email protected]

mailto:[email protected]

Documents

The Linux Audit Framework