Detection, Prevention, and Detection, Prevention, and Containment:Containment:
A Study of grsecurityA Study of grsecurity
Brad SpenglerBrad Spengler
http://www.grsecurity.nethttp://www.grsecurity.net
[email protected]@grsecurity.net
The Problem
The ProblemThe Problem
Bugs in software cause unexpected Bugs in software cause unexpected resultsresults
Unexpected functionality can result from Unexpected functionality can result from errors in design, implementation, or errors in design, implementation, or configurationconfiguration
Bugs can often be wielded for malicious Bugs can often be wielded for malicious purposes by an attackerpurposes by an attacker
Problems With the Current SolutionProblems With the Current Solution
Avoid / Identify / FixAvoid / Identify / Fix
Current state of security is a never ending Current state of security is a never ending rat racerat race
Endless cycle of vulnerability discovery Endless cycle of vulnerability discovery and fixingand fixing
Problems With the Current SolutionProblems With the Current Solution
Ultimate goal of today’s security – removal Ultimate goal of today’s security – removal of software bugs through auditingof software bugs through auditing
Security utopia – greatest result, though Security utopia – greatest result, though impossible to achieveimpossible to achieve
Problems With the Current SolutionProblems With the Current Solution
Auditing is expensive, slow, and requires a Auditing is expensive, slow, and requires a great deal of knowledgegreat deal of knowledge
Auditing provides no guarantees about the Auditing provides no guarantees about the security of the softwaresecurity of the software
Auditing cannot be fully automatedAuditing cannot be fully automated
EXAMPLE: format-string vulnerabilitiesEXAMPLE: format-string vulnerabilities
The (Attainable) Solution
The (Attainable) SolutionThe (Attainable) Solution
DetectionDetection
PreventionPrevention
ContainmentContainment
Advantages of the (Attainable) Advantages of the (Attainable) SolutionSolution
InexpensiveInexpensive
Can be mostly automatedCan be mostly automated
Works for known and unknown bugsWorks for known and unknown bugs
Allows administrators to focus more on Allows administrators to focus more on administration (checking logs..etc) instead administration (checking logs..etc) instead of rushing for the newest patchof rushing for the newest patch
Our solution:Our solution:
grsecuritygrsecurity
Overview of grsecurity
Background on grsecurityBackground on grsecurity
Started in February 2001Started in February 2001
Initial release was for Linux 2.4.1Initial release was for Linux 2.4.1
Originally a port of Openwall to Linux 2.4Originally a port of Openwall to Linux 2.4
Goals of grsecurityGoals of grsecurity
Configuration-free operationConfiguration-free operation
Complete protection against all forms of Complete protection against all forms of address space modification bugsaddress space modification bugs
Feature-rich ACL and auditing systemsFeature-rich ACL and auditing systems
Operation on multiple processor Operation on multiple processor architectures and Operating Systemsarchitectures and Operating Systems
Features of grsecurityFeatures of grsecurity
A robust ACL system with an intelligent A robust ACL system with an intelligent userspace administration tooluserspace administration toolExtensive auditing capabilitiesExtensive auditing capabilitiesMeasures to stop the most common Measures to stop the most common methods of exploiting a system:methods of exploiting a system: Address space modificationAddress space modification Races (specifically filesystem races, most Races (specifically filesystem races, most
common of which are /tmp races)common of which are /tmp races) Breaking a chroot(2) jailBreaking a chroot(2) jail
Features of grsecurityFeatures of grsecurity
Supports sysctl so that it can be included Supports sysctl so that it can be included with Linux distributions and allow the user with Linux distributions and allow the user to modify the options to his/her likingto modify the options to his/her likingNetfilter module that drops connections to Netfilter module that drops connections to unserved TCP and UDP portsunserved TCP and UDP portsMany of the same randomness features as Many of the same randomness features as OpenBSDOpenBSDAn enhanced implementation of Trusted An enhanced implementation of Trusted Path Execution (TPE)Path Execution (TPE)
Detection in grsecurity
Detection in grsecurityDetection in grsecurity
Implemented in two formsImplemented in two forms AuditingAuditing Logging of real attacksLogging of real attacks
Inode and device numbers used wherever Inode and device numbers used wherever possiblepossible
Parent process info loggedParent process info logged
AuditingAuditing
Audited events include:Audited events include: Exec (with arguments)Exec (with arguments) Chdir(2)Chdir(2) Mount(2)/unmount(2)Mount(2)/unmount(2) IPC (semaphore, message queue, shared IPC (semaphore, message queue, shared
memory) creation and deletionmemory) creation and deletion
AuditingAuditing
Signals: SIGSEGV, SIGABRT, SIGBUS, Signals: SIGSEGV, SIGABRT, SIGBUS, SIGILLSIGILL
Failed forksFailed forks Ptrace(2)Ptrace(2) Time changes (stime(2), settimeofday(2))Time changes (stime(2), settimeofday(2)) Execs inside chroot(2)Execs inside chroot(2) Denied capabilitiesDenied capabilities
Prevention in grsecurity
Prevention in grsecurityPrevention in grsecurity
Prevention is implemented through PaX Prevention is implemented through PaX and hardening certain sections of the and hardening certain sections of the kernelkernelHardened syscalls include:Hardened syscalls include: Chroot(2)Chroot(2) Ptrace(2)Ptrace(2) Mmap(2)Mmap(2) Link(2)/symlink(2)Link(2)/symlink(2) Sysctl(2)Sysctl(2)
Prevention in grsecurity - PaXPrevention in grsecurity - PaX
What is PaX?What is PaX? PaX implements non-executable VM pages PaX implements non-executable VM pages
on architectures that do not support the non-on architectures that do not support the non-executable bit (currently only ia-32, more to executable bit (currently only ia-32, more to come)come)
PaX makes use of hardware-supported non-PaX makes use of hardware-supported non-executable bits (still to be tested, but should executable bits (still to be tested, but should work for alpha, parisc, and ia-64)work for alpha, parisc, and ia-64)
PaX provides full address space layout PaX provides full address space layout randomization (ASLR) for ELF binariesrandomization (ASLR) for ELF binaries
Prevention in grsecurity - PaXPrevention in grsecurity - PaX
How does PaX accomplish this?How does PaX accomplish this? Include/asm-<arch>/processor.h is modified to Include/asm-<arch>/processor.h is modified to
support executable and non-executable support executable and non-executable pages (if they don’t already exist)pages (if they don’t already exist)
Rest of kernel is modified to use the non-Rest of kernel is modified to use the non-executable pages, applied to ELF and a.out executable pages, applied to ELF and a.out binaries if they carry the required PaX flags binaries if they carry the required PaX flags (enabled by default)(enabled by default)
Prevention in grsecurity - PaXPrevention in grsecurity - PaX
Non-executable pages are made supervisor in Non-executable pages are made supervisor in the TLB; executable pages are left as userthe TLB; executable pages are left as user
If CPU is in user mode, access to the non-If CPU is in user mode, access to the non-executable pages causes a page-fault which PaX executable pages causes a page-fault which PaX handleshandlesMakes up the core logic of how PaX worksMakes up the core logic of how PaX worksMakes PaX ineffective against kernel overflowsMakes PaX ineffective against kernel overflows
Mmap(2) and mprotect(2) restrictions/featuresMmap(2) and mprotect(2) restrictions/featuresDisallows anonymous mappings with PROT_EXEC Disallows anonymous mappings with PROT_EXEC present – stops one method of arbitrary code present – stops one method of arbitrary code execution (another method, mapping a file with execution (another method, mapping a file with PROT_EXEC, is handled by ACL system)PROT_EXEC, is handled by ACL system)
Prevention in grsecurity - PaXPrevention in grsecurity - PaX
Causes mmaps (applies to libraries) to be mapped Causes mmaps (applies to libraries) to be mapped at random locations below 0x01000000 until it’s at random locations below 0x01000000 until it’s full, then above 0x40000000full, then above 0x40000000
Causes exploits to have to guess the library function Causes exploits to have to guess the library function addressaddress
Makes the address contain a NULL byte, which stops Makes the address contain a NULL byte, which stops ASCII shellcode from calling a library functionASCII shellcode from calling a library function
Keeps non-executable pages from being Keeps non-executable pages from being mprotected to executablemprotected to executable
No performance impactNo performance impact
Prevention in grsecurity - PaXPrevention in grsecurity - PaX Full Address Space Layout Randomization Full Address Space Layout Randomization
(ASLR)(ASLR)Randomizes the base of mmaps, stack, and Randomizes the base of mmaps, stack, and executable (if the binary is ET_DYN)executable (if the binary is ET_DYN)Makes the leftover methods of exploitation a guessing Makes the leftover methods of exploitation a guessing gamegame
With no-execWith no-exec Without no-execWithout no-exec
Stack smashingStack smashing ImpossibleImpossible Guess 16-bitGuess 16-bit
Heap overflowHeap overflow ImpossibleImpossible Guess 32-bitGuess 32-bit
Ret-to-libcRet-to-libc Guess 32 or Guess 32 or 48-bit48-bit
Guess 32 or 48-Guess 32 or 48-bitbit
Prevention in grsecurity - PaXPrevention in grsecurity - PaX
0x0012d00 – 0x00391000 Libraries
0x0fd6b000 – 0x0fefc000 Executable
Executable 0x08048000 -0x08049000
Libraries 0x40000000 – 0x40168000
Stack 0xbfffe0000– 0xc0000000
0xbfff2000 – 0xbfffa000 Stack
PaX with Full ASLR Without PaX
0x40000000 – 0x50000000
0x08048000 – 0x0fd6b000
0x00fefc000 – 0x18048000
0xbfffa000 – 0xc0000000
0xbff00000 – bfff2000
256MB
256MB
1MB
Prevention in grsecurity - PaXPrevention in grsecurity - PaX
Full ASLR can only be bypassed in the case of Full ASLR can only be bypassed in the case of information leak. While there’s nothing that can be information leak. While there’s nothing that can be done about software vulnerabilities that allow done about software vulnerabilities that allow information leaking without crashing, we’ve information leaking without crashing, we’ve implemented the following features to stop local implemented the following features to stop local users from obtaining information about the random users from obtaining information about the random base addresses:base addresses:
Ptrace(2) restrictions in ACL systemPtrace(2) restrictions in ACL system Restricted /procRestricted /proc
For 64-bit architectures, the randomness provided For 64-bit architectures, the randomness provided by full ASLR could be increased to 48/64/80 bits by full ASLR could be increased to 48/64/80 bits (the amount the attacker has to overcome is (the amount the attacker has to overcome is determined by the type of exploit)determined by the type of exploit)
Prevention in grsecurity - PaXPrevention in grsecurity - PaX
What’s in it for me?What’s in it for me? No more arbitrary code executionNo more arbitrary code execution No more stack smashing, heap or bss No more stack smashing, heap or bss
overflow exploitationoverflow exploitation No more return-to-libc exploitationNo more return-to-libc exploitation (Soon) no more arbitrary execution flow (Soon) no more arbitrary execution flow
redirectionredirection
Prevention in grsecurity - PaXPrevention in grsecurity - PaX
What’s coming for this section of What’s coming for this section of grsecurity?grsecurity? New segmentation-based implementation of New segmentation-based implementation of
non-executable pages with an insignificant non-executable pages with an insignificant performance hitperformance hit
Increased stack base address randomness to Increased stack base address randomness to 24 bits24 bits
Binary instrumentationBinary instrumentationStops ret-to-libc by checkpointing execution flow Stops ret-to-libc by checkpointing execution flow changeschangesAbility to handle other vulnerabilities (eg. Stack Ability to handle other vulnerabilities (eg. Stack based overflows, format string, info-leak)based overflows, format string, info-leak)
Prevention in grsecurityPrevention in grsecurity
OpenBSD randomness featuresOpenBSD randomness features Random IP IDsRandom IP IDs Random RPC XIDsRandom RPC XIDs Random RPC privileged portsRandom RPC privileged ports Random PIDsRandom PIDs
Prevention in grsecurityPrevention in grsecurity
Random IP IDsRandom IP IDs Uses Niels Provos’ random IP ID generation Uses Niels Provos’ random IP ID generation
function ported to Linuxfunction ported to LinuxLittle entropy useLittle entropy use
Values are not reused quicklyValues are not reused quickly Useful for preventing OS fingerprinting and Useful for preventing OS fingerprinting and
spoofed scansspoofed scans
Prevention in grsecurityPrevention in grsecurity
Random RPC XIDsRandom RPC XIDs Uses same random IP ID codeUses same random IP ID code Useful for preventing RPC connection Useful for preventing RPC connection
hijackinghijacking
Random PIDsRandom PIDs Uses same random IP ID codeUses same random IP ID code Properties of returned values make the Properties of returned values make the
function almost always return an unused PID function almost always return an unused PID even on heavily loaded serverseven on heavily loaded servers
Prevention in grsecurityPrevention in grsecurity
Prevents filesystem races since getpid() is Prevents filesystem races since getpid() is sometimes used as part of a temporary sometimes used as part of a temporary filenamefilename
Adds additional randomness to programs that Adds additional randomness to programs that use getpid(2) for srandom(3) seedinguse getpid(2) for srandom(3) seeding
Prevention in grsecurityPrevention in grsecurity
Stealth netfilter moduleStealth netfilter module Based on the fact that OS fingerprinting relies Based on the fact that OS fingerprinting relies
greatly on the packets sent in reply to those greatly on the packets sent in reply to those sent to unserved TCP or UDP portssent to unserved TCP or UDP ports
Matches unserved ports dynamically, so it can Matches unserved ports dynamically, so it can be used in shell-server environmentsbe used in shell-server environments
Slows down blocking port-scannersSlows down blocking port-scanners
Prevention in grsecurityPrevention in grsecurity
Problems with chroot(2)Problems with chroot(2) Easy to use it insecurelyEasy to use it insecurely Generally only filesystem-related functions Generally only filesystem-related functions
care if a process is chrootedcare if a process is chrooted Easy for a root user in chroot to break outEasy for a root user in chroot to break out
Prevention in grsecurityPrevention in grsecurity
How we strengthen chroot(2):How we strengthen chroot(2): Make syscalls unrelated to the filesystem Make syscalls unrelated to the filesystem
chroot-awarechroot-awareDeny double-chroots, pivot_root(2)Deny double-chroots, pivot_root(2)
Restrict signals outside of chrootRestrict signals outside of chroot
Deny fchdir(2) outside of chrootDeny fchdir(2) outside of chroot
Deny mountingDeny mounting Enforce chdir(“/”) upon chrootEnforce chdir(“/”) upon chroot Lower capabilities upon chrootLower capabilities upon chroot
Containment in grsecurity
Containment in grsecurityContainment in grsecurity
Trusted Path Execution (TPE)Trusted Path Execution (TPE) Keeps users from executing untrusted Keeps users from executing untrusted
binaries (those not in root-owned non-world binaries (those not in root-owned non-world writable directories)writable directories)
Hardened against evasionHardened against evasionSilent removal of glibc environment variables that Silent removal of glibc environment variables that allow arbitrary code execution (eg. LD_PRELOAD)allow arbitrary code execution (eg. LD_PRELOAD)
TPE checks implemented in mmap(2) (stops TPE checks implemented in mmap(2) (stops /lib/ld.so <executable> evasion)/lib/ld.so <executable> evasion)
Containment in grsecurityContainment in grsecurity
Grsecurity’s ACL systemGrsecurity’s ACL system Process-based : Allowed for a large reduction Process-based : Allowed for a large reduction
in code basein code base ACL parsing handled via userspace, interacts ACL parsing handled via userspace, interacts
with kernel via a /proc entrywith kernel via a /proc entryInclude directiveInclude directiveACL analysisACL analysis
$PATH$PATH /etc/ld.so.conf/etc/ld.so.conf Auto-add libraries for ELF executablesAuto-add libraries for ELF executables /etc/lilo.conf/etc/lilo.conf
Containment in grsecurityContainment in grsecurity
Uses LEX/YACCUses LEX/YACC
Sends data to kernel in ready-to-use structures – Sends data to kernel in ready-to-use structures – further reduces necessary kernel code sizefurther reduces necessary kernel code size
Enable, disable, and administrator modesEnable, disable, and administrator modes Hidden and protected processesHidden and protected processes Read, write, append and execute modes for Read, write, append and execute modes for
file objectsfile objects Inherit and hidden flags for file objectsInherit and hidden flags for file objects
Containment in grsecurityContainment in grsecurity
Capability support (including inheritance)Capability support (including inheritance) Hardened against ACL evasion and privilege Hardened against ACL evasion and privilege
leakingleakingPtrace restriction – user can only ptrace a process Ptrace restriction – user can only ptrace a process if the default ACL allows writing to itif the default ACL allows writing to itGlibc environment variable handlingGlibc environment variable handling
Performs correct handling, not just a denied exec if LD_ Performs correct handling, not just a denied exec if LD_ is foundis found
Checks each path in glibc environment to see if the Checks each path in glibc environment to see if the default ACL allows writing to it; if so, deny the exec and default ACL allows writing to it; if so, deny the exec and log pathname and environment variable usedlog pathname and environment variable used
Applies executable restrictions in mmap(2), not just Applies executable restrictions in mmap(2), not just execveexecve
Containment in grsecurityContainment in grsecurity
Human readable configuration filesHuman readable configuration files Insignificant performance impact due to Insignificant performance impact due to
efficient searching algorithms (hash tables == efficient searching algorithms (hash tables == O(1) )O(1) )
Containment in grsecurityContainment in grsecurity
What’s coming for the ACL system?What’s coming for the ACL system? Redesign to become more modular and allow Redesign to become more modular and allow
quicker implementation of new featuresquicker implementation of new features Intelligent learning mode resulting in a least-Intelligent learning mode resulting in a least-
privilege system with little or no configuration privilege system with little or no configuration necessarynecessary
Support of fine-grained resource restrictions Support of fine-grained resource restrictions and something similar to nergal’s segvguardand something similar to nergal’s segvguard
Time-based ACLsTime-based ACLs Merging of GID-based grsecurity featuresMerging of GID-based grsecurity features Role-Based Access Control (RBAC)Role-Based Access Control (RBAC)
Containment in grsecurityContainment in grsecurity
Domain-based authentication supportDomain-based authentication support
Performance
Performance of ACL systemPerformance of ACL system
Completed 150 runs of 16 dbench Completed 150 runs of 16 dbench processesprocessesAverage throughput with ACL system was Average throughput with ACL system was larger than a clean kernellarger than a clean kernelStandard deviation was 5MB/s, which was Standard deviation was 5MB/s, which was larger than the difference of throughputlarger than the difference of throughputRESULT: The ACL system causes no RESULT: The ACL system causes no noticeable performance hit on filesystem noticeable performance hit on filesystem accessaccess
Performance of ACL systemPerformance of ACL system
Results of kernel compile benchmark:Results of kernel compile benchmark: Total time with ACL system – 265.86 secondsTotal time with ACL system – 265.86 seconds Total time w/o ACL system – 264.94 secondsTotal time w/o ACL system – 264.94 seconds .4% performance hit.4% performance hit
Performance hit only due to execs in Performance hit only due to execs in compiling and making – search is called compiling and making – search is called twice, acl label is copied, acl label is set, twice, acl label is copied, acl label is set, checks are performed on the environmentchecks are performed on the environment
Performance with PaXPerformance with PaX
Memory load latency microbenchmarksMemory load latency microbenchmarks
MySQL benchmarks (real life example)MySQL benchmarks (real life example)
Test system:Test system: Dual AMD XP 1600+Dual AMD XP 1600+ 512MB PC2100 ECC DDR registered RAM512MB PC2100 ECC DDR registered RAM 266mhz FSB266mhz FSB 80GB ATA100 5400RPM HD80GB ATA100 5400RPM HD
Performance with PaXPerformance with PaX2.4.18 memory load latency
0
100
200
300
400
500
600
700
800
900
1000
0.0001 0.001 0.01 0.1 1 10 100 1000
Array Size (MB)
Late
ncy (
ns)
Stride 16
Stride 32
Stride 64
Stride 128
Stride 256
Stride 512
Stride 1024
Stride 2048
Stride 4096
Performance with PaXPerformance with PaXgrsecurity w / PaX memory load latency
0
100
200
300
400
500
600
700
800
900
1000
0.0001 0.001 0.01 0.1 1 10 100 1000
Array Size (MB)
Late
ncy (
ns)
Stride 16
Stride 32
Stride 64
Stride 128
Stride 256
Stride 512
Stride 1024
Stride 2048
Stride 4096
Performance with PaXPerformance with PaX
Athlons encounter less performance hit Athlons encounter less performance hit partially due to their 256 entry DTLB (4KB partially due to their 256 entry DTLB (4KB page x 256 = 1MB)page x 256 = 1MB)PaX starts showing its performance impact PaX starts showing its performance impact when the DTLB is full and expired entries when the DTLB is full and expired entries are replacedare replacedPerformance with PaX can only be Performance with PaX can only be determined by the size and type of determined by the size and type of memory accesses performed by an memory accesses performed by an applicationapplication
Performance with PaXPerformance with PaX
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
% o
f C
PU
tim
e
alter-table ATIS big-tables connect create insert select
Linux 2.4.18 MySQL benchmark
System
User
Performance with PaXPerformance with PaX
0%
20%
40%
60%
80%
100%
% o
f C
PU
tim
e
alter-table ATIS big-tables connect create insert select
grsecurity w / PaX MySQL benchmark
System
User
Performance with PaXPerformance with PaX
1 10 100 1000 10000
Wallclock time (s)
alter-table
ATIS
big-tables
connect
create
insert
select
grsecurity MySQL benchmark
grsecurity w / PaX
Clean 2.4.18
Performance with PaXPerformance with PaX
A result weighted according to an actual A result weighted according to an actual system’s load shows that for MySQL, PaX system’s load shows that for MySQL, PaX caused an overall performance hit of 13%caused an overall performance hit of 13%
Since the memory access patterns of each Since the memory access patterns of each test were different, the performance hits test were different, the performance hits for each test ranged from 3% - 20%for each test ranged from 3% - 20%
For More Information…For More Information…
grsecurity’s ACL documentation: grsecurity’s ACL documentation: http://www.grsecurity.net/gracldoc.htmhttp://www.grsecurity.net/gracldoc.htm
PaXPaX
http://pageexec.virtualave.nethttp://pageexec.virtualave.net
THANKSTHANKS
PaX TeamPaX Team
Tim YardleyTim Yardley
Michael Dalton - grsecurityMichael Dalton - grsecurity
Recommended