Upload
rhoda-stokes
View
218
Download
2
Tags:
Embed Size (px)
Citation preview
Advanced UnixAdvanced Unix
Final Review
December 6, 2005
IPSECIPSEC
OutlineOutline
• IPsec overview• Alphabet soup being served…
• Security Associations (SA) & SPI’s• Authentication Header (AH) protocol• Encapsulating Security Payload (ESP) protocl• Internet Key Exchange (IKE)• IPsec pitfalls• IPsec vs tunneling (PPTP, L2TP)
IPSec OverviewIPSec Overview
IPSec is a suite of protocols for securing network connections – The details and variations are overwhelming
One cause of the complexity is that IPSec provides a mechanism, not policy– A framework that allows implementation
possible that both ends can agree on
Virtual Private Network (VPN)Virtual Private Network (VPN)
• Secure communications between two hosts or networks
• VPN, is the buzzword that solves all you problems
• IPsec is one of the more popular VPN technology's
What can IPSEC ProvideWhat can IPSEC Provide
• Authentication• Integrity• Access control• Confidentiality• Replay protection (Partial)
Types of VPNsTypes of VPNs
• Host To Host• We’ll do this in class
• Host To Security or Secure Gateway• Secure Gateway To Secure Gateway
• Secure Gateway = Firewall or VPN router• Also referred to as Network To Network
Security Associations (SA)Security Associations (SA)
• A group of security settings related to a specific VPN
• Stored in the SPD (Security Policy Database)• Uniquely Identify IPsec sessions by:
• SPI (Security Parameter Index) a unique number that identifies the session
• The destination IP address• A security protocol or encryption method
• Normally AH or ESP
• A shared secret
Types of IPSEC ConnectionsTypes of IPSEC Connections
• Transport Mode• Does not encrypt the entire packet• Uses original IP Header• Faster
• Tunnel Mode• Encrypts entire packet including IP Header
(ESP)• Creates a new IP header• Slower
IKE (Internet Key Exchange)IKE (Internet Key Exchange)
• UDP port 500• Negotiates connection parameters • ISAKMP (Internet Security
Association and Key Management Protocol)
• Oakley (Diffie-Helmen key exchange)
IPsec PitfallsIPsec Pitfalls
• Complicated• many different ways to configure
• Can be configured insecurely• Client security is an issue• Performance in IPv4 implementation
Advantages of IPSecAdvantages of IPSec
• Encrypts the entire packet, including IP Header (not just layer 4 and higher)
• Can Encrypt any protocol• No Impact on users when using Secure
Gateway to Secure Gateway• Acts independent of IP address
IPsec GuidelinesIPsec Guidelines
• Always use:• 3des or blowfish• SHA1 over SHA and MD5• NEVER USE DES• Tunnel Mode• Main Mode• AH and ESP together• Certificates for production environments
OS Support for IPsecOS Support for IPsec
• OpenBSD, FreeBSD, NetBSD• Linux• Solaris• Windows 2000 (Native)• Windows NT/95/98/Me (Add-on)• Cisco IOS (PIX and Routers)• Others as well....
Squid Proxy ServerSquid Proxy Server
Squid FeaturesSquid Features
Its a caching proxy for:– HTTP, HTTPS (tunnel only)– FTP– Gopher
A full-featured Web proxy cache Designed to run on Unix systems Free, open-source software
Squid SupportsSquid Supports
proxying and caching of HTTP, FTP, and other URLs proxying for SSL cache hierarchies ICP, HTCP, CARP, Cache Digests transparent caching extensive access controls HTTP server acceleration SNMP caching of DNS lookups
Other proxies (besides Squid)Other proxies (besides Squid)
Commercial– Netscape Proxy– Microsoft Proxy Server– NetAppliance’s NetCache (shares some
code history with Squid in the distant past)
– CacheFlow (http://www.cacheflow.com/)– Cisco Cache Engine
What is a proxy?What is a proxy?
Firewall device; internal users communicate with the proxy, which in turn talks to the Internet– Gateway for private address space (RFC 1918)
into publicly routable address space
Allows one to implement policy– Restrict who can access the Internet– Restrict what sites users can access– Provides detailed logs of user activity
What is a caching proxy?What is a caching proxy?
Stores a local copy of objects fetched– Subsequent accesses by other users in the
organization are served from the local cache, rather than the origin server
– Reduces network bandwidth– Users experience faster web access
How proxies workHow proxies work
User configures web browser to use proxy instead of connecting directly to origin servers– Manual configuration for older PC based
browsers, and some UNIX browsers (e.g., Lynx)– Proxy auto-configuration file for Netscape 2.x+ or
Internet Explorer 4.x+• Far more flexible caching policy• Simplifies user configuration, help desk support, etc.
How proxies work (user request)How proxies work (user request)
User requests a page: http://www.rose.edu
Browser forwards request to proxyProxy optionally verifies user’s
identity and checks policy for right to access uniforum.chi.il.us
Assuming right is granted, fetches page and returns it to user
SambaSamba
What is SambaWhat is Samba
Samba is an Open Source/Free Software suite that provides file and print services to SMB clients
Samba current version: 3.20bSamba Home Page:
http://www.samba.org
PrerequisitesPrerequisitesThe following installs:
– Samba– samba-client– samba-common– system-config-samba
samba-swat (optional)
Samba Utilities and DaemonsSamba Utilities and Daemons
net nmbd nmblookup smbclient smbd smbpasswd smbstatus smbtree swat (not part of samba) testparm testprns (deprecated and will be removed in a future Samba release)
Samba users, maps, passwordsSamba users, maps, passwords
Usernames - /etc/samba/smbusersPasswords - /etc/samba/smbpasswdDemo:
/etc/samba/smbusers
Quick StartQuick Start
system-config-samba is used to configure samba server on linux computer
Demo: system-config-sambaSamba usersLinux shares
Sendmail and SMTPSendmail and SMTP
OverviewOverview
Introduction to EmailMessage BreakdownSample MessagesExtensions (MIME)MTA’s and Mailbox Protocols
Email StatisticsEmail Statistics
31 billion emails are sent daily, expected to double by 2006
Email generates about one billion Gigabytes of new “information” per year
Spam accounts for about 40% of all email traffic
http://www.spamfilterreview.com
SMTPSMTP
Originated in 1982 (rfc0821, Jon Postel) Goal: To transfer mail reliably and efficiently
SMTPSMTP SMTP clients and
servers have two main components– User Agents – Prepares
the message, encloses it in an envelope. (Eudora for example)
– Mail Transfer Agent (MTA) – Transfers the mail across the internet
SMTPSMTP SMTP also allows the
use of Relays allowing other MTAs to relay the mail
Mail Gateways are used to relay mail prepared by a protocol other then SMTP and convert it to SMTP
What is Mail?What is Mail? Mail is a text file Envelope –
– sender address– receiver address– other information
Message –– Mail Header –
defines the sender, the receiver, the subject of the message, and some other information
– Mail Body – Contains the actual information in the message
Return-Path: <[email protected]>Delivered-To: [email protected]
Received: by mail.eecis.udel.edu (Postfix, from userid 62)id 17FBD328DE; Wed, 5 Nov 2003 11:27:02
Received: from mail.acad.ece.udel.edu (devil-rays.acad.ece.udel.edu [128.4.60.10])by mail.eecis.udel.edu (Postfix) with ESMTP id
5F41832893 for <[email protected]>; Wed, 5 Nov 2003 11:27:01 Received: by mail.acad.ece.udel.edu (Postfix, from userid 62)id 47509456C; Wed, 5 Nov 2003 11:27:01 Received: from stimpy.eecis.udel.edu(stimpy.eecis.udel.edu [128.4.40.17])by mail.acad.ece.udel.edu (Postfix) with SMTP id 7C2943D79 for <[email protected]>; Wed, 5 Nov 2003 11:26:34 Message-Id: <[email protected]>Date: Wed, 5 Nov 2003 11:26:34 From: [email protected]: undisclosed-recipients: ;
MIME-Version: 1.0
This is a test.
Post Office
Mailbox
Post office
and mail route
Receivers
Mailbox
How SMTP worksHow SMTP works
The Essentials
How about a Demo?
Keyword Arguments
HELO Sender’s Host Domain Name
MAIL FROM: Email Address of sender
RCPT TO: Email of Intended recipient
DATA Body of the message
QUIT
How SMTP worksHow SMTP works
The Extras
Keyword Arguments
RSET
VRFY Name to be verified
NOOP
TURN
EXPN Mailing list to expand
HELP Command Name
Status CodesStatus Codes
The Server responds with a 3 digit code that may be followed by text info– 2## - Success– 3## - Command can be accepted with
more information– 4## - Command was rejected, but error
condition is temporary– 5## - Command rejected, Bad User!
Connection EstablishmentConnection Establishment
TCP Connection Establishment
Message ProgressMessage Progress
Connection TerminationConnection Termination
TCP Connection Termination
Problems with SMTPProblems with SMTP
No security– Authentication– Encryption
Only uses NVT (Network Virtual Terminal) 7-bit ASCII format
E-mails can be forged…..E-mails can be forged…..HELO mail.rose.eduMAIL FROM: [email protected] TO: [email protected]: Dr. Art ZennerTo: Professor RichardsSubject: CIT 2243
Professor Richards,By department decree all students in your CIT 2243
Introduction to Unix class are hereby to be given automatic A’s.Thank you,Dr. Art Zenner.
QUIT
Extensions to SMTPExtensions to SMTP MIME – Multipurpose Internet Mail Extensions
– Transforms non-ASCII data to NVT (Network Virtual Terminal) ASCII data
• Text
• Application
• Image
• Audio
• Video
MIME HeadersMIME Headers
Goes between the Email Header and Body– MIME-Version: 1.1– Content-Type– Content-Transfer-Encoding– Content-Id– Content-Description
MIME HeadersMIME Headers
Content-Type – Type of data used in the body of the message
Text – plain, unformatted text; HTML
Multipart – Body contains multiple independent parts
Message – The body is whole mail message, part of a message, or a pointer to a message
MIME HeadersMIME Headers
Image – The message is a stationary image (JPEG or GIF)
Video – The message is an animation (Mpeg)
Audio – The message is 8 kHz standard audio data
Application – The message is a type of data not previously defined
MIME HeadersMIME Headers
Content-Transfer-Encoding – The method used to encode the messages7 bit – no encoding needed
8 bit – Non-ASCII, short lines
Binary – Non-ASCII, unlimited length lines
Base64 – 6 bit blocks encoded into 8-bit ASCII
Quoted-printable – send non-ASCII characters as 3 ASCII characters, =##, ## is the hex representation of the byte
Base64 EncodingBase64 Encoding
Divides binary data into 24 bit blocks Each block is then divided into 6 bit chunks Each 6-bit section is interpreted as one character
incurs a 25% overhead
11001100 10000001 00111001
110011 001000 000100 111001
01111010 01001001 01000101 00110101
(51) (8) (4) (57)
(z) (I) (E) (5)
Quoted-Printable EncodingQuoted-Printable Encoding
Used when the data has a small non-ASCII portion
Non-ASCII characters are sent as 3 characters First is ‘=‘, second and third are the hex
representation of the byte
01001100 10011101 00111001
00111101 00111001 01000100(=) (9) (D)
MIME HeadersMIME Headers
Content-Id – Uniquely identifies the whole message in a multiple message
environment Content-Description – defines whether the
body is image, audio, or video
A Multipart, Encoded MIME Message From: [email protected]: [email protected]: Info on Gibson guitarMIME-Version: 1.0Content-Type: multipart/mixed; boundary=17
- 17Content-Type: text/enriched; charset="us-ascii"Content-Transfer-Encoding: 8bitContent-Description: Greetings As promised, I'm getting back to you about the Gibson Southern Jumbo guitar you were Interested in. I've enclosed a spec sheet on the guitar, which is in Microsoft Word.
I guarantee that you'll love it!
- 17Content-Type: application/octet-streamContent-Transfer-Encoding: base64Content-Description: Spec sheet saved as MS Word file
- 17 -
MIME ExampleMIME ExampleDate: Wed, 04 Apr 2001 00:11:37 -0400From: Meghna Naik <[email protected]>MIME-Version: 1.0To: [email protected]: =?gb2312?B?1tDOxA==?= titleContent-Type: text/plain; charset=gb2312Content-Transfer-Encoding: 7bit
a body text, blah, blah
Mail Transfer Agents (MTA)Mail Transfer Agents (MTA)
MTAs do the actual mail transfersMTAs are not meant to be directly
accessed by users.Other MTA’s are:
– Postfix– Qmail– MS Exchange– CC:Mail– Lotus Notes– ….etc.
SendmailSendmail
It's been said that you aren't a real Unix system administrator until you've edited a sendmail.cf file.
It's also been said that you're crazy if you've done so twice.
What is Sendmail?What is Sendmail?
Definition: Sendmail is the most widely used Mail Transport Agent (MTA) on the internet
MTAs send mail from one machine to another. Sendmail is not a client program, which you
use to read your email. Sendmail is one of the behind-the-scenes
programs which move email over the Internet. – Normally it runs as a background daemon– Can even be run out of the super daemon (xinetd)
ImplementationsImplementations
SMTP Gateway– An SMTP gateway allows users on your network
to communicate with others on the Internet without concern as to which local mail software package exists on your network.
– All incoming mail for your network will pass through this gateway which converts the message into the appropriate format specific to your local mail software.
– Similarly, all mail destined for the Internet from your network will pass through this gateway to be sent across the Internet via SMTP
ImplemetationsImplemetations
SMTP Relay ”Warning Will Rogers”– An SMTP relay is a machine that actually
sends the mail across the Internet.– A common misconception is that SMTP
gateways are the same as SMTP relays. This is not always the case.
– There are SMTP gateways that act as relays themselves, but there are also many that do not. If the latter is the case on your network, you'll need to bounce your mail off one of the relays.
Installation MethodsInstallation Methods
RPM installation– Obtained from installation CDs
Binaries (*.tgz)– Obtained from http://www.sendmail.org
Source Code– Obtained from http://www.sendmail.org
The PiecesThe Pieces
The binary: /sbin/sendmail
The configuration file:/etc/mail/sendmail.cf
Supporting files:/etc/mail/access/etc/mail/aliases…and many more
More PiecesMore Pieces
Email messages are stored in the directory:/var/spool/mail – There is a separate file for each user
Email waiting to be sent./var/spool/mqueue
A log of Email sent and received:/var/log/mail
Sendmail Features Sendmail Features
Sendmail uses DNS (Domain Naming System)– But not 100% dependent: [Joe@[192.168.1.1]
DNS provides Mail Exchange (MX) Info Sendmail can do a DNS double-tap
– Lookup up who the client says they are Sendmail default is mail relay off Realtime Blackhole Lists (RBL) Mail Relay checkers - - Open Mail Relay Db
http://www.ordb.org/submit/
Sendmail Anti-Spam EnhancementsSendmail Anti-Spam Enhancements
Mailscanner– Minimal anti-spam– Anti-virus integration (scan in/outbound)– http://www.sng.ecs.soton.ac.uk/mailscanner/– Or http://www.mailscanner.info
Spam Assassin– Rule based heuristic – Header and text analysis– Blacklist (RBL)– Vipul's Razor (http://razor.sf.net)– http://www.spamassassin.org
Mail Access ProtocolsMail Access Protocols
The MTAs place the email in the user’s mailbox
The Mail Access Protocols are used by the users to retrieve the email from the mailbox– POP3– IMAP4
All Messages
POP3:
IMAP:Mr Smith
Friends
….headers
Whole message
POP vs. IMAP
Post Office Protocol v3Post Office Protocol v3
SimpleAllows the user to obtain a list of
their EmailsUsers can retrieve their emailsUsers can either delete or keep the
email on their systemMinimizes server resources
Internet Mail Access Protocol v4Internet Mail Access Protocol v4
Has more features then POP3 User can check the email header before
downloading Emails can be accessed from any location Can search the email for a specific string
of characters before downloading User can download parts of an email User can create, delete, or rename
mailboxes on a server
ReferencesReferences
RFC’s: – RFC 821 - Simple Mail Transfer Protocol – RFC0822 - Standard for the Format of ARPA
Internet Text Messages – RFC 1521 - MIME (Multipurpose Internet Mail
Extensions)
E-mail Explained– http://www.sendmail.org/email-explained.html
Sendmail ConfigurationSendmail Configuration
Internal SMTP IssuesInternal SMTP Issues
Vrfy name– Used to verify if a mailbox with the given name
exists in an SMTP server
Expn maillist-name– Used to expand the members of the given
maillist name
Both sources of e-mail address for spammers
Must be disabled
SendmailSendmail
An open source mail transfer agent Original version written by Eric Allman in
1980’s at UC Berkeley Descendant of ARPANET delivermail Very flexible
– Supports different transfer and delivery protocols Very complicated
– Difficult to manage– Configured using sendmail.cf, sendmail.mc
Unfortunately, known for it’s bugs
SendmailSendmail
Security measures:– Sendmail restricted shell: smrsh– Standard security checks– SMTP AUTH– SMTP STARTTLS
Rejecting SPAM– Access database– Anti-spamming relay features– Validating senders
SendmailSendmail
Configuring sendmail– /etc/mail/sendmail.cf
• Actual configuration file
– /etc/mail/sendmail.mc• More user friendly configuration file
– Make sendmail.cf from sendmail.mc• m4 /usr/local/share/sendmail/cf/m4/cf.m4
/etc/mail/sendmail.mc > /etc/mail/sendmail.cf
SendmailSendmail
Turning off exploitable features– Find the line in sendmail.cf that contains
• O PrivacyOptions=
– Add noexpn and novrfy• O PrivacyOptions=noexpn novrfy
– Most strict : goaway– Or set the confPRIVACY_FLAGS in
sendmail.mc• define(`confPRIVACY_FLAGS’, `goaway, noexpn,
novrfy, nobodyreturn’)
SendmailSendmail
SMTP server banner– May give away system info
• 220 192.168.1.1 ESMTP Sendmail 8.10.2+Sun/8.10.2; Tue,14 Jan 2003 09: 28:02-0500 (EST)
– Change SmtpGreetingMessage field in sendmail.cf
SendmailSendmail
Precautions against DoS attacks, in sendmail.mc:– Set confMAX_MESSAGE_SIZE to limit
message size– Set confMAX_DAEMON_CHILDREN to
limit number of processesDoes not prevent DoS attacks
SendmailSendmail
Controlled SMTP relaying in sendmail: FEATURE(access_db)– List the domains you are willing to relay from
in /etc/mail/relay-domains• FEATURE(relay_hosts_only)
– Hosts must also be listed• FEATURE(relay_entire_domain)
– Relay all computers in domain• FEATURE(access_db)
– Enables or disables access database• FEATURE(blacklist_recipients)
– Also look up recipients in access database
SendmailSendmail
Controlled SMTP relaying in sendmail:– List the domains you are willing to relay from in
/etc/mail/relay-domains
• FEATURE(dnsbl)– Use realtime black hole list at mail-abuse.org– 1.5.5.192.blackholes.mail-abuse.org IN A 127.0.0.2
• FEATURE(accept_unqualified_senders)– Allow users without domains
• FEATURE(accept_unresolvable_domains)– Allow users with unresolvable domains
• FEATURE(relay_based_on_MX)– Permit any relay directed to your host
SendmailSendmail
Following features make vulnerable to abuse:– FEATURE(relay_local_from).
• Allows relaying if the message claims to originate at your domain.
– FEATURE(loose_relay_check).• turns off checking for explicit routing
– FEATURE(promiscuous_relay).• turn off all checking for relaying.
SendmailSendmail
Access database– In /etc/mail/access– Allow access by individual domains– Two-tuples : key – action– Key:
• Fully or partly qualified host name• Network or subnetwork address• Specific e-mail addresses• Can also include FROM:, TO:, etc.
SendmailSendmail
Actions:– REJECT
• refuse connections from host
– DISCARD• accept the message but silently discard it, sender will
think message is accepted
– OK• Allow access, overrides other checks
– RELAY• Allow access including relaying
– ERROR:### arbitrary message• Reject mail with customized message
SendmailSendmail
Example– cyberpromo.com REJECT– sendmail.org RELAY– [email protected] ERROR:550 Spammers
do not live here anymore– From:[email protected] REJECT– To:[email protected] REJECT– 193.140 RELAY
Generate database from map– makemap hash /etc/mail/access < /etc/mail/access
Sendmail: smrshSendmail: smrsh
The smrsh program is intended as a replacement for /bin/sh in the program mailer definition of Sendmail.
It's a restricted shell utility that provides the ability to specify, through the /etc/smrsh directory, an explicit list of executable programs available to Sendmail.
smrsh effectively limits Sendmail's scope of program execution to only those programs specified in smrsh's directory.
Sendmail: smrshSendmail: smrsh
The sendmail.cf is configured to run /bin/smrsh by default
To prevent duplicate programs, and do a nice job, it is better to establish links to the allowable programs from /etc/smrsh rather than copy programs to this directory.
For example:
cd /etc/smrsh ln -s /usr/bin/procmail /etc/smrsh/procmail
SendmailSendmail
smsrh:– Form an explicit list of executables that
sendmail is allowed to executesendmail.mc :
– FEATURE(`smsrh’)Advised to be used in all sendmail
versions
SendmailSendmail
Enhanced File Security:– Tight rules for opening files– In general, all read directories should be
owned by root– No .forward in unsafe (group or world
writable) directories
SendmailSendmail
Enhanced File Security:– If too restrictive, set the DontBlameSendmail
option in sendmail.mc– define (`confDONT_BLAME_SENDMAIL’,...)
• ForwardFileInUnsafeDirPath
Allow .forward files in unsafe directories.• ForwardFileInUnsafeDirPathSafe
Allow a .forward file that is in an unsafe directory to include references to program and files.
SendmailSendmail
SMTP-Auth in sendmail:– Install an SASL library
• i.e. Cyrus SASL
– Compile sendmail with right options• APPENDDEF(`confENVDEF', `-DSASL')
APPENDDEF(`conf_sendmail_LIBS', `-lsasl') for Cyrus SASLv1• APPENDDEF(`confENVDEF', `-DSASL=2')
APPENDDEF(`conf_sendmail_LIBS', `-lsasl2') for Cyrus SASLv2to site.config.m4
SendmailSendmail
Set options in sendmail.mc– TRUST_AUTH_MECH(`GSSAPI DIGEST-
MD5')dnl– define(`confAUTH_MECHANISMS', `GSSAPI
DIGEST-MD5')dnl– define(`confDEF_AUTH_INFO',
`/etc/mail/auth/auth-info')dnl– DAEMON_OPTIONS(`a')dnl
Requiring SMTP AUTH– Delete all other means of relaying
SendmailSendmail
To use as client,generate an info file:– client-info: AuthInfo:your.isp.net "U:root"
"P:password“ Generate authentication database:
– # makemap hash client-info < client-info Edit configuration file:
– define(`SMART_HOST',`your.isp.net')– define(`confAUTH_MECHANISMS', `DIGEST-MD5 CRAM-
MD5 LOGIN PLAIN')– FEATURE(`authinfo',`hash /etc/mail/auth/client-info')
SendmailSendmail
SMTP STARTTLS in sendmail– Allow relaying based on certificates– Restrict incoming or outgoing connections
Define following variables:– define(`confCACERT_PATH', `/etc/mail/certs/')– define(`confCACERT',
`/etc/mail/certs/CA.cert.pem')– define(`confSERVER_CERT',
`/etc/mail/certs/my.cert.pem')– define(`confSERVER_KEY',
`/etc/mail/certs/my.key.pem')
SendmailSendmail
${verify} : macro that keeps result of verification– OK : verification succeeded.– NO : no cert presented.– NOT : no cert requested.– FAIL : cert presented but could not be verified,
e.g., the cert of the signing CA is missing.– NONE : STARTTLS has not been performed.– TEMP : temporary error occurred.– PROTOCOL : protocol error occurred (SMTP level).– SOFTWARE : STARTTLS handshake failed.
SendmailSendmail
Relaying based on certificates– If sender not verified, usual relaying– If verified, look up the domain of
certificate issuer, and check access database for that domain
• If result is RELAY, relay• If result is SUBJECT, look up the subject
SendmailSendmail
Example– To allow relaying only for a subset of machines that have a cert
signed by• /C=US/ST=California/O=endmail.org/OU=private/CN=
Darth+20Mail+20+28Cert+29/[email protected]
– use: • CertIssuer:/C=US/ST=California/O=endmail.org/OU=private/
CN=Darth+20Mail+20+28Cert+29/[email protected] SUBJECT
• CertSubject:/C=US/ST=California/O=endmail.org/OU=private/CN= DeathStar/[email protected] RELAY
– Received header• (version=${tls_version} cipher=${cipher} bits=${cipher_bits} verify=$
{verify})
SendmailSendmail
Deciding to continue communication– Two-tuples in access map– Key : clients or servers– Values:
• VERIFY : successful verification required• VERIFY:bits : successful verification
required & cipher bits >= bits• ENCR:bits : cipher bits >= bits
– TLS_Srv, TLS_Clt keywords
SendmailSendmail
Example:– TLS_Srv:secure.example.com ENCR:112 – TLS_Clt:laptop.example.com VERIFY:112
E-mail sent to secure.example.com should be encrypted
E-mail sent from laptop.example.com should be authenticated
SendmailSendmail
Known application bugs and exploits:– CERT advisories, www.cert.org
Do not run sendmail as root– Current versions do not
Sendmail X: new generation of sendmail– Similar to Postfix architecture– Not ready for Prime Time
Advanced UnixAdvanced Unix
Apache Web Server
November 29, 2005
Web ServersWeb Servers
Tim Berners-Lee is credited with having created the World Wide Web– he was a researcher at the European High-
Energy Particle Physics lab, the Conseil Européenne pour la Recherche Nucleaire (CERN), in Geneva, Switzerland.
– A tool was needed to enable collaboration between physicists and other researchers
Web ServersWeb Servers
Tim Berners-Lee wrote a proposal called HyperText and CERN in 1989– an extension of the gopher concept but
incorporated many new ideas and features, including:
• HTML (HyperText Markup Language)• HTTP (HyperText Transfer Protocol) • Web browser client software program
– 1989 it was first installed at CERN– 1991 it was fully operational
Web ServersWeb Servers
Main type of web servers exist– For Linux the primary server is Apache
Fedora Core 3 comes with:– Apache– Tux– Stronghold– Zope– BOA– Jigsaw, etc…..
Apache OverviewApache Overview
The “A Patchy” Web server– put together over time by the Apache group– Based on the National Center for
Supercomputing Applications (NCSA) Web daemon.
• The NCSA was created by the National Science Foundation (NSF) and the state of Illinois in 1986 at the University of Illinois
Apache is free, open-source
Apache OverviewApache Overview
Configured with Text filesDependableAvailable for numerous platforms,
– even WindowsNetcraft.Com shows 76,000,000 web sites
– 70% are Apache– 21% are Microsoft(http://news.netcraft.com/archives/web_server_survey.html)
Apache OverviewApache Overview
There are two core versions of Apache– Version 1.3.x
• Fast enough for most sites• Particularly on 1 and 2 CPU systems
– Version 2.0.x• More features• filters• threads• portability• Scales to much higher loads
Testing ApacheTesting Apache
Now if Apache is runningCreate two files
– index.htm– phptest.php
Save files in:– /var/www/html/– Document Root Directory
Index,htmIndex,htm
Looks like this:
Phptest.phpPhptest.php
File looks like this:
Testing ApacheTesting Apache
Open the web browser on the system that apache is configured.
In the Address bar type in the IP Address of the system.
Testing ApacheTesting Apache
Now test Apache from another machine on the network.
Open a web browser then type IP Address in the address bar.
PHPPHP
PHP is a script language for web sites
Comes from PerlGreat for databases and Content
Management Systems (CMS)
PHPPHP
http://<your-ip>/testphp.phpLooks like this:
Apache ConfigurationApache Configuration
Prefork MPMPrefork MPM
Apache 1.3 and Apache 2.0 PreforkEach child handles one connection at a
timeMany childrenHigh memory requirements“You’ll run out of memory before CPU”
Prefork Directives Prefork Directives (Apache 2.0)(Apache 2.0)
StartServersMinSpareServersMaxSpareServersMaxClientsMaxRequestsPerChild
Worker MPMWorker MPM
Apache 2.0 and laterMultithreaded within each childDramatically reduced memory
footprintOnly a few children (fewer than prefork)
Worker DirectivesWorker Directives
MinSpareThreadsMaxSpareThreadsThreadsPerChildMaxClientsMaxRequestsPerChild
KeepAlive RequestsKeepAlive Requests
Persistent connectionsMultiple requests over one TCP socket
Directives:– KeepAlive– MaxKeepAliveRequests– KeepAliveTimeout
Apache 1.3 and 2.0Apache 1.3 and 2.0Performance CharacteristicsPerformance Characteristics
Multi-process,
Multi-threaded,
or Both?
PreforkPrefork
High memory usage Highly tolerant of faulty modules Highly tolerant of crashing children Fast Well-suited for 1 and 2-CPU systems Tried-and-tested model from Apache 1.3 “You’ll run out of memory before CPU.”
WorkerWorker
Low to moderate memory usage Moderately tolerant to faulty modules Faulty threads can affect all threads in
child Highly-scalable Well-suited for multiple processors Requires a mature threading library
(Solaris, AIX, Linux 2.6 and others work well)
Memory is no longer the bottleneck.
Important Performance Important Performance ConsiderationsConsiderationssendfile() supportDNS considerationsstat() callsUnnecessary modules
sendfile() Supportsendfile() Support
No more double-copy Zero-copy* Dramatic improvement for static files Available on
– Linux 2.4.x– Solaris 8+– FreeBSD/NetBSD/OpenBSD– ...
* Zero-copy requires both OS support and NIC driver support.
DNS ConsiderationsDNS Considerations
HostNameLookups– DNS query for each incoming request– Use logresolve instead.
Name-based Allow/Deny clauses– Two DNS queries per request for each
allow/deny clause.
stat() for Symlinksstat() for Symlinks
Options– FollowSymLinks
• Symlinks are trusted.
– SymLinksIfOwnersMatch• Must stat() and lstat() each symlink, yuck!
stat() for .htaccess filesstat() for .htaccess files
AllowOverride– stat() for .htaccess in each path
component of a request– Happens for any AllowOverride– Try to disable or limit to specific sub-
dirs– Avoid use at the DocumentRoot
stat() for Content Negotiationstat() for Content Negotiation
DirectoryIndex– Don’t use wildcards like “index”– Use something like this instead
DirectoryIndex index.html index.php index.shtml
mod_negotiation– Use a type-map instead of MultiViews if
possible
Remove Unused ModulesRemove Unused Modules
Saves Memory– Reduces code and data footprint
Reduces some processing (eg. filters)
Makes calls to fork() faster
Static modules are faster than dynamic
TroubleshootingTroubleshooting
Common pitfalls
and their solutions
Check your error_logCheck your error_log
The first place to lookIncrease the LogLevel if needed
– Make sure to turn it back down (but not off) in production
Check System HealthCheck System Health
vmstat, systat, iostat, mpstat, lockstat, etc...
Check interrupt load– NIC might be overloaded
Are you swapping memory?– A web server should never swap
Check system logs– /var/log/message, /var/log/syslog, etc...
Check Apache HealthCheck Apache Health
server-status– ExtendedStatus (see next slide)
Verify “httpd -V”ps -elf | grep httpd | wc -l
– How many httpd processes are running?
server-status Exampleserver-status Example
Other PossibilitiesOther Possibilities
Set up a staging environment Set up duplicate hardware
Check for known bugs– http://nagoya.apache.org/bugzilla/
Common BottlenecksCommon Bottlenecks
No more File DescriptorsSockets stuck in TIME_WAITHigh Memory Use (swapping)CPU OverloadInterrupt (IRQ) Overload
File DescriptorsFile Descriptors
Symptoms– entry in error_log– new httpd children fail to start– fork() failing across the system
Solutions– Increase system-wide limits– Increase ulimit settings in apachectl
TIME_WAITTIME_WAIT
Symptoms– Unable to accept new connections
– CPU under-utilized, httpd processes sit idle
– Not Swapping
– netstat shows huge numbers of sockets in TIME_WAIT
Many TIME_WAIT are to be expected Only when new connections are failing is it a
problem– Decrease system-wide TCP/IP FIN timeout
Memory Overload, SwappingMemory Overload, Swapping
Symptoms– Ignore system free memory, it is misleading!– Lots of Disk Activity– top/free show high swap usage– Load gradually increasing– ps shows processes blocking on Disk I/O
Solutions– Add more memory– Use less dynamic content, cache as much as possible– Try the Worker MPM
How much free memoryHow much free memorydo I really have?do I really have?Output from top/free is misleading.Kernels use buffersFile I/O uses cachePrograms share memory
– Explicit shared memory– Copy-On-Write after fork()
The only time you can be sure is when it starts swapping.
CPU OverloadCPU Overload
Symptoms– top shows little or no idle CPU time– System is not Swapping– High system load– System feels sluggish– Much of the CPU time is spent in userspace
Solutions– Add another CPU, get a faster machine– Use less dynamic content, cache as much as
possible
Interrupt (IRQ) OverloadInterrupt (IRQ) Overload
Symptoms– Frequent on big machines (8-CPUs and above)– Not Swapping– One or two CPUs are busy, the rest are idle– Low overall system load
Solutions– Add another NIC
• bind it to the first or use two IP addresses in Apache• put NICs on different PCI busses if possible
Virtual HostsVirtual Hosts
Virtual HostingVirtual Hosting
Apache was among the first (the first?) web server to offer Virtual hosting.
With Virtual hosting many URL's can be associated with one IP address– this is useful as IP addresses are a limited
resource.
IIS as supplied free with W2K/XP does not support Virtual Hosting.
Many hosts, one IPMany hosts, one IP
Several Hosts may translate to the same IP address. – IP addresses are a scarce reource.
An Apache server listening on 193.111.200.150 will read the Host: field to see where to look for the page to serve.
Host fieldHost field
http://www.ollieclark.com/acronyms.html
The HTTP request:GET /acronyms.html HTTP/1.1.Host: www.ollieclark.com
Apache users the Host header to see which domain was requested
– this is only available in HTTP/1.1
Apache checks its virtual hosts for the requested Host to see which page to serve or script to run.
An ExampleAn Example
We want to give convenient access to some administrative functions at www.myfirm.co.uk site
We want the URL http://admin.myfirm.co.uk/
to run a script for administering the site. We add a virtual domain admin.myfirm.co.uk
– this is OK as registered .co.uk domain will be myfirm.co.uk.
– In fact 'www' indicates a subdomain
Adding Virtual HostsAdding Virtual Hosts
NameVirtualHost directive specifies an interface on which Apache will accept virtual host requests. – ‘*’ means all interfaces.– can be several
NameVirtualHost directives
– Virtual hosts on the loopback interface
Why?Why?
Why set up virtual hosts on your local computer? Use the Hosts file
– On XP in: C:\WINDOWS\SYSTEM32\DRIVERS\ETC– also on Linux
Add entries:
Then http://admin.myfirm.co.uk/ … will go the local Apache instance which will process the Vhosts as it would in a real set up. Useful for constructing a website locally.
SecuritySecurity
Security – small rantSecurity – small rant
"Security" has three aspects:A. Security. Data is not lost.B. Availability. Data is available to its ownersC. Privacy. Data is not available to others
It is trivial to achieve C on its own. The challenge is to achieve acceptable levels of
A and C while allowing sufficient of B. Advice to keep an Apache web server secure is
often just "Don't allow …".
Access (external)Access (external)
Security as regards visitors to websites hosted by Apache on the web-server.– External security is managed by .htaccess files– and in the main configuration files
An .htaccess file is placed in a directory and manages access to that directory.
.htaccess.htaccess
An .htaccess file may be placed in any directory It controls many features of how Apache treats that directory
– security– execute scripts– use server-side includes
.htaccess files only work if main configuration file has permitted them by an apprpriate AllowOverride directive:
AuthorizationAuthorization
To protect a directory /htdocs/secure we place an .htaccess file in it
This is a text file as above.
Name of the password file
Simplest type of password
Displayed to user
Cannot GET or POST without authorisation
User must give password
Order Allow Deny Order Allow Deny
Three directives really: order, allow, deny Allow directives specify who can access a
resource Deny directives specify who cannot
access a resource Order directive specifies the order in
which the Allow and Deny directives are processed
Order directiveOrder directive
Order directive takes a single argument which is one of:– Deny,Allow– Allow,Deny
Deny,Allow evaluates the Deny directives first and then the Allow directives. So the Allow directives can override the Deny ones. Any request which does not match any directive is allowed. So default is Allow access)
Allow,Deny reverses the ordering. Default is to Deny access.
Allow/Deny directivesAllow/Deny directives
"Allow from" location Recall that the address of the client is supplied location can be a domain name or partial
domain name, an IP address or partial IP address– Allow from comp.leeds.ac.uk would allow
connections originating from within the School– Allow from 129.11 would allow connections fro any
IP address whose first two bytes were 129.11Allow from all is legitimate the Deny directive has the same syntax,
ExampleExample
The example below is one way to allow access to clients in the School of computing
Order Deny,AllowDeny from allAllow from comp.leeds.ac.uk
Order Deny,AllowDeny from allAllow from comp.leeds.ac.uk
Access (internal)Access (internal)
The security situation as regards other users of the web-server.
A web-server has three relevant classes of users:– the administrators (root, wheel)– users (<username>, users)– Apache (nobody, users)
Users manage websites. Apache needs access to the users directories to
retrieve web-pages and execute cgi-scripts
A Typical TaskA Typical Task
We have a script that is going to create and modify the contents of a file.
Visitors to the site will make these modifications
We investigate – file/directory permissions needed to make this
work. – how 'insecure' this leaves the files.
Steps– review file and directory permissions– look at application
PermissionsPermissions
File permissions are 'r', 'w', 'x' set separately for owner, group, and other.– processes run with user and group identity
owner uses owner permissions onlygroup uses group permission onlyother uses other permission only
OctalOctal
Octal digits 0-7 chmod command chmod access file(s) Access can be three octal digits:
– 1st for owner, 2nd for group 3rd for – 4 enables read, 2 enables write, 1 enables execute
So 705 enables rwx for owner, no access group, rx for other, 777 enables everyone rwx, 700 enable rwx for owner but nothing for the group.
PathsPaths
To access a file referenced by a path you must have 'x' permission on every directory on the path.– if 'x' is missing then you cannot list a directory even
To read temp.txt requires 'r' a file
Create and deleteCreate and delete
To create files in a directory a process must have 'w' and 'x' permission on that directory
If you can create a file you can delete any file in the directory– unless the 'sticky bit' is set, then a process can only
delete the files it owns (except the owner of the directory)
ApplicationApplication
Web page visitors.html invites the user to add a comment.
The work is done by visitors.py which opens the file visitors.txt, adds the comment and returns the current contents.
See visitors.html, visitors.py
Sample permissionsSample permissions
Set visitors.py permissions to 755 Set visitors.html to 644 Set visitors.txt to 666 Ser directory of visitors.txt 777 You see these permissions frequently
suggested:– they will work whatever user and group Apache is
running as.– typically Apache runs as user nobody (group
nogroup)
visitors.pyvisitors.py
The script opens visitors.txt for appending. – if the file does not exist it is created– Creation requires write permissions on the
directory Creation permission on the directory
carries with it delete permission– so the script could delete the file if it wanted to.– in fact any Apache script on that server can
delete the file, not just your scripts.
MitigationMitigation
The malicious user needs to know the file system path to the writable directory.
You only need set other permissions for the standard Apache set up. Thus 707, 606, 404 will do.– you can set directory permissions to 705 on your
home directory. Then other users cannot list your directories because they share your group (users, typically)
Some server set ups allow Apache to run as the user who owns the file requested
Advanced UnixAdvanced Unix
Linux Kernel
December 1, 2005
Boot ProcessBoot Process
Boot ProcessBoot Process The basic input/output system (BIOS) starts
and checks for hardware devices.– Stored in the computer’s ROM and described as
firmware.– Finds the hardware devices (diskette drives, CD-
ROM drives, and hard drives) needed by the boot process.
– Loads and initiates the boot program stored in the Master Boot Record (MBR, residing in the first sector of the device), and passes control to the boot program.
First Stage Boot LoaderFirst Stage Boot Loader
Two boot loaders are available: Linux Loader (lilo) and Grand Unified Bootloader (grub)
The first-stage boot loader – reads in the partition table and looks for the
second-stage boot loader on the partition configured as bootable (/boot partition).
– Launches the second stage boot loader.
Second Stage Boot LoaderSecond Stage Boot Loader
Presents the user with different OS kernels it has been configured to boot.
Finds the kernel image in the /boot directory.– The kernel binary is named /boot/vmlinuz-<kernel-version>
Places the appropriate initial RAM disk image, called an initrd, into memory. The initrd is used by the kernel to load drivers necessary to boot the system.
Hands control to the kernel.
grub.confgrub.conf
# grub.conf generated by anaconda## Note that you do not have to rerun grub after making changes to this file# NOTICE: You have a /boot partition. This means that# all kernel and initrd paths are relative to /boot/, eg.# root (hd0,1)# kernel /vmlinuz-version ro root=/dev/hdb3# initrd /initrd-version.img#boot=/dev/hdbdefault=0timeout=10splashimage=(hd0,1)/grub/splash.xpm.gztitle Linux Fedora (2.6.5-1.358smp) root (hd0,1) kernel /vmlinuz-2.6.5-1.358smp ro root=LABEL=/ rhgb quiet initrd /initrd-2.6.5-1.358smp.imgtitle Linux Fedora-up (2.6.5-1.358) root (hd0,1) kernel /vmlinuz-2.6.5-1.358 ro root=LABEL=/ rhgb quiet initrd /initrd-2.6.5-1.358.imgtitle Windows 2000 rootnoverify (hd0,0) chainloader +1
Specifies the default boot image willbe the first hard entry
Grub will wait for 10 seconds for input fromthe user before continuing to boot.
The root partition is the second partitionon the first hard drive.
The KernelThe Kernel
Initializes and configures the computer’s memory and configures hardware attached to the system (processors, I/O subsystems, and storage devices).
Decompresses and mounts initrd to load all necessary drivers.
Mounts the root file system in read-only mode and frees any unused memory.
Starts the init process by running /sbin/init.
Initialization ProcessInitialization Process
Init parses the /etc/inittab file to determine the specifics of what programs to run and at what level.– 0 used to halt the system. The system performs an init 0
command and the system is halted.– 1 Puts he system into single-user mode.– 2 Puts the system into a multiuser mode but does not
support networking.– 3 Puts the system into the standard full multiuser mode but
does not automatically start X.– 4 Unused.– 5 X11; Puts the system into standard multiuser mode with a
graphical (X-based) login.
InittabInittab
id:5:initdefault:– Tells the init program what run level to use
after a reboot.si::sysinit:/etc/rc.d/rc.sysinit
– Tells the init program to run the rc.sysinit script.
– Since the second field is empty, the script will run at boot time for all run levels.
rc.systinitrc.systinit
Setting the path and the hostname, and checking whether networking is activated.
Mounting the /proc file system Setting the kernel parameters Setting the system clock Loading keymaps and fonts Starting swapping Initializing the USB controller along with the attached
devices. Checking the root file system. Remounting the root file system as read-write. Loading modules as appropriate.
Inittab (cont’d)Inittab (cont’d)
Starts the /etc/rc.d/rc script with the appropriate run level.– The rc script executes all of the scripts
pointed to by the symblic links contained in the directory for that run level.
– For example, if the run level is 3, the scripts pointed to by the links in /etc/rc.d/rc3.d are run.
/etc/rc.d/rc3.d/etc/rc.d/rc3.dK01yum K35vncserver K74ypserv S12syslog S28autofs S90xfsK05saslauthd K36lisa K74ypxfrd S13irqbalance S40smartd S95anacronK10dc_server K45named K89netplugd S13portmap S44acpid S95atdK10psacct K50netdump K99readahead S14nfslock S55cups S97messagebusK12dc_client K50snmpd K99readahead_early S18rpcgssd S55sshd S97rhnsdK15httpd K50snmptrapd S00microcode_ctl S19rpcidmapd S56rawdevices S99localK20nfs K50tux S05kudzu S19rpcsvcgssd S56xinetd S99mdmonitorK24irda K50vsftpd S06cpuspeed S20random S80sendmail S99mdmpdK25squid K70aep1000 S08iptables S24pcmcia S85gpmK34yppasswdd K70bcm5820 S09isdn S25netfs S87IIimK35smb K74ntpd S10network S26apmd S90crond
All the files here are only symbolic links to the actual scripts that exist in /etc/rc.d/init.d.
The system first runs the scripts whose names start with K to kill the associated processes /etc/rc.d/init.d/<command> stop
The system runs the scripts whose names start with S to start the processes /etc/rc.d/init.d/<command> start
Changing a K name to start with S (e.g., K20nfs S20nfs) makes Linux start the process rather than kill it.
Inittab (cont’d)Inittab (cont’d)
ca::ctrlaltdel:/sbin/shutdown -t3 -r now– Sets the Ctrl+Alt+Delete key combination to indicate a
reboot of the system.– -t option indicates that the init process waits for 3
seconds after sending the warning message and before sending the kill signal.
pf::powerfail:/sbin/shutdown -f -h +2 "Power Failure; System Shutting Down"
pr:12345:powerokwait:/sbin/shutdown -c "Power Restored; Shutdown Cancelled"
3:2345:respawn:/sbin/mingetty tty3– Initializes the ttys, provides the login and retrieves the user-
input data, and then starts a login process for the user.
Building the KernelBuilding the Kernel
Building the KernelBuilding the Kernel
cd /usr/src/linux-2.4.20-8 make mrproper (optional but recommended) make xconfig
– This command runs a X-based configuration tool that asks you specific question about every kernel configuration.
Building the Kernel (Cont’d)Building the Kernel (Cont’d)
make xconfig, make menuconfig, make config, etc….– Most kernel features have three compilation
options: Y (compiling the option directly into the kernel), N (not compiling the option at all), and M (compiling the option as a kernel module and load it on demand).
– After saving the selection, the configuration file /user/src/linux-2.4.20-8/.config is created.
Building the Kernel (Cont’d)Building the Kernel (Cont’d)
make dep– Creates dependency information, so that the compiler knows
each component’s dependencies and can compile components as appropriate.
make clean– Cleans up some miscellaneous object files.
make bzImage – Can customize the title in Makefile– Compiles the Linux kernel properly.– The result is a kernel file called bzImage and located in
/user/src/linux-2.4.20-8/arch/i386/boot make modules
– Compiles the kernel modules files
Building the Kernel (Cont’d)Building the Kernel (Cont’d)
make modules_install– Installs the kernel modules into the directory path
/lib/modules/2.4.20-8/kernel/drivers. make install
– Copies the new kernel and its associated files to /boot directory.
– Builds a new initrd image and adds new entries to the boot loader configuration file.
Use the command ls -l /boot to make sure the initrd-2.4.20-8.img file was created.
Confirm that the file /boot/grub/grub.conf contains a title section with the same version as the kernel package just installed