Embed Size (px)
Citation preview
Unix Administration
Guntis Barzdins
SYS ADMIN TASKS
Linux System Administration
Setting the Run LevelSystem ServicesUser ManagementNetwork SettingsScheduling JobsQuota ManagementBackup and RestoreAdding and Removing software/packagesSetting a PrinterMonitoring the system (general, logs)Monitoring any specific services running. Eg. DNS, DHCP, Web, NIS, NPT, Proxy etc.
Have you used UNIX before?
• Which OS did Apple choose when it needed a stable OS layer for its Mac OSX?
• Which OS made the biggest impact to the online lives as you know it today?
Process Manipulation
Once you run a program (e.g. vi, myprog,...), that program will suspend the terminal you called it in (the terminal will not be receiving input from you).
You can start the program in the background to avoid this: myprog &
You can suspend a program that is running and send it to background, if you already started it:
Ctrl-z (to suspend) bg (sends the suspended program to the background)
ps (show running processes) top (monitor running processes) kill (kill processes)
& (send process to background) bg (send process to background) fg (get process from background) Ctrl+c (terminate process) Ctrl+z (suspend process)
Intrusion Detection System (IDS)
Open Source Tripwire – is a file integrity-checking program for UNIX/Linux operating systems
Host-based Software that alerts you when important files change
Tripwire keeps a hash value for each designated file When a file is altered/deleted, tripwire will have a new hash value that is
different than the original Replaced by more advanced HIDS: OSSEC, Samhain, AIDE
Tripwire tutorial in a slide
Initial setup download / build / install it modify policy file (e.g. remove unnecessary files)
# vi /etc/tripwire/twpol.txt generate policy file
# twadmin –create-polfile /etc/tripwire/twpol.txt build initial database
# tripwire –init
check periodically# tripwire –check reconcile differences (e.g. software installation)
# tripwire –update –accept-all –twrfile report_file
LINUX Firewall
Linux Security
SELinux
Linux Security
Originally created by NSA to meet US DoD MACMalicious or broken software can have root-level access to the entire system by running as a root process. SELinux (Security Enhanced Linux) provides enhanced security.Through SELinux policies, a process can be granted just the permissions it needs to be functional, thus reducing the risk
SELINUX can take one of these three valuesenforcing - SELinux security policy is enforced.permissive - SELinux prints warnings instead of enforcing.disabled - SELinux is fully disabled.
SELinux Configuration
Linux Security
AppArmor
Less complex and less securePopular in user oriented distributions (Ubuntu, SUSE),
enabled for some potentially vulnerable services by defaultBundle software packages with AppArmor profilesCan create profile file by launching application in learning
mode, can make secure enough profile if application not already compromisedCapabilities: FS open/read/write different modes,
networking (all/tcp/udp), executability etc.
Log files On linux, you can go to /var/log Depends on the application Information shown in log files depend on
the debug level you defined
Configuring Disk Quotas
Linux System Administration
To implement disk quotas, use the following steps:
Enable quotas per file system by modifying /etc/fstabRemount the file system(s)Create the quota files and generate the disk usage tableAssign quotas
Configuring Disk Quotas
Linux System Administration
Enabling Quotas: Edit fstab to enable usrquota
LABEL=/1 / ext3 defaults 1 1
LABEL=/boot /boot ext3 defaults 1 2
LABEL=/users /users ext3 exec,dev,suid,rw,usrquota 1 2
LABEL=/var /var ext3 defaults 1 2
LABEL=SWAP-sda5 swap swap defaults 0 0
Configuring Disk Quotas
Linux System Administration
Remounting the File Systems: Issue the umount command followed by the mount command to remount the file system in which quota has been implemented (umount /users;mount /users)
Creating the Quota Database Files: Use quotacheck command to create quota.user filequotacheck -cu /users
Assigning Quotas per User: assigning the disk quotas with the edquota command (edquota <username>)
Disk quotas for user web_cc (uid 524): Filesystem blocks soft hard
inodes soft hard /dev/sdb1 988612 1024000 1075200
7862 0 0
Linux Filesystem Management
Linux Commands
badblocks Used to search a disk or partition for badblocks. (badblocks device) (badblocks hda)
df Shows the disk free space on one or more filesystems. (df –k, df -h)
du Shows how much disk space a directory and all its files contain. (du <directory>, du –sk <directory>, du –sh <directory>) Find out which users use most space etc. $ du /home -d 1 | sort
fsck Filesystem check. Must not be run on a mounted file system. (fsck <filesystem>)
Linux Filesystem Management
Linux Commands
sync Synchronize data on disk with memory. `sync' writes any data buffered in memory out to disk.
mount Used to mount a filesystem. Complement is umount. (mount <filesystem>, mount –a)
umount Unmounts a filesystem. Complement is mount. (umount <filesystem>)
Native UNIX Backup Utilities
UNIX Systems include 3 core utilities that allow you to backup files to tape or disk.
tar (very simple to use) cpio (a bit more complex) dump (most complex of the three)
Using the tar Utility for Backup
tar usage:tar [x|c]vf [tape device name] [files or directory]
Where: x = extract from a tape c = compress onto tape j = use bzip compression z = use gzip compression
(just like when we tar and untar regular .tar files)
Other UNIX Backup Utilities cpio – has the ability to detect I/O
errors during backup that tar cannot detect. Also has the ability to do things like specify wildcard patters during restore.
dump – very fast, detects I/O errors, allows you to perform incremental backups.
TAR CPIO DUMP
Simplicity of Invocation Very Simple (tar c files)
Needs find to specify file names Simple. Few Options
Recover from I/O errors? None. Write your own utility Resync Option on HP-UX will cause some data loss Automatically skips over bad section
Backup special files Later Revisions Yes Yes
Multi-volume backup Later Revisions Yes Yes
Backup across network? Using rsh only Using rsh only Yes
Append files to backup Yes, (tar –r) No No
Multiple Independent Backups on Single Tape Yes Yes Yes
Ease of listing files on the volume Difficult, Must search entire backup ( tar –t )
Difficult, Must search entire backup ( cpio –it )
Simple, Index at front ( restore –t )
Ease and speed of finding a particular file Difficult, No wildcards, Must search entire volume Moderate, Wildcards, Must search entire volume Interactive. Very easy with commands like cd, ls
Incremental backup No Must use find to locate new/modified files Incremental of whole filesystem only, Mult. Levels
List files as they are being backed up tar cvf 2>logfile cpio –v 2>logfile Only after backup with restore –t >logfile (Dump can show % complete, though.)
Backup based on other criteria No Find can use multiple criteria No
Restore absolute path names to relative location Only by using chroot Limited with cpio -I Always relative to current working directory
Interactive decision on restore Yes or No possible with tar –w Can specify new path or name on each file Specify individual files in interactive mode
Compatibility Multiple platform Multiple platform with ASCII header, not always portable
Readable between some platforms, but cannot be relied on
Primary usefulness Individual user backup, transfer files between filesystems
System backup, transfer files between filesystems System backup
Volume efficiency Medium, usually limited to 10k block size Medium, usually only 5K block size, but can specify larger size on some OSs
High, can usually specify up to maximum block size of device
Wildcards on restore No Yes Only in interactive mode
Simplicity of selecting files for backup from numerous directories
Low, must specify each independent directory, subdirectories included
Medium, find options None, will backup one and only one filesystem
Specifying directory on restore get files in that directory
Yes No, must use "path/*" Yes
Stop reading tape after a restored file is found No No Will stop reading tape as soon as last file is found
Track deleted files No No If you restore with –r, files deleted before last incremental dump will be deleted.
Filesystem efficiency Better Worst (files get a stat from both find and cpio) Best
Limit on path length (Tests done with Solaris native utils 7/99.)
155 characters. Complains "prefix is greater than 155 characters." Gtar has slight workaround.
255 characters. Doesn’t complain. Just truncates pathname to 255 char’s.
1056 characters.
Likelihood that file exists in TOC but not in archive Low Low Medium (since TOC is made first)
rsync Over network and filesystem Secure through SSH
Both ends require rsync executable, no services or daemons required
Incremental backup Delta encoding
Only changed parts of files transmitted
Example rsync -avz [email protected]:/home /backups/server1
Many options
Lost Root Passwd
If you have Lilo installed, type LILO: linux init 1 Change the root passwd, reboot again
If you have installed grub Type ‘e’ to go to edit mode, add init 1 argument at the end
Boot with LiveCD (default Ubuntu etc.) Mount the disk chroot into mounted disk passwd Reboot and remove CD
Linux Services
Linux System Administration
There are 113 daemons, Out of them, the following are most widely used:
apmd : Power Management
autofs : Automount services
crond : Periodic Command Scheduler
cups : Common Unix Printing System
dhcpd : The DHCP server
dovecot : IMAP (Internet Message Access Protocol) and POP3 (Post Office Protocol) server
gpm : Mouse
httpd : Apache Web server
Linux Services
Linux System Administration
iptables : Kernel based Packet Filtering firewall
kudzu: Finds new Hardware
mysqld : MySQL server
named : BIND server
network : Networking
nfs : Network File Share
nfslock : NFS file locking
ntpd : NTP (Network Time Protocol) server
portmap : RPC (Remote Procedure Call) support
postgresql : The Postgresql Database Engine
Linux Services
Linux System Administration
sendmail : Sendmail Mail Server
smb : Samba Network Services
snmpd : Simple Network Management Protocol
squid : Squid Proxy Server
sshd : Open SSH and SFTP server
syslog : System Logging
xinetd : Provides support for telnet, ftp, talk, tftp etc.
ypbind : NIS Server
Automating Unix Administration
You don’t want to spend the whole day making sure that all servers/workstations and its services are fine
Use monitoring tools that can alert you for any problem in the network
mon, nagios, cacti, angel Zabbix – Latvian product
Create scripts to check the status of servers/services and use cron to run it periodically
Mail the result to admin
Example script#!/bin/shmachine="sunfire"down=i=0while [ $i -le 15 ]do sun=$machine"$i" /usr/sbin/ping $sun > /dev/null if [ $? -ne 0 ] then down="$down:$sun" fi i=`echo "$i+1" | bc -l`done
if [ -n "$down" ]thenecho $down | tr : '\012' | /usr/ucb/mail -s "DOWN machines"
exit 0
NFS Architecture
VFS layer hides differences between OS’s It doesn’t matter what OS the client or server implements, UNIX
or Windows. As long as the file systems are compliant with the file system model offered by NFS.
Operations on VFS are either passed to local FS or to NFS Client, which handles files at the remote server.
All client-server communication is done through RPCs, with client and server stubs. Implemented with either UDP or TCP.
NFS Architecture
Stateless vs. Stateful
NFS (Network File System)RCP request Action Idempotent
GETATTR Get file attribute YES
SETATTR Set file attribute YES
LOOKUP File name search YES
ACCESS Check access YES
READLINK Read from symbolic link YES
READ Read file YES
WRITE Write to the file YES
COMMIT Fix server cache data to the disk YES
CREATE Create file NO
REMOVE Remove file NO
RENAME Rename file NO
NFS (Network File System)RCP request Action Idempotent
LINK Create hard link NO
SYMLINK Create symbolic link NO
MKNOD Create special node NO
MKDIR Crate directory NO
RMDIR Remove directory NO
READDIR Read directory YES
READDIRPLUS Extended directory read YES
FSSTAT Get FS dynamic attribute YES
FSINFO Get FS static attribute YES
PATHCONF Get POSIX information YES
NFS (Network File System) Stateless protocol problems:
Local file systems have state. Shared lock’s implemented by user space daemon
rcp.lockd Performance problems, because all file system modification
commands should be fixed on disks before RPC request can be positively answered. In most cases it is 3 I/O operations.
In NFSv3 protocol there is asynchronous writes. Implemented using cookies to control server state during asynchronous writes.
FreeBSD NFS implementationThere are 3 type of leases:
Non-cache lease – define that all file system operations should be take synchronously with server
Read cache lease – let client cache data, not allow to change file.
Write cache lease – let client to cache write operations for lease time. So if client cache write data, then this data will not be written to the server synchronously. When lease time coming to the end client will try to get another lease, but if it’s not possible, then data have to be written to the server.
FreeBSD NFS implementation (read cache lease)
Client A Client BServer
Read req. + lease
Answer
Read sys. call
Read sys. Call(from cache)
Read req.(cache miss)
AnswerLease timeoutRead sys. call
Lease expiredRead lease req.
Answer with same ctimectime the same -
cache validRead sys. Call(from cache)
Lease timeout
Read req.(cache miss)
Answer
Read cache lease for client A
Read req. + lease Read sys. call
Read sys. call
Lease timeout
AnswerClient B added to lease
Read req.(cache miss)
Answer
Time
FreeBSD NFS implementation (write cache lease)
Server Client B
Write system callWrite cached leaseWrite cached leasefor client B Answer
(write cache lease)Write system call(cached leaved records)
Write cached leasereq. before previous lease expired.
Get record lease
Lease update
Answer(write cache lease)
Lease timeout
System call
Lease expiredrecord
answer
record
answer
Lease expirationStopped for a moment becauseof records
Write_slack secondsAfter last records
Time
FreeBSD NFS implementation (non-cache lease)
Client A Server Client B
TimeRead sys. call req.Read req. + lease
Read cache lease for A client
answerRead req.(from cache)
Read req.(miss cache)
answerLease timeout
Read sys. call req. Lease request
Answer (non-cache lease)Read sys. call req.(non-cache lease mode)
Read req.
Read data
Lease expiredWrite sys. call req.Get write cache lease
Answer (non-cache lease)
Write sys. call(async write cached)
Cleanup req.
Write cached data to server
record
answer
answer
record
Release msg. Write sys. call req.Get write cache lease
record
answerSynchronous Writes wihout cache
Starting up NFS There are three key things you need to start
on Linux to make NFS work. /usr/sbin/rpc.portmap /usr/sbin/rpc.mountd /usr/sbin/rpc.nfsd
These things should start up automatically at boot time.
The file that makes this happen is "/etc/rc.d/rc.inet2"rpcinfo -p localhost program vers proto port 100000 2 tcp 111 portmapper 100000 2 udp 111 portmapper 100005 1 udp 679 mountd 100005 1 tcp 681 mountd 100003 2 udp 2049 nfs 100003 2 tcp 2049 nfs
39
Exporting File System
To make parts of your file system accessible over the network to other systems The /etc/exports file must be set up to define which of the local directories will
be available to remote users and how each is used# sample /etc/exports file /home/yourname 192.168.12.1(rw)/master(rw) trusty(rw,no_root_squash) /projects proj*.local.domain(rw) /usr *.local.domain(ro) @trusted(rw) /home/joe pc001(rw,all_squash,anonuid=150,anongid=100) /pub (ro,insecure,all_squash) /pub/private (noaccess)
stop and restart the server# etc/rc.d/init.d/nfs stop# etc/rc.s/init.d/nfs start
The NFS Server Started though rc script:/etc/rc.d/init.d/nfsMust be started after:/etc/rc.d/init.d/portmap
Uses these RPC daemons in /usr/sbin: rpc.nfsd – main component of NFS system rcp.mountd – handles mount requests rpc.quotad – allows for quota enforcement via NFS. All of which are started in the nfs rc script when the system starts
/etc/exports – the main server configuration file
Above utilities are part of knfsd package .rpm package on Linux.
/etc/exports Contains information about the directory
paths and partitions that are sharable and hosts they can be shared with.
i.e. “Any host from .rutgers.edu can access the /home/documents directory on my server”
Entry format:/dir/to/export client1(permissions) client2 (permissions)Sample entry:/tmp iti.rutgers.edu(rw) 185.14.237.4(ro)
Need to run exportfs to inform NFS server process about changes in /etc/exports:> /usr/sbin/exportfs –a (exports all entries)
The NFS Client Requires knfsd-clients .rpm package on
Linux. Necessary services started from:/etc/rc.d/init.d/nfslock
RPC daemons in /sbin handle file locking between client and server:
rpc.locked rpc.statd All are started from the nfslock rc script automatically
Allows clients to mount remote file systems either using the mount command or by placing an entry in the /etc/fstab file.
Local and remote file systems accessible on an NFS client
jim jane joeann
usersstudents
usrvmunix
Client Server 2
. . . nfs
Remote
mountstaff
big bobjon
people
Server 1
export
(root)
Remote
mount
. . .
x
(root) (root)
mount –t nfs Server1:/export/people /usr/studentsmount –t nfs Server2:/nfs/users /usr/staff
FUSE (Filesystem in Userspace)Lets non-privileged users create their own file systems without editing kernel code.
FUSE
Allows to implement anything with file write and read operations and provide it as file system
Encryption – EncFS, TrueCrypt, etc. Network protocols – SSH, FTP,
SFTP, etc. Cloud storage – Dropbox and every
other kind RAM disk
SMB SMB is Microsoft’s protocol to share files
and printers Also renamed CIFS (Common Internet File System) Client/Server, no location transparency Not the same as Samba: an open source implementation of SMB primarily
found on UNIX systems (Linux) SMB usually runs on NetBIOS (naming + sessions + datagram)
NetBIOS + SMB developed for LAN use A number of other services run on top of
SMB In particular MS-RPC, a modified variant of DCE-RPC Authentication for SMB handled by the NT Domains
suite of protocols, running on top of MS-RPCTo know more: Timothy D Evans, NetBIOS, NetBEUI, NBF, NBT, NBIPX, SMB, CIFS Networking
TCP/IP
NetBIOS
SMB
MS-RPC
NT-Domain
Samba Services
File sharing. Printer sharing. Client authentication.
SMB Protocol
Request/response. Runs atop TCP/IP. E.g., file and print operations.
Open close, read, write, delete, etc. Queuing/dequeing files in printer spool.
Network Booting
No need for harddisk(or harddisk with Linux) on every host
High level work flow The system boots up, may be with floppy (could be with hard disk also) Sends dhcp request for IP number, gets one Mounts the root file system over NFS
Requirements for Network Booting
Setup an LAN infrastructure Need to setup nfs server Need to setup dhcp server Build a kernel image for network
booting
Setup an LAN infrastructure
Ethernet Cable
Hub Ethernet Cable
Your m/c to be booted
Your host, NFS server and DHCP server should be on same LAN
NFS server
Setup nfs server• Edit /etc/exports file before starting
the nfs server. • / 10.114.7.115(rw,no_root_squash)• This will export all files with root r/w to host
10.114.7.115• Save your exports file and from the
prompt execute exportfs command• Start the nfs server (nfs daemon)
• E.g. /etc/rc.d/inid.d/nfs start
Setup dhcp server Add in your /etc/dhcpd.conf before starting the dhcp
server.
Set the correct MAC address in /etc/dhcpd.conf as follows:
subnet <subnet address e.g.10.3.31.0> netmask 255.255.255.0 {
}subnet 10.10.10.0 netmask 255.255.255.0 {
host master {hardware Ethernet <Mac address of your Ethernet card>;fixed-address <IP address of your machine e.g.10.10.10.1>;option root-path <your root path>”;
}} Save your /etc/dhcpd.conf file start the dhcpd dameon by “/etc/rc.d/init.d/dhcpd start”
command
Build a kernel image for network booting
Linux Kernel compilation steps: Assumptions: machine x86 (i386); boot loader lilo. Get plain vanilla kernel from www.kernel.org Explode it into a directory (better if can do it in /usr/src/) => tar -zxvf linux-2.x.xx.tar.gz Optional: create a symbolic link ln -s linux-2.x.xx linux cd to linux directory cd /usr/src/linux or cd /usr/src/linux-2.x.xx Select the components support by make menuconfig or make xconfig - save the
configuration Select IP:BOOTP support from Networking options In File system -> Network File System -> Select
NFS File system support and Root file system on NFS
Do Make dep bzImage Make modules modules_install
Build a kernel image for network booting…
Copy the /usr/src/linux/arch/i386/boot/bzImage to /boot
Do mkbootdisk with new kernel as argument
Optional take a coffee or tea break ?
Your CEO announces:• Company is changing name from "Windoze" to "UsefulNix"• TOMORROW!
Your "small part":• Update the company website* to reflect that!
Can you deliver this in time?*: About 20,000 html files.
Just imagine if one day...
Demo (1/2)- UNIX vs. Window
• Task 1 : Open a file. Find occurrences of "Windoze".
Windows: use Ctrl-F at any text editor. UNIX: grep -l Windoze fileName
• Task 2 : Find all files in folder A containing "html".
Windows: Arggghhhh!!! Open all files and check? UNIX: find A -type f | xargs grep -l Windoze
Demo (2/2) - UNIX vs. Window• Task 3 : Open a file. Replace "Windoze" by "UsefulNIX"
Windows: Use Ctrl + H at any text editorUNIX: perl -pi -e 's/Windoze/UsefulNIX/g' fileName
• Task 4 : Find all files in folder A with "html", and replace by "UsefulNIX"Windows: haizzz....UNIX: find A -type f | xargs grep -l Windoze | xargs perl -pi -e 's/Windoze/UsefulNIX/g'
See how powerful UNIX is ^^ & the idea of "achieving complex tasks through small toys“
Let's learn UNIX !!!
