Linux

Arti Srivastava

2

Agenda

1. Introduction of Linux – History, Distributions

2. Exploring Command line tools – shells, redirection, pipes

3. Software Management – RPM, dpkg

4. Hardware configuration– modprobe, lspci, lsmod, insmod, rmmod

5. Managing Files – mkdir, cp, rm, grep, find

6. Administering the system – useradd, usermod, shutdown, crontab, chown

7. Networking – ifconfig, route, nslookup, ping, samba, ftp, http, mail, ssh, scp

3

1. Introduction of Linux – History, Distributions

History

4

In 80’s, Microsoft’s DOS was the dominated OS for PC

Apple MAC was better, but expensive

UNIX was much better, but much, much more expensive. Only for minicomputer for commercial applications

People were looking for a UNIX based system, which is cheaper and can run on PC

Both DOS, MAC and UNIX were proprietary, i.e., the source code of their kernel is protected

No modification is possible without paying high license fees

GNU

5

Established in 1984 by Richard Stallman, who believes that software should be free from restrictions against copying or modification in order to make better and efficient computer programs

GNU is a recursive acronym for “GNU's Not Unix” Aim at developing a complete Unix-like operating system which is free for copying and modificationCompanies make their money by maintaining and distributing the software, e.g. optimally packaging the software with different tools (Redhat, Slackware, Mandrake, SuSE, etc)Stallman built the first free GNU C Compiler in 1991. But still, an OS was yet to be developed

Begin Linux

6

Andrew Tanenbaum developed Minix, a simplified version of UNIX that runs on PC

In Sept 1991, Linus Torvalds, a second year student of Computer Science at the University of Helsinki, developed the preliminary kernel of Linux, known as Linux version 0.0.1

Soon more than a hundred people joined the Linux camp. Then thousands. Then hundreds of thousands

It was licensed under GNU General Public License, thus ensuring that the source codes will be free for all to copy, study and to change.

7

At 25/11/2012 08:31pm, there are 118,240 users and 95,204 machines registered.My guess at the number of Linux users: 63,195,939

World population: 7,096,950,557Internet users: 2,478,272,151

– Taken from linuxcounter.net

Major Software from GNU Project

8

Gcc : c compilerG++: C++ compilerGdb: source code debuggerGNU make: a version of makeBison: a parser generatorBash: command shellGNU emacs: a text editor

Linux directory structure

9

/ - root directory/bin – essential programs/boot – boot information for linux/dev – includes all devices/mnt – storage devices are mounted/proc – fluid data and status of kernel/sbin – sys admin software /etc – admin related config files and folders/home/lib/tmp/usr/var

Linux Variant

10

SuseRedhatDebianMandrakeUbuntuCentosScientificOS

Desktop Applications

11

Word processing (OpenOffice, Koffice)Programming (C, C++, Perl, Python, Java, PHP)Graphics (GIMP)Web browsers (Mozilla, Konquerer)Email (Evolution, Mozilla, KMail)Audio (amarok)Video (mplayer)Games (MAME)

Linux continue to grow…

12

Mobile OS: Android is Linux based Major Virtualization flavour: Base kernel is Linux

Citrix Xen Vmware

Cloud solution - Openstack

13

14

2. Exploring Command line tools – shells, redirection, pipes

Shell

15

Bash – Baurne Again Shell (Default Shell)

Csh

Tcsh

Zsh

In GUI – xterm, kconsole

Shell Configuration Files

16

Login config files: ~/.bashrc

~/.profile

~/.bash_login

~/.bash_logout

Global configuration file /etc/profile

/etc/bash.bashrc

Shell variable Using Environment variables: env

PATH

alias

set

unset

Commands

17

Change the working directory – cd, cd~, cd /, cd /var/log

Display the working directory – pwd

Display a line of text – echo Hello

Execute a program – exec myprog

Time an operation – time lsof Total execution time, user cpu time, system cpu time

Set options – environment variables

Terminate the shell Exit

logout

MAN Page

18

1 Executable programs and shell commands

2 System calls provided by the kernel

3 Library calls provided by program libraries

4 Device files (usually stored in /dev)

5 File formats

6 Games

7 Miscellaneous (macro packages, conventions, and so on)

8 System administration commands (programs run mostly or exclusively

by root)

9 Kernel routines

Redirection

19

> : Creates a new file containing standard output. If the specified file exists, it’s

overwritten.

>> : Appends standard output to the existing file. If the specified file doesn’t exist, it’s created.

2> : Creates a new file containing standard error. If the specified file exists, it’s

overwritten.

2>> : Appends standard error to the existing file. If the specified file doesn’t exist,

it’s created.

&> : Creates a new file containing both standard output and standard error. If the

specified file exists, it’s overwritten.

< : Sends the contents of the specified file to be used as standard input.

<< : Accepts text on the following lines as standard input.

Redirection continued…

20

<> : Causes the specified file to be used for both standard input and standard

output.

tee command

View the command and send the output to another file

lsmod | tee lsmod.txt

Pipes

21

ps aux | grep apache

xargs

find ./ -name “*~” | xargs rm

Less is more

22

more

less

23

3. Software Management – RPM, dpkg, yum, processes

Package Concepts

24

Packages: collection of files

Installed file database

Dependencies

Checksums

Upgrades and uninstallation

Package naming: samba-4.0.12-24.i386.rpm– Package name

– Version no

– Build no

– arhitecture

RPM Operations

25

-i : Installs a package;

-U : Installs a new package or upgrades an existing one

-F or --freshen : Upgrades a package only if an earlier version already exists

-q : Queries a package—finds if a package is installed, what files it contains, and so on

-V or --verify : Verifies a package—checks that its files are present and unchanged

since installation

-e : Uninstalls a package

rpm examples

26

rpm -qa -> lists all the installed packages

rpm -qc {pname} -> list configuartion file names for given package

rpm -qi {pnane} -> Give details of package

rpm -ql {pname} -> lists the files in a package

rpm -qR {pname} -> Lists package dependencies

rpm -qf filename -> List the package name of given file

rpm -qpl {pname} -> lists all the files in a package

rpm -qp {pname} -> list the package with given pname(*/?)

rpm -Va -> Verify all the installed packages

rpm -V {pname} -> Verify specify package

rpm -V -f {filename} {packagename} -> Verify a specified file in a package

rpm continued…

27

extracting data from rpms – rpm2cpio

$ rpm2cpio XXXsrc.rpm > xxx.cpio

$cpio –i –make-directories < xxx.cpio

$rpm2cpio xxxsrc.rpm | cpio –i –make-directories

dpkg dpkg –i xxx.deb

dpkg –r xxx.deb

apt-get install xxx

dpkg

dselect utility – menu driven

aptitude install/update/remove

/etc/dpkg/dpkg.cfg and ~/.dpkg.cfg,

/etc/apt/apt.conf(controls dselect and apt)

Yum

/etc/yum.repos.d

/etc/yum.conf

Yum client Redhat 5: yum

Sles10 : zypper

Sles11 : rug

Yum installation Setting the yum repo

install, upgrade, uninstall

Try this: http://linux.dell.com/repo/hardware/

alien

Covert packages from one format to another

Formats are:

Linux standard base

RPM

deb

stampede(.slp)

Solaris(.pkg)

Slackware(.tgz)

# alien --to-rpm --scripts ./mypkg.deb

Library management

/etc/ld.so.conf

/etc/ld.so.conf.d/*.conf

Temporarily changing the library path LD_LIBRARY_PATH

$ export LD_LIBRARY_PATH=/usr/local/testlib:/opt/newlib

ldd Displaying shared library dependencies : ldd /bin/cat

Understanding the kernel

uname –n -> hostname

uname –s -> kernel name

uname –v -> kernel version

uname –r -> kernel release

uname –m -> machine option

uname –p -> Processor

uname –o -> Operating system

uname –i -> Hardware platform

uname –a -> all information

Process – ps output meaning(columns)

Username

Process ID

Parent Process ID

TTY – identifying terminal

Cpu time

Cpu priority

memory use

Command

Other Process related commands

Dynamic variant of process – top

nice : Run a program with modified scheduling priority. Priority ranges from

-20(most favorable) to +19(least favorable)

renice : alter priority of running processes

kill

nohup

killall

36 Confidential

4. Hardware configuration– modprobe, lspci, lsmod, insmod, rmmod

Hardware

BIOS – resides on the motherboard in ROM – EEPROM/Flash memory

When computer is turned on- BIOS performs POST and initializes hardware and then load boot loader

IRQ - An interrupt request (IRQ), or interrupt, is a signal sent to the CPU instructing it to suspend its current activity and to handle some external event such as keyboard input. /proc/interrupts

I/O addresses (also referred to as I/O ports) are unique locations in memory that are reserved for communications between the CPU and specific physical hardware devices. Like IRQs, These are commonly associated with specific devices and should not ordinarily be shared.

DMA Addresses - Direct memory addressing (DMA) is an alternative method of communication to I/O ports. Rather than have the CPU mediate the transfer of data between a device and memory, DMA permits the device to transfer data directly, without the CPU’s attention. The result can be lower CPU requirements for I/O activity, which can improve overall system performance.

/proc/dma

Common Linux Devices• Linux Device Windows Name Typical IRQ I/O Address

• /dev/ttyS0 COM1 4 0x03f8

• /dev/ttyS1 COM2 3 0x02f8

• /dev/ttyS2 COM3 4 0x03e8

• /dev/ttyS3 COM4 3 0x02e8

• /dev/lp0 LPT1 7 0x0378-0x037f

• /dev/lp1 LPT2 5 0x0278-0x027f

• /dev/fd0 A: 6 0x03f0-0x03f7

• /dev/fd1 B: 6 0x0370-0x0377

Coldplug and Hotplug devices

Colplug devices – Components internal to the computer such as memory, CPU, pci cards etc. resides on the motherboard in ROM – EEPROM/Flash memory

Hotplug devices – Devices which can be added/removed when the system is in running state.

Configuring expansion cards

lspci

setpci

• /usr/share/misc/pci.ids

Learning about kernel modules

lsmod

insmod

modprobe

rmmod

modinfo

Configuring USB devices

USB basics USB 1.0 and USB 1.1 : 12Mbps

USB 2.0 : 48Mbps

USB 3.0 : 3.2 Gbps

USB devices: Scanner, printer, mice, digital camera, keyboard, speakers etc.

lsusb

/proc/bus/usb

usbmgr

/etc/usbmgr/usbmgr.conf

Systems Run Levels

0: Halt

1: single User mode

2: Multi user mode without NFS

3: Full multi user mode

4: unused

5: X11

6: reboot

Filesystems

fdisk

mkfs –t ext3 /dev/sda2

mkswap /dev/hda2

swapon /dev/hda2

fsck

/etc/fstab

df

du

45 Confidential

5. Managing Files – mkdir, cp, rm, grep, find

File types

Regular file

Directory file

Special file Character

Block

Links Soft link Hard link

Sockets

Named pipes

Managing Links

ln [options] source link

Hard link: ln mainfile hlinkfile

Soft link: ln –s mainfile slinkfile

Inode table

Owner of the file

Group of the file

File type

File access permission

Date and time of last access

Date and time of last modification

Number of links to the file

Size of the file

Addresses of blocks where the file is physically present

Surrogate super block and Inode table

Super block – state of the file system Size

No of files it can accommodate

How many mores can be created

sync

How to check file inode no

ls –i filename

File Commands

ls

cp

mv

rm

touch : Last file modification time/inode change time/access time

ln

chmod

chown

mkdir

rmdir

umask

File Commands contd… cut

paste

join

split

expand

unexpand

uniq

sort

head

tail

wc

tr

nl

File Archiving Commands

tar

cpio

gzip

Gunzip

bzip2

Managing file ownership

Changing file ownership: chown

Chown [options] [newowner][:newgroup] filename

Changing file group ownership

chgrp [options] [newgroup] filename

File Access

Changing file permission: chmod

$ ls -l test

-rwxr-xr-x 1 rodsmith users 111 Apr 13 13:48 test– “-” -> file

– d -> firectory

– l -> symbolic link

– p -> Named pipe

– s -> Socket

– b -> Block device

– c -> Character device

SUID

SGID

Sticky bit

Setting the default mode and group

Default permissions are configurable and defined by user mask.(umask)

• Umask Created Files Created Directories

• 000 666 (rw-rw-rw-) 777 (rwxrwxrwx)

• 002 664 (rw-rw-r--) 775 (rwxrwxr-x)

• 022 644 (rw-r--r--) 755 (rwxr-xr-x)

• 027 640 (rw-r-----) 750 (rwxr-x---)

• 077 600 (rw-------) 700 (rwx------)

• 277 400 (r--------) 500 (r-x------)

Managing Disk Quotas

Enabling Quota Support Kernel 2.4.x – quota v1 support

Kernel 2.6.x – quota v2 support

/etc/fstab

/dev/hdc5 /home ext3 usrquota,grpquota 1 1

chkconfig quota on

Setting quota for users, edquota raghu

Locating Files

whereis

which

locate

find

type

Updatedb

/etc/updatedb.conf

Regular Expressions

59

Bracket Expressions: d[aeiou]g => dag, deg, dig, dog, dug

Range Expression : a[1-5] => a1, a2, a3, a4, a5

Any single character except new line: .

Start and end of line: ^ and $

Repetition operators: * -> 0 and more), + -> 1 and more, ? -> 0 and 1

Any single character except new line: .

Multiple possible strings: Linux | Windows

Escaping: backslash \

grep

60

grep [options] regexp [files]

grep openmanage /var/log/messages

grep –i openmanage /var/log/messages

grep –r –i openmanage /var/log

grep –r –i error /var/log

Sed(stream editor)

61

Modifies the contents of files

• sed [options] -f script-file [input-file]

• sed [options] script-text [input-file]

$ sed ‘s/2009/2010/’ cal-2009.txt > cal-2010.txt

awk

62

Print list of all processes of all the users ps -ef | awk '{print $1"=>" $8}‘

Print all the child process of PPID 1. ps -ef | grep -w 1 | grep -v /1 | awk '{print $2"=>"$3"=>"$8}‘

Display and create all the loaded module details in a file lsmod | awk '{print $1}'| xargs modinfo | tee mod.txt

File Systems

63

Ext2/3: Native file system

Reiserfs : Suitable for small files – less than 1K

Vfat : 32bit file system compatible with win

XFS : Journaling file systems –handle large files

JFS : Handles power down and crashes

Swap: virtual memory

Iso9660 : cdfs, dvd

64

65 Confidential

6. Administering the system – useradd, usermod, shutdown, crontab, chown

66 Confidential

Booting Linux and editing files

Boot Loaders BIOS->MBR->Boot loader->OS kernel

LILO

GRUB

Boot messages: /var/log/messages

Command dmesg

LILO – Linux Loader

Configuration file: /etc/lilo.conf

Boot loader location: boot=/dev/hda

Default: default os to boot

Boot Prompt

Boot Timeout

Linux root partition, root=/dev/hda4

Boot in read-only mode

Linux boot image: image=/kernelimage

RAM disk: initrd=oskernel imahe

Extra kernel option: mem=2048

Lilo –c – testing purpose

Lilo –v – output in verpose

LILO prompt

Boot: linux 1 -> To boot in single user mode 1/S/s/single can be typed

If suppose init program is corrupted Boot: linux init=/bin/sh can be used

GRUB

Configuration file: /boot/grub/menu.lst (some read/fedora used grub.conf)

Confidential71

LILO and GRUB

LILO GRUB

/dev/hda (hd0)

/dev/hda1 (hd0,0)

/dev/hdb (hd1)

/etc/lilo.conf /boot/grub/menu.lst

lilo /etc/lilo.conf grub-install /dev/hda

Linux Boot Process

• System Power on-> CPU executes BIOS code->BIOS(Post, check and configure hardware)->Boot loader kicks off->OS kernel is loaded(initialize devices, mount boot partition, init program starts->init selects run level from /etc/inittab and default level run level loads

Inittab entry

• id:runlevels:action:process

# Default runlevel. The runlevels used by RHS are:

# 0 - halt (Do NOT set initdefault to this)

# 1 - Single user mode

# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)

# 3 - Full multiuser mode

# 4 - unused

# 5 - X11

# 6 - reboot (Do NOT set initdefault to this)

Linux Boot Process contd..

inittab file

id:3:initdefault:

5:2345:respawn:/sbin/mingetty tty5

chkconfig : updates and queries run level information for system services.

runlevel : Find the current and previous run level

init : parent of all the processes

telinit: tell init to switch to specified runlevel

shutdown

Life cycle of a process

fork and exec init – process id 1

getty

login

sh

Who or grep or any command running on shell

kill

Linux installation and designing hard disk layout

/ : default root partition

/var : logs related entries

/home: User home directory

/opt: Optional packages

Swap space: typically double of RAM

Managing Users

useradd

usermod

userdel

groupadd

groupmod

gpasswd

Chage: change user password expiry information

usermod

Usermod –l [newlogin] [login]

Usermod –c [comment login]

Usermod –f [no of days] [login] => no of days password to expire

Usermod –L [login] -> lock the password and suspend the user

Usermod –U [login] -> unlock the password

Usermod –e [yyyy-mm-dd] [login] -> change the expiration date

Tuning User and System Environments

Global Configuration file /etc/profile

/etc/bash.bashrc

User Configuration file ~/.profile

~/.bashrc

Configuartion file for added user’s fefault environment setting /etc/skel

Kernel Parameter /etc/sysctl.conf

sysctl

Automate system administration tasks by scheduling jobs

Manage cron and at jobs

Configure user access to cron and at services

The following is a partial list of the used files, terms, and utilities:

/etc/cron.{d,daily,hourly,monthly,weekly}

/etc/crontab

crontab –e

Cronjob Definition

80

* * * * * Command

Minute(0-59)

Hour(0-23)

Day of month(0-31)

Month(1-12)

Day of week(1-7)

System logging

Syslog configuration files: /etc/syslog.conf

/etc/sysconfig/syslog

/var/log

/var/log/messages

dmesg

last

Binary names: syslogd

klogd

Systems Monitoring

Process ps

top

Disk space df

du

Bandwidth tcpdump

netstat

Memory pmap

ps

free

other commands lsof

Systems Monitoring contd…

uptime: Load average is average no of processes waiting to run in 1min, 5min, and 15 mins. Ideally it should be <1.

tload is graphical presentation of uptime.

vmstat-Virtual memory usage

pmap -x pid: mapping of processes with memory resources

scsiinfo

hdparam

Perform security administration tasks

Audit system to find files with the suid/sgid bit set.

Set or change user passwords and password aging information.

Discovering open ports on a system: nmap, netstat

Setting up limits on user logins, processes and memory usage ulimit

Basic sudo configuration and usage /etc/sudoers

Linux Kernel

Module components in source tree: /usr/src/linux

Module components at runtime: /lib/modules/<kernelversion>/kernel.

Download latest stable kernel from www.kernel.org

Unpack the kernel

Compiling a kernel: make config/menuconifg/xconfig

make dep

make bzImage

make modules

make modules_install

make install

Installing the kernel image using boot loader: /boot/grub/grub.conf

Reboot the system and new kernel should be up

Shell Scripting

Variables – strings, numbers, environment and paramter Conditions: shell booleans Control Structures: if, elif, for, while, until, case Lists Functions Commands built into shell Getting the result of a command Here documents

Quoting

Declaration svar = “Hello world”

$svar “$svar” ‘$svar’ \$svar

Environment Variables

$HOME $PATH $PS1 $PS2 $0 $# $$ $1, $2, $3…

$*

$@

Control Structures

Ifif condition

then

Statements

else

Statements

fi

elifif condition

then

Statements

elif

then

statements

else

Statements

fi

Control Structures contd..

forfor variable in values

do

statements

done

whilewhile condition do

statements

done

untiluntil condition

do

statements

done

Control Structures contd..

CaseCase variable in

pattern1 statements;;

pattern2 statements;;

pattern3 statements;;

esac

Debugging scripts

sh –n <script> sh –v <script> sh –x <script>

GUI Based Monitoring tools

Nagios Cacti Zabbix MRTG Nfsen

94 Confidential

7. Networking

Basic Network Configuration

Manually and automatically configure network interfaces

Basic TCP/IP host configuration

The following is a partial list of the used files, terms, and utilities:

/etc/hostname

/etc/hosts

/etc/resolv.conf

/etc/nsswitch.conf

ifconfig

Configuring Network

/etc/sysconfig/network-scripts/ifcfg-eth0

A Sample Network Configuration File DEVICE=eth0 BOOTPROTO=static IPADDR=192.168.29.39 NETMASK=255.255.255.0 NETWORK=192.168.29.0 BROADCAST=192.168.29.255 GATEWAY=192.168.29.1 ONBOOT=yes

# ifconfig eth0 up 192.168.29.39 netmask 255.255.255.0

# route add default gw 192.168.29.1

ifconfig eth0

DNS Entry: /etc/resolv.conf

Configuring routing

# route add -net 172.20.0.0 netmask 255.255.0.0 gw 172.21.1.1

ifup eth0

ifdown eth0

ethtool

Network Port numbers

• Port Number TCP or UDP Purpose Example Linux Servers

20 TCP File Transfer Protocol ProFTPd, vsftpd

21 TCP FTP ProFTPd, vsftpd

22 TCP Secure Shell (SSH) OpenSSH, Dropbear

23 TCP Telnet in.telnetd

25 TCP Simple Mail Transfer Protocol (SMTP)Sendmail, Postfix,

53 TCP and UDP Domain Name System (DNS) BIND;

Diagnosing Network connections

ping

traceroute

netstat

nslookup

dig

Examining raw network traffic– tcpdump

– wireshark

Various tools

telnet

ftp

ssh

scp

ping

Linux Server

Web Server – Apache

Database Server – mysql, oracle

ftp server – proftp, vsftp

File server - samba

Mail server – sendmail, postfix

Linux Firewall

#iptables -t filter -A INPUT -p tcp --dport 22 -j DROP#iptables -t filter -A INPUT -p udp --dport 22 -j DROP#iptables -t filter -A INPUT -p tcp --dport 23 -j DROP #iptables -t filter -A INPUT -p udp --dport 23 -j DROP#iptables -t filter -P OUPUT DROP#iptables -t filter -A OUTPUT -p tcp --dport 80 -j ACCEPT#iptables -t filter -A OUTPUT -p udp --dport 80 -j ACCEPT #iptables -t filter -A OUTPUT -p tcp --dport 53 -j ACCEPT#iptables -t filter -A OUTPUT -p udp --dport 53 -j ACCEPT  # service iptables save#service iptables restart

Exercise

How long the server is running and no of users who are using the system.

Create a user with your name, create a group name training, validity period, home directory, assign training group

Search for files with specific pattern

Display line no 10 to line no 20 , from a file having 30 lines.

print file in reverse

References

www.tldp.org

www.kernel.org

www.linux.org