Understanding Operating Systems 1 Ch. 14 : UNIX Operating System with Linux History –UNIX System V –Linux Design Goals Memory Management Processor Management

Understanding Operating Systems

1

Ch. 14 : UNIX Operating System with Linux

• History

– UNIX System V

– Linux

• Design Goals

• Memory Management

• Processor Management

• Device Management

• File Management

• Network Management

• Security Management

• User Interface


2

UNIX Runs On All Sizes of Computers

Advantages:

1. Portable from large to small systems because written in C.

2. Powerful utilities are brief, single-operation commands that can be combined in single command line.

3. Device independent -- can be configured to run any device because it includes device drivers as part of OS.

Disadvantages

1. No single standardized version of OS.

– AT&T’s UNIX V standard (late 1980s).

2. Commands are so brief that novice users find it unfriendly


3

Year Internal release

External release

Features

1971 Version 1 Based on MULTICS; introduced shell concept, written in assembly language

1972 Version 2 Added pipes and filters 1973 Version 3 Kernel and I/O, first version written in C 1973 Version 4 UNIX V4 1975 Version 6 UNIX V6 First version to become commercially available 1979 Version 7 UNIX V7 More powerful shell added: string variables,

structured programming, trap handling 1980 Release 3.0 UNIX System III First version used in 16-bit microcomputers 1981 Release 4.0 First version available for a mainframe 1983 UNIX System V

Release 1 Added small general-purpose programs

1984 UNIX System V Release 2

Added features from Berkeley version: shared memory, more commands, vi editor, termcap database, flex file names

1989 Open Software Foundation founded 1991 UNIX System V

Release 4 Combined features from BSD, SunOS and Xenix

1993? Novell’s Unix Ware

Novell’s version of System V release 4 designed to work with NetWare

199x? Solaris ? Sun’s version designed to run to Sun workstations, derived from AT&T’s UNIX System V, Release 4


4

UNIX System V

• AT&T entered computer industry by offering line of PCs powered by UNIX System V.

• 1990 -- +24 versions of UNIX (AT&T’s UNIX System V, A/UX, Ultrix, Microsoft’s XENIX & University of California at Berkeley’s UNIX Versions 4.1 bsd, 4.2 bsd, and 4.3 bsd.

• 1991 -- AT&T’s Unix System Labs developed System V release 4 which incorporated almost all features found in System V release 3 plus BSD release 4.3, SunOS, and Xenix. – Other companies established Open Software Foundation.

• 1993 -- Berkeley released 4.4BSD based on AT&T’s UNIX, requiring customers to obtain licenses from AT&T to use it.

• Novell acquired UNIX from AT&T & released own version, UNIX Ware. Designed to interact with Novell’s NetWare system.


5

New Releases of UNIX

• Although newer commands become more difficult to use, they can be adapted to new situations with relative ease.

• Offer full support for LANs.

• Comply with international OS standards.

• System security has been greatly improved & meets many of the U.S. government security requirements.

• Most UNIX systems feature Common Desktop Environment (CDE).– Uniform GUI designed to give UNIX systems a standard look.

• Many “standards,” each supported by coalition of vendors.


6

Linux

• Developed in Finland, by Linus Torvalds in 1991. • Linux was based on UNIX, multi-platform OS.

• Brought speed, efficiency & flexibility of UNIX to a PC environment.

• Open-source program -- source code is freely available to anyone for improvement.

– Under constant development by uncounted people around world.

• Linux is not UNIX, it’s clone.

• Linux’s ncurses (new curses) -- screen handling library that improved on curses libraries featured in UNIX Release V.

• First used by programmers at universities and research environments, but becoming widely adopted in commercial & industrial markets.


7

UNIX Design Goals“OS by programmers for programmers”

1. Develop OS to support software development. – Utilities for which programmers typically need to write code. – Each utility was designed for simplicity.– Designed to be used in combination with each other.

2. Keep algorithms as simple as possible (without becoming rudimentary).– UNIX can be mastered by experienced programmers in weeks.

3. Make OS & any application software developed for it, portable from machine to machine. – Reduces conversion costs & doesn’t cause application packages

to become obsolete with every change in hardware.


8

Linux Design Goals

Function Purpose Multiple processes & multiple processors

Linux can run +1 program at a time using one or several processors.

Multiple platforms Operate on Intel’s x86 processors & other platforms including Alpha, Sparc, certain 6800 platforms, certain MIPS machines, and Power PCs.

Multiple users Allows several users to work on the same machine at same time. Inter-process communications

Linux supports pipes, IPC, sockets, etc.

Terminal management Its terminal management conforms to POSIX standards and it also supports pseudo-terminals as well as process control systems.

Peripheral devices Supports a wide range of devices including sound cards, graphics interfaces, networks, SCSI, etc.

Buffer cache Supports a memory area reserved to buffer the input and output from different processes. Demand paging memory management

Loads pages into memory only when they’re needed.

Dynamic and shared libraries

Dynamic libraries are loaded only when they’re needed & their code is shared if several applications are using them.

Disk partitions Linux allows file partitions used by file systems such as Ext2 and partitions having other formats (MS-DOS, ISO 9660, etc.)

Network protocol It supports TCP/IP and other network protocols.


9

UNIX Memory Management

• Beginning with Version 4, available for multi-users.

• Swapping or demand paging memory management used.

• Best depends on kind of applications that will run.– Small jobs -- swapping. Many large jobs -- demand paging.

• Swapping requires entire program in RAM before execution.– Imposes size restriction on programs.

– Uses round robin policy which slows system when traffic is heavy.

• Paging requires more complicated hardware configurations.– Increases system overhead & under heavy loads,thrashing can occur.

– Advantage of implementing the concept of virtual memory.


10

Typical Memory Layout for Single User-Memory Part UNIX Image

• Image -- computer execution environment composed of: user-memory part, general register values, status of open files, & current directory.

• Must remain in memory during execution of a process

Stack

Data

Program Code

Highest Memory Address

Lowest Memory Address


11

Program Code

• Sharable portion of program. • Because this code will be physically shared by several

processes it must be written in reentrant code. – Code protected -- instructions not modified during normal execution.

– All data references made without use of absolute physical addresses.

• Memory Manager gives program code special treatment. – Space allocated to program code can’t be released until all of

processes using it have completed their execution.

– UNIX uses “text table” to track which processes using which code.

– Memory isn't released until the program code is no longer needed.


12

Data Segments & Stack Segments

• Data segment starts after program code & grows toward higher memory locations as needed by program.

• Stack segment starts at highest memory address & grows downward as subroutine calls & interrupts add info to it.

• Data & stack are non-sharable sections of memory, so when original program terminates memory space is released.

• While each process is in memory, Memory Manager protects them from each other so they don’t overlap .


13

UNIX Kernel

• Part of OS that implements “system calls” to set up memory boundaries so several processes coexist in memory. – Permanently resides in memory – Processes use calls to interact with File Manager & request I/O.

• Kernel -- set of programs that implements most primitive of system’s functions.– Only part of OS to permanently reside in memory.

• Remaining sections of OS handled in same way as any large program. – Pages of OS brought into memory on demand.– Uses least-recently-used (LRU) page replacement algorithm.


14

Linux Memory Management

• Allocates memory space to each process. – Address space divided : process code, process data,

code & shared library data used by process, & stack used by process.

• When process begins execution its segments have fixed size but sometimes process handles variables with unknown number & size. – System calls change size of process data segment

(expanding it to accommodate extra data values or reducing it when certain values positioned at end of data segment no longer needed).


15

Linux Memory Management - 2

• Memory protection based on type of info stored in each region belonging to process address space.

– If process modifies access authorization assigned to a memory region, kernel changes protection info assigned to corresponding memory pages.

• When process requests pages, Linux loads them into memory.

• When kernel needs the memory space, pages are released using a least recently used (LRU) algorithm.


16

Linux Memory Management - 3

• Linux maintains dynamically managed area in memory (page cache) where new pages inserted or old pages deleted. – If necessary, Linux deactivates swap devices without

rebooting.• To track free/busy pages, uses system of page tables.• With certain chip architectures, memory access uses

segments. • Uses VM mechanism provided by processor on which it is

executing, converting virtual addresses into physical addresses.


17

UNIX Processor Management

• Allocates CPU, schedules process, & handles process requests.

– Kernel maintains several tables to coordinate execution of processes & allocation of devices.

• Using predefined policy, Process Scheduler selects process from READY queue & begins its execution for given time slice.

• Process scheduling algorithm picks process with highest priority to be run first; any processes using a lot of CPU time get lower priority.

– System updates compute-to-total-time ratio for each job every second.

– If processes have same computed priority, handled round-robin.

• Effect of negative feedback -- system balances I/O-bound jobs with CPU-bound jobs.


18

UNIX Processor Management Policies

• To load process from READY queue, Processor Manager chooses process with longest time spent on secondary storage.

• To select process to temporarily move out, Processor Manager chooses process that’s either waiting for disk I/O or currently idle.

• If process is waiting for completion of I/O request & isn’t ready to run, UNIX dynamically recalculates all process priorities to determine which inactive but ready process will begin when processor available.

• Policies seem to work well & don’t impact on running processes.

• If disk is used for secondary file storage & as “swapping area,” then heavy traffic significantly slow disk I/O because job swapping may take precedence over file storage.


19

UNIX Uses Process & User Table to Keep System Running Smoothly

• Process table always resides in memory. – Set up when process created; deleted when process terminates.

• Each entry contains: process identification #, user identification #, process memory/secondary storage address, process size, & scheduling info.

• For processes with sharable code, process table maintains text table. – Contains: memory/ secondary storage address of text segment

(sharable code) & count to track # of processes using this code.

• When count =0, code not needed.– Table entry & any memory locations allocated to code segment

released.


20

User Table

• User table resides in memory only while process is active.– Allocated to each active process. – Kept in transient area of memory.

• Contains info needed when process is running: user & group identification # to determine file access privileges; pointers to system’s File Table for every file being used by process; pointer to current directory; & list of responses for various interrupts.

• User table, process data segment and code segment (sharable code, can be swapped into/out of main memory as needed.


21

Synchronization

• UNIX is true multitasking OS.

– Achieves process synchronization by requiring that processes wait for certain events.

– Each event is represented by integers that are equal to address of table associated with event.

• Race occurs if event happens during process’s transition between deciding to wait for event & entering WAIT state.

– Isn’t a problem in single-processor environments.

– May pose a problem in multiprocessor environments.


22

fork and wait Commands

• fork -- gives user capability of executing 1 program from another.

– Gives 2nd program all attributes of 1st program.

– Splits program into 2 copies, both running from statement after fork.

– Process id (pid) generated so each process has unique ID number.

• wait -- allows programmer to synchronize process execution by suspending parent until child is finished.


23

exec Commands (execl, execv, execls, execlp, execvp)

• Used to start execution of new program from another program.

• Successful exec call overlays 2nd program over 1st.– Leaves only 2nd program in memory. – 2nd program considered new process but has pid of 1st program.

• No return from successful exec call.– Concept of parent-child doesn’t hold.

• Programmer can use fork, wait, & exec commands in this order to create parent-child relationship & then have child be overlaid by another program that, when finished, awakens parent so that it can continue its execution


24

Linux Processor Management

• Linux scheduler scans list of processes in READY state & uses predefined criteria to choose which process to execute.

• 3 different scheduling policies: 1 for "normal" processes, and 2 for "real time" processes.

• Each process has associated process type, fixed priority & variable priority.

– Used by scheduler to determine which scheduling policy used on processes in READY queue.


25

Process Types in Linux

SCHED_FIFO for non-preemtible "real time" processes

SCHED_RR for preemptible "real time" processes

SCHED_OTHER for "normal" processes

• SCHED_INFO -- executed immediately; scheduler selects process with highest priority & executes it. – Non-preemptible; normally runs to completion.

• SCHED_RR -- executed using round robin with small time quantum.

• SCHED_OTHER -- executed only when "real time" processes do not exist in READY queue. – dynamic priorities set by user-specified level using system calls

"nice" and "setpriority," & by factor computed by system.


26

Linux Supports Pipes & Clones

• Linux supports pipes to allow executing processes to exchange data.

• Extension permits process clones to be created. – Clone process is created using primitive clone, by duplicating

parent process. – Allows both processes to share same segment of code & data.– Any modification of 1 1 visible to other, unlike classical

processes.

• Ability to clone processes allows implementation of servers where several threads may be executing. – Operations may simply share data, without making use

of inter-process communication mechanisms.


27

Organization of Table of Processes in Linux

• Each process is referenced by descriptor.– Approximately 70 fields describing process attributes

& info needed to manage process. – Kernel dynamically allocates descriptors when

processes begin execution. – All process descriptors organized in doubly linked list.– Descriptors of ready/in execution processes put in

another doubly linked list with fields indicating "next run" and “previously run."

• Several macro instructions used by scheduler to manage & update process descriptor lists as needed.


28

Process Synchronization in Linux

• Provides wait queues & semaphores to allow 2 processes to synchronize with each other.

• Wait queue -- linked and circular list of process descriptors.

• Semaphores used to solve problems of mutual exclusion & problems of producers & consumers. – Contains 3 fields: semaphore counter, number of waiting

processes, & list of processes waiting for semaphore. – Semaphore counter may contain only binary values.– Semaphore counter takes value of number of units which are

accessible concurrently.


29

UNIX Device Management : Device Drivers

• Innovative feature is treatment of devices—device independent.– Treats each I/O device as special type of file. – Every device assigned name & descriptors that identify devices,

contain info about them, & stored in device directory.

• Device drivers -- subroutines working with OS to supervise transmission of data between main memory & peripheral unit.

• Incorporation of driver into kernel done during system configuration. – Config.c automatically creates conf.c file for any hardware config. – Contains parameters that control resources & 2 tables (bdevsw,

cdevsw).


30

Each Physical Device Identified By

• Major device number -- used as index to array to access appropriate code for specific device driver.

• Minor device number -- passed to device driver as argument & is used to access one of several identical physical devices.

• Class

• Each class has Configuration Table with array of entry points into device drivers. – Only connection between system code & device drivers.– Allows systems programmers to create new device drivers quickly

to accommodate differently configured systems


31

UNIX Divides I/O System Into 2 Separate Systems

1. Block I/O system (structured I/O system). – Used for devices addressed as sequence of 512-byte blocks.

– Allows Device Manager to use buffering to reduce I/O traffic.

– UNIX has from 10 to 70 buffers for I/O, & buffer-related info.

2. Character I/O system (unstructured I/O system).– Device drivers that implement character lists.

– Subroutine puts a character on list, or queue, and another subroutine retrieves character from list.

– E.g., terminal -- 2 input (raw, canonical) & 1 output queue.

• Some devices actually belong to both classes.


32

Linux Device Drivers

• Building on UNIX architecture, each device has special section in kernel (device driver).– Includes all instructions necessary to communicate with

device. • When new device is developed, it can be used with Linux

by writing its own device driver.• Instructions for talking to devices are loaded as needed

(seldom used devices) or can be kept in memory all the time when OS system boots.

• Although device files may be kept anywhere on file system, kept in /dev directory.


33

Linux Device Classifications

• Devices defined as either character mode or block mode.

• Information is kept in device file.

• Since there may be +1 device of same type in system, each device is identified by 2 device numbers.

– Major number -- device driver to be used.

– Minor number -- device number.

– Assigned to ensure each device has unique code.


34

UNIX File Management : Types of Files

1. Directories -- files used by system to maintain hierarchical structure of file system. • Users can read info, but only system can modify.

2. Ordinary files -- where users store information. • Protection is based on user’s requests (read, write, execute, &

delete functions).

3. Special files -- device drivers that provide interface to I/O hardware. • Appear as entries in directories. • Part of file system & reside in /dev directory. • Name of each special file indicates type of device with which it’s

associated.


35

UNIX File Structure

• Stores files as sequences of bytes.

• Doesn’t impose any structure on them.

– Text files are strings of characters with lines delimited by line feed, or new line, character.

– Binary files are sequences of binary digits grouped into words as appear in memory during execution.

• Structure of files is controlled by programs that use them, not by system.


36

UNIX File Management System Organizes Disk Into 512-Byte Blocks

1. Disk divided into 4 basic regions:

2. Address 0 reserved for booting.

3. Size of disk & boundaries of other regions.

4. i-list -- list of file definitions using combination of major & minor device numbers & i-numbers to uniquely identify file.

5. Free blocks available for file storage -- kept in linked list where each block points to next available empty block.

– As files grow, noncontiguous blocks linked to chain.


37

UNIX File Storage

• Whenever possible files are stored in contiguous empty blocks.

• Allocation is very simple.

• No need to compact files until become large & more dispersed—so that file retrieval becomes cumbersome.

– Compaction to bring retrieval time back to normal.


38

i-list

• Each entry in i-list called i-node with 13 disk addresses.

– 1st 10 addresses point to 1st 10 blocks of a file.

– If file > 10 blocks, 11th address points to block with addresses of next 128 blocks of file.

• Each i-node contains info on specific file (e.g., , owner’s identification, protection bits, physical address, file size, time of creation, last use/last update, number of links, & of file is directory, ordinary file, or special file).


39

File Names

• UNIX is case sensitive.

• Most UNIX versions allow file names with unlimited length.

• Some system programs (e.g., compilers) expect files to have specific “suffixes”.

• UNIX supports a hierarchical tree file structure.

– Root directory is identified by slash (/).

– Names of other directories are preceded by (/) symbol, which is used as a delimiter.


40

A Few Rules Apply to All Path Names

1. If path name starts with a slash, path starts at root directory.

2. A path name can be either one name or list of names separated by slashes.

3. Using 2 periods (..) in path name will move you upward in hierarchy (closer to root).

4. Spaces are not allowed within path names.


41

File Directories

Access control

No. of links

Group Owner No. of bytes

Date Time File name

drwxrwxr-x 2 journal comp 128 Jan 10 19:32 chap7 drwxrwxr-x 2 journal comp 128 Jan 15 09:59 chap8 -rwxr-xr-x 1 journal comp 11904 Jan 6 11:38 ms-dos -rwxr--r-- 1 journal comp 12556 Jan 20 18:08 unix -rwx------ 1 journal comp 10362 Jan 17 07:32 vax

• Information for each file: access control, number of links, name of group and owner, byte size of file, date & time of last modification, and file name.


42

Data Structures for Accessing Files

• Info presented in directory isn’t all kept in same location.

• UNIX divides the file descriptors into parts.

– Hierarchical directories contain only name of file & “i-number.”

• All i-nodes are stored in reserved part of device where directory resides, usually in Block 1.

• Each i-node has room for 13 pointers (0–12).

– 1st 10 block # stored in i-node list relate to 1st 10 blocks of file.

– For files larger than 138 blocks, 12th entry points to a block that contains a list of 128 indirect blocks.

– For extremely large files of more than 16,522 blocks, 13th entry points to a “triple indirect block.”


43

Open/Create/Link/Delete Files

• File opened -- its device, i-number, & read/write pointer are stored in System File Table & indexed by the i-node.

• File created -- i-node is allocated to it, & directory entry with file name & its i-node number is created.

• File linked -- directory entry is created with new name & original i-node number, & link-count field in the i-node is incremented by one.

• Delete shared file -- link-count field in i-node is decremented by 1. – When count =0, directory entry is erased & all disk blocks

allocated to file, along with its i-node block, are deallocated.


44

Linux File Management

• All Linux files organized in directories connected to each other in treelike structure.

• Linux specifies 5 types of files used by system to determine what file is to be used for.

File type File functionsDirectory Contains lists of file names.Ordinary file Contains data or programs belonging to users.Symbolic link Contains path name of another file that it is linking

to.Special file Assigned to device controller located in kernel.

When accessed, physical device associated with it isactivated & put into service.

Named pipe Used as communication channel among severalprocesses to exchange data. Creation is same as anysort of file. Carried out via a C function namedmkfifo.


45

Linux File Names & File Directories

• File names can be up to 256 characters long and contain alphabetic character, underscores, & numbers.

• Linux actually uses only 1st 32 or 64 characters of name.

• File directories contain same info, in same order as UNIX.

• Linux supports—for each type of member, owner, group, or world—3 types of file permissions: read, write, & execute.


46

Linux Data Structures

• Linux support several file types.

– Some inherent to this OS, such as Ext2FS.

– Others from other OS such as UNIX System V.

• Kernel has layer of software to maintain interface between system calls related to files & file management code.

– Virtual File System (VFS).

– Any process-initiated system call to files is directed to VFS.

– Performs file operations independent of format of file system involved.

– VFS then redirects request to module managing file.


47

UNIX User Interface

Command Stands for Action to be performed (filename) Run File Run/Execute file with that name. ls List Directory Show a listing of file names in directory. ls -l Long List Show a comprehensive directory list. cd Change Directory Change working directory. cp Copy Copy a file into another file or directory. rm Remove Remove/delete a file or directory. mv Move Move or rename a file or directory. more Show More Type the file’s contents to the screen. lpr Print Print out a file. date Date Show date and time. date -u Universal

Date/Time Show date and time in universal format.

mkdir Make Directory Make a new directory. grep “Global Regular

Expression/Print” Find a specified string in a file.

cat Concatenate Create a file or append to existing file. format Format Format a volume. diff Different Compare two files. pwd Path Working

Directory Show the path name of this directory.


48

Command Files (Script Files)

• Used to automate repetitious tasks.

– Each line of file is valid UNIX instruction and can be executed by user simply by typing sh and the name of the script file.

– E.g., bash (Bourne Again Shell), csh (C shell) & ksh (Korn shell).

• Script files used to automate repetitive tasks & to simplify complex procedures.

• If script file included in user’s configuration file, automatically executed every time user logs on.

– Exact name of user configuration file varies from system to system.

– Common names are .profile and .login.


49

Redirection

• To send output to a file or to another device, use symbol > between command & output destination.

– ls > myfiles

• To concatenate files (copy their contents into file), use cat file1 file2 > newfile

• Symbol >> appends new file to existing file.

• Reverse redirection takes input for program from existing file. mail ann roger < memo


50

Pipes

• Pipes & filters allow you to redirect output or input to selected files or devices based on commands given to command interpreter.

– UNIX does that by manipulating I/O devices as special files.

• Pipe -- open file connecting 2 programs: info written to it by 1 program may be read immediately by other, with synchronization, scheduling, & buffering handled automatically by system.

– Programs are executing concurrently, not one after other.

– who | sort

• Pipeline -- several programs simultaneously processing same I/O stream.

– who | sort | lpr


51

Filters

• UNIX has many programs that read some input, manipulate it in some way, & generate output – filters.

• wc (word count) -- counts lines, words, & characters in file.

• sort -- contents of file sorted & displayed on screen or accepts input from keyboard & directs output to screen. – sort -f sortednames– sort -n sortednums– sort -nr sortednums– sort +2f sortedempl


52

Other Commands

• man – displays on-line manual supplied with OS. • grep -- looks for specific patterns of characters.

– grep Pittsburgh maillist– grep -v Pittsburgh maillist– grep -c Pittsburgh maillist

• pg or more -- displays output one screen at a time. • nohup -- starts its execution & then log off system without having to

wait for it to finish. – nohup cp oldlargefile newlargefile &

• nice – lowers priority of a program if it uses large number of resources & you’re not in hurry for results.– nice cp oldlargefile newlargefile &


53

Linux User Interface

• Linux can be used in command-based mode.

• Also has built-in GUI support for X-Windows developed at MIT as part of Athena project.

– Windows Application Binary Interface (Wabi)

• Linux tape archive utility is significant improvement over UNIX tar utility.

– Features built-in error recovery.

– Accommodates both command line interface and menu-driven GUI.

Documents

Understanding Operating Systems 1 Ch. 14 : UNIX Operating System with Linux History –UNIX System V –Linux Design Goals Memory Management Processor Management