308
SYSTEM PROGRAMMING IN LINUX UTTAM K. ROY Dept. of Information Technology, Jadavpur University, Kolkata

System Programming in Linux

Embed Size (px)

Citation preview

Page 1: System Programming in Linux

SYSTEM PROGRAMMING

IN LINUX

SYSTEM PROGRAMMING

IN LINUX

UTTAM K. ROY

Dept. of Information Technology,

Jadavpur University, Kolkata

Page 2: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

2System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Introduction to Operating Systems and Linux InternalsGeneral Introduction to Operating SystemsProcess managementSchedulingMemory ManagementFile Management

Linux Systems ProgrammingFiles and DirectoriesInput/OutputSystem FilesProcess ControlSignalsInterprocess CommunicationNetwork programming

Linux System AdministrationLinux Shells and CommandsSystems managementManaging File SystemsNetwork Management

Topics to be discussed

Page 3: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

3System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Silberschatz Galvin, “Operating System Concepts”, Addison Wesley publication

• Richard Stones & Neil Matthew, “Beginning Linux Programming”, A Wrox publication

• Andrew S. Tanenbaum, “Operating Systems”

• Daniel P. Bovet, Marco Cesati, “Understanding the Linux Kernel”, O’Reilly publication

• Sumitabha Das, “Unix Shell Programming”, A Tata McGraw-Hill publication

• Olaf Kirch, Terry Dawson, “Linux Network Administrator's Guide”, A O’relly publication

Books

• William Stallings, “Operating Systems”, Prentice-Hall India

Page 4: System Programming in Linux

INTRODUCTION

TO

OPERATING SYSTEM

INTRODUCTION

TO

OPERATING SYSTEM

Page 5: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

5System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Computer System Organization

Computer-system operationOne or more CPUs, device controllers connect through common bus providing access to shared memoryConcurrent execution of CPUs and devices competing for memory cycles

Page 6: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

6System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Components of a Computer System

Page 7: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

7System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

A computer system consists ofhardware

system programs

application programs

Components (Contd.)

Page 8: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

8System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Computer system componentsHardware – provides basic computing resources

• CPU, memory, I/O devices

Operating system• Controls and coordinates use of hardware among various

applications and users

Application programs – define the ways in which the system resources are used to solve the computing problems of the users

• Word processors, compilers, web browsers, database systems, video games

Users• People, machines, other computers

Components (Contd.)

Page 9: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

9System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

What is an Operating System?

A program that acts as an intermediary between a user of a computer and the computer hardware.

Operating system goals:Execute user programs and make solving user problems easier.

Make the computer system convenient to use.

Puspose:Use the computer hardware in an efficient manner.

Page 10: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

10System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Operating System Functions? Process Management

The creation and deletion of both user and system process

The suspension and resumption of process

The provision of mechanisms for process synchronization

The provision of mechanism for deadlock handling

Main Memory ManagementKeep track of which parts of memory are being used and by whom

Decide which processes are to be loaded into the memory when memory space becomes available

Allocates and deallocates memory as and when needed

File ManagementCreation and deletion of files

Creation and deletion of directories

The support of primitives for manipulating files and directories

The mapping of files onto secondary storage

The backup of files on stable(non-volatile) storage media

Page 11: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

11System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Operating System Functions? I/O System Management

A memory management component including buffering, caching and spoolingA general device driver interfaceDrivers for specific hardware interface

Secondary Storage ManagementFree space managementStorage allocationDisk Scheduling

NetworkingCommunication task

Command Interpreter SystemCommand line interpreter

Page 12: System Programming in Linux

PROCESS

MANAGEMENTPROCESS

MANAGEMENT

Page 13: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

13System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Process Management

Process ManagementCreating and deleting both user and system processes

Suspending and resuming processes

Providing mechanisms for process synchronization

Providing mechanisms for process communication

Providing mechanisms for deadlock handling

Page 14: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

14System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Process—Basic Concept

A process is a program in execution. It is a unit of work within the system. Program is a passive entity, process is an active entity.

Process needs resources to accomplish its taskCPU, memory, I/O, files

Initialization data

Process termination requires reclaim of any reusable resources

Page 15: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

15System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Single-threaded process has one program counter specifying location of next instruction to execute

Process executes instructions sequentially, one at a time, until completion

Multi-threaded process has one program counter per thread

Typically system has many processes, some user, some operating system running concurrently on one or more CPUs

Concurrency by multiplexing the CPUs among the processes / threads

Process—Basic Concept

Page 16: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

16System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Process States

Possible process statesrunning—Instructions are being executedBlocked—Process is waiting for some event to occurReady—The process in waiting to be assigned to a processor

Transitions between states shown

Page 17: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

17System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Process Control Block(PCB)

Fields of a Process Control Block entry

Page 18: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

18System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Scheduling—Basic Concept

Three level scheduling

Page 19: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

19System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Process Scheduling

Multiprogramming needed for efficiencySingle user cannot keep CPU and I/O devices busy at all times

Multiprogramming organizes jobs (code and data) so CPU always has one to execute

A subset of total jobs in system is kept in memory

One job selected and run via job scheduling

When it has to wait (for I/O for example), OS switches to another job

Context Switch

Page 20: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

20System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Process Scheduling

Timesharing (multitasking) is logical extension in which CPU switches jobs so frequently that users can interact with each job while it is running, creating interactive computing

Response time should be small enoughEach user has at least one program executing in memory processIf several jobs ready to run at the same time CPU schedulingIf processes don’t fit in memory, swapping moves them in and out to runVirtual memory allows execution of processes not completely in memory

Page 21: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

21System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Basic ConceptMaximum CPU utilization obtained with multiprogramming

CPU–I/O Burst Cycle – Process execution consists of a cycle of CPU execution and I/O wait

CPU burst distribution

CPU Scheduling

Page 22: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

22System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Scheduling

Bursts of CPU usage alternate with periods of I/O waita CPU-bound processan I/O bound process

Page 23: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

23System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Histogram of CPU-burst Times

Page 24: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

24System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

CPU Scheduler

Selects from among the processes in memory that are ready to execute, and allocates the CPU to one of them

CPU scheduling decisions may take place when a process:1. Switches from running to waiting state2. Switches from running to ready state3. Switches from waiting to ready4. Terminates

Scheduling under 1 and 4 is nonpreemptive

All other scheduling is preemptive

Page 25: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

25System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Dispatcher

Dispatcher module gives control of the CPU to the process selected by the short-term scheduler; this involves:

switching context

switching to user mode

jumping to the proper location in the user program to restart that program

Dispatch latency – time it takes for the dispatcher to stop one process and start another running

Page 26: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

26System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Scheduling Criteria CPU utilization

keep the CPU as busy as possible

Throughput# of processes that complete their execution per time unit

Turnaround time amount of time to execute a particular process

Waiting time amount of time a process has been waiting in the ready queue

Response time amount of time it takes from when a request was submitted until the first response is produced, not output (for time-sharing environment)

Page 27: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

27System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Scheduling Algorithm Goals

Page 28: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

28System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

First-Come, First-Served (FCFS) Scheduling

Process Burst TimeP1 24

P2 3

P3 3

Suppose that the processes arrive in the order: P1 , P2 , P3

The Gantt Chart for the schedule is:

Waiting time for P1 = 0; P2 = 24; P3 = 27 Average waiting time: (0 + 24 + 27)/3 = 17

P1 P2 P3

24 27 300

Page 29: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

29System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

FCFS Scheduling (Cont.)

Suppose that the processes arrive in the order P2 , P3 , P1

The Gantt chart for the schedule is:

Waiting time for P1 = 6; P2 = 0; P3 = 3 Average waiting time: (6 + 0 + 3)/3 = 3 Much better than previous case Convoy effect short process behind long process

P1P3P2

63 300

Page 30: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

30System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Shortest-Job-First (SJR) Scheduling

Associate with each process the length of its next CPU burst. Use these lengths to schedule the process with the shortest time

Two schemes: nonpreemptive – once CPU given to the process it cannot be preempted until completes its CPU burstpreemptive – if a new process arrives with CPU burst length less than remaining time of current executing process, preempt. This scheme is know as the Shortest-Remaining-Time-First (SRTF)

SJF is optimal – gives minimum average waiting time for a given set of processes

Page 31: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

31System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Process Arrival Time Burst TimeP1 0.0 7

P2 2.0 4

P3 4.0 1

P4 5.0 4 SJF (non-preemptive)

Average waiting time = (0 + 6 + 3 + 7)/4 = 4

Example of Non-Preemptive SJF

P1 P3 P2

73 160

P4

8 12

Page 32: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

32System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example of Preemptive SJF

Process Arrival Time Burst TimeP1 0.0 7

P2 2.0 4

P3 4.0 1

P4 5.0 4 SJF (preemptive)

Average waiting time = (9 + 1 + 0 +2)/4 = 3

P1 P3P2

42 110

P4

5 7

P2 P1

16

Page 33: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

33System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Determining Length of Next CPU Burst

Can only estimate the length Can be done by using the length of previous CPU

bursts, using exponential averaging

:Define 4.

10 , 3.

burst CPU next the for value predicted 2.

burst CPU of lenght actual 1.

1

n

th

n nt

.1 1 nnn t

Page 34: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

34System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Prediction of the Length of the Next CPU Burst

Page 35: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

35System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Examples of Exponential Averaging =0

n+1 = n

Recent history does not count

=1 n+1 = tn

Only the actual last CPU burst counts

If we expand the formula, we get:n+1 = tn+(1 - ) tn -1 + …

+(1 - )j tn -j + …

+(1 - )n +1 0

Since both and (1 - ) are less than or equal to 1, each successive term has less weight than its predecessor

Page 36: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

36System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Priority Scheduling A priority number (integer) is associated with each process

The CPU is allocated to the process with the highest priority (smallest integer highest priority)PreemptiveNonpreemptive

SJF is a priority scheduling where priority is the predicted next CPU burst time

Problem Starvation – low priority processes may never execute

Solution Aging – as time progresses increase the priority of the process

Page 37: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

37System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Round Robin (RR)

Each process gets a small unit of CPU time (time quantum), usually 10-100 milliseconds. After this time has elapsed, the process is preempted and added to the end of the ready queue.

If there are n processes in the ready queue and the time quantum is q, then each process gets 1/n of the CPU time in chunks of at most q time units at once. No process waits more than (n-1)q time units.

Performanceq large FIFO

q small q must be large with respect to context switch, otherwise overhead is too high

Page 38: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

38System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example of RR with Time Quantum = 20

Process Burst TimeP1 53

P2 17

P3 68

P4 24 The Gantt chart is:

Typically, higher average turnaround than SJF, but better response

P1 P2 P3 P4 P1 P3 P4 P1 P3 P3

0 20 37 57 77 97 117 121 134 154 162

Page 39: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

39System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Time Quantum and Context Switch Time

Page 40: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

40System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Turnaround Time Varies With The Time Quantum

Page 41: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

41System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Multilevel Queue Ready queue is partitioned into separate queues:

foreground (interactive)background

Each queue has its own scheduling algorithmforeground – RRbackground – FCFS

Scheduling must be done between the queuesFixed priority scheduling

• serve all from foreground then from background—Possibility of starvation.

Time slice • each queue gets a certain amount of CPU time which it can schedule amongst its processes; e.g., 80% to foreground in RR and 20% to background in FCFS

Page 42: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

42System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Multilevel Queue Scheduling

Page 43: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

43System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Multilevel Feedback Queue

A process can move between the various queues

aging can be implemented this way

Multilevel-feedback-queue scheduler defined by the following parameters:

number of queuesscheduling algorithms for each queuemethod used to determine when to upgrade a processmethod used to determine when to demote a processmethod used to determine which queue a process will enter when that process needs service

Page 44: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

44System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example of Multilevel Feedback Queue

Three queues: Q0 – RR with time quantum 8 milliseconds

Q1 – RR time quantum 16 milliseconds

Q2 – FCFS

SchedulingA new job enters queue Q0 which is served RR. When it gains

CPU, job receives 8 milliseconds. If it does not finish in 8 milliseconds, job is moved to queue Q1.

At Q1 job is again served RR and receives 16 additional milliseconds. If it still does not complete, it is preempted and moved to queue Q2.

Page 45: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

45System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Multiple-Processor Scheduling CPU scheduling more complex when multiple CPUs are available

Homogeneous processors within a multiprocessorAny available processor can be used to run any process

Heterogeneous processorsprogram complied must run on that processor

Load sharing Use separate queue for each processor

• One could be idle while another was very busy

Use Common ready queue• Each processor picks a process—may result inconsistency • Use one processor as scheduler—creates master-slave relationship

Asymmetric multiprocessing only one processor accesses the system data structuresOthers executes user processesalleviating the need for data sharing—avoid inconsistency

Page 46: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

46System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Real-Time Scheduling Hard real-time systems

required to complete a critical task within a guaranteed amount of time

Soft real-time systems requires that critical processes receive priority over less fortunate ones

Schedulable real-time system Given

m events (processes)process i occurs within period Pi and requires Ci seconds

Then the load can only be handled if

1

1m

i

i i

C

P

Page 47: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

47System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Algorithm Evaluation Criteria used in selecting an algorithm

Maximum CPU utilization under the constraint the maximum response time is 1 second

Maximum throughput such that average turnaround time linearly proportional to execution time

Deterministic ModelingEvaluates using a predetermined work load

• can not cope with the dynamic behavior of the system

Queueing ModelsUses mathematical analysis

• Complicated—hence used unrealistic assumtions• Uses some assumptions—that may not occur• An approximation of real system and accuracy is questionable

SimulationExpensive—takes hours of computer time

Page 48: System Programming in Linux

PROCESS

SYNCHRONIZATIONPROCESS

SYNCHRONIZATION

Page 49: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

49System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Process Synchronization

Background

The Critical-Section Problem

Simple Solutions

Solutions using Synchronization Hardware

Semaphores

Solutions of Classic Problems of Synchronization

Page 50: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

50System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Background Cooperating Processes

Processes that can affect or be affected by other processesImplementation uses shared logical address space or shared data using files

Concurrent access to shared data may result in data inconsistency

Maintaining data consistency requires mechanisms to ensure the orderly execution of cooperating processes

Suppose that we wanted to provide a solution to the consumer-producer problem

Page 51: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

51System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Producer-Consumer Problem

while (true) {… /* produce an item to nextp */while (count == BUFFER_SIZE) ; // buffer is full, do nothingbuffer [in] = nextp;in = (in + 1) % BUFFER_SIZE;count++;…

}

Producer process

while (true) {…while (count == 0) ; // buffer is empty, do nothingnextc = buffer[out];out = (out + 1) % BUFFER_SIZE;count--;/* consume the item from nextc…

}

Consumer process

Representative of operating system process

Page 52: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

52System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Race Condition count++ could be implemented as

register1 = count register1 = register1 + 1 count = register1

count-- could be implemented as

register2 = count register2 = register2 - 1 count = register2

Consider this execution interleaving with “count = 5” initially:S0: producer execute register1 = count {register1 = 5}S1: producer execute register1 = register1 + 1 {register1 = 6} S2: consumer execute register2 = count {register2 = 5} S3: consumer execute register2 = register2 - 1 {register2 = 4} S4: producer execute count = register1 {count = 6 } S5: consumer execute count = register2 {count = 4}

Race conditionOutcome depends upon the order of execution

Page 53: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

53System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Solution to Critical-Section Problem Critical Section

segment of code where shared variables are used

Solution must satisfy the following criteria

Mutual Exclusion If process Pi is executing in its critical section, then no other processes can be executing in their critical sections

Progress If no process is executing in its critical section and there exist some processes that wish to enter their critical section, then the selection of the processes that will enter the critical section next cannot be postponed indefinitely

Bounded Waiting A bound must exist on the number of times that other processes are allowed to enter their critical sections after a process has made a request to enter its critical section and before that request is granted

Assume that each process executes at a nonzero speed No assumption concerning relative speed of the N processes

Page 54: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

54System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Two process solutionSolution 1 Assume that the LOAD and STORE instructions are atomic; that is, cannot be interrupted. The two processes share a variable:

int turn; The variable turn indicates whose turn it is to enter the critical section.

while (true) {…while (turn != i) ;

<CS>

turn = j;

REMAINDER SECTION}

Structure of process i

Page 55: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

55System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Two process solutionSolution 2 The two processes share a variable:

Boolean flag[2] The flag array is used to indicate if a process is ready to enter the critical section. flag[i] = true

implies that process Pi is ready!

while (true) {…flag[i]=true;while (flag[j]) ;

<CS>

flag[i]=false;

REMAINDER SECTION}

Structure of process i

Page 56: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

56System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Solution 3

Two process solution

while (true) {…flag[i]=true;turn=jwhile (flag[j] && turn==j) ;

<CS>

flag[i]=false;

REMAINDER SECTION}

Structure of process i

Page 57: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

57System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Synchronization Hardware Many systems provide hardware support for critical section code

Uniprocessors – could disable interruptsCurrently running code would execute without preemptionGenerally too inefficient on multiprocessor systems• Operating systems using this not broadly scalable

Modern machines provide special atomic hardware instructions• Atomic = non-interruptible

Either test memory word and set valueOr swap contents of two memory words

Page 58: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

58System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

TestAndSet Instruction Definition:

bool TestAndSet (bool &target) { bool temp = target; target = true; return temp: }

Solution using TestAndSetShared boolean variable lock., initialized to false.

while (true) {while ( TestAndSet (lock ))

; /* do nothing<CS>lock = FALSE;// remainder section

}

Page 59: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

59System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Swap Instruction Definition:

void Swap (bool &a, bool &b) { bool temp = a; a = b; b = temp: }

Solution Shared Boolean variable lock initialized to false; Each

process has a local Boolean variable key. Solution:

while (true) { key = true; while ( key == true) Swap (lock, key ); // critical section lock = false; // remainder section

}

Page 60: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

60System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Semaphore Synchronization tool that does not require busy waiting Semaphore S – integer variable Two standard operations modify S: wait() and signal()

Originally called P() and V() Less complicated Can only be accessed via two indivisible (atomic) operations

wait (S) { while S <= 0

; // no-op S--; }

signal (S) { S++; }

Page 61: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

61System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Semaphore as General Synchronization Tool Counting semaphore – integer value can range over an unrestricted domain

Binary semaphore – integer value can range only between 0 and 1; can be simpler to implementAlso known as mutex, locks

Can implement a counting semaphore S as a binary semaphore

Provides mutual exclusionSemaphore S; // initialized to 1wait (S);

Critical Sectionsignal (S);

Page 62: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

62System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Solution of other Synchronization Problem Two processes P1 and P2 are running concurrently: P1 with a statement S1 and P2 with a statement S2

Requirement:S2 be executed only after S1

Solution:Use a a semaphore variable sync initialized to 0

……

S1 wait(sync)

Signal(sync) S2

……

Process P1

Process P2

Page 63: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

63System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Semaphore Implementation Must guarantee that no two processes can execute wait () and signal () on the same semaphore at the same time

Thus, implementation becomes the critical section problem where the wait and signal code are placed in the crtical section.Could now have busy waiting in critical section implementation

• But implementation code is short• Little busy waiting if critical section rarely occupied

Note that applications may spend lots of time in critical sections and therefore this is not a good solution.

Page 64: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

64System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Semaphore Implementation with no Busy waiting

With each semaphore there is an associated waiting queue. Each entry in a waiting queue has two data items:

value (of type integer)

a list of processes L waiting on this semaphore

Two operations:block – place the process invoking the operation on the appropriate waiting queue.

wakeup – remove one of processes in the waiting queue and place it in the ready queue.

Page 65: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

65System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Semaphore Implementation with no Busy waiting (Cont.)

Implementation of wait:

wait (S){ value--;

if (value < 0) { add this process to waiting queueblock();

}}

Implementation of signal:

Signal (S){ value++;

if (value <= 0) { remove a process P from the waiting queuewakeup(P);

}}

Page 66: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

66System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Deadlock and Starvation Deadlock – two or more processes are waiting indefinitely for an event

that can be caused by only one of the waiting processes

Let S and Q be two semaphores initialized to 1

P0 P1

wait (S); wait (Q); wait (Q); wait (S);

. .

. .

. . signal (S); signal (Q); signal (Q); signal (S);

Starvation – indefinite blocking. A process may never be removed from the semaphore queue in which it is suspended.

Page 67: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

67System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Classical Problems of Synchronization

Bounded-Buffer Problem Readers and Writers Problem Dining-Philosophers Problem

Page 68: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

68System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Bounded-Buffer Problem

N buffers, each can hold one item Semaphore mutex initialized to the value 1 Semaphore full initialized to the value 0 Semaphore empty initialized to the value N.

Page 69: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

69System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Bounded Buffer Problem (Cont.)

The structure of the producer process

while (true) {

// produce an item

wait (empty); wait (mutex);

// add the item to the buffer

signal (mutex); signal (full);

}

The structure of the consumer process

while (true) {

wait (full); wait (mutex);

// remove an item from buffer

signal (mutex); signal (empty); // consume the removed item

}

Page 70: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

70System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Readers-Writers Problem A data set is shared among a number of concurrent

processesReaders – only read the data set; they do not perform any updatesWriters – can both read and write.

Problem – allow multiple readers to read at the same time. Only one writer can access the shared data at the same time.

Shared DataData setSemaphore mutex initialized to 1.Semaphore wrt initialized to 1.Integer readcount initialized to 0.

Page 71: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

71System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Readers-Writers Problem (Cont.)

The structure of a writer process

while (true) { wait (wrt) ; // writing is performed

signal (wrt) ;}

The structure of a reader process

while (true) { wait (mutex) ; readcount ++ ; if (readercount == 1) wait

(wrt) ; signal (mutex) // reading is performed

wait (mutex) ; readcount - - ; if (redacount == 0) signal (wrt) ; signal (mutex) ;

}

Page 72: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

72System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Dining-Philosophers Problem

• Shared data Bowl of rice (data set)Semaphore chopstick [5] initialized to 1

Page 73: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

73System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Dining-Philosophers Problem (Cont.)

The structure of Philosopher i:

While (true) { wait ( chopstick[i] );

wait ( chopStick[ (i + 1) % 5] );

// eat

signal (chopstick[ (i + 1) % 5] ); signal ( chopstick[i] );

// think}

Page 74: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

74System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Dining-Philosophers Problem (Cont.) Deadlock free solution

Allow at most four philosophers to be sitting simultaneously at the tableAllow a philosopher to pick up chopsticks only both are available

While (true) { wait(mutex) wait ( chopstick[i] );

wait ( chopStick[ (i + 1) % 5] );signal(mutex) // eat

wait (mutex)signal (chopstick[ (i +

1) % 5] ); signal ( chopstick[i] );signal(mutex)

// think}Asymmetric solution

• Odd philosopher will pick first left then right while even philosopher picks up first left and then right chopstick

Page 75: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

75System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Problems with Semaphores• Correct use of semaphore operations:

• signal (mutex) …. wait (mutex)• Mutual exclusion requirement is violated

• wait (mutex) … wait (mutex)• Violation of progress condition• Possibility of deadlock

• Omitting of wait (mutex) or signal (mutex) (or both)• Violation of mutual exclusion or deadlock

Page 76: System Programming in Linux

DEADLOCKSDEADLOCKS

Page 77: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

77System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Deadlocks

The Deadlock Problem System Model Deadlock Characterization Methods for Handling Deadlocks Deadlock Prevention Deadlock Avoidance Deadlock Detection Recovery from Deadlock

Page 78: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

78System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

The Deadlock Problem In a multiprogramming Environment several processes compete for finite number of resources

A set of blocked processes each holding a resource and waiting to acquire a resource held by another process in the set.

Example 1System has 2 disk drives.P1 and P2 each hold one disk drive and each needs another one.

Example 2semaphores A and B, initialized to 1

P0 P1

wait (A); wait(B)wait (B); wait(A)

Example 3Each philosopher picks left chopstick in dining philosopher problem

Page 79: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

79System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Bridge Crossing Example

Traffic only in one direction.

Each section of a bridge can be viewed as a resource.

If two cars approach each other at the crossing, neither can start until the other has

gone(Dead locks)

If a deadlock occurs, it can be resolved if one car backs up (preempt resources and

rollback).

Several cars may have to be backed up if a deadlock occurs.

Starvation is possible.

Page 80: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

80System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

System Model Resource types R1, R2, . . ., Rm

CPU cycles, memory space, I/O devices

Each resource type Ri has Wi instances.Example—two CPU, Three printers, 2 disk drives

Process may requests as many resources as it requires to perform its designated task

Must be less the total number of resources available

Each process utilizes a resource as follows:Request—If the request can not be granted immediately, the requesting process must wait Use Release

Page 81: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

81System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Deadlock Characterization

Mutual exclusion: at least one resource must in non-sharable mode i.e. only one process at a time can use that resource.

Hold and wait: a process holding at least one resource is waiting to acquire additional resources held by other processes.

No preemption: a resource can be released only voluntarily by the process holding it, after that process has completed its task.

Circular wait: there exists a set {P0, P1, …, Pn} of waiting processes such that P0 is waiting for a resource that is held by P1, P1 is waiting for a resource that is held by P2, …, Pn–1 is waiting for a resource that is held by Pn, and Pn is waiting for a resource that is held by P0.

Deadlock can arise if four conditions hold simultaneously.

Page 82: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

82System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Resource-Allocation Graph Used to visualize resource allocation pattern Helps to describe deadlock more precisely

A set of vertices V and a set of edges (directed) E

V is partitioned into two types:

P = {P1, P2, …, Pn}, the set consisting of all the processes in the system.

R = {R1, R2, …, Rm}, the set consisting of all resource types in the system.

E is partitioned into two types:

request edge – directed edge Pi Rj

assignment edge – directed edge Rj Pi

Page 83: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

83System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Resource-Allocation Graph (Cont.)

Process

Resource Type with 4 instances

Pi requests an instance of Rj

Pi is holding an instance of Rj

Pi

Pi

Rj

Rj

Page 84: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

84System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example of a Resource Allocation Graph The sets P, R and E

P = {P1, P2, P3}

R = {R1, R2, R3, R4}

E = {P1 R1, P2 R3, R1 P2 ,

R2 P2, R2 P1, R3 P3}

Resource InstancesOne instance of resource type R1

Two instances of resource type R2

One instance of resource type R3

Three instances of resource type R3

Page 85: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

85System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Basic Facts

If graph contains no cycles no deadlock.

If graph contains a cycle if only one instance per resource type, then deadlock.

if several instances per resource type, possibility of deadlock.

Page 86: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

86System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Resource Allocation Graph With A Deadlock

Page 87: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

87System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Graph With A Cycle But No Deadlock

Page 88: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

88System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Methods for Handling Deadlocks Three methods for dealing with the deadlock problem

Ensure that the system will never enter a deadlock state.• Deadlock prevention—ensure that one of the necessary conditions can not hold• Deadlock avoidance—systems additional information about resource usage pattern during the execution

Allow the system to enter a deadlock state and then recover.

Ignore the problem and pretend that deadlocks never occur in the system; used by most operating systems, including UNIX.

Page 89: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

89System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Deadlock Prevention

Mutual Exclusion – not required for sharable resources; must hold for nonsharable resources.

Hold and Wait – must guarantee that whenever a process requests a resource, it does not hold any other resources.

Two possibilitiesRequire process to request and be allocated all its resources before it begins execution

allow process to request resources only when the process has none.

• Low resource utilization; possibility of starvation • Data inconsistency

• ensure that one of the necessary conditions can not hold

• Restrain the ways request can be made.

Page 90: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

90System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Deadlock Prevention (Cont.)No Preemption – Two possibilities

If a process that is holding some resources requests another resource that cannot be immediately allocated to it, then all resources currently being held are released.

If a process requests some resources that are not available as they are allocated to other waiting processes, then preempt desired resources from waiting processes

• Preempted resources are added to the list of resources for which the process is waiting.• Process will be restarted only when it can regain its old resources, as well as the new ones that it is requesting.

Page 91: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

91System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Deadlock Prevention (Cont.) Circular Wait – impose a total ordering of all resource types, and require that each process requests resources in an increasing order of enumeration.

F(tape drive)=1F(disk drive)=5F(printer)=12

Facts:IIf a process currently holds resource type R i , the process can request another resource type R j if F(Rj) > F(Ri)

ORIf a process requests a resource type R j , if it has released any resource type R i such the F(Ri) F(Rj)

Proof(by contradiction):Set of deadlocked process {P0, P1, P2, …, Pn}

F(R0) < F(R1) < F(R2) <…< F(Rn) < F(R0) which is impossible

So there can be no circular wait—no deadlock

Page 92: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

92System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Deadlock Avoidance

Simplest and most useful model requires that each process declare the maximum number of resources of each type that it may need.

The deadlock-avoidance algorithm dynamically examines the resource-allocation state to ensure that there can never be a circular-wait condition.

• Requires that the system has some additional information available.

• With this additional information, system can decide whether a newly arrived request can be satisfied (served) or not

Page 93: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

93System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Safe State state is defined by the number of available and allocated

resources, and the maximum demands of the processes.

System is in safe state if there exists a sequence <P1, P2, …, Pn> of ALL the processes in the systems such that for each Pi, the resources that Pi can still request can be satisfied by currently available resources + resources held by all the Pj, with j < i.

That is:If Pi needs some resources that are not immediately available, then

Pi can wait until all Pj have finished.

When Pj is finished, Pi can obtain needed resources, execute, return

allocated resources, and terminate.

When Pi terminates, Pi +1 can obtain its needed resources, and so on.

When a process requests an available resource, system must decide if immediate allocation leaves the system in a safe state.

Page 94: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

94System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Basic Facts If a system is in safe state no deadlocks.

If a system is in unsafe state possibility of deadlock.

Avoidance ensure that a system will never enter an

unsafe state.

Example: 12 disk drives Snapshot at time T0:

Maximum needs Current NeedsP0 10 5

P1 4 2

P2 9 2 The sequence <P1, P0, P2> is safe At time T1 P2 requests one more disk drive Satisfying the request leaves the system in an unsafe state Request can not be granted

Page 95: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

95System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Safe, Unsafe , Deadlock State

Page 96: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

96System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Avoidance algorithms

Single instance of a resource type. Use a resource-allocation graph

Multiple instances of a resource type. Use the banker’s algorithm

Page 97: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

97System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Resource-Allocation Graph Scheme

Claim edge Pi Rj indicated that process Pj may request resource Rj; represented by a dashed line.

Claim edge converts to request edge when a process requests a resource.

Request edge converted to an assignment edge when the resource is allocated to the process.

When a resource is released by a process, assignment edge reconverts to a claim edge.

Resources must be claimed a priori in the system.

Page 98: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

98System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Resource-Allocation Graph

Page 99: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

99System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Resource-Allocation Graph Algorithm

Suppose that process Pi requests a resource Rj

The request can be granted only if converting the request edge to an assignment edge does not result in the formation of a cycle (including the claim edge )in the resource allocation graph

Page 100: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

100System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Unsafe State In Resource-Allocation Graph

Page 101: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

101System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Banker’s Algorithm

Multiple instances.

Each process must a priori claim maximum use.

When a process requests a resource it may have to wait.

When a process gets all its resources it must return them in a finite amount of time.

Page 102: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

102System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Data Structures for the Banker’s Algorithm

Available: Vector of length m. If Available [j] = k, there are k instances of resource type Rj available.

Max: n x m matrix. If Max [i,j] = k, then process Pi may request at most k instances of resource type Rj.

Allocation: n x m matrix. If Allocation[i,j] = k then Pi is currently allocated k instances of Rj.

Need: n x m matrix. If Need[i,j] = k, then Pi may need k more instances of Rj to complete its task.

Need [i,j] = Max[i,j] – Allocation [i,j].

Define XY if X[i] Y[i] i

Let n = number of processes, and m = number of resources types.

Page 103: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

103System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Safety Algorithm1. Let Work and Finish be vectors of length m and n, respectively. Initialize:

Work = AvailableFinish [i] = false for i = 0, 1, …, n- 1.

2. Find and i such that both: (a) Finish [i] = false(b) Needi Work

If no such i exists, go to step 4.

3. Work = Work + Allocationi

Finish[i] = truego to step 2.

4. If Finish [i] == true for all i, then the system is in a safe state.

Algorithm requires an order of O(m x n2) operations to detect whether the system is in safe state.

Page 104: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

104System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Resource-Request Algorithm for Process Pi

Requesti = request vector for process Pi. If Requesti [j] = k then process Pi wants k instances of resource type Rj.

1. If Requesti Needi go to step 2. Otherwise, raise error condition, since process has exceeded its maximum claim.

2. If Requesti Available, go to step 3. Otherwise Pi must wait, since resources are not available.

3. Pretend to allocate requested resources to Pi by modifying the state as follows:

Available = Available – Request;Allocationi = Allocationi + Requesti;

Needi = Needi – Requesti;

If safe the resources are allocated to Pi. If unsafe Pi must wait, and the old resource-allocation state is restored

Page 105: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

105System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example of Banker’s Algorithm

5 processes P0 through P4; 3 resource types: AvailableA (10 instances), B (5instances), and C (7 instances).

Snapshot at time T0:Allocation Max Available

A B C A B C A B CP0 0 1 0 7 5 3 3 3 2

P1 2 0 0 3 2 2

P2 3 0 2 9 0 2

P3 2 1 1 2 2 2

P4 0 0 2 4 3 3

Page 106: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

106System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example (Cont.)

The content of the matrix Need is defined to be Max – Allocation.

NeedA B C

P0 7 4 3

P1 1 2 2

P2 6 0 0

P3 0 1 1

P4 4 3 1

The system is in a safe state since the sequence < P1, P3, P4, P2, P0> satisfies safety criteria.

Page 107: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

107System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example: P1 Request (1,0,2)

Check that Request Available (that is, (1,0,2) (3,3,2) true.

Allocation Need AvailableA B C A B C A B C

P0 0 1 0 7 4 3 2 3 0

P1 3 0 2 0 2 0

P2 3 0 1 6 0 0

P3 2 1 1 0 1 1

P4 0 0 2 4 3 1 Executing safety algorithm shows that sequence <

P1, P3, P4, P0, P2> satisfies safety requirement. Can request for (3,3,0) by P4 be granted? Can request for (0,2,0) by P0 be granted?

Page 108: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

108System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Deadlock Detection

Allow system to enter deadlock state

System must provide

Detection algorithm• Checks the state of the system to determine whether a

deadlock has occured

Recovery scheme

Page 109: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

109System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Single Instance of Each Resource Type

Maintain wait-for graphNodes are processes.

Pi Pj if Pi is waiting for Pj.

Periodically invoke an algorithm that searches for a cycle in the graph. If there is a cycle, there exists a deadlock.

An algorithm to detect a cycle in a graph requires an order of n2 operations, where n is the number of vertices in the graph.

Page 110: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

110System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Resource-Allocation Graph and Wait-for Graph

Resource-Allocation Graph Corresponding wait-for graph

Page 111: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

111System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Several Instances of a Resource Type

Available: A vector of length m indicates the number of available resources of each type.

Allocation: An n x m matrix defines the number of resources of each type currently allocated to each process.

Request: An n x m matrix indicates the current request of each process. If Request [ij] = k, then process Pi is requesting k more instances of resource type. Rj.

Page 112: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

112System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Detection Algorithm1. Let Work and Finish be vectors of length m and n, respectively Initialize:

(a) Work = Available(b) For i = 1,2, …, n, if Allocationi 0, then

Finish[i] = false;otherwise, Finish[i] = true.

2. Find an index i such that both:

(a) Finish[i] == false(b) Requesti Work

If no such i exists, go to step 4.

3. Work = Work + Allocationi

Finish[i] = truego to step 2.

4. If Finish[i] == false, for some i, 1 i n, then the system is in deadlock state. Moreover, if Finish[i] == false, then Pi is deadlocked.

Algorithm requires an order of O(m x n2) operations to detect whether the system is in deadlocked state.

Page 113: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

113System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example of Detection Algorithm Five processes P0 through P4;

three resource types A (7 instances), B (2 instances), and C (6 instances).

Snapshot at time T0: Allocation Request Available

A B C A B C A B CP0 0 1 0 0 0 0 0 0 0

P1 2 0 0 2 0 2

P2 3 0 3 0 0 0

P3 2 1 1 1 0 0

P4 0 0 2 0 0 2

Sequence <P0, P2, P3, P1, P4> will result in Finish[i] = true for all i.

Page 114: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

114System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example (Cont.)

P2 requests an additional instance of type C.RequestA B C P0 0 0 0

P1 2 0 1

P2 0 0 1

P3 1 0 0

P4 0 0 2 State of system?

Can reclaim resources held by process P0, but insufficient resources to fulfill other processes; requests.

Deadlock exists, consisting of processes P1, P2, P3, and P4.

Page 115: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

115System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Detection-Algorithm Usage When, and how often, to invoke depends on:

How often a deadlock is likely to occur?

How many processes will need to be rolled back?• one for each disjoint cycle

Run deadlock detection detection algorithm every time a request can not be granted

Considerably overhead Run once per hour Run when CPU/resource utilization drops below 40% If detection algorithm is invoked arbitrarily, there may be

many cycles in the resource graph and so we would not be able to tell which of the many deadlocked processes “caused” the deadlock.

Page 116: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

116System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Recovery from Deadlock: Process Termination

Abort all deadlocked processes.

Abort one process at a time until the deadlock cycle is eliminated.

In which order should we choose to abort?Priority of the process.How long process has computed, and how much longer to completion.Resources the process has used.Resources process needs to complete.How many processes will need to be terminated. Is process interactive or batch?

Page 117: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

117System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Recovery from Deadlock: Resource Preemption

Selecting a victim – minimize cost.

Rollback – return to some safe state, restart process for that state.

Starvation – same process may always be picked as victim, include number of rollback in cost factor.

Page 118: System Programming in Linux

Memory Management

Memory Management

Page 119: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

119System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Memory Management

Background Swapping Contiguous Memory Allocation Paging Structure of the Page Table Segmentation Example: The Intel Pentium

Page 120: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

120System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Objectives

To provide a detailed description of various ways of organizing memory hardware

To discuss various memory-management techniques, including paging and segmentation

To provide a detailed description of the Intel Pentium, which supports both pure segmentation and segmentation with paging

Page 121: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

121System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Background

Program must be brought (from disk) into memory and placed within a process for it to be run

Main memory and registers are only storage CPU can access directly

Register access in one CPU clock (or less) Main memory can take many cycles Cache sits between main memory and CPU

registers Protection of memory required to ensure correct

operation

Page 122: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

122System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Base and Limit Registers

A pair of base and limit registers define the logical address space

Page 123: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

123System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Binding of Instructions and Data to Memory

Address binding of instructions and data to memory addresses can happen at three different stages

Compile time: If memory location known a priori, absolute code can be generated; must recompile code if starting location changesLoad time: Must generate relocatable code if memory location is not known at compile timeExecution time: Binding delayed until run time if the process can be moved during its execution from one memory segment to another. Need hardware support for address maps (e.g., base and limit registers)

Page 124: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

124System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Multistep Processing of a User Program

Page 125: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

125System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Logical vs. Physical Address Space

The concept of a logical address space that is bound to a separate physical address space is central to proper memory management

Logical address – generated by the CPU; also referred to as virtual addressPhysical address – address seen by the memory unit

Logical and physical addresses are the same in compile-time and load-time address-binding schemes; logical (virtual) and physical addresses differ in execution-time address-binding scheme

Page 126: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

126System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Memory-Management Unit (MMU)

Hardware device that maps virtual to physical address

In MMU scheme, the value in the relocation register is added to every address generated by a user process at the time it is sent to memory

The user program deals with logical addresses; it never sees the real physical addresses

Page 127: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

127System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Dynamic relocation using a relocation register

Page 128: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

128System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Dynamic Loading

Routine is not loaded until it is called Better memory-space utilization; unused routine

is never loaded Useful when large amounts of code are needed

to handle infrequently occurring cases No special support from the operating system is

required implemented through program design

Page 129: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

129System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Dynamic Linking

Linking postponed until execution time Small piece of code, stub, used to locate

the appropriate memory-resident library routine

Stub replaces itself with the address of the routine, and executes the routine

Operating system needed to check if routine is in processes’ memory address

Dynamic linking is particularly useful for libraries

System also known as shared libraries

Page 130: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

130System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Swapping

A process can be swapped temporarily out of memory to a backing store, and then brought back into memory for continued execution

Backing store – fast disk large enough to accommodate copies of all memory images for all users; must provide direct access to these memory images

Roll out, roll in – swapping variant used for priority-based scheduling algorithms; lower-priority process is swapped out so higher-priority process can be loaded and executed

Major part of swap time is transfer time; total transfer time is directly proportional to the amount of memory swapped

Modified versions of swapping are found on many systems (i.e., UNIX, Linux, and Windows)

System maintains a ready queue of ready-to-run processes which have memory images on disk

Page 131: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

131System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Schematic View of Swapping

Page 132: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

132System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Contiguous Allocation

Main memory usually into two partitions:Resident operating system, usually held in low memory with interrupt vectorUser processes then held in high memory

Relocation registers used to protect user processes from each other, and from changing operating-system code and data

Base register contains value of smallest physical addressLimit register contains range of logical addresses – each logical address must be less than the limit register MMU maps logical address dynamically

Page 133: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

133System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

HW address protection with base and limit registers

Page 134: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

134System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Contiguous Allocation (Cont.)

Multiple-partition allocationHole – block of available memory; holes of various size are scattered throughout memoryWhen a process arrives, it is allocated memory from a hole large enough to accommodate itOperating system maintains information about:a) allocated partitions b) free partitions (hole)

OS

process 5

process 8

process 2

OS

process 5

process 2

OS

process 5

process 2

OS

process 5

process 9

process 2

process 9

process 10

Page 135: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

135System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Dynamic Storage-Allocation Problem

First-fit: Allocate the first hole that is big enough Best-fit: Allocate the smallest hole that is big enough; must search entire list, unless ordered by size

Produces the smallest leftover hole Worst-fit: Allocate the largest hole; must also search entire list

Produces the largest leftover hole

How to satisfy a request of size n from a list of free holes

First-fit and best-fit better than worst-fit in terms of speed and storage utilization

Page 136: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

136System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Fragmentation

External Fragmentation – total memory space exists to satisfy a request, but it is not contiguous

Internal Fragmentation – allocated memory may be slightly larger than requested memory; this size difference is memory internal to a partition, but not being used

Reduce external fragmentation by compactionShuffle memory contents to place all free memory together in one large blockCompaction is possible only if relocation is dynamic, and is done at execution timeI/O problem

• Latch job in memory while it is involved in I/O• Do I/O only into OS buffers

Page 137: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

137System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Paging

Logical address space of a process can be noncontiguous; process is allocated physical memory whenever the latter is available

Divide physical memory into fixed-sized blocks called frames (size is power of 2, between 512 bytes and 8,192 bytes)

Divide logical memory into blocks of same size called pages

Keep track of all free frames To run a program of size n pages, need to find n

free frames and load program Set up a page table to translate logical to

physical addresses Internal fragmentation

Page 138: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

138System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Address Translation Scheme

Address generated by CPU is divided into:

Page number (p) – used as an index into a page table which contains base address of each page in physical memory

Page offset (d) – combined with base address to define the physical memory address that is sent to the memory unit

For given logical address space 2m and page size 2n

page number page offset

p d

m - n n

Page 139: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

139System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Paging Hardware

Page 140: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

140System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Paging Model of Logical and Physical Memory

Page 141: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

141System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Paging Example

32-byte memory and 4-byte pages

Page 142: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

142System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Free Frames

Before allocation After allocation

Page 143: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

143System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Implementation of Page Table

Page table is kept in main memory Page-table base register (PTBR) points to the

page table Page-table length register (PRLR) indicates

size of the page table In this scheme every data/instruction access

requires two memory accesses. One for the page table and one for the data/instruction.

The two memory access problem can be solved by the use of a special fast-lookup hardware cache called associative memory or translation look-aside buffers (TLBs)

Some TLBs store address-space identifiers (ASIDs) in each TLB entry – uniquely identifies each process to provide address-space protection for that process

Page 144: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

144System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Associative Memory

Associative memory – parallel search

Address translation (p, d)If p is in associative register, get frame # out

Otherwise get frame # from page table in memory

Page # Frame #

Page 145: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

145System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Paging Hardware With TLB

Page 146: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

146System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Effective Access Time

Associative Lookup = time unit Assume memory cycle time is 1 microsecond Hit ratio – percentage of times that a page number is found in the associative registers; ratio related to number of associative registers Hit ratio = Effective Access Time (EAT)

EAT = (1 + ) + (2 + )(1 – )= 2 + –

Page 147: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

147System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Memory Protection

Memory protection implemented by associating protection bit with each frame

Valid-invalid bit attached to each entry in the page table:

“valid” indicates that the associated page is in the process’ logical address space, and is thus a legal page

“invalid” indicates that the page is not in the process’ logical address space

Page 148: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

148System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Valid (v) or Invalid (i) Bit In A Page Table

Page 149: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

149System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Shared Pages

Shared codeOne copy of read-only (reentrant) code shared among processes (i.e., text editors, compilers, window systems).

Shared code must appear in same location in the logical address space of all processes

Private code and data Each process keeps a separate copy of the code and data

The pages for the private code and data can appear anywhere in the logical address space

Page 150: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

150System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Shared Pages Example

Page 151: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

151System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Structure of the Page Table

Hierarchical Paging

Hashed Page Tables

Inverted Page Tables

Page 152: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

152System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Hierarchical Page Tables

Break up the logical address space into multiple page tables

A simple technique is a two-level page table

Page 153: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

153System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Two-Level Page-Table Scheme

Page 154: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

154System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Two-Level Paging Example

A logical address (on 32-bit machine with 1K page size) is divided into:a page number consisting of 22 bits

a page offset consisting of 10 bits Since the page table is paged, the page number is further divided into:

a 12-bit page number

a 10-bit page offset Thus, a logical address is as follows:

where pi is an index into the outer page table, and p2 is the displacement within the page of the outer page table

page number page offset

pi p2 d

12 10 10

Page 155: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

155System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Address-Translation Scheme

Page 156: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

156System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Three-level Paging Scheme

Page 157: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

157System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Hashed Page Tables

Common in address spaces > 32 bits

The virtual page number is hashed into a page table. This page table contains a chain of elements hashing to the same location.

Virtual page numbers are compared in this chain searching for a match. If a match is found, the corresponding physical frame is extracted.

Page 158: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

158System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Hashed Page Table

Page 159: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

159System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Inverted Page Table

One entry for each real page of memory Entry consists of the virtual address of the

page stored in that real memory location, with information about the process that owns that page

Decreases memory needed to store each page table, but increases time needed to search the table when a page reference occurs

Use hash table to limit the search to one — or at most a few — page-table entries

Page 160: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

160System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Inverted Page Table Architecture

Page 161: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

161System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Segmentation

Memory-management scheme that supports user view of memory

A program is a collection of segments. A segment is a logical unit such as:

main program,procedure, function,method,object,local variables, global variables,common block,stack,symbol table, arrays

Page 162: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

162System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

User’s View of a Program

Page 163: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

163System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Logical View of Segmentation

1

3

2

4

1

4

2

3

user space physical memory space

Page 164: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

164System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Segmentation Architecture

Logical address consists of a two tuple:<segment-number, offset>,

Segment table – maps two-dimensional physical addresses; each table entry has:

base – contains the starting physical address where the segments reside in memorylimit – specifies the length of the segment

Segment-table base register (STBR) points to the segment table’s location in memory

Segment-table length register (STLR) indicates number of segments used by a program; segment number s is legal if s < STLR

Page 165: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

165System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Segmentation Architecture (Cont.)

ProtectionWith each entry in segment table associate:

• validation bit = 0 illegal segment• read/write/execute privileges

Protection bits associated with segments; code sharing occurs at segment level

Since segments vary in length, memory allocation is a dynamic storage-allocation problem

A segmentation example is shown in the following diagram

Page 166: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

166System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Segmentation Hardware

Page 167: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

167System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example of Segmentation

Page 168: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

168System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example: The Intel Pentium

Supports both segmentation and segmentation with paging CPU generates logical address

Given to segmentation unit• Which produces linear addresses

Linear address given to paging unit• Which generates physical address in main memory• Paging units form equivalent of MMU

Page 169: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

169System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Logical to Physical Address Translation in Pentium

Page 170: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

170System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Intel Pentium Segmentation

Page 171: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

171System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Pentium Paging Architecture

Page 172: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

172System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Linear Address in Linux

Broken into four parts:

Page 173: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

173System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Three-level Paging in Linux

Page 174: System Programming in Linux

LINUX COMMANDSLINUX COMMANDS

Page 175: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

175System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

In this session, you will learn to: Use the basic Unix commands like

pwd

date

who

ls

man Use “man” pages

Objectives

Page 176: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

176System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

pwdDisplays the current working directory.

dateDisplays the current date and time

Simple commands

Page 177: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

177System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

whoDisplays the names of all the users who have currently logged in

who am iDisplays the name of the current user.

Simple commands

Page 178: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

178System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• ls• Syntax :ls [options] [file….]• options: -l list in long format• -a list all file including those

beginning with a dot• -i list inode no of file in first

column• -s reports disk blocks occupied

by file• -R recursively list all sub

directories• -F mark type of each file• -C display files in columns

Listing contents of directory

Page 179: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

179System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Meta Characters

Purpose Example

* Match with one or more characters or none $ ls –l *.c file* ? Match with any single character $ ls –l file? […] Match with any single character within the

brackets $ ls –l file[abc]

; Command separator $ cat file1; cat file2 | Pipe two commands $ cat abc | wc ( ) Group commands

Useful when the output of the command group has to be redirected

$ (echo “==== x.c ====”; cat x.c) > out

`command ̀ Execute the command enclosed within back quotes. Useful when the output of a command into a variable in a shell script

count=`expr $count + 1 ̀ assuming count has value 3, this increments the value of count

‘string’ Quote all characters with no substitution (ex. no special meaning for $)

echo ‘expr $count + 1’ displays expr $count + 1

“string” Quote all characters with substitution. The characters $, \ (back slash) and back quote have special meaning.

echo “expr $count + 1” displays expr 3 + 1 assuming the variable count has value 3

Meta characters

Page 180: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

180System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

$ ls –l

total 6 -rwxr-xr-x 1 user1 projA 12373 Dec 15 14:45 a.out drwxr-xr-x 2 user2 projD 4096 Dec 22 14:00 awkpro -rw-r--r-- 1 user1 projA 12831 Dec 12 13:59 c -rw------- 1 user1 projA 61440 Dec 15 11:16 core -rw-r--r-- 1 user3 projC 255 Dec 20 14:29 cs

File type

File access permissions

Link countGroup id

Date & time of modification

User id File sizein bytes

File name

Listing contents of directory

Page 181: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

181System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• The Unix manual, usually called man pages, is available on-line to explain the usage of the Unix system and commands.

• Syntax:• man [options] command_name

Common Options-k keyword list command synopsis line for all keyword

matches-M path path to man pages-a show all matching man pages (SVR4)

• info command_name - help for the internal commands• help -–command_name– gives command synatx

Getting help on commands

Page 182: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

182System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

In this session, you have learned to …• use the basic Unix commands like

• pwd• date• who• ls• man

• use “man” pages

Summary

Page 183: System Programming in Linux

FILES AND

DIRECTORIESFILES AND

DIRECTORIES

Page 184: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

184System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

In this session, you will learn to:set file permissions using the chmod command

use directory-related commands namely mkdir, rmdir, cd commands

use file-related commands namely cp, mv, rm commands

access advanced file permissions using commands umask, suid, sgid, linking files, stickybit

create and edit files using the vi editor

Objectives

Page 185: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

185System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Refers to the permissions associated with a file with respect to the following

Permission LevelsUser (owner) (u)

Group (wheel, staff, daemon, etc.) (g)

World (guest, anonymous and all other users) (o)

Permission SettingsRead (r)

Write (w)

Execute (x)

File access permissions

Page 186: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

186System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

No read permission does not allow the user to:List the contents of directory

Remove the directory No Write permission does not allow the user to :

copy files to the directory

remove files from the directory

rename files in the directory

make a subdirectory

remove a subdirectory from the directory

move files to, and from the directory

File access permissions

Page 187: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

187System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

No execute permission does not allow the user to:

display the contents of a directory file from within the directory

change to the directory

display a file in the directory

copy a file to, or from the directory

File access permissions

Page 188: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

188System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

chmod u+x file_name Syntax: chmod <category> <operation> <permission>

<filename(s)> or chmod <octal number> filename

Octal Number4 - for read2 - for write1 - for execution

$ chmod 744 xyzthis will set rwx for user, r– for group, r—for others

Changing permissions - chmod

Page 189: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

189System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Command Syntax mkdir [OPTION] DIRECTORY $mkdir <path>/<directory> $mkdir –m <directory> $mkdir –p

<directory1>/<directory2>/<directory3>

Example: mkdir project1

This creates a directory project1 under current directory

Note: Write and execute permissions are needed for the user to create a directory

Directory creation

Page 190: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

190System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

rmdir command removes directory Syntax

rmdir <directory name> Example Removes project1 directory in the current directory

rmdir project1 Remove multiple directories rmdir pos1 pos2 Remove the directory recursively rmdir –p dir1/dir2/dir2 Rule: rmdir can be executed to remove a directory

if it is empty and not the current directory

Directory removal

Page 191: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

191System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

cd command is used to change the directory

cd - take to the home directorycd .. - takes to the parent directorycd / - takes to the root directory

Command - cd

Page 192: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

192System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

File Operation

Copying a file

Moving a file

Removing a file

Displaying a fileand concatenating files

Command

cp

mv

rm

cat

File related commands

Page 193: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

193System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Used to copy files across directories Syntax cp <source file> <new file name>

Example cp file1 file2

Command - cp

Page 194: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

194System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

-p Copies the file and preserves the following attributes

• owner id• group id• permissions• last modification time

-r recursive copy; copy subdirectories under the directory if any

-iinteractive; prompts for confirmation before overwriting the target file, if it already exists

Command - cp

Page 195: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

195System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Used to move a file, or rename a file

Preserves the following details• owner id• group id• permissions• Last modification time

-f suppresses all prompting (forces overwriting of target)

-i prompts before overwriting destination file

Command - mv

Page 196: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

196System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Used to remove a file

• Syntax : rm file(s) -f suppresses all prompting

-i prompts before overwriting destination file

-r will recursively remove the file from a directory (can be used to delete a directory along with the content )

Caution: Use “i” option along with “r” to get notified on deletion

Command – rm

Page 197: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

197System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

$ ls –l -rwxr-xr-x 1user1 training 12373 Dec 15 14:45 a.out -rwxr-xr-x 3 user1 faculty 4096 Dec 24 11:56 awkpro

$chown user2 a.out

$ls –l -rwxr-xr-x 1user2 training 12373 Dec 15 14:45 a.out -rwxr-xr-x 3 user1 faculty 4096 Dec 24 11:56 awkpro

$ chgrp training awkpro

$ls –l -rwxr-xr-x 1user2 training 12373 Dec 15 14:45 a.out -rwxr-xr-x 3 user1 training 4096 Dec 24 11:56 awkpro

Command – chown & chgrp

Page 198: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

198System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

umask value is used to set the default permission of a file and directory while creating

umask command is used to see the default mask for the file permission

Default umask value will be set in the system environment file like /etc/profile

umask 022 will set a mask of 022 for the current session

The file permission after setting this umask value will be 644And the directory permission will be 755

Command - umask

Page 199: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

199System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Linking files

Hard Link (in the same filesystem)$ ln /usr/bin/clear /usr/bin/cls

Hard link uses the same inode number

Soft Link (in different filesystems also used to link directories)

$ ln –s /usr/bin/clear /home/user1/cls

Command - ln

Page 200: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

200System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Set user ID (SUID)This means that if the SUID bit is set for any application then your user ID would be set as that of the owner of application/file rather than the current user, while running that application

“set user ID” bit can be set in one of the two ways:• chmod u+s <filename>

• chmod 4755 <filename>

• The leftmost octal number 4 indicates “set user ID” bit to be set, other octal digits indicate regular file permissions. This is meaningful for executable files only.

Special permission bits

Page 201: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

201System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Set group id (SGID)Just like SUID, setting the SGID bit for a file sets your group ID to the file's group while the file is executing

“set group ID” bit can be set in one of the two ways:• chmod g+s <filename>

• chmod 2755 <filename>

• The leftmost octal number 2 indicates “set group ID” bit to be set, other octal digits indicate regular file permissions. This is meaningful for executable files only.

Special permission bits

Click to edit Master text styles Second level

• Third level• Fourth level

• Fifth level

Page 202: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

202System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Sticky bit (SVTX)Typically set to a directory that is shareable

Any user can create a file in such sharable directory

Only owner of the file or super user (root) can remove a file from the directory

“sticky” bit can be set in one of the two ways:• chmod +t <directoryname>

• chmod 1555 <directoryname>• The leftmost octal number 1 indicates “sticky” bit to be set, other

octal digits indicate regular file permissions.

Special permission bits

Page 203: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

203System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

vi is a visual editor used to create and edit text files.A screen-oriented text editor

Included with most UNIX system distributions

Command driven

Categories of commands includeCursor movement

Editing commands

Search and replace commands

The vi editor is invoked by the following command: vi <filename>

vi editor

Page 204: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

204System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Backspace

h

Space

lj k

the quick brown fox

ww w

the quick brown fox

2 w

the quick brown fox

bb b

the quick brown fox

$

the quick brown fox

^

Navigation

Page 205: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

205System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Text insertion / replacement

i - inserts text to the left of the cursor

a - inserts text to the right of the cursor

I - inserts text at the beginning of the line

A - appends text at end of the line

o - opens line below

O - opens line above

R - replaces text from cursor to right

s - replaces a single character with any number of

characters

S - replaces entire line

Editing commands

Page 206: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

206System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Deletion

x - to delete character at cursor position

3x - to delete 3 characters at cursor position

dw - to delete word

2dw - to delete 2 word

dd - to delete a line

2dd - to delete 2 lines

Editing commands

Page 207: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

207System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

YankingY - copy line into buffer3Y - copy 3 lines into bufferp - copy buffer below cursorP - copy buffer above cursor

Save and quit:w - to save:w! - to name a file (:w! filename -> save as):x - save and quit:q - cancel changes:q! - cancel and quit

Editing commands

Page 208: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

208System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

The following commands are applicable for vi editor in Linux

/pat searches for the pattern pat and places cursor

where pattern occurs.

/ repeat last search

:%s/old/new/g to change every occurrence in the whole file.

:#,#s/old/new/g where #,# are replaced with the numbers of the two lines.

Search & replace commands

Page 209: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

209System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

In this session, you have learned how to …use file permissions using the chmod commanduse directory-related commands namely mkdir, rmdir, cd commands use file-related commands namely cp, mv, rm commandsaccess advanced file permissions using commands umask, suid, sgid, linking the files, stickybitcreate and edit files using the vi editor

Summary

Page 210: System Programming in Linux

Chapter 4

UNIX Utilities

Page 211: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

211System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

In this session, you will learn to:use the Unix Utilities like

• cat, echo, touch, more, file, wc, cmp, comm, find

employ redirection operators

use filters like• sort, grep, cut, head, tail, tr, and paste

use communication commands• telnet, ftp

use backup commands• zip/gzip and tar

Objectives

Page 212: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

212System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• cat command takes the input from the keyboard, and sends the output to the monitor

• We can redirect the input and output using the redirection operators

$ cat>file1 Type the content herepress <ctrl d> $ cat file1Displays the content of the file$cat>>file1 (will append to the content of the

file)

cat

Page 213: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

213System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

touch is used to change the time stamp of the file

Syntax: touch [options] file Options:

a to change the access timem to change the modification timec no create if not exists

touch <file> will change the time of change of the file if the file exists

If the file does not exist, it will create a file of zero byte size.

touch

Page 214: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

214System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

echo command is used to print output to the screen

echo “This is an example”This is an example

x=10echo $x10

read command allows to read input from user and assign it to the variable specified.

read x

echo & read

Page 215: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

215System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

moreAllows user to view one page-full of information at a time.

fileUsed to display the type of the file

ttyPrints the terminal’s name

General purpose utilities

Page 216: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

216System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

wcA filter used to count the number of lines, words, and characters in a disk file or from the standard input.

-l - displays the number of lines

-w - displays the number of words

-c - displays the number of characters

General purpose utilities

Page 217: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

217System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

cmpReturns the offset and the line number of the first position where the two files differ.

commcol1 - unique lines of first file

col2 - unique lines of second file

col3 - common lines

General purpose utilities

Page 218: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

218System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

diffIndicate the differences between the files

a Lines added

d Lines deleted

c Lines changed

General purpose utilities

Page 219: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

219System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Lets user to search set of files and directories based on various criteria

Syntax: find [path...] [expression] [path]

where to search

[expression]what type of file to search (specified with –type option)What action to be applied (–exec, –print, etc.)Name of the files (specified as part of –name option, enclosed in “ “)

Example find . –name “*.c” -print

lists all files with .c extension from the current dir & its subdirectories

find

Page 220: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

220System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Finding files on the basis of file size

– size [+ –]n[bc]

• n represents size in bytes (c) or blocks (b) of 512 bytes

• find . –size 1000c lists all files that are exactly 1000 bytes in size

• find . –size +1000c lists all files that are more than 1000 bytes in size

• find . –size –1000c lists all files that are less than 1000 bytes in size

find

Page 221: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

221System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Finding files on the basis of access time (atime) or modified time (mtime)

– atime [+-]n

– mtime [+-]n

• n represents number of days ( actually 24 * n hours)• find . –atime 2 lists files accessed exactly

2 days ago• find . –atime +2 lists files accessed more

than 2 days ago• find / –mtime –2 lists files modified less

than 2 days ago

find

Page 222: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

222System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Applying a command on files matching the criteria with –exec and –ok options

– exec command {} \;

• command is command to be applied on the matching files (does not prompt user)

• find . -name “*.dat” –exec ls –l {} \;• Long listing of all files with .dat extension in the current and its

subdirectories

-ok command {} \;

• Functionality is similar to –exec, but prompts user before applying the command on the file matching the criteria.

find

Page 223: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

223System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• pr• Used to display a file in a format to be printed.• Breaks up a file into pages with a header, text and

footer area

• Options• -l to alter the length of the file• -h to set the header• -t to suppress the header and the footer• -n to set the line number

pr

Page 224: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

224System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Standard Input fileKeyboard, file descriptor is 0

Standard Output fileMonitor, file descriptor is 1

Standard Error fileMonitor, file descriptor is 2

Standard files

Page 225: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

225System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

< file redirect standard input from file > file redirect standard output to file 2> file redirect standard error to file 2>&1 merge standard error with standard

output $ cat > abc

$ ls –l > outfile

$ cat xyz abc > outfile 2> errfile

$ cat xyz abc > outfile 2>&1

I/O redirection

Page 226: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

226System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Filters are programs that takes its input from the standard input file, process it, and sends it to the standard output file.

Commonly used filter commandssort

grep

cut

head

tail

paste

Filters

Page 227: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

227System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Sorts the contents of the given file based on the first char of each line.

-n --numeric sort -r --Reverse sort -t -Specify delimiter for fields

+num -specify sorting field numbers +num -num - to specify the range

can specify the characters in the field

sort

Page 228: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

228System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

grep -Global Regular Expression Printer is used for searching regular expressions

Syntaxgrep <options> <pattern> <filename(s)>

grep

Page 229: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

229System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

-c displays count of the number of occurrences

-n displays line numbers along with the lines

-v displays all lines except lines matching pattern

-i Ignores case for matching

grep options

Page 230: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

230System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

* - matches 0 or more characters

[^pqr] - Matches a single character which is not p ,q or r

^pqr -Matches pqr at the beginning of the line

pqr$ -Matches pqr at the end of the line

“.” - Matches any one character

\ - ignores the special meaning. grep “New\[abc\]” filename

Patterns

Page 231: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

231System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Displays the top n lines of the file

Can specify top n lines to be displayed

$ head -3 file1

Filter command - head

Page 232: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

232System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Displays the end of the file Can specify last n lines to be displayed

$ tail -3 file1

Can also specify the line number from which the data has to be displayed

$ tail +5 file1

Filter command - tail

Page 233: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

233System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

tr - translate filter used to translate a given set of characters

usage :

tr [a-z] [A-Z] <filename

option -s can be used to squeeze the repeated characters.

Filter command - tr

Page 234: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

234System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Useful options for tr

-s charSqueeze multiple contiguous occurrences of the character

into single char

-d charRemove the character

Filter command - tr

Page 235: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

235System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Allows the output (only the standard output) of a command to be sent as input to another command.

• Multiple pipes may appear in one command line.

Example:

$ cat * | wc

$ cat fil1 | head | wc -l

Command piping

Page 236: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

236System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• tee command allows the normal output to the standard output, as well as to a file

• Useful to capture intermediate output of a long command pipeline for further processing, or debugging purpose.

• Example• who | tee userlist• cat - | tee file1 | wc -l

Filter command - tee

Page 237: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

237System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Used to extract specified columns from the output of certain commands

-c used to extract characters -d Delimiter for fields -f Field no. Examples cut -c2-5 file1 cut -d “|” -f2,3 file1

Filter command - cut

Page 238: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

238System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Paste is used to fix two cut portions of the file

vertically

-s Pastes the contents of file2 below file1

-d Specify delimiter

$ paste -d”|” file1 file2

Filter command - paste

Page 239: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

239System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

telnet hostname

or

telnet <ip address>

telnet

Page 240: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

240System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

ftp hostname

put/send: Transfer to Other hosts

put [local_file [remote_file]]

get/recv: Transfer From Other hosts

get [remote_file [local_file]]

mput: Multiple Transfer to Other hosts

ftp

Page 241: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

241System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

gzip,

Usage is very similar to compress and pack utilities in Unix:

gzip [-vc] filename

where -v displays the compression ratio.

-c sends the compressed output to standard output and leaves

the original file intact.

gunzipgunzip can uncompress files originally compressed with compress.

Compression utilities

Page 242: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

242System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Tar is an archiving utility to store and retrieve files from an archive, known as tarfile.

• Though archives are created on a tape, it is common to have them as disk files as well.

– tar c|t|x [vf destination] source...

Examples: Create a new tar file containing all .dat files (assuming

a.dat, b.dat and c.dat exist) $ tar –cf mytar *.dat

Tape archive - tar

Page 243: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

243System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• In this session, you have learned to:• use the Unix Utilities like

• cat, echo, touch, more, file, wc, cmp, comm, find

• employ redirection operators• use filters like

• sort, grep, cut, head, tail, tr, and paste• communication commands

• telnet, ftp• backup commands

• zip/gzip and tar

Summary

Page 244: System Programming in Linux

Chapter 5

Process

Page 245: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

245System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• In this session, you will learn to:• Define a process• Use process-related commands like

• ps, kill, sleep • Start a Background process• Use background and foreground-related commands

like• bg, fg, jobs , nice , nohup

Objectives

Page 246: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

246System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Process - a program in execution

• When program is executed, a new process is created.

• The process is alive till the execution of the program is complete.

• Each process is identified by a number called pid.

Process

Page 247: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

247System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

As soon as the user logs in, a process is created which executes the login shell.

Login shell is set for each login in /etc/passwd file.

Login shell

Page 248: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

248System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• The ps command is used to display the characteristics of a process

• It fetches the pid, tty, time, and the command which has started the process.

• -f lists the pid of the parent process also.• -u lists the processes of a given user• -a lists the processes of all the users• -e lists all the processes including the system

processes

ps

Page 249: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

249System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Enables the user to do more than one task at a time.

If the command terminates with an ampersand (&), UNIX executes the command in the background

Shell returns by displaying the process ID (PID) and job id of the process

Background process

Page 250: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

250System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

nohup

Lets processes to continue to run even after logout

The output of the command is sent to nohup.out if not redirected

$ nohup command args

$ nohup sort emp.lst &[1] 21356

nohup: appending output to `nohup.out'

Background process

Page 251: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

251System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

wait command

can be used when a process has to wait for the output of a background process

The wait command, can be used to let the shell wait for all background processes terminate.

$ waitIt is possible to wait for completion of one specific process as well.

Background process

Page 252: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

252System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• jobs• List the background process

• fg % <job id>• Runs a process in the foreground

• bg %<job id>• Runs a process in the background

Controlling background process

Page 253: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

253System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• nice• Used to reduce the priority of jobs

Process priority

Page 254: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

254System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• kill: Kills or terminates a process

• kill command send a signal to the process• The default signal is 15 ( SIGTERM)

• kill -9 (SIGKILL)• Terminates the process abruptly

kill

Page 255: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

255System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• In this session, you learned to:

• Define a process• Use process-related commands like

• ps, kill, sleep • Start a background process• Use background and foreground-related

commands like• bg, fg, jobs

Summary

Page 256: System Programming in Linux

Chapter 6

UNIX Shell Programming

Page 257: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

257System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• In this session, you will learn to:• Use Shell variables• Write scripts to process positional parameters• Use “test” command• Use “if” construct• Use “for” loop• Use “while” loop• Use “case” construct • Define and use functions• Debug shell scripts

Objectives

Page 258: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

258System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Bourne shell sh

• C shell csh

• Korn shell ksh

• Bourne again shell bash • (shell distributed with linux)

Flavors of UNIX shell

Page 259: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

259System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Displays the shell prompt and reads the command typed by the user.

• Interprets the command and classifies it as an internal (built-in), or an external command.

• If it is NOT a built-in command, searches for the command in the PATH-specified directories, and executes that command if it is found.

Command processing

Page 260: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

260System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Parent shell process

(bash)

User Mode

Kernel Mode

$ vi test.c

command typed by user

Child shell process

(bash)

fork

Exec of “vi test.c”

Shell features

Page 261: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

261System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Each shell, apart from the basic features, provides additional features such as:

• Maintaining command history (C, korn and bash)

• Renaming (aliasing) a command (C, korn, bash)

• Command editing (C, korn and bash)

• Programming language (all shells)

Additional shell features

Page 262: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

262System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• some UNIX shells support command history • facility to keeps track of commands that were executed• facility to rerun previously executed commands• bash shell supports the following

!! recall the last command and execute it.

!num execute nth command where n is the

the num specified after !

History

Page 263: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

263System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• alias can be used to give new name to an existing command

• A better name that represents a single command or a sequence of commands to be executed, often with appropriate options

• alias is an internal command

alias newname=command

$ alias l=‘ls –l’

The unalias command cancels previously defined alias.

alias

Page 264: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

264System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• When the user enters a command string, the shell parses the string into following components:

• Command (the first part of the string, till the first space char)

• Command arguments (the subsequent parts of the string)

• For example, given the command-string “ls –l *.c”, this string contains the “ls” command and two arguments “-l” and “*.c”.

File name substitution

Page 265: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

265System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• In arguments of a command, the shell recognizes certain characters – such as *, ?, [ ], and - as special characters and expands these characters into a filename list before executing the command.

• To see how this works, enter following commands while in /bin directory

$ ls a*

$ ls ??

File name substitution

Page 266: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

266System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Allows

• Defining and referencing variables• Logic control structures such as if, for,

while, case• Input and output

Shell programming

Page 267: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

267System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• A variable is a name associated with a data value, and it offers a symbolic way to represent and manipulate data variables in the shell. They are classified as follows

• user-defined variables• environment variables• predefined variables

• value assigned to the variable can then be referred to by preceding the variable name with a $ sign.

Shell variables

Page 268: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

268System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• The shell provides the facility to define normal, and environment variables.

• A normal variable can be only used in the shell where it is defined.

• An environment variable can be used in the shell where it is defined, plus any child shells invoked from that shell.

Shell variables

Page 269: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

269System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• To define a normal variable, use the following syntax:• variable_name=value

• Examples:$ x=10$ textline_1=‘This line was entered by

$USER’$ textline_2=“This line was entered by

$USER”$ allusers=`who`$ usercount=`who | wc –l`

Using normal variables

Page 270: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

270System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Once variables are defined, one can use the echo command to display the value of each variable:

$ echo $x$ echo $textline_1$ echo $textline_2$ echo $allusers$ echo $usercount

Using normal variables

Page 271: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

271System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• To define an environment variable, use following syntax:

variable_name=value• export variable_name

• Examples:• $ x=10; export x• $ allusers=`who` ; export allusers

Using environment variables

Page 272: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

272System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• PATH

• BASH_ENV

• HOME

• PWD

• SHELL

• TERM

• MAIL

• USER

• LOGNAME

• PS1

• PS2

Built-in environment variables

Page 273: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

273System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

#! /bin/bash # # The above line has a special meaning. It must be the # first line of the script. It says that the commands in # this shell script should be executed by the bash # shell (/bin/bash). # --------------------------------------------------------------- echo “Hello $USER….” echo “Welcome to programming shell scripts..” # ---------------------------------------------------------------

Sample shell script

Page 274: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

274System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

There are two ways of executing a shell script:

By passing the shell script name as an argument to the shell. For example:

sh script1.sh

If the shell script is assigned execute permission, it can be executed using it’s name. For example:

./script1.sh

Executing shell scripts

Page 275: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

275System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• parameter can be passed to a shell script

• parameters are specified after the name of the shell script when invoking the script.

• Within the shell script, parameters are referenced using the predefined variables $1 through $9.

• In case of more than 9 parameters, other parameters can be accessed by shifting.

Passing parameters to scripts

Page 276: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

276System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Following are built-in variables supported• $0, $1…$9 - positional arguments• $* - all arguments• $@ - all arguments

Built-in variables

Page 277: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

277System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• Consider following shell script:

----------------------script2.sh-------------------------- echo “Total parameters entered: $#” echo “First parameter is : $1” echo “The parameters are: $*” shift echo “First parameter is : $1” ------------------------------------------------------------

• Execute the above script using the “script2.sh these are the parameters” command.

Passing parameters to scripts

Page 278: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

278System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• The shell parameters are passed as strings.

• to pass a string containing multiple words as a single parameter, it must be enclosed within quotes.

• For example,

$ script2.sh “this string is a single parameter”

Passing parameters to scripts

Page 279: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

279System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Arithmetic operations within a shell script can be performed using expr command.

Example,• x=10• y=5

number_1 = `expr $x + $y`

number_2 = `expr $x - $y`

number_3 = `expr $x / $y`

number_4 = `expr $x \* $y`

number_5 = `expr $x % $y`

Doing arithmetic operations

Page 280: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

280System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

The general syntax of test command is:

test <expression>

• The expression can be formed using a combination of shell variables and the operators supported by the test command. These operators provide facility to compare numbers, string and logical values, file types and file access modes.

Using the test command

Page 281: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

281System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

To compare two integers using test following operators are available:

-eq (equal to)

-ne (not equal to)

-lt (less than)

-le (less than or equal to)

-gt (greater than)

-ge (greater than or equal to)

Using the test command

Page 282: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

282System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Please note that you can only compare two integers using test.

General syntax

test [expression]

test integer1 operator integer2OR

[ integer1 operator integer2 ]

Using the test command

Page 283: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

283System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

To compare two strings using the test command, following operators are available:

• string1 = string2 (equal to, please note it is a single =)• string1 != string2 (not equal to)• string1 (string is not NULL)• -n string1 (string is not NULL and exists)• -z string1 (string is NULL and exists)

Using the test command

Page 284: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

284System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

The syntax for this string comparison is:

test string1 operator string2

OR

[ string1 operator string2 ]

OR

test operator string

OR

[ operator string ]

Using the test command

Page 285: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

285System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

To check a file type/access permissions using the test command, following operators are available:

• -s file (file is not empty and exists)• -f file (Ordinary file and exists)• -d file (file is a directory and exists)• -r file (file is readable and exists)• -w file (file is write-able and exists)• -x file (file is executable and exists)

Using the test command

Page 286: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

286System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

To check a file type/access permissions using the test command, following operators are available:

• -b file (file is a block device and exists)• -c file (file is a character device and exists)• -p file (file is a named pipe and exists)• -g file (file has sticky bit set)• -u file (file has setuid bit set)• -t file_des (file descriptor is standard

output)

Using the test command

Page 287: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

287System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• It is possible to combine conditions by using following operators:

• -a (logical AND operator)• -o (logical OR operator)• ! (logical NOT operator)

Combining conditions

Page 288: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

288System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

The syntax for this is:

test expression_1 –a expression _2, OR[ expression _1 –a expression _2 ]

test expression_1 –o expression _2, OR[ expression_1 –o expression_2 ]

test ! expression _1 OR

[ ! expression _1 ]

Combining conditions

Page 289: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

289System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Bash shell provides the if command to test if a condition is true. The general format of this command is:

if condition

then

command

fi

The condition is typically formed using the test command.

Condition checking in scripts

Page 290: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

290System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

# to check if the current directory is the same as your home directory curdir=`pwd` if test “$curdir” != “$HOME” then echo your home dir is not the same as your pesent working directory else echo $HOME is your current directory fi

Example

Page 291: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

291System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

The complex form of if statement is as follows:

if condition_1

then

command

elif condition_2

then

command

else

command

fi

Checking multiple conditions

Page 292: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

292System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Example:

for i in 1 2 3 4 5 do echo -n $i \* $i =

" " echo `expr $i \*

$i ` done

The Bash shell provides a for loop.

The syntax of this loop is:

for variable in list do command … command done

Using for loop

Page 293: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

293System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

----------------------script.sh--------------------------

#! /bin/sh

usernames=`who | cut –d “ “ –f1`

echo “Total users logged in = $#usernames”

#

for user in ${usernames}

do

echo $user

done

------------------------------------------------------------

Example

Page 294: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

294System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

The Bash shell provides a while loop. The syntax of this loop is:

while condition do command … command done

Using while loop

Page 295: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

295System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Shell script checks for a blank/non blank string eg: read nam while [ -z “$nam” ] do read nam done echo the string is $nam

the above piece of code keeps accepting string variable nam until it is non zero in length.

Example

Page 296: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

296System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Shell script to compute factorial of a given number

#!/bin/bash n=$1 if [ $n -eq 0 ]; then fact=0 else fact=1 while [ $n –ne 0 ] do fact=`expr $fact \* $n` n=`expr $n – 1` done fi echo $fact

Example

Page 297: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

297System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

- THE CASE STATEMENT case value in pattern1) command command;; pattern2) command command;; patternn) command;; esac

case statement

Page 298: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

298System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

#!/bin/bash echo enter a choice read choice case $choice in 1) echo enter 2 nos read num1 read num2 res=`expr $num1 + $num2` echo result is $res;; 2) who;; *) echo invalid choice;;

esac

Example

Page 299: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

299System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

#!/bin/bash read number case $number 1) echo 1st break;; 2) echo 2nd break;; 3) echo 3rd break;; *) echo ${number}th break;; esac

Example

Page 300: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

300System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Shell functions are a way to group commands for later execution using a single name for the group. They are executed just like a "regular" command.

Shell functions are executed in the current shell context; no new process is created to interpret them. Functions are declared using this syntax:

[ function ] name () { command-list; }

Functions

Page 301: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

301System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Shell functions can accept arguments Arguments are passed in the same way as given to commands Functions refer to arguments using $1, $2 etc., similar to the way shell scripts

refer to command line arguments

Functions

Page 302: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

302System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Function to convert standard input into upper case

toupper() { tr A-Z a-z }

This function can be used as

$ cat abc | toupper

Functions

Page 303: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

303System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Two options help in debugging shell scripts

“-v” (verbose) option: causes shell to print the lines of the script as

they are read.

$ bash –v script-file

“-x” (verbose) option: prints commands and their arguments as

they are executed.

$ bash –x script-file

Debugging shell scripts

Page 304: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

304System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Comparison between

• A solution in C• A shell solution written like a C program• A proper “shell/unix” solution

e.g: The problem is to count the no of lines in a file ( the

file is called the_file)

Programming in C vs shell

Page 305: System Programming in Linux

#include <stdio.h> void main(void){ int lcount=0; FILE *infile; char line[500]; infile=fopen("the_file","r"); while(!feof(infile)) { fgets(line,500,infile); lcount ++; } printf("no of lines is %d\n",lcount);}

count=0while read line do count=`expr $count + 1`done < the_file

echo Number of lines is $count Solution using existing commands

$ wc –l the_file

Programming in C vs shell

C program to count lines in a file Solution in Bash

Page 306: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

306System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

• In this session, you have learned to:

• Use Shell variables• Write scripts to process positional parameters• Use “test” command• Use “if” construct• Use “for” loop• Use “while” loop• Use “case” construct • Define and use functions• Debug shell scripts

Summary

Page 307: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

307System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

UNIX for the Impatient, Paul W. Abrahams & Bruce R. Larson (Addison-Wesley Publishing Company, 1992, ISBN 0-201-55703-7).

UNIX in a Nutshell for BSD 4.3: A Desktop Quick Reference For Berkeley (O'Reilly & Associates, Inc., 1990, ISBN 0-937175-20-X). (A handy reference for BSD.)

UNIX in a Nutshell: A Desktop Quick Reference for System V & Solaris 2.0 (O'Reilly & Associates, Inc., 1992, ISBN 0-56592-001-5). (A handy reference for SysV and Solaris 2.)

The UNIX Programming Environment, Brian W. Kernighan & Rob Pike (Prentice Hall, 1984). (A classic. For serious folks.)

Bibliography

Page 308: System Programming in Linux

Advanced Java Programming

[ 04/08/23 09:09 ][ 04/08/23 09:09 ]

308System Programming in Linux

SYSTEM

PROGRAMMING

IN LINUX

SYSTEM

PROGRAMMING

IN LINUX| U. K. Roy |

Thank You