View
4
Download
1
Category
Preview:
Citation preview
WashingtonWASHINGTON UNIVERSITY IN ST LOUIS
Final Review
Fred Kuhns
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 2
Computer Systems• Computer Systems
– Hardware and software combine to solve specific problems– Two software categories: 1) Application software, 2) System software
• Goals of an OS– User Environment: Execution environment, Error detection and
handling, Protection and security, Fault tolerance and failure recovery– Resource Management: Time and space management;
Synchronization and deadlock handling; Accounting and status information
• Driving Concepts– Abstraction: manage complexity, create an extended machine.– Virtualization: Permits controlled sharing and isolation (enables
process model and isolation)– Resource management: implicit/explicit allocation and policy control.
Performance critical, competing goals of maximizing utilization, minimizing overhead, providing fair/predictable access to system resources.
• Resource Sharing: apparent (logical) concurrency or true (parallel) concurrency– abstract machines transparently share resources– concurrent programs may also elect to explicitly share resources– Space-multiplexed orTime-multiplexed
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 3
Common Abstractions: Process and Resource
Process P0
Process P1
Process PN
...
File F0
File F1
File FM
...
memory Ri
display Rj
...
ResourcesProcesses
Operating System
CPU DRAM DISKS NETother
otherother
An application consists of one or more processes, a set of resources and state
CPU
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 4
Computations, Processes and Threads
• Computation: sequential execution, implements algorithm, applied to data set.
• Program: encoding of an algorithm using a programming language (i.e. C or C++).
• program requests resources from OS, example system call interface.
• Process: program in execution and embodies all resources and state of a running program (CPU, memory, files,...)
• Modern process model separates execution context from other resources. – Thread: Dynamic object representing an execution path and
computational state. Remaining resources are shared amongst threads in a process
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 5
Implementing Operating Systems• Common functions:
– Device management– Process, thread and resource management– Memory management– File management
• Issues in OS design– Performance – should be low overhead– Exclusive use of resources
• protection and security – sharing, authorizing, authenticating– correctness – does what it is supposed to do– maintainability – universal goal– commercial – they are used by people and organizations– standards and open systems – conforming to standards
• Common implementation mechanisms– Protection and processor modes– Trusted control program – kernel– Method for processes to request services of the OS – system calls
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 6
Policy and Mechanism
• A recurring theme in OS design is ensuring exclusive use of system resources (real and abstract)
• Administrators and developers define policies to define resource sharing and isolation
• Mechanisms implement these policies• Common mechanisms:
– hardware enforced processor modes to control execution of privileged instructions (such as for I/O or memory)
– Core OS modules are trusted and have access to protected operations and all resources. This kernel of the OS is responsible for enforcing policies.
– Well defined and protected mechanism for requesting services of the OS. There are two approaches: system calls or message passing.
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 7
OS Techniques
• Controlling access to hardware resources is done using hardware supported privilege levels, typically two: user and system
• Privileged operations, resource management, protection enforcement (isolation), and sharing performed by a trusted control program: the Kernel
• Users request OS services using a well defined interface that validates user and notifies OS of request: two common methods:– System calls: trap changes mode and executes privileged
code in context of calling process– Message passing: interface constructs message and sends to
another system (i.e. privileged) process
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 8
An Example: an OS Kernel• Trusted program that runs directly on the hardware• Loaded at boot time and initializes system,
– creates some initial system processes. – remains in memory and manages the system
• Resource manager/mediator - process is key abstraction.– Time share (time-slice) the CPU,– coordinate access to peripherals,– manage virtual memory.– Synchronization primitives.
• Well defined entry points:– syscalls, exceptions or interrupts.
• Performs privileged operations.• Kernel Entry
– Synchronous - kernel performs work on behalf of the process:• System call interface (UNIX API): central component of the UNIX API• Hardware exceptions - unusual action of process
– Asynchronous - kernel performs tasks that are possibly unrelated to current process.• Hardware interrupts - devices assert hardware interrupt mechanism to notify kernel of events
which require attention (I/O completion, status change, real-time clock etc) – System processes - scheduled by OS (swapper and pagedaemon)
• Synchronization within Kernel– Traditional: Kernel is re-entrant but with only one thread/processes active at any given
time, Nonpreemptive. Blocking operations, Masking interrupts– Modern: threaded kernels, synchronization primitives within kernel, locking granularity
is issue
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 9
PC
CPU
Buffers
Devices
Memory
Instruction
InstructionInstruction
DataData
DataData
...
..
IR MAR
MBR
I/O AR
I/O BR
executionunit ...
...
MBR - Memory Buffer RegisterI/O AR - Input/Output Address RegisterI/O BE - Input/Output Buffer Register
PC - Program CounterIR - Instruction RegisterMAR - Memory Address Register
0
N
1
Reg NReg 1
Reg 0
Zooming in on Computer Architecture
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 10
Instruction Cycle with Interrupts
Fetch NextInstruction
Fetch NextInstruction
ExecuteInstruction
ExecuteInstructionSTARTSTART
HALTHALT
Fetch Cycle Execute Cycle
Check for &Process Int
Check for &Process Int
Interrupt Cycle
Inte
rru
pts
En
able
d
InterruptsDisabled
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 11
Interrupts – devices notify CPU of some event
Processor
Timer
command status rt-counter
Bus
clock handlerX
Device table
dispatcher(interrupt handler)
What about multiple interrupts?
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 12
OS and I/O Hardware Management• Two fundamental operations: 1) Processing data, 2) Perform I/O• Large diversity in I/O devices, capabilities and performance• Goal: provide simple/consistent interface, efficient use, max
concurrency• Mechanisms: device drivers provide standard interface to I/O
devices• Kernel I/O tasks: Scheduling, Buffering, Caching, Spooling,
Reservations• Common concepts: Port, Bus, Controller• Accessing a device: Direct I/O instructions and Memory-mapped I/O• Common Techniques:
– Synchronous: Direct I/O with polling, aka Programmed I/O– Asynchronous: Interrupt Driven I/O– Direct memory access (DMA)– Memory Mapped I/O
• Process I/O interface models: Blocking, Nonblocking and Asynchronous
• Improving performance: Reduce context switch, data copy; use DMA; joint scheduling of multiple resources.
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 13
Process Model
• Define process: ?• Traditional unit of control for synchronization, sharing,
communication and deadlock control• Explicit versus implicit resource allocations• Know state diagrams• Understand how process execution is interleaved on the system
CPU(s)• Be able to describe a context switch• Understand process creation and termination• Cooperating processes
– Independent processes cannot affect or be affected by one another– Cooperating processes can affect or be affected by one another– Advantages of process cooperation: Information sharing, Computation
speed-up, Modularity, Convenience• Dangers of process cooperation
– Data corruption, deadlocks, increased complexity– Requires processes to synchronize their processing
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 14
Example: Process Representation
Process P1
Process P3
Process P2
init P0
Hardware State (registers)
Program counter
Memory base register
…
Process State (logical)
Open file table
Pending Requests
Memory mappings
…
Process P2 State
Kernel Process Table
P0: HW state; resources
PN: HW state; resources
P2: HW state; resources
…
…
System Memory
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 15
5 State Model – More realistic
Ready
Blocked
New Running
Exit
terminate
dispatch
pause
event
admit
wait
• New: The process is being created.• Running: Instructions being executed.• Blocked (aka waiting): Must wait for some event to
occur.• Ready: Runnable but waiting to be assigned to a
processor.• Exit : The process has finished execution.
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 16
BlockedSuspend
ReadySuspend event
Suspended Processes(possibly on backing store)
Ready
Blocked
New Running
Exit
terminate
dispatch
preempt
event
admit
waitactivate
suspend
suspend
suspend
activate
admit
Suspending a Process
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 17
3 Process Example
time
entity
P1
P2
P3
System(dispatch)
...
BlockedRunning Ready
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 18
Process Scheduling
• Long-term scheduler: job scheduler– selects which processes should be brought into the ready
queue.– invoked infrequently (seconds, minutes)– controls the degree of multiprogramming
• Medium-term scheduler – allocates memory for process.– invoked periodically or as needed.
• Short-term scheduler: CPU scheduler– selects which process should be executed next and allocates
CPU.– invoked frequently (ms)
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 19
IPC (Message Passing)• Purposes: Data Transfer, Sharing Data, Event notification,
Resource Sharing and Synchronization, Process Control• Mechanisms: message passing and shared memory
• Message passing systems have no shared variables.• Two operations for fixed or variable sized message:
– send(message)– receive(message)
• Communicating processes must establish a communication link and exchange messages via send and receive
• Communication link: physical or logical • implementation methods for logical communication links::
1. Direct or Indirect communications 2. Symmetric or Asymmetric communications 3. Automatic or Explicit buffering4. Send-by-copy or send-by-reference5. fixed or variable sized messages
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 20
Communication Methods
• Direct : Processes must name each other explicitly : Symmetric v. asymmetric addressing– Properties of communication link: automatic link establishment,
at most one link between processes and must know peers ID. – Disadvantages: a process must know the name or ID
• Indirect : Use mailboxes (aka ports) with unique ID– Properties of communication link: must share mailbox, >2
processes per mailbox or > 1 mailbox between processes.– Ownership: process versus system mailbox, ownership and read
permissions.
• Synchronous versus asynchronous: – blocking send, nonblocking send, blocking receive,
nonblocking receive• Buffering: Messaging system must temporarily buffer
messages. Zero capacity, Bounded capacity, Unbounded capacity
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 21
Multiprocessors
• Advantages : performance and fault tolerance• Classifications: tightly or loosely coupled• Memory access schemes: UMA, NUMA and NORMA• Cache consistency problem
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 22
Typical SMP System
cache MMU
CPUcache MMU
CPUcache MMU
CPUcache MMU
CPU
I/Osubsystem
Issues:• Memory
contention• Limited bus BW• I/O contention• Cache coherence
MainMemory
50ns
Typical I/O Bus:• 33MHz/32bit (132MB/s)• 66MHz/64bit (528MB/s)
500MHz
System/Memory Bus
ether
scsi
video
Bridge
System Functions(timer, BIOS, reset)
INT
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 23
Threads• Dynamic object representing an execution
path and computational state.– Effectiveness of parallel computing depends on the
performance of the primitives used to express and control parallelism
– Useful for expressing the intrinsic concurrency of a program regardless of resulting performance
• Benefits and drawbacks for the models we studied: User threads, Kernel threads and Scheduler Activations
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 24
Threading Models
• User threads := Many-to-One• kernel threads := One-to-One• Mixed user and kernel := Many-to-Many
P PP PP P
Many-to-One One-to-One Many-to-Many
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 25
Solaris Threading Model (Combined)
L LL
P P
Process 1
user
kernel
hardware
L
P
Process 2
...
Int kthr
......
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 26
CPU Scheduling
• Understand CPU-I/O burst cycle• Components: short-term scheduler, dispatcher• Criteria: CPU utilization, Throughput, Turnaround time,
Waiting time, Response time• Algorithms: FIFO, Shortest-Job-First, Priority-based,
Round-robin, Multilevel Queue, Multilevel feedback queue
• Priority-based scheduling, static versus dynamic priorities– Solution to starvation problem?
• Preemptive versus nonpreemptive schemes• Typical goals: Interactive Systems, Batch Systems,
Real-time Systems• timers and clock handler• Priority inversion
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 27
Histogram of CPU-burst Times
From Silberschatz, Galvin and Gagne, “Operating Systems Concepts”, 6 th eidtion, Wiley, 2002.
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 28
Concurrency: Origins and problems
• Context– Processes need to communicate– Kernel communication/controlling hardware resources (for example I/O
processing)• Issues:
– How is information exchanged between processes (shared memory or messages)?
– How to prevent interference between cooperating processes (mutual exclusion)?– How to control the sequence of process execution (conditional synchronization)?– How to manage concurrency within the OS kernel?
• Problems:– Execution of the kernel may lead to concurrent access to state– Deferred processing pending some event– Processes explicitly sharing resource (memory)
• Problems with shared memory– Concurrent programs may exhibit a dependency on process/thread execution
sequence or processor speed (neither are desirable!)– race condition – who’s first, and who goes next. affects
program results.– There are two basic issues resulting from the need to support concurrency: – Mutual exclusion: ensure that processes/threads do not interfere with
one another, i.e. there are no race conditions. In other words, program constraints (assumptions) are not violated.
– Conditional synchronization. Processes/threads must be able to “wait” for the data to arrive or constraints (assertions) to be satisfied.
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 29
Race Conditions - Example
Example 1int y = 0, z = 0;thread A { x = y + z} thread B {y = 1; z = 2}
Results:x = {0, 1, 2, 3}load y into R0
load z into R1
set R0 = R0 + R1
set R0 -> x
• There are 4 cases for x:– case 1: task A runs to completion
first loading y=0, z=0. x = 0 + 0 = 0
– case 2: Task B runs and sets y to 1, then Task A runs loading y=1 and z=0. x = 1 + 0 = 1
– case 3: Task A runs loading y=0, then Task B runs to completion, then Task A runs loading z=2. x = 0 + 2 = 2
– case 4: Task B runs to completion, then Task A runs loading y=1, z=2,x = 1 + 2 = 3
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 30
Critical Section Problem
• Entry/exit protocol satisfies:• Mutual Exclusion: At most one
process may be active within the critical section CS.
• Absence of deadlock (livelock): if two or more processes attempt to enter CS, one will eventually succeed.
• Absence of Unnecessary delay: Processes neither within nor competing for CS (terminated or executing in Non-CS) can not block another process from entering CS.
• Eventual entry (no starvation, more a function of scheduling): if a processes is attempting to enter CS it will eventually be granted access.
Task A { while (True) { entry protocol; critical section; exit protocol; non-critical section; }}
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 31
Necessary Conditions for Deadlock• 1) Mutual exclusion:
– One process holds a resource in a non-sharable mode. – Other processes requesting resource must wait for resource to be
released.
• 2) Hold-and-wait:– A process must hold at least one allocated resource while awaiting
one or more resources held by other processes.
• 3) No preemption:– Resources not forcibly removed from a process holding it, the holding
process must voluntarily released it.
• 4) Circular wait– a closed chain of processes exists, such that each process holds at
least one resource needed by the next process in the chain.
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 32
Example of a Resource Allocation GraphR1 R2
P1 P2 P3
R3 R4
Initial: No Deadlock
R1 R2
P1 P2 P3
R3 R4
P2 releases R1, P3 requests R3
(No Deadlock:Why not?)
R1 R2
P1 P2 P3
R3 R4
P3 requests R3
(Deadlock)
• If graph contains no cycles – no deadlock.
• If graph contains a cycle – if only one instance per resource type, then deadlock.– if several instances per resource type, possibility of deadlock.
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 33
Approaches to Deadlock Handling
• Ensure system never enters a deadlocked state: Use protocol to prevent or avoid deadlock:– deadlock prevention scheme - ensure that at least one of
the necessary conditions cannot hold. – deadlock avoidance scheme - requires the OS know in
advance the resource usage requirements of all processes. Then for each request the OS decides if it could lead to a deadlock before granting.
• Allow the system to enter a deadlock state and then recover (detection and recovery).– system implements an algorithm for deadlock detection, if so
then recover
• Assume deadlocks never occur in the system– used by most operating systems, including UNIX.
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 34
Recall the Von Neumann Architecture
Arithmetic-Logical Unit(ALU) Control Unit
Primary Memory Device ControllerDeviceDevice Controller
DeviceDevice ControllerDeviceDevice Controller
Device
Central Processing Unit (CPU)
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 35
Memory Management
• Understand the principle of locality (temporal and spatial locality)
• Understand impact of Stride-k reference patterns• Understand cache• Memory Manager Functions: Allocate memory to processes,
Map process address space to allocated memory, Minimize access times while limiting memory requirements
• Understand program creation steps and typical process address map: compile, link and load
• Partitioning schemes and fragmentation– varia ble sizes and placement algorithms: best-fit, worst-fit,
next-fit, first-fit.
• Placement and relocation• Paging and segmentation
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 36
Cache/Primary Memory Structure
MemoryAddress
1
3
0
2
2n - 1
Block
Block
Word Length
SetNumber
Set 0
Set S-1
E lines per setS = 2s sets in the cacheB = 2b data Bytes per lineM = 2m = max memory address t = m – (s+b) tag bits per line1 valid bit per line (may also require a dirty bit)Cache size = C = B * E * Sm address bits (= t + s + b)
...
Tag BlockV
Tag BlockV
Tag BlockV
Tag BlockV
t bits s bits b bits
Address
Cache
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 37
Typical Process Address Space
ProcessAddress space
Low Address(0x00000000)
High Address(0x7fffffff)
Initialized Data
Text (shared)
Unitialized Data
Heap (Dynamic)
stack (dynamic)
Environment
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 38
Memory Management: Requirements
• Relocation– Why/What:
• programmer does not know where the program will be placed in memory when it is executed
• while the program is executing, it may be swapped to disk and returned to main memory at a different location
– Consequences/Constraints:• memory references must be translated in the code to actual physical
memory address• Protection - Protection and Relocation are interrelated
– Why/What:• Protect process from interference by other processes• processes require permission to access memory in another processes
address space.– Consequences/Constraints:
• impossible to check addresses in programs since the program could be relocated
• must be checked at run time• Sharing - Sharing and Relocation are interrelated
– allow several processes to access the same data– allow multiple programs to share the same program text
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 39
Lastallocatedblock (14K)
Before After
8K 8K
12K 12K
22K
18K
6K 6K
8K 8K
14K 14K
6K
2K
36K20K
Next Fit
Free block
Allocated block
Best Fit
First Fit
Variable Partition Placement Algorithmalloc 16K block
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 40
Hardware Support for Relocation
Interrupt tooperating system
Process image inmain memory
Relative address
Absoluteaddress
Process Control Block
Program
Data
Stack
Adder
Comparator
Base Register
Bounds Register
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 41
An Example: Paged Virtual Memory
page0
page1
page2
page3
pagen
P1 virtualaddress space
page0
page1
page2
page3
pagen
P2 virtualaddress spaceFrame 0
Frame 1
Frame 2
Frame 3
Frame 4
Frame 5
Frame 6
Frame 7
Physicaladdress space
AddressTranslation
resident
Non-resident
Working set
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 42
Example Paging System
Swap
UFS
app1
Disk
DRAM
CPU
Unitialized data
Allocated virtual pages
Text and initialized data
Stack and heap
app1Address space
Low Address(0x00000000)
High Address(0x7fffffff)
Initialized Data
Text (shared)
Unitialized Data
Heap (Dynamic)
stack (dynamic)
Environment
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 43
Buffer Cache
process
Copy
CopyVirtual Memory System
process
Copy
P1 pages
Process P1 Process P1
mmap(): Address spaceread/write: Copy
Traditional ApproachVM Approach
File Mapping - read/write interface
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 44
Paging and Segmented Architectures
• Memory references are dynamically translated into physical addresses at run time– A process may be swapped in and out of main memory such that it
occupies different regions• A process may be broken up into pieces that do not need to be
located contiguously in main memory– All pieces of a process do not need to be loaded in main memory
during execution• Example Program execution:
– Resident set: Operating system brings into main memory portions of the memory space
– If process attempts to access a non-resident page then the MMU raises an exception causing the OS to block process waiting for page to be loaded.
– Steps for loading a process memory pages:• Operating system issues a disk I/O Read request• Another process is dispatched to run while the disk I/O takes place• An interrupt is issued when disk I/O complete which causes the
operating system to place the affected process in the Ready state
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 45
Know the following
• Define thrashing• working set model• Three policies used with paging systems: fetch,
placement and replacement• Understand the MMU operation• Know the static and dynamic paging algorithms
– static: Optimal, Not recently used, First-in, First-out and Second chance, Clock algorithm, Least Recently Used, Least Frequently Used
– Dynamic: working set algorithm
• How does page buffering help?
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 46
Address Translation Overview
Virtualaddress
MMU
context table pointercontext
Page tables
physicaladdressTLB cacheCPU
Y bitsX bitsvirtual page numberVirtual address offset in page
Z bitsframe numberM R control bitsPage Table Entry (PTE)
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 47
Example 1-level address Translation
12 bits20 bitsvirtual page number
Virtual address
offset in page
current page table register(Process) Page Table
add
frame numberM Rcontrol bits
DRAMFrames
X offset
Frame X
PTE
offset with table
start of page table
physical address of frame
offset within frame
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 48
Secondary Storage
• See homework problems• Know the disk scheduling algorithms: FCFS, SSTF,
SCAN (elevator algorithm), C-SCAN, LOOK and C-LOOK
Fred Kuhns (04/08/23) CS422 – Operating Systems Concepts 49
Remaining material
• Review Class Notes and Homework Problems• FS, Networking, Remote FS and RPC.
Recommended