Operating Systems Unit 10: – Networking – Distributed File Systems Operating Systems

Operating Systems

Unit 10:– Networking– Distributed File Systems

Operating Systems

COP 5994 - Operating Systems 2

Networking concepts

• host– single computer– local CPU, memory, disks

• link– connecting media– attributes:

• bandwidth, latency

COP 5994 - Operating Systems 3

Network Topologies

COP 5994 - Operating Systems 4

Communication Protocols

Data Transport

Application

COP 5994 - Operating Systems 5

Data Transport layers

4: Transport layer• End-to-end communication• Relies on network layer to determine proper

path from one end of communication to the other

3: Network layer• Moving data between computers

2: Link layer• Provides an interface between the network layer

and the underlying physical medium of the connection

1: Physical layer

COP 5994 - Operating Systems 6

2: Link Layer

• first software layer on top of physical medium

• unit: frame• responsibilities:

– deliver frame – detect errors– correct transmission errors

COP 5994 - Operating Systems 7

2: IEEE 802.3 CSMA/CD (Ethernet)• Carrier Sense Multiple Access

with Collision Detection protocol– transceiver tests delivery medium for

availability• due to delays in medium, it is possible that multiple

transceivers begin transmitting simultaneously

– if transceivers detect collision:• continues to transmit bytes for a specific period of

time to ensure that all transceivers become aware of the collision

• waits for a random interval before attempting to retransmit

COP 5994 - Operating Systems 8

2: Token Ring

• operates on ring networks • employs tokens to gain access to

the transmission medium

• token is empty frame that is circulated between machines

COP 5994 - Operating Systems 9

2: Token Ring example scenario 1/2

COP 5994 - Operating Systems 10

2: Token Ring example scenario 2/2

COP 5994 - Operating Systems 11

2: Fiber Distributed Data Interface (FDDI)

• operates over fiber-optic cable– support more transfers at greater

speeds over larger distance

• built on two token rings– the second usually being reserved for

backup

COP 5994 - Operating Systems 12

2: IEEE 802.11 (Wireless)

• similar to Ethernet:– Carrier Sense Multiple Access

with Collision Avoidance (CSMA/CA)– require each sender broadcast a

Request to Send (RTS) to entire network

– upon receiving an RTS• receiver broadcasts a Clear to Send (CTS)

message to the entire network if the medium is available

COP 5994 - Operating Systems 13

3: Network Layer

• routes packet to next host toward destination

• determines next host– address information– network topology– link quality

• strength of signal, error rate and interference– Interference is broadcast throughout networks

using various router protocols, such as Routing Information Protocol (RIP)

COP 5994 - Operating Systems 14

3: Internet Protocol (IP or IPv4)

• hosts have unique IP address– 32-bit number written as quad– network

• first n bits of IP number, written as “/n”• 8 - class A, 16 - class B, 24 - class C• more than 24 - class D

– netmask• 32 bit number with first n bits all 1, rest 0

– broadcast• network number (first n bits), rest all 1

– gateway IP

131.94.134.39

/24

255.255.255.0

131.94.134.255

131.94.134.1

COP 5994 - Operating Systems 15

3: Internet Protocol (IP or IPv4)

• host names are mapped to IP address via the Domain Name System (DNS)– need IP numbers of DNS servers

• private IP numbers– used locally– Network Address Translation (NAT)

COP 5994 - Operating Systems 16

3: IPv4 packet

COP 5994 - Operating Systems 17

3: Internet Protocol version 6 (IPv6)

• larger address space: 128 bit number– written as 8 groups of 4 hex digits2001:468:701:3800:206:5bff:fe53:2a87

COP 5994 - Operating Systems 18

3: IPv6 new features

• quality of service• authentication and privacy• anycast address

– used to send packet to one of a group

• multicast– used to send packet in all of a group

COP 5994 - Operating Systems 19

4: Transport Layer

• Connection-oriented approach• Hosts send each other control information through

handshaking to set up a logical end-to-end connection

• Imposes reliability on unreliable networks• Guarantees that data sent from sender will arrive

at intended receiver undamaged and in correct sequence

• Connectionless approach• Two hosts do not handshake before transmission• No guarantee that sent messages will be received

in their original order, or at all

COP 5994 - Operating Systems 20

4: Transmission Control Protocol (TCP)

• connection-oriented transmission protocol– guarantees that segments sent from a

sender will arrive at the intended receiver undamaged and in correct sequence

– handles error control, congestion control, and retransmission

– allows protocols like HTTP and FTP to send information into network as simply and reliably as writing to a file on the local computer

COP 5994 - Operating Systems 21

4: User Datagram Protocol (UDP)

• Connectionless User Datagram Protocol– provides minimum overhead necessary

for the transport layer– no guarantee that UDP datagrams will

reach their destination in their original order, or at all

COP 5994 - Operating Systems 22

Application Layers

• Application layer protocols– Specify rules that govern remote

interprocess communication– Determine how processes should

interact

• protocols interact with remote resources– via Uniform Resource Identifier (URI)

COP 5994 - Operating Systems 23

Application Layer protocols

URL: Uniform Resource Locator– special form of URIsyntax: protocol://host:port/path

• Common protocols– ftp– http– smtp– ldap– sip

COP 5994 - Operating Systems 24

Application Layer protocol: RPC

Remote Procedure Callgeneral mechanism to enable function invocation across host boundaries

• extension of local function call model

COP 5994 - Operating Systems 25

Application Layer protocol: RPC

COP 5994 - Operating Systems 26

RPC portmapper

• common Unix implementation• translates RPC program numbers

– TCP/IP port numbers– UDP/IP port numbers

• common RPC programs– NIS– NFS

COP 5994 - Operating Systems 27

Distributed File Systems

• Networked file systems – Allow clients to access files stored on

remote computers

• Distributed file systems– Special examples of networked file

systems that allow transparent access to remote files

COP 5994 - Operating Systems 28

Distributed File System Concepts

• location transparency– user is unaware of the physical location of a

file within a distributed file system– user sees only a global file system

• caching & consistency– Clients keep a local copy of a file and flush

modified copies of it to the server from time to time

– Because there are multiple copies of the same file, files can become inconsistent

COP 5994 - Operating Systems 29

Distributed File System Concepts

• scalability– Distributed file systems are designed to share

information among large groups of computers– New computers should be able to be added to

the distributed system easily

• security– Ensuring secure communications– Access control

• fault tolerance

COP 5994 - Operating Systems 30

Distributed File System Concepts

• server state– can be either stateful or stateless– stateful system

• server keeps state information of the client requests so that subsequent access to the file is easier

– stateless system• client must specify which file to access in

each request

COP 5994 - Operating Systems 31

Network File System

• developed by Sun MicroSystems• early 1980s

• dominant for Unix environments• Versions

– NFS-3– NFS-4, IETF standard

COP 5994 - Operating Systems 32

NFS Architecture

COP 5994 - Operating Systems 33

Network File System (NFS)

• NFS versions 2 and version 3 – assume a stateless server

implementation

– if the server crashes: • client simply retries until the server

responds, or• aborts

– if server resumes, no need to rebuild state

COP 5994 - Operating Systems 34

Network File System (NFS)

• NFS-4 – Stateful– enables faster access to files

– if the server crashes• all the state information of the client is lost• client needs to rebuild its state on the

server before retrying

COP 5994 - Operating Systems 35

NFS-4 Caching: Delegation

• efficient client-caching scheme:– server temporarily transfers control of

file to client• read delegation: no other client can write• write delegation: no other client can read or

write

– If another client requests a delegated file, the server will revoke the delegation and request that the original client flush the file back to server

COP 5994 - Operating Systems 36

Andrew File System

• developed by Carnegie Mellon University– after first names of A. Carnegie & A. Mellon– early 1980s

• design goals:– large scale– secure– efficient

• Versions: AFS-1, AFS-2, AFS-3– basis for DFS of OSF, predecessor to Coda

COP 5994 - Operating Systems 37

Andrew File System (AFS)

• location transparency– all files appear as a branch of a

traditional UNIX file system at each workstation

• built on RPC• security:

– uses Kerberos authentication server– access control lists for file/directories

COP 5994 - Operating Systems 38

AFS Structure

COP 5994 - Operating Systems 39

AFS caching

• files are cached on clients local disk– notifies clients via callback that files

are no longer valid– client must invalidate its file and

request the most recent version

COP 5994 - Operating Systems 40

Coda File System

• developed by Andrew team• design goal: fault tolerance• mid 1980s

COP 5994 - Operating Systems 41

Coda File System Concepts

• Volumes – logical pieces of the file system – replicated physically across multiple file

servers

• Volume storage group (VSG)– servers that hold the same volume

• Available volume storage group (AVSG)– reachable members of the VSG

• Preferred Server– member of AVSG, currently serving files

COP 5994 - Operating Systems 42

Coda volume structure

COP 5994 - Operating Systems 43

Coda read/write scenario

• to read, check cache for file– if found, done– else

•get file from PS•get file versions from other members of AVSG• if conflict, then AVSG members agree on new

versions, and update their copies

• to write,– send file to all members of AVSG– track members of AVSG that have written

file

COP 5994 - Operating Systems 44

Coda inconsistency scenario

COP 5994 - Operating Systems 45

Coda client disconnect scenario

• When connected to Coda, clients cache files so they can be accessed when disconnected (hoarding stage)

• When disconnected, clients enter the emulation stage where all file requests are serviced from the cache, if the file is resident (error otherwise)

• When reconnected, file updates are sent to the server asynchronously (reintegration stage)

COP 5994 - Operating Systems 46

Sprite File System

• Part of Sprite distributed OS– Developed at UC Berkeley, mid 1990s

• Sprite file system characteristics– Emulates a UNIX file system

• even remote access to I/O devices

– Every client has the exact same view of the hierarchy

COP 5994 - Operating Systems 47

Sprite File System domains

per client

COP 5994 - Operating Systems 48

Sprite read scenario

– to open a file, the client first checks its cache, then makes a request to server

– if the server is unable to satisfy the request from its cache, it reads the data from disk

– both caches retain a copy of file

COP 5994 - Operating Systems 49

Sprite write scenario

– client writes to its cache– updated pages are flushed to server every

30 seconds

– lazy write-back strategy

COP 5994 - Operating Systems 50

Sprite caching protocol

• sequential write sharing– upon opening file for write, client

checks file version with server• might cause reload of cached file from

server

• concurrent write sharing– if 2 clients open file for write, caching

is turned disabled

COP 5994 - Operating Systems 51

Common Internet File System

• started by IBM,no propagated by Microsoft

• other name: server message block (smb)

• built on top of NetBIOS– name service– session management– datagram distribution

• also supports printing

COP 5994 - Operating Systems 52

CIFS concepts

• Universal Naming Convention (UNC)\\host\path\file

• server publishes shares– workgroup elects browse master

• client maps share to drive letters

COP 5994 - Operating Systems 53

CIFS caching

• enabled via locking concept• opportunistic lock (oplock)

– exclusive oplock• client is granted abritrary buffering rights

– batch oplock• client keeps lock despite no local accessor

– level II oplock• multiple clients read, no writer

COP 5994 - Operating Systems 54

CIFS features

• user/share authentication– plain password– challenge/response protocol

with password encryption

• replicated virtual volumes– volumes may span servers– volumes can be transparently moved– volumes may be replicated

COP 5994 - Operating Systems 55

Agenda for next week:

– Chapter 19: Security

– Read ahead !