Distributed system
Distributed system1DefinitionA distributed system is a
collection of indepent entities that cooperate to solve problem
that cannot be individually solved
Message passing Message passing
Message passing
processormemoryprocessormemoryprocessormemoryThe characterized
of distributed systemYou must know that when the one of computers
in distributed system is crash the distributed its never heard and
its continue his work
A collection of computers that do not share common memory that
communicate by messages passing over communication network and
where each computer has its own memory and runs its own operating
system typically the computers are semi-autonomous and loosely
coupled while they cooperate to address problem
A collection of independent computers that appears to the user
of the system as a single coherent system.4- A term that describes
a wide range of computers from weakly coupled system such as wide
area network(WAN) to strongly coupled systems such as local area
network (LAN) to very strongly coupled system such as
multiprocessor systemCharacterized distributed system over
communication networkNo common physical clockThis is an important
assumption because it introduces the element of distributed in the
system and gives rise to the inherent a synchrony( ) Between the
processors
2. No shared memoryThis is a key feature that requires message
passing for communication this feature implies() the absence()Of
the common physical clock
3. Geographical separationIts not necessary for the processors
to be on a wide area network(WAN) recently the Network/cluster of
work station(NOW/COW) configuration connecting processor on LANThe
NOW configuration is becoming popular because of the low-cost and
high speed. The google search engine based on the NOW architecture
4. Autonomy and heterogeneityThe processors are loosely coupled in
that they have different speeds and each can be running a different
operating system but they cooperate with another by offering
services or solving a problemThe motivation for using distributed
systemInherently distributed computations ( )In many applications
such as money transfer in banking, that reach consensus() among
parties( ) that are geographically distant.
2. Resource sharing( ) Resources such as peripherals, complete
data sets in databases, special libraries, as well as data
(variable/files) cannot be fully replicated at all the sites
because it is often neither practical Therefore, such resources are
typically distributed across the system. For example, distributed
databases such as DB2 partition the data sets across several
servers in addition to replicating them at a few sites 3. Access to
geographically remote data and resources ( )In many scenarios, the
data cannot be replicated at every site participating in the
distributed execution because it may be too large or too sensitive
to be replicated. For example, payroll data within a multinational
corporation is both too large and too sensitive to be replicated at
every branch office/site. It is therefore stored at a central
server which can be queried by branch offices. Similarly, special
resources such as supercomputers exist only in certain() locations,
and to access such supercomputers, users need to log in
remotely.
4. Enhanced reliability( ) A distributed system has the inherent
potential( ) to provide increased reliability because of the
possibility of replicating resources and executions, as well as ( )
the reality that geographically distributed resources are not
likely to crash/malfunction( ) at the same time under normal
circumstances. Reliability entails several aspects:Availability:
the resource should be accessible at all times;integrity: the
value/state of the resource should be correctfault-tolerance:the
ability to recover from system failuresWhen the system meet one of
the failure model that explain later
5. Increased performance/cost ratio (( / By resource sharing and
accessing geographically remote data and resources, the
performance/cost ratio is increased. Although higher throughput has
not necessarily been the main objective behind using a distributed
system any task can be partitioned across the various computers in
the distributed system6. Scalability() As the processors are
usually connected by a wide-area net- work, adding more processors
does not effected for the communication network.
7. Modularity and incremental expandability( ) Heterogeneous
processors may be easily added into the system without affecting
the performance, as long as those processors are running the same
middleware algo- rithms. Similarly, existing processors may be
easily replaced by other processors.
Types of MultiprocessorsThere are two types of multiprocessors
on the way their memory is organized. 1.Tightly coupled
multiprocessors.2.loosely coupled multiprocessors.
Tightly coupled multiprocessors1. Tightly coupled
multiprocessors
Tightly coupled multiprocessorsIn tightly coupled main memory is
shared by all processors i.e. HIGH DEGREE OF RESOURCE SHARING.
This is used to speed up the execution of a single large program
in time critical operations.
all processors share main memory &access to I/O devices and
are interconnected by a bus.
It is characterized by uniform memory access time.
Loosely coupled multiprocessors
2.Loosely coupled multiprocessors
Loosely coupled multiprocessorsIn loosely coupled multiprocessor
each processor has its own memory (i.e. faster access to its local
or own memory).
processor interact through interconnection structure or by
message passing using standard primitive functions send() &
receive().
The access to remote memory attached to other processors takes
longer time due to added delay through the interconnection
network.
It is characterized by non-uniform memory access time.
DIFFERNCE BETWEEN TIGHTLY AND LOOSELY COUPLED
MULTIPROCESSOR1)Tightly coupledMultiprocessor systemscontain
multiple CPUs that are connected at the bus level. These CPUs
mayhave access to a central shared memory or may participate() in a
memory hierarchy with both local and shared memory (NUMA).
1)Loosely-coupledmultiprocessor system(often referred to as
clusters) are basedon multiple standalone() single or dual process
or interconnected viaa high speed communication system 2)
Tightly-coupled systems perform better & are physically smaller
than loosely-coupled system3) More expensive.4) In a
tightly-coupled system, the delay Experienced , when a message is
sent from one computer to another is short and data rate is high;
that is, the number of bits per second that can be transferred is
large.(2) Loosely coupled system is just Opposite of tightly
coupled system.3) Less expensive.(4)In a loosely-coupled system,
the opposite is true: The inter-machine message delay is large and
the data rate is low.DIFFERNCE BETWEEN TIGHTLY AND LOOSELY COUPLED
MULTIPROCESSOR5) For example, two CPU chips on the same printed
circuit board and connected by wire etched onto the board are
likely to be tightly Coupled.
5) For example two computers connected by a2400 bit/sec modem
over the telephone system are certain tobe loosely coupled.Relation
to parallel multiprocessor/multicomputer systems
The main question is the system still a distributed system, or
does it become a parallel multiprocessor system? To better answer
these questions, we first examine the architecture of parallel
systems
Characteristics of parallel systems
A parallel system may be classified as belonging to one of three
types:
1. A multiprocessor system: is a parallel system in which the
multiple processors have direct access to shared memory which forms
a common address space. The architecture is shown in the following
Figure . Such processors usually do not have a common clock.
pc1pc3pc2SharedmemoryCharacteristics of parallel systemsA
multiprocessor system usually corresponds ()to a uniform memory
access (UMA) architecture in which the access latency, waiting
time, to complete an access to any memory location from any
processor is the same.The processors are in very close physical
proximity and are connected by an interconnection network The
communication across processors is traditionally through read and
write operations on the shared memorythe processors usually run the
same operating system, and both the hardware and software are very
tightly coupled.
Characteristics of parallel systemsExamples of multiprocessor
systemOmega network and butterfly(banyan) network :each of which is
a multi-stage network formed of 2 2 switching elements. Each 2 2
switch allows data on either of the two input wires to be switched
to the upper or the loweroutput wire. In a single step, however,
only one data unit can be sent on an output wire. So if the data
from both the input wires is to be routed to the same output wire
in a single step, there is a collisionEach 2 2 switch is
represented as a rectangle in the following figure
Characteristics of parallel systems
Characteristics of parallel systems2. A multicomputer parallel
system: Is a parallel system in which the multiple processors do
not have direct access to shared memory. The memory of the multiple
processors may or may not form a common address space The
processors are in close physical proximity and are usually very
tightly coupled (homogenous hardware and software), and connected
by an interconnection network. Such computers usually do not have a
common clockA multicomputer system that has a common address space
usually corresponds to a non-uniform memory access (NUMA)
architecture in which the latency to access various shared memory
locations from the different processors varies.
Characteristics of parallel systemsExamples of multicomputer
systemWrap-around 2D-mesh(torus) and Hypercube : For a k k mesh
whichwill contain k2 processors, the maximum path length between
any two processors is 2(k/2 1).While A k-dimensional hyper- cube
has 2k processor-and-memory units . Each such unit is a nodein the
hypercube, and has a unique k-bit label. Each of the k dimensions
is associated with a bit position in the label. The labels of any
two adjacent() nodes are identical() except for the bit position
corresponding to the dimension in which the two nodes differ.
Characteristics of parallel systems
Characteristics of parallel systems3. Array processors: Belong
to a class of parallel computers that are physically very tightly
coupled, and have a common system clock (but may not share memory
and communicate by passing data using messages). The primary and
most effective use of parallel systems is for obtaining a higher
throughput by dividing the computational workload among the
processors. The tasks that are most amenable( ) to higher speedups
on parallel systemsFlynns taxonomy
Flynn identified four processing modes, based on whether the
processors execute the same or different instruction streams at the
same time, and whether or not the processors processed the same
(identical) data at the same timeSingle instruction stream, single
data stream (SISD)This mode corresponds to the conventional
processing in the von Neumann paradigm with a single CPU, and a
single memory unit connected by a system busSingle instruction
stream, multiple data stream (SIMD)This mode corresponds to the
processing by multiple homogenous processors which execute in
lock-step on different data itemsMultiple instruction stream,
single data stream (MISD)This mode corresponds to the execution of
different operations in parallel on the same data. This is a
specialized mode of operation with limited Multiple instruction
stream, multiple data stream (MIMD)In this mode, the various
processors execute different code on different data. This is the
mode of operation in distributed systems as well as in the vast
majority( ) of parallel systems. There is no common clock among the
system processors
. MIMD architectures are most general and allow much flexibility
in partitioning code and data to be processed among the
processors
OSI Open Systems Interconnect
Rapid growth of computer networks caused compatibility
problemsISO recognized the problem and released the OSI model in
1984OSI stands for Open Systems Interconnection and consists of 7
LayersThe use of layers is designed to reduce complexity and make
standardization easier
The 7-layers7. Application6. Presentation5. Session4.
Transport3. Network2. Data Link1. PhysicalMnemonics7.)
(A)pplication(A)ll6.) (P)resentation(P)eople5.) (S)ession(S)eem4.)
(T)ransport(T)O3.) (N)etwork(N)eed2.) (D)ata Link(D)ata1.)
(P)hysical(P)rocessingDivision of LayersUpper LayersLower
LayersMiddle Layer7. Application6. Presentation5. Session4.
Transport3. Network2. Data Link1. Physical The first three layer
which called upper layers it is specifiedFor user application and
programming
The middle layer it is work as interface between upper layer and
lower layer
While the lower layer it is specified for bit transfer from data
and swap it between networksThe Role of Layers in Point-to-point
Communication7. Application1. Physical7. Application1.PhysicalNode
aNode bENCAPSULATIOnDEENCAPSULATIOn
7-layersApplication layer:In this layer the user will control on
it directly The Purpose of using this layer it is convert User
application to network service interfaceExample E-mail
servicesPresentation Layer:This layer response for Data formatting
and give it to the next layer in suitable wayThe purpose of this
layer is to formats data for exchange between points of
communicationEx: Between nodes in a network
7-layersSession Layer:Sometimes called the dialog controller,
this layer establishes, maintains, and terminates sessions between
applicationsThis layer is response for define the devices and its
name and gives repot about the connection In this layer defines
checkpoints for acknowledgements during sessions between
applications
7-layersTransport Layer:in this layer the data will separate
into segments and it is Repackage proper( ) and efficient delivery
of packages the package should be Without errorIn sequenceWithout
duplication
7-layersNetwork layer:It is responsible to address messages and
translate the logical address to the physical address that the
network understand it Ex: nganesa ----------> 102.13.345.25In
this layer it choice Best path at the time of transmissionSo the
routers its work here7-layersData Link Layer:The purpose of this
layer is Manages the flow of data over the physical media and its
responsible for error-free transmission over the physical mediaIn
this layer the data will be separate into smaller parts called
Frame in this layer two parts will add into data first one called
Header the second one is called Trailer
DataheaderTrailer7-layersPhysical Layer:Purpose deals with the
transmission of 0s and 1s over the physical mediaTranslation of
bits into signals
Computer Networks LAN ,WANWAN
LAN
Wide Area NetworkLocal Area NetworkStands ForLarge geographic
areas (e.g., cities, states, nations)Local areas only (e.g., homes,
offices, schools)Covers
WAN (Wide Area Network) is a computer network that covers a
broad area (e.g., any network whose communications links cross
metropolitan, regional, or national boundaries over a long
distance).LAN (Local Area Network) is a computer network covering a
small geographic area, like a home, office, school, or group of
buildings.Definition
Less speed (150 mbps)High speed (1000 mbps)SpeedComputer
Networks LAN ,WANWANs have a lower data transfer rate compared to
LANs.LANs have a high data transfer rate.Data transfer ratesThe
Internet is a good example of a WANThe network in an office
building can be a LANExampleComputers connected to a wide-area
network are often connected through public networks, They can also
be connected through leased lines or satellites.One LAN can be
connected to other LANs over any distance via telephone lines
Connection
Layers 3 devices Routers, Multi-layer SwitchesLayer 2 devices
like switches and bridges. Layer 1 devices like hubs and
repeaters.ComponentsComputer Networks LAN ,WANWANs tend to be less
fault tolerant as they consist of large number of systems.LANs tend
to have fewer problems associated with them, as there are smaller
number of systems to deal with.Fault Tolerance
Experiences more data transmission errors as compared to
LANExperiences fewer data transmission errorsData Transmission
ErrorWANs (like the Internet) are not owned by any one
organizationTypically owned, controlled, and managed by a single
person or organization.OwnershipFor WANs since networks in remote
areas have to be connected the set-up costs are higherIf there is a
need to set-up a couple of extra devices on the network, it is not
very expensive to do that.Set-up costsComputer Networks LAN
,WANHave a large geographical range generally spreading across
boundaries and need leased telecommunication linesHave a small
geographical range and do not need any leased telecommunication
linesGeographical Spread
Maintaining WAN is difficult because of its wider geographical
coverage and higher maintenance costs.Because it covers a
relatively small geographical area, LAN is easier to maintain at
relatively low costs.Maintenance costs
Low bandwidth is available for transmission.High bandwidth is
available for transmission.Bandwidth
More congestionLess congestion(Congestion(EthernetEthernet is a
certain type of a local area network (LAN) which was developed in
1972 in the renowned PARC-research facility of Xerox in Palo Alto
by Robert Metcalfe. In the meantime the companies Intel, DEC and
Xerox have specified a common standard that has been established in
the IEEE-standard 802.3.
Medium
Medium
Medium
Ethernet Technologies: 10Base2Thin coaxial cable in a bus
topology
10BaseT and 100BaseT10/100 Mbps rateT stands for Twisted
PairHub(s) connected by twisted pair facilitate star topology
Distance of any node to hub must be < 100M
Collision
CSMA/CD
Collision
Access method: CSMA/CD
Back-Off AlgorithmIf a collision has occurred, the stations try
to send again after a certain period of time.After the first
collision there a two different back-off times available, from
which one is chosen at random. Transmission probability is 50%After
the second consecutive collision there are four different back-off
times available, from which one is chosen at random.The
transmission probability now is 75%
Segmentation
62 Segmentation
Segmentation
Client server modelServer: Is a process implementing specific
service For example database serviceClient: Is a process that
request service from a server by sending it request and
subsequently waiting for servers replay
This known client server interaction also known Request- reply
behavior
The communication between a client and a server can be
implemented by means of simple connectionless protocolRequest reply
behaviorClient waiting for result Request Reply
Server provide Time service
When a client request a service ,it simply packages a massage to
the server , identifying the service it wants , along with the
necessary input data Request reply behaviorThe massage is then send
to the server.After the massage is sent the client get suspend
until it get replyThe server now is weak up and provide service ,
its work on the data and then send reply to the client After sent
replay the server will be free to get more requestAnd the client
get resume and continue its work
Request reply behaviorUsing connectionless protocol has the
advantage of being efficient as long as messages do not get lost or
corruptedMaking the protocol resistant to occasional transmission
failures is not trivial The only thing we can do is possibly let
the client resend the request when no reply message comes in.The
problem is that the client cannot detect whether the original
request message was lost or that transmission of the reply failedIf
the reply lost then resending a request may result in performance
the operation twiceExample if operation was something like transfer
10,000 $ As the alternative many client server systems use a
reliable connection oriented protocol . Although this solution is
not appropriate in local area network because it give low
performance but its works perfectly fine on the wide area
network
Reliable connection oriented protocol: when a client request
service it first sets up a connection to the sever before sending a
request . The sever use same connection to send the reply massage .
After that the connection get downClient Server ArchitecturesThe
client server can be organize in two type of machine
1. A client machine containing only the programs that
implementing on the user interface level2. A server machine
containing the rest that is the programs implementing the
processing and data level
The problem with this organization is not really
distributedEverything is handled by the server, while the client is
no more than a dumb terminal three tiered architectureUser
interface wait for result (presentation)
REQUEST Return result operation wait for data Application
Request Return server data data Database server
At first time the client that in user interface level will send
request to the sever to get the resultAfter sending request the
client get suspend until the result get back
When the request reach to the Application server the server get
resume three tiered architectureThe server continue its work with
processing level suppose the request needed some data to
perform
Now the server will send data request for the data level and get
suspend until the data reach
The database level will send the data to processing level and
the server continue its work
After performing the request it will be sent to client and the
client get resume and continue its work .Remote Procedure
CallRemote Procedure Call (RPC) is aprotocolthat one program can
use to request a service from a program located in another computer
in a network without having to understand network details.
(Aprocedure callis also sometimes known as afunction callor
asubroutine call.) RPC uses theclient/servermodel.
The requesting program is a client and the service-providing
program is the serverRPC b/w client and serverClient waiting for
result
Call remote Return Procedure from call
Server Call local procedure time and return result Client and
Server StubsSuppose that a program needs to read some data from a
file. The programmer puts a call to read in the code to get the
data in the traditional (single-processor) system.
The read routine is extracted from the library by the linker and
inserted into the object program .After the programmer calling
procedure then mean put the parameter in a stackAll the above
procedures its controlled by client stubeIts pass a parameter into
message & send it to the serverClient and Server StubsWhen the
message arrives at the server, The server operating system pass
message into server stub.
server stube its a piece of code that transforms request coming
over the network into local procedure callsThe server performs its
work and returns the result to the caller in usual wayWhen the
message gets back to the client machine the clients OS sees the
address of the client and then the message puts into waiting buffer
and client process unblocked.Client and Server Stubs
Asynchronous RPCWhen a client calls a remote procedure the
client will block until a reply is returned this request-reply
behavior is unnecessary when there is no result to return and only
leads to blocking the client while it could have proceeded and have
done useful workTo support such situations,RPC system provide
facilities that called asynchronous RPCs by which client
immediately continues after issuing the RPC request . With
asynchronous RPCs the server immediately sends a reply back to the
client the moment the RPC request is received The reply acts as an
acknowledgement to the client The client will continue without
blocking as soon as received the server acknowledgementAsynchronous
RPCThe following figure shows the interaction between client and
server using asynchronous RPC
Client Wait for acceptance Call remote Return from call
procedure Request Accept Request
Server call local Time procedure
Message-Oriented Communication
There are two type of communicationPersistent versus
transientSynchronous versus asynchronous
Persistent: messages are held by the middleware comm. service
until they can be delivered (e.g., email)Sender can terminate after
executing sendReceiver will get message next time it runs
Transient: messages exist only while the sender and receiver are
running
Message-Oriented CommunicationAsynchronous: (non-blocking)
sender resumes execution as soon as the message is passed to the
communication/middleware software
Synchronous: sender is blocked until The OS or middleware
notifies acceptance of the message, orThe message has been
delivered to the receiver, orThe receiver processes it &
returns a response
Message-Oriented Communication
(a): persistent asynchronous communication(b): persistent
synchronous communication (c): Transient asynchronous
communication(d): receipt-based Transient synchronous
communication(E): delivery-based Transient synchronous
communication(F): response-based Transient synchronous
communication
