Chapter 5

Middleware

Topics Server structures External data representation Remote procedure call (RPC) Interface definition language (IDL) Java RMI Java security Failures and service guarantees Stateless and stateful servers Naming and name look-up service (DNS)

RPC & RMI

Layered Communication Architecture

Server Structures

Server Components

Examples Ex1: Disk server

Listener and one worker. After servicing current request, worker checks list.

Ex2: Window server X Window server manages several windows:

issues select to constantly watch events on keyboard and mouse. If, e.g., right_mouse_button_down is detected, it creates worker thread, which pops up appropriate menu and executes selected command.

Web Server

Dispatcher thread

Web server process

Worker threads

Web page cache

User space

kernelNetwork

Hard Disk

•Dispatcher thread: reads incoming requests•Worker threads: downloads requested pages

Spreadsheet

Thread 1:Update page 1

Listen to keyboard

Go to page 600

Thread 2:reformatting

All threads in the same process share same resources

Thread 3:Backup

Kernel

Produce-Consumer Relation Listener is producer Workers are consumers If the request list is empty, worker thread calls

method wait( ) in monitor on condition list_nonempty

When listener places new request, it invokes notify( ), which wakes up waiting worker thread, if any.

Other ways for mutual exclusion/synchronization?

UNIX inetd Daemon (listener)

Invoked at boot time Consulting /etc/inetd.conf file, creates one socket for

each service and binds port number. Invokes select on socket descriptors

select ( …, *readfds, *writefds, …, *timeout) readfds points to bit pattern of file descriptors (“1” if fd is of

interest). Upon return, bit pattern indicates fd’s which are ready for

input. inetd invokes accept on active descriptors and forks

worker thread to serve request; Go back to select.

/etc/inetd.conf

Design Options Single-threaded server (for simple services) Listener and one thread per request

Upon receiving request, listener spawns worker. Upon completion worker dies. No contention for shared queue Thread creation/destruction overhead

Listener and fixed number of worker threads Each worker thread checks request buffer and services

request. Idles if no request. Limited concurrency

Real-life example: SQL Server 2000

Remote Procedure Calls

invocation invocationremote

invocationremote

local

local

local

invocation

invocation

A B

C

D

E

F

•Purpose: call the procedures that are locally unavailable.

•Any other way(s)?

RPC Flow

Invocation Semantics Maybe: the invoker cannot tell whether a

remote method has been executed once or not al all.

Supported by CORBA At-least-once: re-transmission of request until

reply arrives. Require idempotent operation Supported by Sun RPC

At-most-once: the invoker receives either a result or an exception.

Require more fault-tolerance measures Supported by Java RMI and CORBA

Sun RPC Message Format Uses UDP

Other UNIX versions use TCP At-least-once call semantics XDR

Request xid (unique tag) prog (100003 for NFS protocol) vers (version no.: 2, 3 or 4 for NFS) proc (procedure no.; see Fig. 5.8 of Text) Authentication info (client machine name, userID, etc.) Arguments for procedure ….

Restrictions Not support interface name Single input parameter Not scale well

RPC Components Interface defined in Interface Definition

Language (IDL) Client stub and server stub (also called

skeleton) make remote calls transparent, i.e., hide communication from users.

Stub generator (or ID compiler) generates stubs by compiling interface definition.

Issues to be considered Binding Security

IDL No existing language has all types in all other

languages C cannot be IDL: char* in C cannot be translated uniquely to

pointer to character, or pointer to array of characters. New I/O types, in, out, inout, are useful in IDL. CORBA, Sun XDR, DCOM, DCE, MiG (subset of

Matchmaker), Courier, have their own IDLs. Java can function as IDL for Java programs CORBA allows client and server programs to be

written in different languages.

IDL Example (Sun XDR)const MAX = 1000;typedef int FileIdentifier;typedef int FilePointer;typedef int Length;struct Data {

int length;char buffer[MAX];

};struct writeargs {

FileIdentifier f;FilePointer position;Data data;

};

struct readargs {FileIdentifier f;FilePointer position;Length length;

};

program FILEREADWRITE { version VERSION {

void WRITE(writeargs)=1; 1Data READ(readargs)=2; 2

}=2;} = 9999;

Stubs Marshalling (serializing or flattening)

and unmarshalling (de-serializing) Also required for persistent object stores

Data format conversion between heterogeneous systems (Big endian, little endian, ASCII, EBCDIC, etc.) In Sun, converted to XDR. Two ways to do data conversion

Client Stub

Marshall procedure id and parameters. Send message to server and wait by

blocking receive. Unmarshall returned result Deliver it to calling process.

Server Stub (Skeleton) Unmarshall message and extracts

procedure id and parameters Security checking

Invoke called procedure using standard procedure call logging

Compose reply message (marshalling) Send it back to client

Stub Generator (Compiler) Stub generator generates stubs by compiling

interface definition. E.g.,rmic (Java), rpcgen (SunOS), MiG (Mach), uuigen (OSF DCE).

Generated stubs are linked to client and server programs Server and client stubs could be in different

languages. Only interfaces are included in stubs.

rpcgen

Locating Server Sun RPC uses portmapper at well-known port

(111) on each computer It lists program#, version#, and port# used by each

local service On start up, server registers with local portmapper,

providing this info. Client sends prog# and version# to remote

portmapper to find server port Q: why don’t specify the port number in the

interface?

Server Port Lookup

1: register program, version, and port;

2: look up port number;

3: port# returned;

4: request/reply

Q: How to broadcast messages to multiple instances of a service on different machines?

Portmapper Interface Definition

program PMAP_PROG {

version PMAP_VERS {

void PMAPPROC_NULL(void) = 0;

bool PMAPPROC_SET(mapping) = 1;

int PMAPPROC_GETPORT(mapping)= 3;

} = 2;

} = 100000;

Authentication

Needed for request and reply UNIX style: uid, gid Digital signature Kerberos style

Widely used in intranet (campus, company)

The default authentication service in Windows 2000

General Service Lookup

Steps for RPC

Define the RPC Interface in a .x file.

Use rpcgen to compile the .x file Code the server implementation Build and run the server Write a client

Step 1: Define the RPC Interface in MyRPCService.x file

struct aTypeStruct { int i; double d; };typedef struct aTypeStruct aType; program MyRPCService { version ThisVersion {

aType function_foo(int) = 1; int function_bar(aType) =2;

}=1; }=34567;

Step 2: Use rpcgen to compile the MyRPCService.x file % rpcgen MyRPCService.x

MyRPCService.h --- the common header file for the RPC, read the generated comments.

MyRPCService_xdr.c --- the XDR routine for struct aType. This file will be used by both the client and the server.

MyRPCService_cln.c --- The generated client stub. An RPC client application calls the functions defined in this file.

MyRPCService_svc.c --- The generated server stub. MyRPCService_imp.tmp.c --- The server

implementation template, you copy this file and fill in the server implementation.

MyRPCService.hpp ---- The C++ class for the client MyRPCService.mak ---- Makefile, you need to edit this

Step 3: Code the server implementation

the RPC server programmer must implement the functions defined in MyRPCService.x file to do the real work. rpcgen generates a server

implementation template file, you can copy it over and fill in the details.

Step 4: Build and run the server

Compile the MyRPCService_xdr.c, MyRPCService_svc.c and MyRPCService_imp.c files

Link with the pwrpc32.lib library to build executable file

Start the portmapper if necessary Start the server we just built

Step 5: Write a client (simplified version)

int i; aType a, *ap; CLIENT * cl = clnt_create

("server_host_name", program_number, version_number,

"tcp"); ap = function_foo_1(&i, cl); i = function_bar_1(&a, cl);

Summary of RPC Widely used before. Now HTTP plays the

major role.

Strict request-reply paradigm Not good for certain relational database

applications

Complexity of server Responsible for almost all stuff: security, logging,

service

Difficult to implement complex service

Java RMI

interfaceremote

m1m2m3

m4m5m6

Data

implementation

remoteobject

{ of methods

•Don’t need a special IDL

•Must extend the java.rmi.Remote interface

Java RMI Example

import java.rmi.*…….public interface Find extends Remote {

public String findLine ( String keyword )

throws RemoteException;}

Server Implementationpublic class FindImpl extends UnicastRemoteObject implements Find {

public FindImpl ( String aFile ) throws RemoteException { } //end constructor

public String findLine ( String keyword ) {

This method finds if keyword is in aFile

} //end method

public static void main ( String args[ ]) {

try {

System.setSecurityManager ( new RMISecurityManager () ); FindImpl server = new FindImpl(aFile); Naming.rebind(“127.0.0.1:2099/FindServer”, server); …..}

catch {……..}

} //end method

} //end class

Java Registry Overview

Stub Generator

rmic Generate both server and client stubs Move server stub to the codebase Move client stub to the client machine

Required for compilation.

Service Lookup And Sending Request by Client

public class FindClient {

public static void main (String args[] ) {

……

String name = “rmi://” + “hostName:2099” + “/FindService”;

Find obj = ( Find ) Naming.Lookup (name);

String results = obj.findLine ( args[0] );

……

}

}

Compiling and Running

Classes Supporting Java RMI

RemoteServer

UnicastRemoteObject

<servant class>

Activatable

RemoteObject

Java Security

In Java, executable contents can be shipped to other hosts and executed, e.g., applets. Implication? A new problem?

Degree of Trust The base classes

i.e., Java’s built-in classes, e.g., java.lang.*

Local classes (on CLASSPATH) Remote classes (remote for the

standpoint of the client) e.g., applets

Trusted classes: base and local Untrusted class: remote

Default Restrictions on Applets Run in a restricted environment (e.g.,

sandbox) Not allowed to access the local file

systems Except for loading trusted classes that are

locally available Can only access data/programs from the

remote machine where it came from. Prevent using the client as the platform to

attack other machines

Sandbox

Sandbox (more details)

Security Policy System policy file

System wide, apply to every user E.g., java.home/lib/security/java.policy Grant all permissions to standard extensions Allow anyone to listen on un-privileged ports Allow any code to read certain “standard”

non-security-sensitive properties.

System Policy File

//standard extensions get all permissions by defaultgrant codeBase file://${java.home}/lib/ext/*

{ permission java.security.AllPermission;};

grant{permission java.net.SocketPermission “localhost:1024-”, “listen”;

permission java.util.PropertyPermission “java.version”, “read”;

permission java.util.PropertyPermission “java.vendor.url”, “read”;

permission java.util.PropertyPermission “os.name”, “read”;

permission java.util.PropertyPermission “os.version”, “read”;

};

User Policy File

Apply to a single user E.g. /user.home/.java.policy System policy is loaded first, then the

user policy is added to it If neither policy is present, a built-in

sandbox policy is used.

User Policy Filegrant{permission java.net.SocketPermission “*:1024-”, “connect, accept”;permission java.net.SocketPermission “*:80”, “connect”;

};

//FilePermission

grant{permission java.io.FilePermission “*C:\\ann\\classes\\-”, “read”;permission java.io.FilePermission “*C:\\jones\\classes\\-”, “read, execute”;};

//By owner

grant signedBy “tiko”

{ permission java.net.SocketPermission “localhost:1024-”, “connect”;

};

Security Manager (SM)

Enforce the security policy An object of class

java.lang.SecurityManager Check if an operation is permitted

Yes: simply return No: throw a SecurityException

AppletSecurityManager Installed in browsers (e.g., IE) Part of JVM initialization Check applets downloaded over network Target: preventing

Uninvited retrieval of data checkRead method

Overriding trusted built-in classes Using client machine to attack other systems Changing the environment Adware, data-miner?

Check Methods for RMISecurityManager public synchronized void

checkConnect(String host, int port) public synchronized void checkListen(int port) public synchronized void checkRead(String

file) public synchronized void checkExec(String

command) Security exception thrown by SM if policy is

violated

Installing SM Method 1:

Java –Djava.security.manager myApp Java –Djava.security.manager=MySecMgr

myApp Method 2:

System.setSecurityManager (new SecurityManager())

System.setSecurityManager( ) method in java.lang.System class is normally used to install non-default SM.

RMI Security RMI runtime requires that SM be explicitly set

by System.setSecurityManager (new RMISecurityManager( )) before loading any stubs or other classes over network.

This SM enforces policy similar to AppletSecurityManager

Can be overridden by java -Djava.security.policy==mypolicy

where mypolicy may contain:grant { permission java.security.AllPermission;};

Both client and server should be started this way.

JVM

Class Loader

Gate keeper: check and control byte code Primordial class loader

Written in a native language Platform dependent

Java.lang.ClassLoader loadClass() defineClass()

ClassLoaders Primordial class loader

For classes part of the Java API URLClassLoader

For classes from CLASSPATH AppletClassLoader

Each applet has its own loader Search order: 1) Java built-in classes 2) local

classes 3)Remote classes RMIClassLoader

Loading stub classes & associated utility classes

Class Verifier File integrity check

Lengths match Class integrity check

Having a non-final superclass, etc Bytecode integrity check

Illegal instructions Incorrect types, illegal accesses Over(under)-flowing the program stack

Server Addressing (host-addr, port#)

IP addresses: used in Internet and universally accepted.

Port# indicates the port on which the server is listening

1-1023: reserved for system-provided services

In UNIX, register the service by calling the portmapper which listens on the well-known port 111.

rmiregister in Java.

Name Service

Problems with numeric addresses: Difficult to remember Tightly-bounded with host

Solution: assigning names to hosts Require name server for name

resolution Similar to rmiregistry, but more scalable

DNS

Work in large-scale distributed systems

Features: Hierarchical textual names Distributed name-mapping under local

control Little overhead of changing database Scalable, robust and adequate

performance

Internet Domain Names

E.g.: dogwood.css.sfu.ca Each node (component) represents a

domain. Top-level domains: partitioned both

organizationally and according to geography.

E.g., com, edu, int, us, uk, ca

A Partial Tree

Root

13 Root Servers (as of 2000)

Zones and Their NSs

Distributed NSs

Main issue: robust, no-bottleneck, scale well

For each zone: A primary NS One or more secondary NSs

Loading-share robust

Resource Records (RR)

wormhole.cs.sfu.ca 2419200 IN A 199.60.3.48 wormhole.cs.sfu.ca 86400 IN MX 1 fornax.cs.sfu.ca www.cs.sfu.ca 2419200 IN CNAME adhara.cs.sfu.ca

[www.cs.sfu.ca is an alias] Address mapping for a NS is recorded as a pair:

. 3600000 IN NS A.ROOT-SERVERS.NET.

A.ROOT-SERVERS.NET. 3600000 IN A 198.41.0.4

Domain_name

Time_to_live

Class Type Value

RR TypesDomain_na

meTime_to_liv

eClass Type Value

Recursive Name Resolution

Iterative Name Resolution

Iterative name resolution

A user on momiji.cs.sfu.ca is looking for a host called xyz.lcs.mit.edu.

Failures in DS

1: Unable to locate the server 2: Server crash 3: Request message lost 4: Reply message lost 5: Client crash, after sending a

request 6: Focus on server failures

Failure Handling: Example

The farmer types in command to sprinkle his farm with100Kl of water. Assume his home (client) machine is reliable, but server machine and communications are not.

State of the Client

S0: no reply outstanding and can send a new request

S1: waiting for a reply

S0 S1

Request

Reply

At Server Side 3 possible actions/events

S: activate sprinkler A: send acknowledgement C: crash S and A are not atomic.

Two possible sequences SA: Response-based synchronous communication AS: Receipt-based or Delivery-based

Context: Client is in state S1, and it receives a re-start message from the server.

Q: what should the client do?

Server Adopts AS

Server Adopts SA

Implications If a system affects the state of an external

object, recovery from system crash is possible only by larger system containing the external

object, and only if larger system retains enough info.

E.g., a robot equipped with video camera Different from database transactions

Recovery is possible if external object is not affected by crash.

Idempotent Operations

Can be made repeatable without changing the end effect Pay the bill ignore this bill if already

paid Increment X change X to 15 Read file F at the current pointer

Read file F at the position 56 Not all operations are inherently non-

repeatable

Stateful & Stateless File Server Stateful:

Maintain a list of open files Current position of file pointer Better performance. But how about a crash?

Stateless: Client maintain the states Each request must be self-contained Lower performance. But good at handling

crash? Q: how about file sharing?

Stable Storage Model No disks work all the time with no errors Achievable: either correctly writes the data

or does nothing, leaving the existing data intact.

Possible errors (model assumptions): Incorrect write: detected by ECC Bad sector: ignore the chance that two

independent drives have the same sector going bad

No disaster Guarantee: at system crash, either write

correct data or leave the old data in place

Model: Hardware

A pair of identical disks Corresponding blocks work together

to form one error-free block If no error, the corresponding blocks

on both drives are the same If error occurs, at least one block can

provide the correct data (either new or old)

Model: Stable Writes/Reads Stable writes

Write the block in drive 1read and verify Keep writing until it successes

Write the block in drive 2read and verify Stable reads

Read the block from drive 1. If incorrect (detected by examining ECC), the corresponding block in drive 2 must contain the correct data. Why?

Stable writes leave two good copies Both copies go bad: impossible (assumption 2)

Model: Crash Recovery Compare corresponding blocks

Both good: nothing is done One is bad: overwrite the bad copy Both good but different: block 1 overwrites block2.

Why?

Old Old

Disk1 2

Crash

Bad

Old

Disk1 2

Crash

New Old

Disk1 2

Crash

New

Bad

Disk1 2

Crash

New New

Disk1 2

Crash