11 October 2007Kaiser: COMS W4156 Fall 20071 COMS W4156: Advanced Software Engineering Prof. Gail Kaiser [email protected]

11 October 2007 Kaiser: COMS W4156 Fall 2007 1

COMS W4156: Advanced Software Engineering

Prof. Gail Kaiser

[email protected]

http://york.cs.columbia.edu/classes/cs4156/



Software Process• Requirements –

– As specified by the customer or user• Architectural Design –

– Selection of architectural elements (components), their interactions (connections), and the constraints on those elements (possibly in part required and/or enforced by component model framework)

– Selection should satisfy the requirements – Selection should serve as the base for the modular design

• Modular Design – – Detailed algorithms and data structures– Should satisfy the requirements and the architecture

• Implementation – – Representation of algorithms and data structures in code– Should satisfy the requirements, architecture and modular design

• …


What is Design?From Merriam-Webster Online Dictionary (http://www.m-w.com/dictionary/design)

Main Entry: 1de·sign Pronunciation: di-'zInFunction: verbEtymology: Middle English, to outline, indicate, mean, from Anglo-French & Medieval Latin; Anglo-French designer to designate, from Medieval Latin designare, from Latin, to mark out, from de- + signare to mark -- more at SIGN

transitive verb1 : to create, fashion, execute, or construct according to plan : DEVISE, CONTRIVE2 a : to conceive and plan out in the mind <he designed the perfect crime> b : to have as a purpose : INTEND <she designed to excel in her studies> c : to devise for a specific function or end <a book designed primarily as a college textbook>3 archaic : to indicate with a distinctive mark, sign, or name4 a : to make a drawing, pattern, or sketch of b : to draw the plans for <design a building>

intransitive verb1 : to conceive or execute a plan2 : to draw, lay out, or prepare a design - de·sign·ed·ly /-'zI-n&d-lE/ adverb


What is Design?

• The realization of an imagined state

• To work out a solution in one’s mind

• The specification that guides production

• The transition from possible solutions to a specific one

• To devise a solution to a perceived problem

• …

Design

designer project customer

outcome space

design space

CD

CC

DF SP

A design product is a point in the design spacethat represents a set of decisions that together delineate

a set of possible outcomes in the outcome space

C = conceivableF = feasibleD = desirableSP = still possible

Design

designer project customer

outcome space

design space

CD

CC

DF

C = conceivableF = feasibleD = desirableSP = still possible

A successful design product restricts the set of still possibleoutcomes to those that are both feasible and desirable

DFD


Software Design• Concerned with making major decisions, often of

a structural nature• Shares with programming a concern for

abstracting information representation and processing sequences, but at different level

• Builds coherent, well-planned representations of programs that concentrate on – the interrelationships of parts at the higher levels– the logical operations at the lower levels


Software Design

• Architectural design - “How does it all fit together?”– High-level partitioning of a software system into

separate modules (components)– Focus on the interactions among parts (connections)– Focus on structural properties (configuration)

• Modular design - “How does it work?”– Detailed design of a component (unit)– Focus on the internals of a component and

computational properties (data structures, algorithms) Important note: The “component-level design”

for the 1st progress report is here part of what is called “architectural design”

Example

Architecturaldesign

Moduledesign

Provided Interface

Audio EncoderRequired Interface

Provided Interface

TunerRequired Interface

Provided Interface

Audio PlayerRequired Interface

Com

pre

ssor

Provided Interface

Enco

der

Reader

Required Interface


Compare to Building Architecture

• Overall shape of the physical structure• Manner in which the various components of the

building are integrated to form a whole• The way in which the building fits into its

environment and meshes with other buildings in its vicinity

• Degree to which the building meets its stated purpose and satisfies the needs of its owner

Building Architecture Has Many Levels


Architectural Design

Elements• Floors• Walls• Rooms

Types• Office building• Villa• Aircraft hanger

Elements• Components• Interfaces• Messages

Types• Office automation• Game• Space shuttle

control

Buildings

Software


Architectural Design

Styles• Colonial• Cape Cod• Ranch

Rules and regulations• Electrical• Structural

Styles• Pipe and filter• Layered• Implicit invocation

Rules and regulations• Use of interfaces• Conforms to

Component Model

Buildings

Software


Software Architecture

• A high-level model of a (non-physical) thing– Describes critical aspects of the thing– Understandable to many stakeholders– Allows evaluation of the thing’s properties before it is built– Provides well understood tools and techniques for constructing

the thing from its blueprint

• A representation that enables a software engineer to– Analyze the effectiveness of the design in meeting its stated

requirements– Consider architectural alternatives at a stage when making

design changes is still relatively easy– Reduce the risks associated with the construction of the

software


aTruck aShip aAirplane theWarehouseCo llecti on

theVehicleCollection

UML-A Generated Dependency Class:theRouter Dependency (1.0)

theStorage

aVehicle


availableVehicleCollection

UML-A Generated Association Class:theVehicleCollection Generalization (1.0)UML-A Generated Association Class:theVehicleCollection Generalization (1.0)UML-A Generated Association Class:theVehicleCollection Generalization (1.0)UML-A Genera ted As socia tion C lass: theVehicleC ollec tion Genera lization (1.0)UML-A Generated Association Class:theVehicleCollection Generalization (1.0)UML-A Generated Association Class:theVehicleCollection Generalization (1.0)UML-A Generated Association Class:theVehicleCollection Generalization (1.0)UML-A Generated Association Class:theVehicleCollection Generalization (1.0)UML- A Generated Ass ociati on Cl ass:theVehi cleCo llection Generali zation (1.0 )UML-A Generated Association Class:theVehicleCollection Generalization (1.0)


availableGoods

aPort

aPortC ollec tion

aSurp lus aDifficiency

theTimeNeeded

theGoods

UML-A Generated Association Class:aWarehouse Association (0.5)


UML-A Generated Association Class:availableGoods Association (0.5)

aRouteCollection

UML-A Generated Association Class:aRoute Association (0.25)UML-A Generated Association Class:aRoute Association (0.25)UML-A Generated Association Class:aRoute Association (0.25)UML-A Generated Association Class:aRoute Association (0.25)

UML-A Generated Dependency Class:theRouter Dependency (0.5)UML-A Generated Dependency Class:theRouter Dependency (1.0)


theAWT

aVehiceDialog aWarehouseDialog aPortDialog aRouterDialog

aWarehouse

UML-A Generated Association Class:aDifficiency Association (1.0)UML-A Generated Association Class:aDifficiency Association (1.0)UML-A Generated Association Class:aDifficiency Association (1.0)UML-A Generated Association Class:aDifficiency Association (1.0)

UML-A Generated Association Class:aDifficiency Association (1.0)U ML-A Generated Association Class:aD ifficiency Associ ation (1.0)UML-A Generated Association Class:aDifficiency Association (1.0)UML-A Generated Association Class:aDifficiency Association (1.0)U ML-A Generated Association Class:aD ifficiency Associ ation (1.0)UML-A Generated Association Class:aDifficiency Association (1.0)UML-A Genera ted Associ ation C lass:aSurp lus Associ ation (1.0)UML-A Generated Association Class:aSurplus Association (0.5)

UML-A Generated Associ ation Class:aRoute Association (0.5)

aLocation

UML-A Generated Association Class:aNavPoint Association (1.0)



UML-A Generated Association Class:aNavPoint Association (0.5)UML-A Generated Association Class:aNavPoint Association (0.5)

UML-A Generated Association Class:aWarehouse Association (0.5)aNavPoint


UML-A Generated Association Class:aWarehouse Association (0.5)UML-A Generated Association Class:aWarehouse Association (0.5)



UML-A Generated Association Class:aRoute Association (0.5)

aRoute

UML-A Genera ted Dependency C lass :aRou teCol lection Ass ociation (0.25)



UML-A Generated Dependency Class:aRouteCollection Association (0.5)

UML-A Generated Association Class:aNavPoint Association (1.0)UML-A Generated Association Class:aNavPoint Association (1.0)UML-A Generated Association Class:aNavPoint Association (1.0)UML-A Generated Association Class:aNavPoint Association (1.0)UML-A Generated Association Class:aNavPoint Association (1.0)



UML-A Generated Dependency Class:theRouter Association (0.25)


theCargoRouter

UML-A Generated Association Class:theRouter Association (0.25)

UML-A Genera ted As socia tion C lass: theWarehouseCo llection Dependency ( 0.25)



t heRou ter

UML-A Generated Association Class:theWarehouseCollection Dependency (0.5)

UML-A Generated Association Class:theWarehouseCollection Dependency (0.5)

UML-A Genera ted Dependency Class :aRouteCollection Ass ociation (0.5)UML-A Generated Association Clas s:theWarehouseCollec tion Dependency (0.5)

UML-A Generated Association Class:theVehicleCollection Dependency (0.5)UML-A Generated Association Class:availableVehicleCollection Dependency (0.5)UML- A Generated Generaliz ation Class :avail ableVehicleCollection Dependenc y (1.0 )


UML-A Generated Dependency Class:theRouter Association (0.5)UML-A Generated Dependency Class:theRouter Association (1.0)


UML-A Generated Dependency Class:theWarehouseCollection Dependency (1.0)

UML-A Generated Dependency Class:theRouter Association (1.0)UML-A Generated Dependency Class:theRouter Association (1.0)

What is the Problem?This is a simple

software system!


aTruck aShipaAirplane

availableVehicleCollection

theAWT

aVehiceDialogaWarehouseDialog

aPortDialog

aRouterDialog

aRouteCollection

aVehicle

theVehicleCollection

theCargoRouter

theRouter

theTim eNeeded

aRoute

aDeficiency

theWarehouseCollection

aNavPoint

theStorage

RefrigeratedStorage

RegularStorage

availableGoods

aPortCollection

aLocation

theGoods

aWarehouse

aSurplus

aPort

Design Abstraction


VehicleDel iveryPort

CargoRouter

RouterConn

GraphicsBinding : GraphicsBinding

GraphicsConn

Warehouse

ClockConn

Clock : Clock

10: notification9: notification

5: request

3: request4: request

2: notification

1: request

7: request

6: notification

8: request

Architectural Abstraction


Software Architecture: Essentials• Components

– What are the main parts?– What aspects of the requirements do they correspond to?– Both processing elements and data elements– Can be simple or composite– Examples: abstract data types, filters, databases, GUIs, servers

• Connections– How do components communicate?– Model interactions among components and rules that govern

those interactions– Don’t forget initialization/finalization dependencies– Examples: shared variables, procedure calls, messages,

multicast, pipes


Software Architecture: Essentials• Configurations

– What is the topology?– Connected graph of components and connectors that

describes architectural structure– Defines proper connectivity, concurrent and

distributed properties, adherence to architectural style

• Constraints on change (load bearing walls)– Architectural erosion – due to violations of the

architecture – increases problems and brittleness– Architectural drift – due to insensitivity about the

architecture – leads to inadaptability then disasters


Comparison to Programming (of Modules)

Architecture Modules

• interaction among parts • implementation of parts

• structural properties • computational properties

• system-level performance • algorithmic performance

• outside module boundary • inside module boundary


Comparison to Hardware Architecture

• Two important features:– Relatively small number of design elements.– Scale is achieved by replication of these design elements.

• Similarity between Software and Hardware Architectures:– Analogies to organization and configuration

• Differences between Software and Hardware Architectures:– Software requires large number of design elements.– Scale is achieved (in many cases) by adding more distinct

elements.


Comparison to Network Architecture

• Nodes and Connections act as the design elements.

• Features:– Two components – Nodes and Connections– Only a few topologies are considered (star, ring, grid).

• In software architecture,– Two components – Processes and Interprocess

communication.– Large number of possible topologies, many without

any proper names.


We Can Do Anything…

Provided Interface

Big ComponentRequired Interface

Provided Interface

Tiny ComponentRequired Interface

Provided Interface

B ComponentRequired Interface

Provided Interface

Mr. ComponentRequired Interface

Provided Interface

A ComponentRequired Interface

Provided Interface

Yet ComponentRequired Interface

Provided Interface

Some ComponentRequired Interface

Provided Interface

One ComponentRequired Interface

Provided Interface

Mrs. ComponentRequired Interface


…But Style Has Proven to Help

• Architectural “styles” restrict the way in which components can be connected– Prescribe patterns of interaction– Promote fundamental principles

• Rigor, separation of concerns, anticipation of change, generality, incrementality

• Low coupling among elements• High cohesion within elements

• Architectural styles are based on success stories– Almost all compilers are build as “pipe-and-filter”– Almost all network protocols are build as “layers”


Separation of Concerns

• “The programmer is having to do several things at the same time:

1. describe what is to be computed; 2. organize the computation sequencing into small steps; 3. organize memory management during the computation.

• Ideally, the programmer should be able to concentrate on one of the three tasks (describing what is to be computed) without being distracted by the other two, more administrative, tasks.” [Chris Reade, Elements of Functional Programming, 1989]

• Component services aim to remove additional administrative tasks (e.g., security, transactions, persistence) from the programmer’s concern


Cohesion and Coupling• Cohesion is a property or characteristic of an

individual unit• Coupling is a property of a collection of units• High cohesion GOOD, high coupling BAD• Design for change

– Reduce interdependency (coupling): You don't want a change in one unit to ripple throughout your system

– Group functionality (cohesion): Easier to find things, intuitive metaphor aids understanding


Cohesion

• The measure of strength of the association of elements within a unit

• Cohesion scale:– Functional cohesion (strongest, most desirable)– Sequential cohesion– Communicational cohesion– Procedural cohesion– Temporal cohesion– Logical cohesion– Coincidental cohesion (weakest, undesirable)


Cohesion

• Coincidental cohesion - elements have no meaningful relationship to one another

• Logical cohesion - elements perform similar activities, but the activities to be executed are chosen from outside the unit

• Temporal cohesion - elements are related in time (all should be done together)


Cohesion

• Communicational cohesion - elements perform different functions, but each function references the same input or output information

• Sequential (or informational) cohesion - elements are related such that output data from one serves as input data to the next

• Functional cohesion - elements contribute to a single, well-defined task


Coupling

• A measure of the interdependence of one software unit to another

• Coupling scale– Content coupling (worst)– Common coupling– Control coupling– Stamp coupling– Data coupling (best / minimal

needed coupling)

– No direct coupling


Coupling

• No Direct Coupling - Independent

• Data Coupling - Communicate by passing parameters

• Stamp Coupling - Communicate via a passed data structure that contains more information than necessary for the two units to perform their functions


Coupling

• Control Coupling - Communicate using at least one "control flag"

• Common Coupling - Share the same global data area

• Content Coupling – – One unit changes a statement in another (usually

applicable only to interpreted languages)– One unit references or alters data contained inside

another– One unit branches into another


…But Style Has Proven to Help

• Architectural “styles” restrict the way in which components can be connected– Prescribe patterns of interaction– Promote fundamental principles

• Rigor, separation of concerns, anticipation of change, generality, incrementality

• Low coupling among elements• High cohesion within elements

• Architectural styles are based on success stories– Almost all compilers are build as “pipe-and-filter”– Almost all network protocols are build as “layers”


Hierarchy

Connections are function or method calls

Provided Interface

Main componentRequired Interface

Provided Interface

SubcomponentRequired Interface

Provided Interface


Provided Interface


Provided Interface


Provided Interface


Provided Interface


Provided Interface


Provided Interface



Client-Server

Connections are remote procedure calls or remote method invocations

Provided Interface

ClientRequired Interface

Provided Interface


Provided Interface


Provided Interface

ServerRequired Interface


Blackboard

Connections are encapsulated global variables (tuple spaces)

Provided Interface

BlackboardProvided Interface

One “Part”Required Interface

Provided Interface

My “Bit”Required Interface

Provided Interface

Her “Piece”Required Interface

Provided Interface

A “Part”Required Interface

Provided Interface

Another “Piece”Required Interface

Provided Interface

His “Bit”Required Interface


Peer-to-Peer

Connections are remote procedure calls or remote method invocations

Provided Interface

PeerRequired Interface

Provided Interface


Provided Interface



Pipe-and-Filter

Connections are pipes (also called data flows)

Pro

vid

ed

In

terf

ace

Filt

er

Req

uir

ed

In

terf

ace

Pro

vid

ed

In

terf

ace

Filt

er

Req

uir

ed

In

terf

ace

Pro

vid

ed

In

terf

ace

Filt

er

Req

uir

ed

In

terf

ace

Pro

vid

ed

In

terf

ace

Filt

er

Req

uir

ed

In

terf

ace


Implicit Invocation

Connections are events on the software bus

Provided Interface

ComponentRequired Interface

Provided Interface


Provided Interface


Provided Interface


Provided Interface

BusRequired Interface


Layers

Connections are function or method calls + “something in between”

Provided Interface

Layer 4Required Interface

Provided Interface


Provided Interface


Provided Interface


Provided Interface


Provided Interface


Provided Interface


Provided Interface



Another Kind of Layers

Connections are function or method calls

extended

basic

corecore


Many Other (Variants of) Styles

• Interpreters (“little languages”)• Batch sequential (degenerate case of pipe/filter)• Distribution broker (e.g., CORBA)• Plug-in (e.g., web browsers, Eclipse)• Representational State Transfer (e.g., Web)• Shared Nothing (e.g., google)• Three-tier (e.g., application servers)• …

Example – Compiler

Lexer

Parser

Semantor

Optimizer

CodeGenerator

Lexer Parser Semantor

InternalRepresentation

OptimizerCode

Generator

Sequential Parallel


Example - CASE toolset

Projectrepository

Designtranslator

Programeditor

Designeditor

Codegenerator

Designanalyser

Reportgenerator


Example - Version management system

Operatingsystem

Database system

Object management

Version management


Example - Film and picture library

Catalogueserver

Catalogue

Videoserver

Film clipfiles

Pictureserver

Digitizedphotographs

Hypertextserver

Hypertextweb

Client 1 Client 2 Client 3 Client 4

Wide-bandwidth network

Example – Packing robot control system

Visionsystem

Objectidentification

system

Armcontroller

Grippercontroller

Packagingselectionsystem

Packingsystem

Conveyorcontroller


Choosing the Right Style

• Does a certain style makes sense?– The Internet as a blackboard - Does that scale?– Stock exchange as a layers - How to deal with the

continuous change?– Math as hierarchy - How to properly call different

modules for different functions?

• Draw a picture of the major entities• Look for the natural paradigm• Look for what “feels right”


Mixing Styles May Be Necessary

Blackboard + Client-Server

Provided Interface



Provided Interface

My “Bit”Required Interface

Provided Interface

Her “Piece”Required Interface

Provided Interface

A “Part”Required Interface

Provided Interface

Another “Piece”Required Interface

Provided Interface

His “Bit”Required Interface



Blackboard + Implicit Invocation

Provided Interface



Provided Interface

A “Piece”Required Interface

Provided Interface

Some “Bit”Required Interface

Provided Interface

BusRequired Interface



Pipe-and-Filter + Hierarchy

PipeFCalls

Provided Interface

Audio EncoderRequired Interface

Provided Interface

TunerRequired Interface

Provided Interface

Audio PlayerRequired Interface


Architectural Models• Different architectural models may be produced during

the design process

• Each model presents different perspectives on the architecture– Static structural model that shows the major system

components– Interface model that defines sub-system interfaces– Dynamic process model that shows the process structure of the

system– Deployment model shows the relationship between system

elements and hosts

Important note: 1st iteration progress report seeks all of these


Going from Design to Engineering Tasks

• System design – captures the essence of the solution– describes what the software system should do

• “how does it fundamentally approach solving the problem?”• focuses more on desirability

– typically represents an intermediate “design in progress”– typically captures the overall framing of the solution

• Implementation design – captures the full solution– describes what the implementer should do

• “how do we make the approach reality?”• focuses more on feasibility

– typically represents a final “completed design”– typically captures a fully detailed roadmap

Important Note: Your set of engineering tasks likely to be sketchy in 1st iteration plan, more detailed/specific in 1st iteration progress report, but unlikely to be “complete” and “fully detailed” until the 1st iteration final report (expect change, you probably won’t get it right the first time)


Going from Requirements to Engineering Tasks

• At highest level, most of your systems will follow N-tier architecture – Implies certain infrastructure tasks, e.g., set up application server,

database, design data schema• Each functional requirement implies that some component will

implement it– Implies certain design tasks, to group together requirements into

cohesive components – Implies certain design tasks, to expose that functionality via interfaces

(component interfaces and/or external interfaces) Important Note: Your set of engineering tasks likely to

be sketchy in 1st iteration plan, more detailed/specific in 1st iteration progress report, but unlikely to be “complete” and “fully detailed” until the 1st iteration final report (expect change, you probably won’t get it right the first time)


First Iteration PlanDue Next Week!

• Tuesday 16 October, 10am

• Assignments posted on course website

• Submit via CourseWorks

• First Iteration Plan

• Samples here and here – warning: the assignment has changed somewhat since last year, do not blindly “pattern match”!


Upcoming Deadlines

• First iteration progress report due October 23rd • First iteration demo week October 30th –

November 8th • First iteration final report due November 9th • Midterm Individual Assessment posted Friday

November 9th • Midterm Individual Assessment due Friday

November 16th


COMS W4156: Advanced Software Engineering

Prof. Gail Kaiser

[email protected]


Documents

11 October 2007Kaiser: COMS W4156 Fall 20071 COMS W4156: Advanced Software Engineering Prof. Gail Kaiser [email protected]