(Some) Software Engineering Research at NJIT Sergio Bogazzi, Yao Fei Chen, GuanJie Jiang, BoYu, Ali Mili

(Some) Software Engineering Research at NJIT

Sergio Bogazzi, Yao Fei Chen, GuanJie Jiang, BoYu, Ali Mili

Professor Ali Mili 2

Projects

Software Architecture Analysis

Verification and Validation of Online Adaptive Systems

Software Engineering Trends

Analyzing Redundancy

Software Architecture Metrics

A Study of Change and Error Propagations


Project Objective

Defining and Investigating Quality Metrics for Software Architectures.Applying these Metrics to Large Scale Software Architectures.Correlate the Metrics with Independent Observations of Quality.


Research Plan

Define and validate computable metrics for software architectures based on information theoretic measuresAutomate the process of computing these metrics.Apply the process and metrics to a NASA case study.Validate the correctness and usefulness of these metrics.


Selected Quantitative Factors

Error Propagation. Significance for reliability.Change Propagation. Significance for Maintainability.Requirements Propagation. Signifi- cance for Evolvability.


What are these Metrics useful for?

By looking at the Error Propagation Metrics, the architect would be able to identify components that have high potential to propagate errors through the system,

The error propagation matrices, are also very useful in direction of test plans, critical components that show high tendency to propagate error to other system components should be more thoroughly tested.


What are these Metrics useful for? (cont’d)

By looking at the Change Propagation Metrics (CPM), the architect would be able to identify components that have high potential to propagate the changes inside them to other components.From CPM, the architect would also be able to identify components that have high potential to be affected by the changes in other components


Project Overview

Define Information theoretic metrics Coupling, cohesion, redundancy

Define quantitative factors that are relevant to qualitative attributes of the architecture

Error PropagationChange Propagation

Establishing analytical and/or empirical relationship between the metrics and the quantitative factorsAutomate computation of the metrics


Premises of Our Approach

Architectural Level precludes a logic, semantic-based approach; hence we use a stochastic approach.Information Theory has a wide range of functions that quantify random variables.We use analytical and empirical means to correlate metrics to qualitative properties.


Recent AccomplishmentsFurther Analysis of the Correlation between Error Propagation and Computable Metrics.Initiating the Analysis of Change Propagation in the Context of State Based Components.Automation of the derivation of metrics from architectural descriptions (in UML, by Nicholay) and from source code (by Sergio).


Information Coupling and Cohesion Metrics

We treat coupling in the true architectural sense, as a property of the connector between components.Coupling in our approach becomes a function of an ordered pair of components characterizing their informational interdependence.Cohesion of a component measures its internal information flow.


Definition of Quantitative Factors: Error Propagation

The error propagation (EP) is a measure of the likelihood that an error in the message sent by A will propagate into B.

S

X

A B

xx` x` xOR

Error


Static Error Propagation Matrix SEPM

(Flattened architecture of a NASA case study)

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10C1 1 0.11 0.38 0.44 0.58 0.59 0 0 0 0C2 0.18 1 0 0 0 0 0 0.5 0 0C3 0.11 0.26 1 0 0 0 0 0 0 0C4 0.17 0.15 0 1 0 0 0 0 0 0C5 0 0.57 0 0 1 0 0 0 0 0C6 0 0.18 0 0 0 1 0 0 0 0C7 0.23 0 0 0 0 0 1 0 0 0C8 0 0 0 0 0 0 0 1 0 0C9 0 0 0 0 0 0 0 0 1 0

C10 0.08 0 0 0 0 0 0 0 0 1

B

A

Error Propagation Coefficient (static) = 0.0503


Dynamic Error Propagation Matrix DEPM

(Flattened architecture of a NASA case study)

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10C1 1 0.11 0.15 0.3 0.76 0.73 0 0 0 0C2 0.37 1 0 0 0 0 0 0.52 0 0C3 0.03 0.32 1 0 0 0 0 0 0 0C4 0.03 0.19 0 1 0 0 0 0 0 0C5 0 0.67 0 0 1 0 0 0 0 0C6 0 0.28 0 0 0 1 0 0 0 0C7 0.32 0 0 0 0 0 1 0 0 0C8 0 0 0 0 0 0 0 1 0 0C9 0 0 0 0 0 0 0 0 1 0

C10 0 0 0 0 0 0 0 0 0 1

B

A

Error Propagation Coefficient (dynamic) = 0.0534


Future Work Empirical Validation of Error Propagation Analysis

ExtractModel Info.

UML Model DesignExperiments

CorruptModel

Simulateand Log

Normal Log

Error Logs

AnalyzeLogs

Empirical Error Propagation

Components,Connectors

Connector,Message

MATLAB

SummitBASIC

FaultModel


Change Propagation

We define Change Propagation from component A to component B as the probability that a change in A due to corrective/ defective maintenance requires a change in B to maintain the overall function of the system.We are conducting empirical experiment and trying to propose an analytical formula that approximate change propagation


Analyzing Change Propagation: Simple Model Model architectural components as input-output transformers C : IC OC, from the set of inputs (IC ) of the component C to the set of its outputs (OC ).

Consider elementary two-component (pipeline?) architecture, where the first component A “feeds” its output to the second component B as its input. The functionality of the architecture can be presented in the form of the following diagram:


Empirical work

In our experiment, we randomly select a large amount of changes within each component, then go into the source code level to see, if the change will propagate to other components. By computing for each pair of components (A, B), the number for which change in A cause a change in B, we are able to derive the change propagation matrix.


C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 Changes

C1 1 1/11 2/11 1/11 1/11 2/11 1/11 1/11 3/11 11

C2 1 2/3 3

C3 1/5 1 4/5 5

C4 1 2/2 2

C5 1 0/2 1

C6 1/3 1 1/3 3

C7 4/8 1 3/8 8

C8 1/3 1/3 1 2/3 1/3 3

C9 1/8 1/8 4/8 2/8 1/8 1 1/8 2/8 8

C10 1/8 1 6/8 8

C11 1/3 1/3 1 1/3 3

C12 1 0

C13 2/2 1 2

C14 1 0

C15 1/1 1 1/1 1

C16 2/5 4/5 2/5 2/5 1 1/5 5

C17 1/1 1 1

C18 1/3 1/3 1/3 1/3 1 3

Sum 67

Change Propagation Matrix

The experiment is still undergoing


Automated Tool Simplify application of the theory and methods developed to industrial-strength systemsGive project manager an ability to get a glimpse into the quality of software architecture Project Manager ViewConfigure the tool for handling specific ADLs and CASE tools Analyst View


Benefits

The developments of techniques for measuring error/change propagation help analysts identify trouble spots in the architecture (advances the state of the art)The development of automated tools help the analyst apply these techniques on large architectures (advances the state of the practice)


Future Work

Use a tool such as SIAT TM, or Understand TM to measure static information theoretic metricsCompare the measures obtained to those obtained at the design levelCompare information coupling and cohesion measures with traditional measures based on McCabe.Perfect the tools


Verification and Validation of Online Adaptive Systems

Jian GuanJie, Ali Mili, NJITBojan Cukic, Yan Liu, WVU


The Intelligent Flight Control System (IFCS)

Real-Time PID

estimatedderivatives

derivativeerrors

Controllerpilotinputs

Sensors

controlcommands

baselinederivatives

BaselineNeural Network

OnlineNeuralNetwork

derivativecorrections

analog

IFCS Project: NASA DFRC; NASA ARC; Boeing; ISR; WVU.


Traditional V&V Techniques

Fault Avoidance

Fault Removal

Fault Tolerance

All are inapplicable to adaptive learning systems


Refinement-based V&V of adaptive systems

Functional Envelope

Monotonic Learning

Safe Learning


Functional Envelope

hR hF

FhX Y

X Y

H

F R

Rh


Initial Weights Input Iteration Times with Output

10 20 50 100 500 2000 Converge

0=1.0 7285

(1,1) 0.95718 0.88763 0.64715 0.57213 0.50373 0.12861 0.04999388

(1,0) 0.88929 0.75727 0.49946 0.48526 0.51578 0.88881 0.95669980

(0,1) 0.88985 0.76131 0.50934 0.48949 0.51579 0.88868 0.95675480

(0,0) 0.74329 0.58170 0.41602 0.45580 0.50102 0.09960 0.03909299

0=0.5 7560

(1,1) 0.70029 0.58756 0.53496 0.52377 0.51087 0.14863 0.04999296

(1,0) 0.60074 0.51103 0.48290 0.48596 0.49385 0.87160 0.95670090

(0,1) 0.60987 0.52137 0.48944 0.48869 0.49424 0.87141 0.95675580

(0,0) 0.55051 0.49504 0.48686 0.49964 0.51760 0.11487 0.03909090

0=0.0 8926

(1,1) 0.50565 0.50740 0.50880 0.50976 0.51116 0.50880 0.04999402

(1,0) 0.48353 0.48525 0.48714 0.48836 0.48878 0.49985 0.95669870

(0,1) 0.49364 0.49367 0.49203 0.49037 0.48888 0.50006 0.95675415

(0,0) 0.51421 0.51434 0.51342 0.51227 0.51134 0.51613 0.03909125

0 [-0.3, 0.3] 8942

(1,1) 0.51080 0.51098 0.51101 0.51096 0.51118 0.50909 0.04999226

(1,0) 0.48304 0.48416 0.48627 0.48794 0.48876 0.49888 0.95670134

(0,1) 0.49480 0.49397 0.49197 0.49027 0.48885 0.49911 0.95675653

(0,0) 0.51010 0.51027 0.51051 0.51073 0.51137 0.51662 0.03908928


Monotonic Learning----MLP


Monotonic Learning----DCS


Safe Learning

hFh , hR( )S


Novelty Detection


Experimental Results of Novelty Detection Using Association Rule Learning

Programming Language Trends:

An Empirical Study

Yao Fei Chen, Ali MiliYao Fei Chen, Ali Mili

New Jersey Institute of TechnologyNew Jersey Institute of Technology


Introduction

As part of the project of monitoring software engineering technical trends, we are trying to research the evolution of high level programming language.

What are the possible factors which can affect programming language trends?

How can we quantify these factors?

How can we watch/predict/adapt to/affect the trends based the factors?


Possible Factors Which Affect TrendsIntrinsic factors are the factors which can be used to describe the general design criteria of programming language.

GeneralityOrthogonalityReliabilityMaintainabilityEfficiencySimplicityImplementabilityMachine IndependenceExtensibilityExpressivenessInfluence/Impact


Extrinsic factors are the factors which are not directly related to the general attributes of a programming language, but still can affect the trend of programming language.

Institutional Support

Industrial Support

Governmental Support

Organizational Support

Grassroots Support

Technology Support


Quantifying Factors

Quantifying intrinsic factors:

All intrinsic factors should be considered when one designs a programming language. So, we will try to go over all features of a programming language to check if it matches these factors. Then, we will assign a score to each factor for each programming language.

Quantifying extrinsic factors:

We will do survey for every extrinsic factor and assign scores for them.


Models Construction&Validation

Use statistics method to construct&validate Models

F (I1, I2, …, In)

I1

I2

I3

InOm

O1

O2Factors that might affect the trend

Factors that can be used to evaluate the trend

Function used to describe the relationship among factors.


Evaluating Programming Languages

By analyzing a set of programming language, we expect that we can find out statistics models and use them to predict the future trend of a programming language.

COBOL

FORTRAN

LISP

ALGOL

PASCAL

C

C++

JAVA

ADA

ML

APL

MODULA

EIFFEL

PROLOG

SMALLTALK

SCHEME

Analyzing Redundancy

Ali Mili, NJITBojan Cukic, WVUJules Desharnais, Laval University


Outgrowth of Dryden Project

What is Redundancy?

Does Redundancy characterize a state or its representation?

Is all redundancy amenable to state redundancy?

Can we talk about redundancy without Fault Tolerance?

Instant Redundancy vs Temporal Redundancy


Three Views of Redundancy

Non-Injectivity of Representation Function

Duplication of State information

Scale of Fault Tolerant Capabilities.


Non Surjectivity

Representation function from States to Representations

Total (representability): integers.

Injective (precision): reals.

Surjective (Non-redundancy)

No representation functions satisfy all. Most functions satisfy none.


Duplication of State Information

Axiomatization of functions quantifying duplication

Zero for redundancy free state

One for simple duplication for redundancy free state.

N-1 for N-modular redundancy

Continuous real values in-between.


Scale of Fault Tolerant Capabilities

Levels of Correctness:

Correctness, Maskability, Recoverability, Non-recoverability.

Recovery is necessary and sufficient: Recoverability.

Sufficient conditions of recoverability.

Documents

(Some) Software Engineering Research at NJIT Sergio Bogazzi, Yao Fei Chen, GuanJie Jiang, BoYu, Ali Mili