Which role might model-based engineering play in software certification? Selo Sulistyo

Which role might model-based engineering play in software certification?

Selo Sulistyo

2

1. Software Certification

2. Model-based Engineering (MBE)

3. Roles of MBE in Software Certification

4. Summary

Outline

3

Software certification is the term used to certify the quality of a software application/product against the functional and non-functional specifications before its release to production/ end-user. (Eushiuan Tran, 1999)

Definition

Software Certification

4

The software certification process establishes the level of confidence in a software system in the context of its functional and safety requirements.

Why is a certification needed?


5

Virtual Machine

byteCode

Generates

is executed by

What is being certified?Software Certification

Operating System

Programming Languages

Source Code

Is used to produce application

Running Application

Machine Codeis executed and managed byis executed and managed byis executed and managed by

Is compiled using

Compiler Generates

Assembly

Assembler

Development process

Computer Hardware

Product

6

Product basedExample: ISO IEC 14598. This international standard provides guidance for the practical implementation of software product evaluation when several parties need to understand, accept and trust evaluation results.

Process-based Example:IEC 61508 (Functional safety of electrical/electronic/

programmable electronic safety-related systems)

DO-178B (Software Considerations in Airborne Systems and Equipment Certification).

Certification Approaches


7

Verification and validation are part of the certification process for software system (Eushiuan Tran, 1999)

Certification process


Verification Validation Certification

Developer Functional Expert Certification Authority

8Development Testing

Was the product built right?

Was the right product built?

iteration

When and how the verification and validation are done?


artifactsrequirements

Requirement analysis

design

implementation

verification

deployment

Validation

Testing

Deve-lopment

9

The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase.

Was the product built right?

Verification does not ensure the application:– Solves an important problem– Correctly reflects the workings of a real world

process

Verification


10

The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements.

Was the right product built?

Validation


11

Verification and Validation Techniques


12

An example : DO-178B


13

An example : DO-178B


14



3. Role of MBE in Software Certification

4. Summary

Outline

15

An approach to engineering that uses models as an integral part of the technical baseline that includes

the requirements, analysis, design, implementation, and verification

of a capability, system, and/or product throughout the acquisition life cycle.

Definition

Model-based Engineering

Models

requirements


design

implementation

verification

deployment

Validation

16

A physical, mathematical, or otherwise logical representation of a system, entity phenomenon or process

• Models are scoped to purpose• Models are appropriate to the context (e.g.,

application domain, life cycle phase)• Models integrated or interoperable across

domains and across lifecycle

What is a model?


17

Software Model


model

system

Modeling

language

is written in

describes

18

Jos B. Warmer and Anneke Kleppe defined Modeling Maturity levels (MML)

MML 0: No Specification MML 1: Textual Specification MML 2: Text with Models MML 3: Models with Text MML 4: Precise Models MML 5: Models only

The use of model in software development


requirements


design

implementation

verification

deployment

Validation

Testing

Deve-lopment

19

Testing

Deve-lopment

MML 0 No Specification

My software

…

No specification


requirements


design

implementation

verification

deployment

Validation

20

Testing

Deve-lopment

Specification of software is written down in one or more natural language documents.

MML 1 Textual Specification


requirements


design

implementation

verification

deployment

Validation

21

Testing

Deve-lopment

Specification of software in one or more natural language documents plus several high-level diagrams to explain the overall architecture.

+

MML 2 Text with Models


requirements


design

implementation

verification

deployment

Validation

22

Testing

Deve-lopment

Specification of software is written down in one or more models. Additional natural language text is used to explain the background and motivation of the models.

+

MML 3 Models with Text


requirements


design

implementation

verification

deployment

Validation

23

Testing

Deve-lopment

Specification of software is written down in one or more models.

Natural language text is used to explain the background and motivation of the models.

The models are precise enough to have a direct link with the actual code.

MML 4 Precise Models


requirements


design

implementation

verification

deployment

Validation

24

Testing

Development

The models are precise and detailed enough to allow complete code-generation.

The code is invisible (as assembler is today).

Modeling language High level programming language.

MML 5 Models only


requirements


design

implementation

verification

deployment

Validation

25

MML 0: No Specification MML 1: Textual Specification MML 2: Text with Models MML 3: Models with Text MML 4: Precise Models MML 5: Models only

Modeling Maturity Levels (MML)

MBE !


26

OMG’s Model-driven Architecture


The current state of the art in MDE is much influenced by the ongoing standardisation activities around the OMG Model Driven Architecture® (MDA®).

MDA is a framework which defines a model-driven approach to software systems development.

Focus on the solution of the problem rather than the implementation problem

Raising the level of abstraction, dealing with complexity

An automatic code generation

27

Models in MDA (Model-Driven Architecture)


CIM: Computational-independent

models

PIM: Platform-independent Models

PSM: Platform-specific Models

RequirementAnalysis

Implementation

Low-level design

High-level design

PIM

Code

PSM

TT

TT

Code

PSM

TT

TT

Code

PSM

TT

TT

CIM

TT

TT: Transformation Tool

28



3. Role of MBE in Software Certification

4. Summary

Outline

29

How does model-based engineering produce applications?

Modeling Language

Models

Tools and Production processes

Transformation tool

e.g Code Generator

??

Role of MBE in Software Certification

Operating System


Source Code


Source Code

I s used to produce application


Is compiled using

Compiler Generates

Assembly

Assembler



Is compiled using

Compiler Generates

Assembly

Assembler

I s compiled using

Compiler Generates

Assembly

Assembler

Compiler Generates

Assembly

Assembler

Computer Hardware

Virtual Machine

byteCode

Generates

is executed by

Virtual Machine

byteCode

Generates

is executed by

Virtual Machine

byteCode

Generates

Virtual Machine

byteCode

Generates

is executed byRunning Application

Products

30

Role of MBE in Software CertificationCertification

One central goal of MBE is to enable analysis of the system, thus ensuring the quality of the system already on the model level.

early verification and validation

31

requirements


design

implementation

Role of MBE in Software CertificationWhen and how the verification and validation are done?

How to verify these software

models?

Certification Authority

verification

deployment

Validation

Model Verification

Model Validation

Verified and Validated Software Models

…..

Models

Was the model built right?

Was the right model built?

Was the model built right?

32

Role of MBE in Software CertificationModel verification

Verification is done to ensure that:

– The model is developed correctly and conformance with standard

– The algorithms have been implemented properly

– The model does not contain errors or bugs

Verification does not ensure that the model:

– Solves an important problem

– Meets a specified set of model requirements

– Correctly reflects the workings of a real world process

33

Role of MBE in Software CertificationVerification techniques

Verification techniques that can be used:

– Model checking

– proof of correctness

– consistency checking

This will ensure that the models are

Consistent

Correct against to language syntax of the language

34

Role of MBE in Software CertificationModel validation

Validation ensures that the model meets its intended requirements in terms of the methods employed and the results obtained. Compliance with requirements

The ultimate goal of model validation is to make the model useful in the sense that the model addresses the right problem, provides accurate information about the system being modeled, and to make the model actually used.

35

Role of MBE in Software Certification

B

B

BA

A

A

CIM

PIM

Code

PSM

TT

TT

TT

Model verification and validation

Code

PSM

TT

TT

B

Product familyProduct family

B

A

A: Verification

- Consistency

- Conformance with standard

B: Validation

- Compliance with requirements

- Traceability

TT: Transformation Tool

36

Models provides a view to a complex problem and its solutions.

Models are less risky, cheaper to develop and easier to understand than the implementation of a genuine target system

By assuming that the transformation tools is verified then the verification process can be done in model level, that obviously it saves time and cost.

It also improves the quality of design and software code by better assuring the traceability between artifacts.

Role of MBE in Software CertificationBenefits MBE in software certification

37

MBE can play role on:

Process-based certification.

Early verification process. Activities for verification, validation, and test with Model-Based Engineering can be applied at every stage of the development process.

The model traceability will minimize the certification efforts, especially for product families. reducing certification and re-certification times,

Saving time and cost since the verification is done on higher abstraction levels, but it may be a complicated process with the current MDE technology.

Summary

38

Thank You

39

1. Jos Warmer, The role of OCL in the MDA, Klasse Objecten, http://www.klasse.nl

2. Eushiuan Tran, Verification/Validation/Certification, Carnegie Mellon University, 18-849b Dependable Embedded Systems, Spring 1999

3. Rajwinder Kaur Panesar-Walawege, Mehrdad Sabetzadeh, Lionel Briand, Using Model-Driven Engineering for Managing Safety Evidence: Challenges, Vision and Experience, Simula Research Laboratory, University of Oslo, Lysaker, Norway

4. Ewen Denney and Bernd Fischer, Software Certification and Software Certificate Management Systems, USRA/RIACS, NASA Ames Research Center, Moffett Field, CA 94035, USA

5. OSMAN BALCI, A Methodology for Certification of Modeling and Simulation Applications, ACM Transactions on Modeling and Computer Simulation, Vol. 11, No. 4, October 2001, Pages 352–377.

6. Damian Dechev, Bjarne Stroustrup, Model-based Product-Oriented Certification, Texas A&M University College Station, TX 77843-3112

References

Documents

Which role might model-based engineering play in software certification? Selo Sulistyo