55
1 | 58 Embedded and Networked Systems Lecturer Mitra Nasri ([email protected]) Appointment by email TAs Grzegorz (Greg) Krukiewicz-Gacek ([email protected]) Fanis Floros ([email protected]) Appointment by email IN4390 Quantitative Evaluation of Embedded Systems Lecture 1 Introduction

Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

  • Upload
    others

  • View
    13

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

1 | 58Embedded and Networked Systems

LecturerMitra Nasri ([email protected])Appointment by email

TAsGrzegorz (Greg) Krukiewicz-Gacek

([email protected])Fanis Floros

([email protected])

Appointment by email

IN4390 Quantitative Evaluation of Embedded SystemsLecture 1

Introduction

Page 2: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

2 | 58Embedded and Networked Systems

Agenda

• Course outline

• Definitions and concepts • Embedded systems specifications • Common requirements

• Modeling and model-based design

Page 3: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

3 | 58Embedded and Networked Systems

Course outline

• Lectures• Theory, instructions, examples, quizzes

• Exam • Written exam with open-ended questions

• Practicum (lab)• 3 main assignments• Tools: ROS and Petri-net• Requirement: basic knowledge of C++ and Linux

Page 4: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

4 | 58Embedded and Networked Systems

Course organization

• Lectures• Tuesday 8:45 – 10:30• Thursday 10:45 – 12:30• No lecture on Dec. 19th

• Practicum (Lab)• Thursday 13:45 – 17:45 (starts from Nov. 21)• Teams of 2

Note 1: Your presence in the lab is not mandatory, however, some assignments require you to demo your implementation to the TAs.

Note 2: Both team members must equally contribute to the assignments and reports. Otherwise it would be a case of plagiarism.

Page 5: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

5 | 58Embedded and Networked Systems

http://www.st.ewi.tudelft.nl/koen/in4390/

Where to find information

• Brightspace• Upload assignments• Register your group

• Main webpage of the course• Announcements• Lecture materials• Extra materials• Quiz solutions• Example exams

Page 6: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

6 | 58Embedded and Networked Systems

Final grade = min(10, E + C𝟏𝟏𝟏𝟏𝟏𝟏

) * M

Grading

E = Exam

M = {pass=1 | fail=0} (mandatory assignments)

C = Customizable points

• You fail if you have not passed your mandatory assignments, or your final grade is smaller than 6. • Mandatory assignments must be uploaded to Brightspace before their due dates. Some of these assignments

require a live demo.

Page 7: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

7 | 58Embedded and Networked Systems

Mandatory assignments

• Assignment 0 • Paper reading followed by questions on Brightspace • Related to Lect. 1 and 2

• Assignment 1 (Lab 1)• Measurement-based performance evaluation of ROS 2.0 communication

mechanisms• Related to Lect. 2, 3 and 4

• Assignment 2 (Lab 2)• Behavior modeling and performance analysis using Petri-nets• Related to Lect. 5 and 6

• Assignment 3 (Lab 3)• Deriving a petri-net model from a given ROS application • Answering questions regarding what to expect from the system if certain

parameters/configurations change• Related to Lect. 1, 2, 5, 6, and 13

Autoware or AirSim(autonomous car or drone)

Page 8: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

8 | 58Embedded and Networked Systems

Use cases for Lab 3 (tentative)

Autoware.org

Autoware(an open source framework

for autonomous driving)

AirSim(an open framework for

autonomous cars or drones)

• Both provide simulation framework for the system and the physical world

• Both are based on ROS

Page 9: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

9 | 58Embedded and Networked Systems

Grading

Extra lab assignment

Class activities

Tool presentation

ROS Implementation

Quiz

Projects

The background image comes from here

You can build your own grading scheme according to your own preferences

We are going to use Gamification!

Trion Worlds

Page 10: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

10 | 58Embedded and Networked Systems

Grading: customizable points• Sources of customizable points:

1. Extra (optional) questions in the assignments • 120 points

2. Quiz• 15 points per quiz for those who answer all questions correctly (4~5 quizzes)

3. Questions/answers during the class • 5 points per question (max one answer per student per lecture)

4. Tool demo• 80 points (maximum one per team, FCFS)

5. Project • 100~200 points (maximum one per team, FCFS)

• You may acquire at most 400 customizable points

• Note that teams may do either a demo or a project (but not both)

300 ≤ E ≤ 8000 ≤ C ≤ 400 Final grade = min(10, E + C

𝟏𝟏𝟏𝟏𝟏𝟏) * M

𝑀𝑀 ∈ fail = 0, pass = 1

Page 11: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

11 | 58Embedded and Networked Systems

Grading: customizable points

4. Tool demos• Investigate a certain tool from a given list • Teams will coordinate the tool that they want to present with the TAs.• A short report (of up to 5 pages) • Present the tool in the class • Deadline: at the end of the quarter (will be announced soon)

5. Projects • Projects involve implementation and evaluation using existing tools and frameworks (such as Microsoft

AirSim, Autoware, etc.)• A list of topics will be available soon• Teams will submit a 1-page project proposal

• Explain what the project is about, what are the goals, milestones, steps, and deliverables

• Proposals will be evaluated and adjusted• Deadline: at the end of the quarter (will be announced soon)

Page 12: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

12 | 58Embedded and Networked Systems

Relations to the study program

System Validation

Functional correctness

QEES

Evaluation of non-functional properties such as performance, dependability, …

Real-Time Systems

Evaluation (and correctness) of timing properties such as response-time

Quarter 1

Quarter 2

Quarter 3

Embedded Systems Lab

Quarter 4

Designing and evaluating an embedded system

Page 13: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

13 | 58Embedded and Networked Systems

Goals of QEES

• Use models to design, analyze, and evaluate a system

• Compare alternatives• To provide quantitative information about alternatives

• Determine the impact of a feature on overall system performance

• Pin-point bottlenecks

• System tuning/optimization• To find the best parameters

that produce the best performance

Example: The number of packets lost on two links was measured for our file sizes as shown below:

Which link is better?

Page 14: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

14 | 58Embedded and Networked Systems

Lectures’ organization

• Introduction to modeling and model-based design (1 lecture)

• Measurement-based performance evaluation and statistics 101 (1 lecture)

• Design of experiments (2 lectures)

• Petri-nets and data-flow networks (2 lectures)

• Markov models (2 lectures)

• Queueing theory (2 lectures)

• Memory efficiency and optimization (1 lecture, tentative)

• Design-space exploration and optimization (1 lecture)

• Presentations by students, exam examples, FAQ (1 lecture)

Page 15: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

15 | 58Embedded and Networked Systems

Organization of the contents

Prof. Koen [email protected]• Lect. 9 and 10: operational laws and queuing theory• Lect. 13: design space exploration and optimization• Lect. 14: student presentations, exam preparation, FAQ

Marco [email protected]• Lect. 7 and 8: Markov models

Mitra [email protected]• Lect. 1: Modeling and model-based design• Lect. 2: Measurement-based performance evaluation

Lect. 5: Petri-nets• Lect. 6: Data-flow networks• Lect. 12 (tentative): Memory efficiency• Lect. 14: student presentations, exam preparation, FAQ

Lydia [email protected]• Lect. 3 and 4: Experiment design

Lecturers

There is no Lect. 11

Page 16: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

16 | 58Embedded and Networked Systems

Sources• Books

• The Art of Computer Systems Performance Analysis• Raj Jain • John Wiley & Sons Canada, Ltd., 1 edition, 1991.

• Embedded System Design • Peter Marwedel• The second edition (feel free to get the third edition)

• Introduction to Embedded Systems: A Cyber-Physical Systems Approach• Edward Lee and Sanjit Seisha (UC Berkeley)• Second Edition, 2017• https://ptolemy.berkeley.edu/books/leeseshia/

• Measuring Computer Performance: A Practitioner’s Guide• David J. Lilja - Hardcover.• Cambridge University Press, 2000.

• Papers• Basic Concepts and Taxonomy of Dependable and Secure Computing

This page will be updated soon.Do not yet buy your book!

Page 17: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

17 | 58Embedded and Networked Systems

Agenda

• Course outline

• Definitions and concepts • Embedded systems specifications • Common requirements

• Modeling and model-based design• Modeling• Design process• Model-based design

Sources: • [Book]: Marwedel (chapter 1)• [Paper]: Basic Concepts and Taxonomy of Dependable and Secure Computing

Page 18: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

18 | 58Embedded and Networked Systems

Embedded Systems (and Quantitative Properties)

Page 19: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

19 | 58Embedded and Networked Systems

What is an embedded system?

It is an information processing system that is embedded into an enclosing

physical product.

Unlike a PC or servers, an embedded system “interacts” with its physical world.

Have you heard about Cyber-physical systems (CPS)?

Page 20: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

20 | 58Embedded and Networked Systems

Cyber-physical systems (CPS): A simple rename or a purposeful concept?

[Wiki]: A CPS converts goals defined digitally (the cyber domain) to the

physical world.

[wiki] CPS v.s. ES: A CPS is typically designed as a network of interacting

elements with physical input and output instead of as standalone devices.

Are you convinced?

Mechanical engineer

Software developer

Hardware engineer

Controlengineer

System integratorEvery team tends to assume a perfect output

from the other teams, e.g., zero execution time, perfect controller, perfect hardware, etc.

Page 21: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

21 | 58Embedded and Networked Systems

Concepts and definitions

SystemA system is an entity that interacts with other entities, i.e., other systems, including hardware, software, humans, etc.

FunctionThe function of a system is what the system is intended to doand is described by the functional specification in terms of functionality and performance

The boomerang drone’s functions:- Normal mode (receives signals from user and battery

is not low)- Move up, down, left, right- Increase or decrease speed - Land- Take picture- Send picture

- Hold mode (no signal)- Stay still and wait for signal

- Safe-return mode (no signal from user for 2 minutes)- Follow the path back if there is no signal- Detect obstacles on the way- Avoid obstacles

- Low battery mode (battery is low)- Land safely if the battery is low

BehaviorThe behavior of a system is what the system does to implement its function. The behavior, for example, can be described by a sequence of states

StructureThe structure of a system is what enables it to generate the behavior.

Signal

No signal for 2 minutesS

B

NSignalNo signal

Low battery

Low batteryH

Low battery

Page 22: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

22 | 58Embedded and Networked Systems

BehaviorThis is a state diagram, which is a “model” that

describes how the system changes modes

Is this enough to describe the behavior?

Decision making

This is a component diagram, which is a “model” to show the dependencies and interactions between SW/HW components

Signal

No signal for 2 minutesS

B

NSignal No signal

Low battery

Low batteryH

Low battery

Flight control

Note: this diagram is symbolic and is not accurately model our prior example.

Page 23: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

23 | 58Embedded and Networked Systems

V-Model for system development

customer

Product development

requirementsVerification and validation

End product

Page 24: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

24 | 58Embedded and Networked Systems

Specifications

The boomerang drone’s functions:- Normal mode (receives signals from user and battery

is not low)- Move up, down, left, right- Increase or decrease speed - Land- Take picture- Send picture

- Hold mode (no signal)- Stay still and wait for signal

- Safe-return mode (no signal from user for 2 minutes)- Follow the path back if there is no signal- Detect obstacles on the way- Avoid obstacles

- Low battery mode (battery is low)- Land safely if the battery is low

This is a (very informal) functional specification

SpecificationsThey describe the functional and non-functional requirements of the system

Page 25: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

25 | 58Embedded and Networked Systems

Specifications

SpecificationsThey describe the functional and non-functional requirements of the system

What examples do you have in mind for non-functional requirements?

[Wiki] A non-functional requirement (NFR) is a requirement that specifies criteria that can be used to judge the operation of a system, rather than specific behaviors.

Read more here: https://en.wikipedia.org/wiki/Non-functional_requirement

Page 26: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

26 | 58Embedded and Networked Systems

Quantitative properties

Quantitative properties are key selling points (next to the functional correctness)

Slide course (Bart Theelen): https://www.win.tue.nl/~pcuijper/QEES/Guest%20Lecture%20Bart%20Theelen.pdf

Page 27: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

27 | 58Embedded and Networked Systems

Quantitative properties in industry

Quantitative properties are key selling points (next to the functional correctness)

Slide course (Bart Theelen): https://www.win.tue.nl/~pcuijper/QEES/Guest%20Lecture%20Bart%20Theelen.pdf

Page 28: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

28 | 58Embedded and Networked Systems

Examples of quantitative measures

• Extrema (worst/best case)• Maximum occupancy of a memory• Minimum time until next failure• Peak power consumption• Worst-case response time• Worst-case end-to-end delay

• Reachability (expected time until something happens)• Expected time until next failure• Expected time until first output is produced

• Long-run average• Processor load• Throughput of a communication network• Mean time between failures (MTBF)• Average power consumption

Slide course (Bart Theelen): https://www.win.tue.nl/~pcuijper/QEES/Guest%20Lecture%20Bart%20Theelen.pdf

Page 29: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

29 | 58Embedded and Networked Systems

Quantitative properties

Response time

Performance

Safety

Security

ReliabilityPrivacy

Availability

PortabilityDurability

Extensibility

Understanding these requirements is what makes you a valuable,

highly-paid, and highly-wantedengineer!

Dependability

Read more here: https://en.wikipedia.org/wiki/Non-functional_requirement

Life time

Energy consumption

Quantitative properties are key selling points (next to the functional correctness)

Page 30: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

30 | 58Embedded and Networked Systems

Quantitative properties

Response time

Safety

Security

ReliabilityPrivacy

Availability

PortabilityDurability

Extensibility

Understanding these requirements is what makes you a valuable,

highly-paid, and highly-wantedengineer!

Read more here: https://en.wikipedia.org/wiki/Non-functional_requirement

Life time

Energy consumption

Quantitative properties are key selling points (next to the functional correctness)

Performance

Dependability

Page 31: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

31 | 58Embedded and Networked Systems

Common performance measures• Response time

• Response time is the total amount of time it takes to respond to a request for service• Latency

• Latency is a time delay between the cause and the effect of some physical change in the system being observed

• Bandwidth• In computer networking, bandwidth is a measurement of bit-rate of available or consumed data

communication resources, expressed in bits per second or multiples of it (bit/s, kbit/s, Mbit/s, etc.)• Throughput

• the rate of production or the rate at which something can be processed• Scalability

• Scalability is the ability of a system, network, or process to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth

• Power consumption• The amount of electricity used by the computer

• Compression ratio• Size and weight• Transistor count

Source: https://en.wikipedia.org/wiki/Computer_performance

Page 32: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

32 | 58Embedded and Networked Systems

Dependability

“The dependability and security specification of a system must include the requirements for theattributes in terms of the acceptable frequency and severity of service failures for specified classes offaults and a given use environment. One or more attributes may not be required at all for a givensystem” [TPDS00].

The ability to deliver service that can justifiably be trusted• The stress is on the need for justification (e.g., through rigorous

evaluation or proof) of trust

Definition 1

The ability to avoid service failures that are more frequent and more severe than is acceptable

Definition 2

[TPDS00] “Basic Concepts and Taxonomy of Dependable and Secure Computing”, 2004.

Page 33: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

33 | 58Embedded and Networked Systems

Service/function failure

• Fault -- often referred to as Bug [TPDS00] • A static defect in software (incorrect lines of code) or hardware • A fault is the adjudged or hypothesized cause of an error

• Error• An incorrect internal state (unobserved)• An error is the part of total state of the system that may lead to its subsequent

service failure

• Failure• External, incorrect behavior with respect to the expected behavior (observed)

[TPDS00] “Basic Concepts and Taxonomy of Dependable and Secure Computing”, 2004.

Fault Error Failure

Page 34: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

34 | 58Embedded and Networked Systems

Fault, error, and failure

First we need to know the desired behavior

What is this? A fault? An error? Or a failure?

image: Bernd Bruegge & Allen H. Dutoit. Object-Oriented Software Engineering: Using UML, Patters, and Java

Page 35: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

35 | 58Embedded and Networked Systems

Fault, error, and failure

First we need to know the desired behavior

What is this? A fault? An error? Or a failure?

image: Bernd Bruegge & Allen H. Dutoit. Object-Oriented Software Engineering: Using UML, Patters, and Java

“A design without specifications cannot be right or wrong, it can only be surprising!”

[Lee and Seshia]

Under this usage, this implementation might not be

a fault :-)

Page 36: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

36 | 58Embedded and Networked Systems

Fault, error, and failure

Assume it is a rail for a normal train

It is a fault

What is this? A fault? An error? Or a failure?

Fault examples: • Misconfiguration (e.g., misconfigured user permissions)• Hardware faults (a memory cell that is always zero)• Physical faults (caused by the environment)

• At high radiation, memory bits may randomly flip• At high temperature, pressure sensor produces noisy data• In the tunnels, the GPS does not work

image: Bernd Bruegge & Allen H. Dutoit. Object-Oriented Software Engineering: Using UML, Patters, and Java

Page 37: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

37 | 58Embedded and Networked Systems

……

……

Fault, error, and failureA fault

(like a bug in the code)

It is not activated as long as it is not on the program path

Page 38: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

38 | 58Embedded and Networked Systems

……

……

Fault, error, and failure

An error(a fault that is on the

program path)

Page 39: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

39 | 58Embedded and Networked Systems

……

……

Fault, error, and failure

A failure(an error that caused an observable deviation of

correct behavior)

Page 40: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

40 | 58Embedded and Networked Systems

Taxonomy of faults

• Development Faults• All fault classes occurs during the development• This includes bugs, misconfigurations, etc.

• Physical Faults• All fault classes that affect hardware or physical system• This includes hardware defects, etc.

• Interaction Faults• All external faults • This includes interface mismatch between components

Read the rest from: “Basic Concepts and Taxonomy of Dependable and Secure Computing”.

Page 41: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

41 | 58Embedded and Networked Systems

Dependability and security tree

“Basic Concepts and Taxonomy of Dependable and Secure Computing”

Page 42: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

42 | 58Embedded and Networked Systems

Addressing faults at different stages

Patching/fixing Redundancy Testing

Bernd Bruegge & Allen H. Dutoit. Object-Oriented Software Engineering: Using UML, Patters, and Java

Fault prevention

Better design, better tools, ….

Fault detection

Testing, debugging, …

Fault removal

Fixing, patching, …

Fault tolerance

Redundancy, isolation, …

Fault forecasting

Estimating how many faults might

still be there

Page 43: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

43 | 58Embedded and Networked Systems

Maintainability• Ability to undergo modifications and repairs• It is the probability that a failing system can be

repaired within a certain interval of time• 𝑴𝑴(𝒅𝒅) = probability of system working correctly 𝑑𝑑 time units after an error occurred.

Dependability attributes

“Basic Concepts and Taxonomy of Dependable and Secure Computing”

Availability• Readiness for correct service• It is the probability that the system is available at

time 𝑡𝑡• High availability requires high maintainability and

reliability• 𝑨𝑨(𝒕𝒕): probability of system working at time 𝑡𝑡

Reliability• Continuity of correct service• It is the probability that the system will not fail if it

was working from the beginning• 𝑹𝑹(𝒕𝒕) = probability of system working correctly at

time 𝑡𝑡 provided that it was working at 𝑡𝑡 = 0

Safety• Absence of catastrophic consequences on the

user(s) and the environment

Integrity• Absence of improper system alterations

Page 44: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

44 | 58Embedded and Networked Systems

Assignment 0

• Read the following paper (all chapters)• “Basic Concepts and Taxonomy of Dependable and Secure Computing”• https://ieeexplore.ieee.org/document/1335465

• Answer the questions that will be uploaded on Brightspace• Due date is 8:45am Tuesday Nov. 26

• Notes• The exam will include chapters 1 to 4 of the paper. • Questions that will be asked on Brightspace are similar to what will appear in the exam. • There will be a quiz (with optional points) from the first four chapters of the paper on

Nov. 19th

Page 45: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

45 | 58Embedded and Networked Systems

Agenda• Course outline

• Definitions and concepts • Embedded systems specifications • Common requirements

• Modeling and model-based design• Modeling• Design process• Model-based design• Models of computation (MoC)

Sources: • [Book]: Lee and Seshia (chapter 1)• [Book]: Marwedel (chapter 1 and 2)

Page 46: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

46 | 58Embedded and Networked Systems

Modeling and Model-Based Design

Page 47: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

47 | 58Embedded and Networked Systems

Modeling, design, analysis

Modeling is the process of gaining a deeper understanding of a system through imitation.

Models specify what a system does.

Design is the structured creation of artifacts.

It specifies how a system does what it does. This includes optimization.

Analysis is the process of gaining a deeper understanding of a system through dissection.

It specifies why a system does what it does (or fails to do what a model says it should do).

Page 48: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

48 | 58Embedded and Networked Systems

Modeling, design, analysisReal system

http://ctms.engin.umich.edu/CTMS/index.php?example=InvertedPendulum&section=SimulinkModeling https://www.nomagic.com/mbse/images/whitepapers/SysML_Inverted_Pendulum_System.pdf

Models can be used to describe specification

This mathematical model expresses the physics of the plant

Models can be used to design the system Models can be used to analyze the system

Page 49: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

49 | 58Embedded and Networked Systems

Why is modeling important?• Systems getting more complex

• Abstraction is needed to understand the system or to design it

• Partitioning into subsystems is needed

Design space

Specifications

Intermediate models

implementation

Abst

ract

ion

gapDesign step

Abst

ract

ion

leve

l

Page 50: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

50 | 58Embedded and Networked Systems

Why is modeling important?• Models are a good base for communication between

engineers• Engineers think in diagrams

• Models are important for documentation• Standardized semantics• Formal language with (hopefully) only one meaning

• Models abstract from the very detailed implementation• Allow focusing on most important aspects

• Models can be used when the system architecture is not yet ready at early design stages in order to:• Evaluate functional and non-functional requirements• Find design faults

Page 51: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

51 | 58Embedded and Networked Systems

Model-driven v.s. model-based design

An interesting readIBM Harmony: https://www.ibm.com/support/knowledgecenter/SSB2MU_8.3.0/com.btc.tcatg.user.doc/topics/atgreqcov_SecSysControllerHarmony.htmlOther source: http://www.win.tue.nl/~pcuijper/docs/QEES/DF/QEES%20introcollege%202013-2014.pptx

Model-driven design: use models to automate the arrows

Model-based design: use models to get a grip on the boxes

Example: The IBM Harmony process (for IoT) is a model-driven design.

Page 52: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

52 | 58Embedded and Networked Systems

When do we create a model?

To design a given specification To understand, evaluate, analyze, or optimize a given system

Use modeling to design the

system

End user

Development team Use modeling to

understand, analyze, or optimize the system

Quantitative evaluationand optimization

End user

Development team

Page 53: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

53 | 58Embedded and Networked Systems

Models of computation (MoC)

MoC categories:

State-based MoCs• Finite state machines• Hierarchical state machines• Process state machines• Pushdown automata• Turing machine

Source: http://users.ece.utexas.edu/~gerstl/ee382v_s14/notes/lecture5.pdf

A model of computation (MoC) is a model that describes how an output of a function is computed given an input

Process-based MoCs• Process networks

• Kahn process network• Data flow networks

• Petri nets• Synchronous data flow

• Process calculi

Signal

No signal for 2 minutesS

B

NSignalNo signal

Low battery

Low batteryH

Low battery

Page 54: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

54 | 58Embedded and Networked Systems

Recap: an overview of the courseIntroduction

Measurement-based performance evaluation

Statistical modeling

Modeling concurrent programs

(design and performance analysis)

Modeling state-based programs and queues

(performance analysis)

Modeling and model-based design (1 lecture)

Measurement-based evaluation and statistics 101 (1 lecture)

Design of experiments (2 lectures)

Petri-nets and data-flow networks (2 lectures)

Markov models (2 lectures)Operational laws and queueing theory (2 lectures)

Design-space exploration and optimization (1 lecture)

Memory efficiency and optimization (1 lecture)

Team presentations, exam examples, FAQ (1 lecture)

Model-based design optimization

(tentative)

Page 55: Introduction - TU Delft• Lect. 1: Modeling and model-based design • Lect. 2: Measurement-based performance evaluation Lect. 5: Petri-nets • Lect. 6: Data-flow networks • Lect

55 | 58Embedded and Networked Systems

Questions?Don’t forget Assignment 0