Workshop Performance engineering and architecting · 2017-10-10 · 25 October 3, 2017 Workshop Performance engineering and architecting In 20 minutes, as a group, map your performance

06/10/2017

1

© 2017 Embedded Systems Innovation by TNO

Workshop Performance engineering and architecting Twan Basten Jeroen Voeten

October 3, 2017


Workshop agenda

10 min – Introduction

• Examples

• Goal/organization

20 min – Group discussion 1

• Performance challenges

20 min – Performance architecting and engineering landscape

20 min – Group discussion 2

• Mapping of performance challenge on performance landscape and process

05 min – Conclusion

October 3, 2017 Workshop Performance engineering and architecting 2

06/10/2017

2


Introduction

Twan Basten



Warehouse performance

• warehouse: storage racks, different units to move and collect products

• varying collection of products, and

• varying orders

Simulation and visualization • configurable simulator • visualization of flows

How to organize products in a warehouse minimizing order fulfillment time?


06/10/2017

3


Paper path engineering for professional printers The paper path consists of • physical components: segments

transporting, turning, accelerating, decelerating, printing, heating or cooling sheets

• scheduler: decides sheet release times, duplex sheet merge times, maintenance

• productivity: 150 sheets A4/minute duplex, steady state

• diverse print jobs like complete magazines • time-consuming reconfiguration and

maintenance

How to design a paper path that maximizes productivity? Performance engineering • analytical performance estimators • critical path analysis



Data path performance engineering for printers

Performance numbers show linear relations between

• bitmap size and execution time

• cores and execution time

Predictions multiprocessor platforms

? +

+

Ink

Paper

Data

Print Engine Documents Data path: COTS components • multi-core CPUs • embedded processing (CPU, GPU, …) • interconnect • storage

How to design a cost-effective data path with sufficient productivity ?

Target: 100+ images per minute

Analytical model • regression analysis • design-space exploration


06/10/2017

4


Data path performance engineering for printers

Ink

Paper

Data

Print Engine Documents

Does an extra GPU provide the desired productivity increase?

Refactor the copy path

scan export

rip print

Copy path

Scan path

Print path

Discrete-event simulation • building a prototype is expensive • throughput as a function of latency of GPU processing

• Reduces load on 4-core CPU • Potential bottleneck at GPU



Motion control timing for lithography systems

How to deploy and schedule high-end motion control applications on an execution platform guaranteeing a latency of at most 50 µs?

-

wafer stage

position sensors

position measurement

setpoint profile

power actuators

position control

≤ 𝟓𝟎𝝁𝒔

Application • 50 servo controllers • 4,000 control tasks • 20,000 task dependencies

Platform • 10 multi-core processors • packet-switched interconnect

Requirements • sample frequency: 20 kHz • IO latency: 50 μs

Performance engineering • dataflow analysis • simulation


06/10/2017

5


From system performance to embedded performance



Performance Engineering and Architecting

Observations

• Performance is a cross-cutting concern

• Performance often emerges in late development phases

• Optimizing performance is expensive and difficult

Performance engineering and architecting: Controlled performance in early development phases → Performance by construction

Benefits

• Less rework → Shorter time-to-market

• Less over-dimensioning → Improved cost-performance ratio


06/10/2017

6


Goal of the workshop today

Identification of common performance challenges

now and in the future

as a basis for

• Special Interest Group (SIG) on performance

• training

• projects



Group discussion 1: Performance challenges

20-minute group discussion:

• as a group identify three performance challenges from your domains

• write them on post-its and put these on the sheets

• use additional post-its to describe challenge details October 3, 2017 Workshop Performance engineering and architecting 12

06/10/2017

7


Performance architecting and engineering landscape

Jeroen Voeten



Outline

• Performance engineering and architecting: ESI approach

• Model-based performance engineering: process

• Model-based performance engineering: landscape

• Example: high-end cut-sheet Océ printer


06/10/2017

8


Performance Engineering and Architecting

Goal

• Control performance in early design phase

• Risk reduction: effectively and efficiently

• Less over-dimensioning: improved cost-performance ratio

ESI Approach

• Model-based: models used in development process to

• predict performance impact of architecting / engineering choices

• explore architecting/ engineering alternatives

• Model-driven: models used as pivotal information carriers to

• predict and explore performance

• drive the implementation: performance-by-construction



Model-based performance engineering: process

Requirements w.r.t. performance.

1

Model design alternatives to satisfy

the requirements.

4

Predict the past: Measurements on

existing system are used for calibration and

validation of model X. Gives prediction

accuracy and builds trust.

3

Validation: Retrospective model validation builds

experience.

6

Initial model based on requirements.

2

Explore the future: Predict the future with the models for the design alternatives and analyze results following the requirements. Results are

interpreted and feedback is given to development process.

5


06/10/2017

9


Model-based performance engineering: landscape

• expert knowledge • requirements • architecture • design • measurements

• discrete • data flow • best/worst case

• discrete • control flow • best/worst case

• discrete • data & control flow • stochastic

• continuous


Domain models • Coherent domain

conceptualization • Textual/graphical views

and editing front-ends

Aspect models • Different kinds of

mathematical models • Excelling in different

domains and for different questions

Artefacts • Documents, white

board pictures, files • Human & machine

processable

transformations transformations

• Y-chart pattern • explicit variation points

Application Platform

Mapping

performance prediction


Domain models • Coherent domain

conceptualization • Textual/graphical views

and editing front-ends

Aspect models • Different kinds of

mathematical models • Excelling in different

domains and for different questions

Artefacts • Documents, white

board pictures, files • Human & machine

processable

transformations transformations

Model-based performance engineering: landscape & process


Requirements

1

Initial model

2

Predict the past

3

Model design alternatives

4 Explore the

future

5

Validation

6

06/10/2017

10


Platform-cost reduction

Goal Reduce HW costs: move to a cheaper platform Achieve required throughput for actuator Building a prototype requires significant (SW engineering) effort. Can a model be used to reduce the risks?

Requirements

Ink

Paper

Data

Print Engine Documents





Model relations with regression Application - Execution time: relate processing time per page on single core to data size Platform - Execution time: relate execution time to using more cores

Requirements

Initial model


Platform A Application

Mapping

prediction

06/10/2017

11




Apply model for reference platform Calibrate model using measurements Validate the model for reference platform determine error between measurements and prediction

Requirements

Initial model

Predict the past

Prediction reference platform

Initial model Measurements

Reference platform



Platform performance scaling Predict processing times of application on alternative hardware platforms. We used a publicly available CPU performance benchmark: https://www.cpubenchmark.net/singleThread.html

Requirements

Initial model

Predict the past



Platform X Application

Mapping Y

Cross-platform prediction October 3, 2017 Workshop Performance engineering and architecting 22

https://www.cpubenchmark.net/singleThread.html

06/10/2017

12



Explore domain Using the benchmark scores and CPU costs search cost-effective solutions that achieve productivity

Requirements

Initial model

Predict the past


Explore the future

Result: Model indicates cost-effective target platform that achieves performance


Platform X Application

Mapping Y

Prediction platforms




Compare predictions and measurements At selected target platforms, compare measurements and predictions Preferred platform: around 3% difference observed Alternative platforms: around 10% difference observed

Explore the future

Validation


Requirements

Initial model

Predict the past


06/10/2017

13


Group discussion 2: Mapping of performance challenges


In 20 minutes, as a group, map your performance challenges on the landscape

• use post-its and put these on the sheets


Conclusion


• Round-table feedback:

• common grounds for SIG, training, projects?

06/10/2017

14


Performance engineering and architecting

Thank you for participating

Jeroen Voeten, Twan Basten, Jacques Verriet,

Martijn Hendriks, Tjerk Bijlsma

TNO-ESI October 3, 2017
