SPECTR - duttgroup.ics.uci.eduduttgroup.ics.uci.edu/wp-content/uploads/2018/05/SPECTR-formal... ·...

FormalSupervisoryControlandCoordinationforMany-coreSystemsResourceManagement

AmirM.Rahmani BryanDonyanavard TiagoMück Kasra MoazzemiAxelJantsch Onur Mutlu NikilDutt

SPECTR

23rd ACM International Conference on Architectural Support for Programming Languages and Operating SystemsWilliamsburg, VA, March 2018

ExecutiveSummary• Motivation:

• Formalsupervisorycontroltheory(SCT) cancombine thestrengths ofclassicalcontroltheorywithheuristicapproachestoefficientlymeetchangingruntimegoals.

• SCTenableshierarchical control andfacilitatesautomaticsynthesisofthehigh-levelsupervisorycontrolleranditspropertyverification.

• Problem:Currentresourcemanagementtechniquesdonotoffer1)robustness,2)formalism,3)efficiency,4)coordination,5)scalability,and6)autonomyalltogether.

• Goal:Addressallsixkeychallengesinheterogeneousmultiprocessors(HMPs)resourcemanagement,inparticularscalability andautonomy

• OurProposal:SPECTR usesSCTtechniquessuchasgainscheduling toallowautonomyforindividualcontrollers,andmodulardecompositionofcontrolproblemstomanagecomplexity.

• Evaluation:1. WeimplementSPECTRonanExynos platformcontainingARM’s

big.LITTLE-based HMP2. SPECTRcanmanagemultipleinteractingresources(e.g.,chippowerand

processingcores)inthepresenceofcompetingobjectives(e.g.,satisfyingQoS vs.powercapping)

3. SPECTRachievesupto8x and6x bettertargetQoS andpower trackingoverstate-of-the-art,respectively(inourcasestudy).

2

SPECTR Outline

3

MotivationMIMOControl TheoryforCoordinatedManagementUnaddressedChallengesinResourceManagement

AutonomyScalability

SupervisoryControlTheory(SCT)viaSPECTRScalabilityand AutonomythroughSCTSPECTR OverviewCaseStudy

CaseStudyResultsSummary

SupervisorSynthesisProcess

SummaryResults

SPECTR Outline

4

MotivationMIMOControl TheoryforCoordinatedManagementUnaddressedChallengesinResourceManagement

AutonomyScalability

SupervisoryControlTheory(SCT)viaSPECTRScalabilityand AutonomythroughSCTSPECTR OverviewCaseStudy

CaseStudyResultsSummary

SupervisorSynthesisProcess

SummaryResults

● Severalconflicting goals/constraints

● Multipletunableknobs

● Adhocheuristics○ Canbesub-optimal○ Noformalmethodology○ Noguarantees

ResourceManagementinMany-coreSystems

5

ChallengesinResourceManagement

Majoron-chipresourcemanagementapproachesandthekeyquestionstheyaddress(∗ =partiallyaddressed)

Methods Robustness Formalism Efficiency Coordination Scalability Autonomy

A Machinelearning ✔ ✔ ✔

B Estimation/Modelbasedheuristics

✔ ✔

C SISOControlTheory ✔ ✔ ✔ ✹

D MIMOControlTheory ✔ ✔ ✔ ✔

E SupervisoryControlTheory[SPECTR]

✔ ✔ ✔ ✔ ✔ ✔

Guaranteed coordination ofmultipleconflictinggoalshasbeenrecentlyaddressedforsingle-core systemsviaMIMO

controltheory[ISCA’16].Whatifgoalchangesatruntime(Autonomy)?Canweofferasystematicdesignflowforhierarchicalcontrol(Scalability)?

6

TheGoal

SPECTR Outline

7

MotivationMIMOControl TheoryforCoordinatedManagementUnaddressedChallengesinResourceManagement

AutonomyScalability

SupervisoryControlTheory(SCT)viaSPECTRScalabilityand AutonomythroughSCTSPECTR OverviewCaseStudy

CaseStudyResultsSummary

SupervisorSynthesisProcess

SummaryResults

Benefits:● Simultaneouslyandrobustly trackmultipleobjectives

8

MIMOControlTheoryforCoordination

Benefits:● Simultaneouslyandrobustly trackmultipleobjectives

Shortcomings:● Thegoal isfixed atdesign-time

9

MIMOControlTheoryforCoordination

Theweighted TrackingErrorCostmatrix isfixed.

FPS:Power <=1:10whenMaximizingFPSunderaPowercap

Benefits:● Simultaneouslyandrobustly trackmultipleobjectives

Shortcomings:● Thegoal isfixed atdesign-time● DoesNOT scale whenhavingseveralcontrolinputsand

measuredoutputs.

10

MIMOControlTheoryforCoordination

SPECTR Outline

11

MotivationMIMOControl TheoryforCoordinatedManagementUnaddressedChallengesinResourceManagement

AutonomyScalability

SupervisoryControlTheory(SCT)viaSPECTRScalabilityand AutonomythroughSCTSPECTR OverviewCaseStudy

CaseStudyResultsSummary

SupervisorSynthesisProcess

SummaryResults

12

TheAutonomyChallenge:AnExample

Whatifthegoalchangesatruntime?

Weneedtheabilitytoswitchmodesatruntime

AMIMOcontrollerdesignedwithhigherpriorityonQoSoverpower

Poweriscapped.QoS isnotmet!

QoS ismet.Powerviolation!

AMIMOcontrollerdesignedwithhigherpriorityonpoweroverQoS

13

System(x)

Black-boxIdentificationof

SystemDynamics

TheScalabilityChallenge:Example1

Whatifthe# controlinputsandmeasuredoutputs islarge?

14Weneedtolimitthesystemsize

System for 2 x 2 MIMO

Asetofworkloads

Asetofworkloads

TheScalabilityChallenge:Example1

Prediction is very accurate

Prediction greatly diverges from reality

F level 1

F level N

1 core is active

All cores are active

15

Whatifthe# controlinputs andmeasuredoutputs islarge?

System(x)

Controller

TheScalabilityChallenge:Example2Controller

DesignComplexity

16

Usingonelargecontrollerisnotfeasible!

TheScalabilityChallenge:Example2

Howmanyoperations areexecutedineachcontrolepochforasinglelargeMIMOcontrollingNcores?

SPECTR Outline

17

MotivationMIMOControl TheoryforCoordinatedManagementUnaddressedChallengesinResourceManagement

AutonomyScalability

SupervisoryControlTheory(SCT)viaSPECTRScalabilityand AutonomythroughSCTSPECTR OverviewCaseStudy

CaseStudyResultsSummary

SupervisorSynthesisProcess

SummaryResults

UsingSupervisoryControlTheorywe…

● Provideautonomy viaadaptation inresponsetochangesinpolicy○ Computecontrolparametersfordifferentpolicies

offline

● Providescalability viadecomposition ofsystemintomultiplesubsystemsorganizedinahierarchy○ Supervisor provideshigh-levelmanagement

SPECTR

18

ScalabilityviaSupervisoryControl

High-levelPlantModel(Phi)

Plant(Plo)

SupervisoryController(Chi)

Low-levelController(Clo)

Comhi_lo

Conlo

Inflo

Inflo_hi

Conhi

Infhi

High-level Virtual Control

Combininglogicwithdiscrete

dynamics

19Low-levelTraditionalControlLoop

Representsanabstraction ofthePlant

Informationchanneltoupdatethehigh-levelmodel

SupervisoryControllerusesthischanneltocontrolthehigh-levelmodel

Theactual controlhappensviadiscreteevent-basedcommands

AutonomyviaSupervisoryControl

ThisSCTtechnique iscalledGainScheduling

FPS:Power <=1:10

FPS:Power <=10:1

20

Controlparameterspre-designedtoprioritize onemeasured

outputovertheother(s)

Trackingpower is10xmoreimportant than

trackingQoS

TrackingQoS is10xmoreimportant than

trackingpower

SPECTR Outline

21

MotivationMIMOControl TheoryforCoordinatedManagementUnaddressedChallengesinResourceManagement

AutonomyScalability

SupervisoryControlTheory(SCT)viaSPECTRScalabilityand AutonomythroughSCTSPECTR OverviewCaseStudy

CaseStudyResultsSummary

SupervisorSynthesisProcess

SummaryResults

Sub-plant1 Sub-plant2 Sub-plantN

SupervisoryController

VariableGoalsandPolicies

High-levelPlantModel

ClassicController1

ClassicController2

ClassicControllerN

Userinputs

Con_lo1

Inf_lo1

Con_lo2

Inf_lo2

Con_loN

Inf_loN

Gains1 Refs1Gains2 Refs2

GainsN RefsN……

……

Con_hiInf_hi

Inf_lo_hiLeaf

Cont

rolle

rs

Phys

ical

Plan

t

Systemevents

SPECTR

SPECTRoverview

22

Puttinghierarchicalcontrolandgainschedulingtogether!

Thesupervisorupdatesgoalsandallocatesresourcesatruntime

SPECTR Outline

23

MotivationMIMOControl TheoryforCoordinatedManagementUnaddressedChallengesinResourceManagement

AutonomyScalability

SupervisoryControlTheory(SCT)viaSPECTRScalabilityand AutonomythroughSCTSPECTR OverviewCaseStudy

CaseStudyResultsSummary

SupervisorSynthesisProcess

SummaryResults

● 8-corebig.Little HMP● Twosetofapplications:

○ AforegroundapplicationwithQoSrequirements(e.g.,FPS)

○ AnumberofbackgroundapplicationswithnoQoSrequirements

CaseStudy

24

ODROID-XU3 platform contains an Exynos 5422 Octa-core SoC

● Controlknobs:per-cluster DVFS,numberofidlecores● Systemgoals:

○ MeettheQoSrequirementoftheforegroundapplication○ Ensurethetotalsystempower alwaysremainsbelowthe

ThermalDesignPower(TDP)○ Minimizeenergyconsumption

CaseStudy

25

SPECTR Outline

26

MotivationMIMOControl TheoryforCoordinatedManagementUnaddressedChallengesinResourceManagement

AutonomyScalability

SupervisoryControlTheory(SCT)viaSPECTRScalabilityand AutonomythroughSCTSPECTR OverviewCaseStudy

CaseStudyResultsSummary

SupervisorSynthesisProcess

SummaryResults

5stepstodesign andverify asupervisor:

SupervisoryController

SupervisorSynthesisProcess

27

SupervisoryController

Step1:PlantModel

28Manuallymodeledsub-plants Synthesizedplant

MultiplecharacteristicsareautomaticallysynthesizedusingSCTtools

E.g.Supremica

To develop possible actions inhigh-level plant model in a form ofdiscrete-event dynamics

PrioritizeQoS:Thepowerreference isupdatedtomeettheQoS referenceinanenergy-efficientmanner.

Powerbudgetviolationgeneratesacritical eventandresultsingainswitchingtowardsthepower-drivengoal.

SupervisoryController

Step2:IntendedBehaviorSpecification

29

ForbiddenState

A specification defines theaccepted and forbidden states viarestrictions on the behavior of theplant model.

This examplespecificationprevents exceedingthe power budget forno more than threecontrol intervals.

Note:ThemodelinStep1hasno limitations!(e.g.,onexceedingthepowerbudget)

SupervisoryController

Steps3-5:SynthesisandVerification

30

The synthesizer generates aminimally-restrictive controllerfor the given plant model andspecifications.

Automatically generatedandverifiedusing synchronouscompositionoperationsin

Supremica SCTtool.

SupervisoryController

Steps3-5:SynthesisandVerification

31

SCTtools(e.g.,Supremica)alsoverify thenon-blocking andcontrollability propertiesofthesyntheziedcontroller.

Non-blocking: Acceptedstates(e.g.,idealstates)canalwaysbereached.

Stable

Controllability: Thereisapathtotheacceptedstatesfromeveryothervalidstate.

SPECTR Outline

32

MotivationMIMOControl TheoryforCoordinatedManagementUnaddressedChallengesinResourceManagement

AutonomyScalability

SupervisoryControlTheory(SCT)viaSPECTRScalabilityand AutonomythroughSCTSPECTR OverviewCaseStudy

CaseStudyResultsSummary

SupervisorSynthesisProcess

SummaryResults

● QoSapplications:○ PARSECapplications:x264,bodytrack,canneal,

streamcluster○ Data-intensivemachinelearningworkloads:k-

means,KNN,leastsquares,linearregression

● ComparedSPECTRwiththreealternativeresourcemanagers○ MM-Pow: 2x2MIMOs(onepercluster)with

gainsoptimizedtotrackpower○ MM-Perf: 2x2MIMOs(onepercluster)with

gainsoptimizedtotrackperformance/QoS○ FS:singlesystem-wide 4x2MIMOwithgains

optimizedtowardspower

Evaluatedresourcemanagerconfigurations

Fixed-Objective

33

Executionscenariowiththreephases(x264):

1. Phase1- SafePhase:only theQoS applicationruns;powerlimitedbyTDP

2. Phase2- Emergencyphase:powerlimitsetto1WbelowTDPtoemulateathermalemergency

3. Phase3- Workloaddisturbancephase:powerlimitrestoredtoTDP,butnowseveralbackgroundtasksstart,interferingwiththeQoSapplication

ExperimentalResults– ContollerEvaluation

34

EnoughpowertotrackQoS

Onlyfeasibletotrackpower

TrackQoSunderapowercap

MM-Pow

MM-Perf

MM-FS

SPECTR

● QoStask:x264● Controller:MM-Pow(power-oriented)

○ 2x2MIMOs(onepercluster)withgainsoptimizedtotrackpower

ExperimentalResults-- ContollerEvaluation

35

~40%morethennecessaryFPSinPhase1

Wastingenergy!

UnderapowercapWastedperformance

ItworksfineinPhase2 and3 byfocusingonpowercapping!

36

ExceedsTDPby~30%inPhase3!

TrackingFPS Exceedingpowerlimit

ExperimentalResults-- ContollerEvaluation● QoStask:x264● Controller:MM-Perf(performance-oriented)

○ 2x2MIMOs(onepercluster)withgainsoptimizedtotrackQoS

ItworksfineinPhase1 and2 byfocusingonQoS tracking!

● QoStask:x264● Controller:FS(large4x2power-oriented)

○ Singlesystem-wide 4x2MIMOwithgainsoptimizedtowardspower

37

~40%morethannecessaryFPSinphase1(akintoMM-POW)

SluggishresponseWastedperformance

ExperimentalResults-- ContollerEvaluation

LongersettlingtimeduetolargeMIMOcontroller.

● QoStask:x264● Controller:SPECTR

38

Let’sTakeacloserlookateachphase

ExperimentalResults-- ContollerEvaluation

SafePhase:QoSApponlySPECTR focusesonsatisfyingFPSwiththeminimumpower

39

<5%FPSsteadystateerror(minimalwastedperfomance)

PowerbelowTDP

ExperimentalResults-- ContollerEvaluation

EmergencyPhase:TDPreducedinresponsetothermaleventSPECTR satisfiesthereferenceFPSandpower

40

<5%FPSsteadystateerror

ExperimentalResults-- ContollerEvaluation

DisturbancePhase:TDPreturnedtonormal,backgroundtasksaddedSPECTR focusesonpowercapping

41

NoTDPviolations,but~23%FPSsteadystateerror

(impossibletotrackwithoutviolatingTDP)

ExperimentalResults-- ContollerEvaluation

● Accuracy oftheidentifiedsystemmodelsofdifferentsizedMIMOcontrollers

● Amodeloutputwithintheconfidenceinterval indicatesthatthedeterministic componentofthemodeloutputwillbenearthetrueoutput.

42

ExperimentalResults-- Scalability

Confidencelevelof99% Confidencelevelof99%

Outsidetheconfidenceinterval

Black-boxsystemidentificationisnotfeasibleforlargeandcomplexMIMOsystems!

Withintheconfidenceinterval

● AdetailedContollerEvaluationon:○ PARSECapplications:bodytrack,canneal,streamcluster○ Data-intensivemachinelearningworkloads:k-means,

KNN,leastsquares,linearregression

● ModelAccuracyAnalysisofdifferentsizedMIMOcontrollers:○ 2x2->feasibleandefficient○ 4x2->feasiblebutsluggish○ 10x10->notfeasible

● Furtherdiscussionon:○ Controllerresponsiveness(settlingtime)○ Controllerstability 43

OtherResultsinthePaper

SPECTR Outline

44

MotivationMIMOControl TheoryforCoordinatedManagementUnaddressedChallengesinResourceManagement

AutonomyScalability

SupervisoryControlTheory(SCT)viaSPECTRScalabilityand AutonomythroughSCTSPECTR OverviewCaseStudy

CaseStudyResultsSummary

SupervisorSynthesisProcess

SummaryResults

● Resourcemanagersneedtooffer 1)robustness,2)formalism,3)efficiency,4)coordination,5)scalability,and6)autonomyalltogether

● SPECTR offersthemall!○ SPECTRadapts to changinggoalsatrutime○ SPECTR decomposesthecontrolproblemsto manage

itscomplexity

● SPECTR achieves upto8x and6x bettertargetQoS andpower tracking overstate-of-the-art,respectively(inourcasestudy)

● SPECTRisapplicabletoanyresourcetypeandobjectiveaslongasthemanagementproblemcanbemodeledusingdynamical systemstheory

45

Summary

FormalSupervisoryControlandCoordinationforMany-coreSystemsResourceManagement

AmirM.Rahmani BryanDonyanavard TiagoMück Kasra MoazzemiAxelJantsch Onur Mutlu NikilDutt

SPECTR

23rd ACM International Conference on Architectural Support for Programming Languages and Operating SystemsWilliamsburg, VA, March 2018

47

Step1

Step2

Step3

Step9

Step6

Step5

Step7 Step8

SPECTRDesignFlow

48

49

Steady-stateErrorforAllBenchmarks

Steady-stateerrorforallbenchmarks,groupedbyphase.Anegativevalueindicatestheamountofpower/QoS exceedingthereferencevalue(bad),apositivevalueindicatestheamountofpowersaved(good)orQoS degradation(bad)

50

ModelAccuracy• Autocorrelationof

residualsforidentifiedsystemmodelsofdifferentsizedMIMOcontrollers.

• Weshowasingleperformanceandpoweroutputforeachmodeledsystemacrossmultiplesampleinputs.