Upload
vuongdieu
View
233
Download
0
Embed Size (px)
Citation preview
Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Cyber-Physical V&V Challenges for the Evaluation of State of the Art Model CheckersResearch in Quantum Enabled V&V Technology
July 12-14, 2016
Chris ElliottFlight Controls / Quantum Computing
2Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Overview
I. Quantum Enabled V&V Overview
II. Overview of 10 V&V Challenge Problems
III. End to End Analysis Example
IV. Summary
3Copyright 2016, Lockheed Martin Corporation. All rights reserved.
What is it? • QVTrace*: This technology is a method for Software Verification &
Validation using Quantum Computer Assisted Formal Methods.
RequirementsAnd Implementation
(Software Code)
Classical Computation
D-Wave Adiabatic Quantum Computer
Defects (Bugs)Req/Code
InconsistencyReport to Designer
Quantum V&V
*Product Developed by QRA Inc.
• Target Users are System/Software Design Teams interested in:- Reducing development costs - Improving final product quality
Who will use it?
Quantum Enabled V&V
4Copyright 2016, Lockheed Martin Corporation. All rights reserved.
LM QA Solves a Quadratic Unconstrained Binary Optimization Problem
D-Wave Adiabatic Quantum ComputerCurrent State-of-the-Art, DW-2X: 1152q Washington
5Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Quantum Superposition, Entanglement Enable Unique Optimization
Quantum Optimizationwith Superconducting Qubits
6Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Early 2010LM ID’s
Quantum asKey Tech
Nov 2010Early Access
To QC
Mar 2011USC/ISI/LMTeam for QC Center
Jan 2012USC-LM
QC Operational
Mar 2013QC Upgrade
March 2016QC Upgrade
128 q DW1 “Rainier”512 q DW2 “Vesuvius”
1152 q DW2X “Washington”
QE-V&V Timeline
7Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Overview of Challenge Problems• LM Aero Developed Set of 10 V&V Challenge Problems
• Goal: - Foster Collaboration in S5 Community (Ponder, Present, Publish)- Evaluate & Improve State-of-the-Art Formal Methods Toolsets
• Each Example in Package Includes:- Simulink Model Built in Matlab® R2012B- Parameters, if any, for Simulating Model (.mat)- Documentation Containing Description and Requirements
• Difficult due to Transcendental Functions, Nonlinearities and Discontinuous Math, Vectors, Matrices, States
Challenges Built with Commonly Used Blocks
8Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Overview of Challenge Problems1. Triplex Signal Monitor2. Finite State Machine3. Tustin Integrator4. Control Loop Regulators5. Nonlinear Guidance Algorithm6. Feedforward Cascade Connectivity Neural Network7. Abstraction of a Control Allocator (Effector Blender)8. 6DOF with DeHavilland Beaver Autopilot*9. System Safety Monitor10. Euler Transformation
Flight Control and Vehicle Management System Inspired Problems
9Copyright 2016, Lockheed Martin Corporation. All rights reserved.
1. Triplex Signal Monitor
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
Sensor A
Sensor B
Sensor C
Airborne Redundancy Management
OnlineMonitoring
10Copyright 2016, Lockheed Martin Corporation. All rights reserved.
2. Finite State Machine
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
Discrete Interwoven Modes in Integrated Cyber-Physical System
Flight ControlEmbedded System
IntegratedSensor
11Copyright 2016, Lockheed Martin Corporation. All rights reserved.
3. Tustin Integrator
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
Fundamental Modeling and Simulation Component
NumericalIntegration
12Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Kp
Gain
Kd
Gain1
Ki
Gain3
1s
Integrator
s
s+1
Transfer Fcn
4. Control Loop Regulators
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
CommandAuthority?Feedback
ErrorSynthesis
PIDArchitecture
Attributes of Multi-Axis Control Law of Output Commands
13Copyright 2016, Lockheed Martin Corporation. All rights reserved.
5. Nonlinear Guidance Algorithm
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
3D Vector Mathematics for Outer Loop Intercept Guidance
Aim Point Validity?
14Copyright 2016, Lockheed Martin Corporation. All rights reserved.
5. …Nonlinear Guidance Algorithm
Block TypesFor NL Guidance
Recent Focus on Import of Common Algorithmic Operators (Primitives)
15Copyright 2016, Lockheed Martin Corporation. All rights reserved.
6. Neural Network
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
2x10x10x1 Feedforward Cascade Connectivity NN
Output Features?
Inputs Layer 1 Layer2 Output
2-y
1-x
Network Topology
Inpu
ts
0.0 - 0.293130.29313 - 0.586260.58626 - 0.879380.87938 - 1.17251.1725 - 1.46561.4656 - 1.75881.7588 - 2.0519
PositiveNegative
-0.2
2
0
0.2
0.4
1
z
0.6
0.8
2
1
y
1.50 10.5
x
0-1 -0.5-1
-1.5-2 -2
Truth Model
16Copyright 2016, Lockheed Martin Corporation. All rights reserved.
… and More
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
Cyber-Physical V&V Challenge ProblemsLM Aeronautics Quantum Information Science Research Team 2015
Copyright © 2015 Lockheed Martin Corporation
AD
ID
AP Eng
HDG Mode
ALT Mode
HDG Ref
Turn Knob
ALT Ref
Pitch Wheel
Aileron Cmd
Elevator Cmd
Rudder Cmd
Autopilot
AC Bus
ID
AD
Sensors
Aileron
Elevator
Rudder
Flap
Throttle
Rudder Trim
Controls
Signal Conditioning
trim_flap
Constant
trim_throttle
Constant1
trim_rudder
Constant2trim_hdgref
Constant3
trim_turnknob
Constant4
trim_altref
Constant5
trim_pitchwheel
Constant6
DeHavilland Beaver Airframe
EnvBus
Environment
DeHavilland Beaver model originally based on work created by
Marc Rauw for Delft University of Technology, http://www.dutchroll.com
and subsequently modified by the Mathworkshttp://www.mathworks.com/matlabcentral/fileexchange/
FLIGHT CONTROL DemonstrationAuthor: elliocmModel Version: 1.80Date: 21-Sep-2015 15:23:12
boolean
Data Type Conversion1
boolean
Data Type Conversion2
boolean
Data Type Conversion3
1
Constant7APeng
HDGmode
HDGref
TurnKnob
17Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
End to End Analysis (Tustin)Cyber-Physical V&V Challenge Problems
LM Aeronautics Quantum Information Science Research Team 2015Copyright © 2015 Lockheed Martin Corporation
1
yout
2reset
1
xin
3T
xin
T
TL
BL
reset
ic
yout
TustinIntegrator
(Limited, Resettable, States)
4ic
5
TL
6
BL
cmd
Definitions:• Normal operation: the integrator is not in reset mode, and the
output is within the specified limits (TL and BL).• ypv: prior yout value• xinpv: prior xin input value• SP: Saturation Point
Input Signal to Be Integrated
Discrete Time Step
Top Limit
Bottom Limit
Boolean Reset
Initial Condition Upon Reset
Output Signal
Documentation Provides ICD, Definitions, and Requirements
18Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
End to End Analysis (Tustin)
TUSTIN INTEGRATOR (LIMITED, RESETTABLE, STATES)
Product
~=
Switch
upu
loy
SaturationDynamic
1
yout
.5
Gain
5reset
1xin
2T
4BL
3TL
6ic
z1
Unit Delay
z1
Unit Delay1
[TL]
Goto
TL
BL
TLc
BLc
bounds
[BL]
Goto1
[TL]
From
[BL]
From1
19Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
End to End Analysis (Tustin)
~=
Switch1
<
Relational Operator
1TL
2BL
1TLc
~=
Switch2
2BLc
20Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Requirements: 1. When Reset is True and the Initial Condition (ic) is bounded by the provided Top and
Bottom Limits (BL <= ic <= TL), the Output (yout) shall equal the Initial Condition (ic).2. The Output (yout) shall be bounded by the provided Top and Bottom limits (TL and BL)3. When in normal operation, the output shall be the result of the equation, yout = T/2*(xin
+ xinpv)+ ypv4. The Output of this function shall approximate the integration of the value of the input
signal over time within a specified tolerance, defined in subtests below:a. After 10 seconds of Computation at an execution frequency of 10 hz, the Output
should equal 10 within a +/- 0.1 tolerance, for a Constant Input (xin = 1.0), and the sample delta time T = 0.1 seconds when in normal mode of operation.
b. Over a 10 second computational duration at an execution frequency of 10 hz, the Output should equal the sine of time t, sin(t), where time is defined as a vector from 0 to 10 by increments of 0.1 seconds within a +/- 0.1 tolerance for an input equal to the cosine of time t, cos(t), with the sample delta time T = 0.1 seconds when in normal mode of operation.
End to End Analysis (Tustin)
Requirements Properties (Tests) is At Least Half the Challenge
21Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Requirements: 1. When Reset is True and the Initial Condition (ic) is bounded by the provided Top and
Bottom Limits (BL <= ic <= TL), the Output (yout) shall equal the Initial Condition (ic).2. The Output (yout) shall be bounded by the provided Top and Bottom limits (TL and BL)3. When in normal operation, the output shall be the result of the equation, yout = T/2*(xin
+ xinpv)+ ypv4. The Output of this function shall approximate the integration of the value of the input
signal over time within a specified tolerance, defined in subtests below:a. After 10 seconds of Computation at an execution frequency of 10 hz, the Output
should equal 10 within a +/- 0.1 tolerance, for a Constant Input (xin = 1.0), and the sample delta time T = 0.1 seconds when in normal mode of operation.
b. Over a 10 second computational duration at an execution frequency of 10 hz, the Output should equal the sine of time t, sin(t), where time is defined as a vector from 0 to 10 by increments of 0.1 seconds within a +/- 0.1 tolerance for an input equal to the cosine of time t, cos(t), with the sample delta time T = 0.1 seconds when in normal mode of operation.
End to End Analysis (Tustin)
Detailed Formal Property Derivation:# 1. When Reset is True and the Initial Condition (ic) is# bounded by the provided Top and Bottom Limits (BL<=ic<=TL),# the Output (yout) shall equal the Initial Condition (ic).# If the Initial Condition is not bound by the Limits # during a Reset, the Output shall equal the saturation # point (nominally with TL>=BL, ic>=TL impl SP==TL and ic<=BL implSP==BL. # Off-nominally with TL<BL, ic, ic>=BL impl SP==BL and ic<=TL implSP==TL.((reset and ic<=TL and ic>=BL) impl yout == ic); #1a ((reset and ic>=TL and ic>=BL and TL>=BL) impl yout == TL); #1b ((reset and ic<=BL and ic>=BL and TL>=BL) impl yout == BL); #1c((reset and ic>=BL and ic<=TL and TL<BL) impl yout == BL); #1d((reset and ic<=TL and ic>=BL and TL<BL) impl yout == TL); #1e
22Copyright 2016, Lockheed Martin Corporation. All rights reserved.
End to End Analysis (Tustin)
Requirements Properties (Tests) is At Least Half the Challenge
Requirements: 1. When Reset is True and the Initial Condition (ic) is bounded by the provided Top and
Bottom Limits (BL <= ic <= TL), the Output (yout) shall equal the Initial Condition (ic).2. The Output (yout) shall be bounded by the provided Top and Bottom limits (TL and BL)3. When in normal operation, the output shall be the result of the equation, yout = T/2*(xin
+ xinpv)+ ypv4. The Output of this function shall approximate the integration of the value of the input
signal over time within a specified tolerance, defined in subtests below:a. After 10 seconds of Computation at an execution frequency of 10 hz, the Output
should equal 10 within a +/- 0.1 tolerance, for a Constant Input (xin = 1.0), and the sample delta time T = 0.1 seconds when in normal mode of operation.
b. Over a 10 second computational duration at an execution frequency of 10 hz, the Output should equal the sine of time t, sin(t), where time is defined as a vector from 0 to 10 by increments of 0.1 seconds within a +/- 0.1 tolerance for an input equal to the cosine of time t, cos(t), with the sample delta time T = 0.1 seconds when in normal mode of operation.
Detailed Formal Property Derivation:# Over a 10 second computational duration at an execution frequency of 10 hz, the Output should equal the sine of time t, sin(t), where time is defined as a vector from 0 to 10 by increments of 0.1 seconds within a +/- 0.1 tolerance for an input equal to the cosine of time t, cos(t), with the sample delta time T = 0.1 seconds when in normal mode of operation(xin{0}==1 and xin{1}==0.995 and … xin{100}==-0.83907 and T{all}==0.1 and reset{never} and (TL{all}>=BL{all}) and (yout{all}>BL{all}) and (yout{all}<TL{all})) impl (abs(yout{0}-0)<=0.1 and abs(yout{1}-0.099833)<=0.1 … and abs(yout{98}--0.36648)<=0.1 and abs(yout{99}--0.45754)<=0.1 and abs(yout{100}--0.54402)<=0.1);
Analytic vs Numerical
|Tustin Error| < .05
10 s
10 s
23Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Description: this challenge problem involves the verification of a redundancy management system using quantum simulation techniques. The p
End to End Analysis (Triplex)
Signal A
Signal B
Signal C
Threshold Level
Persistence Limit (Duration Trigger)
Fault Code
Given These Conditions, Prove the Correct Fault Report
FC: 0-nofail, 1-branchC, 2-branchB, 4-branchA# detailed formal property(abs(ia{all}-ib{all})>Tlevel{all} or abs(ia{all}-ic{all})>Tlevel{all} and PC>PClimit and PClimit{all}==1 and Tlevel{all}==1) impl (FC{3}==4);
24
25
26
0 0.5 1 1.5 2 2.5 30
5000
10000
t [sec]
inpu
ts
iaibic
0 0.5 1 1.5 2 2.5 3-1
-0.5
0
0.5
1
t [sec]
FC
Counter Example Data As a Test Harness to Model
27
0 1 2 30
5000
10000
t [sec]
inpu
ts
iaibic
0 1 2 3-1
-0.5
0
0.5
1
t [sec]
FC
0 1 2 30
1
2
3
4
t [sec]
inpu
ts
|ia-ib||ia-ic||ib-ic|
This is a ValidDefect DiscoveredBy QVTrace v0.9.1
Closer Inspection Yields a Problem
28
0 1 2 3 4 5 6 7 8 9 10-4
-2
0
2
t [sec]
inpu
ts
iaibic
0 1 2 3 4 5 6 7 8 9 100
0.5
1
1.5
2
t [sec]
FC
FC: 0-nofail, 1-branchC, 2-branchB, 4-branchA
Nominal Behavior
29
0 1 2 3 4 5 6 7 8 9 10-4
-2
0
2
t [sec]
inpu
ts
iaibic
0 1 2 3 4 5 6 7 8 9 10-1
-0.5
0
0.5
1
t [sec]
FC
FC: 0-nofail, 1-branchC, 2-branchB, 4-branchA
Faulty Behavior
30Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Summary and Path Forward• Round 1 V&V Challenge Problems In Use to Develop Novel QE-V&V
• Requirements Formalization is Difficult Alone and Reduces Defects - Requirements Properties (Tests) is At Least Half the Challenge- Beneficial to Front Load Design Process with Formalization- Need Near if Not Equivalent “Primitives” Capability in Properties- Interested in Deploying Challenges Requirements to SPeAR
• Goals:- Publish Results on Current Round of Challenges - Round 2 V&V Challenge Problems To Increase Complexity Further - Transition Formal Methods Analysis Process/Tools to Programs- Interested? Contact: Chris Elliott,
[email protected], 817-935-3054
Thank You
31Copyright 2016, Lockheed Martin Corporation. All rights reserved.
32Copyright 2016, Lockheed Martin Corporation. All rights reserved.
Dr. Edward H. “Ned” AllenChief Scientist and LM Senior FellowLockheed Martin Corporation
Mr. Greg TallantProgram Manager and LM Fellow
Lockheed Martin AeronauticsSkunk Works
Biography slide
Chris ElliottQuantum Apps Team
Lockheed Martin AeronauticsSkunk Works
Mr. Peter StanfillQuantum Apps TeamLockheed Martin AeronauticsSkunk Works
Dr. Kristen PudenzQuantum Apps Team
Lockheed Martin AeronauticsSkunk Works