Formal Methods for

Cybersecurity

Hélène Kirchner

16/01/2020 1

French Institute For Computer Science

and Applied Mathematics

Tunis

CYBERSECURITY STRATEGY OF THE EU

“For new connected technologies to take off, including e-payments, cloud computing or machine-to-machine communication, citizens will need trust and confidence. Unfortunately, […] almost a third of Europeans are not confident in their ability to use the internet for banking or purchases”

Tunis

CONNECTEDOBJECTS

•  20165.5BILLION

•  202020.8BILLION

2013,2017

16/01/2020 3

• What do we want to trust ?

•  Creditcard,passport•  Phone,Computer•  Car,Plane•  Medicaldevices(pacemaker)•  Infrastructures:water,electricity,publictransports,•  Internet,IoT•  Electronicvoting

Tunis

- 4

!7

16/01/2020 Tunis

What are Formal Methods useful for?

DualuseofFormalMethods(logicalsystem:syntax,deduction,models,semantics)-  tobuildreliablesoftware:givetrustandcertifysoftware-  toanalyseandunderstandexistingsoftwareorprocesses:provideexplainability

andconformitytoexpectations•  Chasingbugsinsoftware:bugsaresourceofvulnerabilities•  Modelingenvironment(conforming,adversarial,uncertain,unknown)and

expected(mis-)behaviour•  Riskanalysis:accidentalorintentionalthreats,faultsorattacks•  Recoverymechanisms•  Certification•  (Safety/Security/Privacy)bydesign

16/01/2020 -5 Tunis

Cyber systems requirements

withimpactonuser’strustandempowerment:•  Safety•  Security•  Privacy•  Transparency•  Accountability•  Certification•  Ethics

16/01/2020 -6 Tunis

01

16/01/2020 Tunis 7

Safety

•  Formalmodelingandverificationmethodshavebeensuccessfulinimprovingthesafetyandsecurityofsoftwareinareassuchasaeronautics…

•  …itisessentialtoimproveonthefoundationsandinterconnectionoftoolsandformalismsforinteractiveandautomatedprogramverificationsuchasCoq,Why3,F*,TLA+,aswellasvariousstaticanalyzers,suchasAstrée.Theautomationandtheexpressivityofthesetoolsmustbeimprovedsothattheycanscaletotheverificationoflargersoftwaresystems,andcertifybothqualitativeandquantitativeproperties.

PS2018-2022Inria

16/01/2020 - 8

Scientific strategic plan

Tunis

void mpi_swap(mpi *X, mpi *Y) { mpi T; memcpy((void *)(& T),(void const *)X,sizeof(mpi)); memcpy((void *)X,(void const *)Y,sizeof(mpi)); memcpy((void *)Y,(void const *)(& T),sizeof(mpi)); return; }

Value Plug-In

Frama-C Kernel

Path-Crawler Plug-In

WP Plug-In

Coq, Alt-Ergo,

Colibri, Why

Impact & Slicing Plug-In

Frama-C Open Source Distribution

Aoraï Plug-In

E-ACSL Plug-In

Invariant generation and RTE checks

Sequence properties

Functional verifications

Runtime monitoring …

/*@ requires \valid(X); requires \valid_read(Y); ensures (\result ≡ 0 ∧ *Y ≡ \old(*X) ∧ *X ≡ \old(*Y));

•  CODEANALYSES

SecureFlow

Plug-In

Information flows

Test case generation

Slidecredit:F.KirchnerTunis

02

16/01/2020 Tunis 10

Security

Cybersecurity relies on

16/01/2020 - 11

•  Cryptography:confidentiality,integrity,anonymity,authenticity

•  SecurityPolicy:setofrulesthatspecifyhowsensitiveandcriticalresourcesareprotected

•  Prevention:toearlydetectvulnerabilities•  Cyber-resilience:capacitytotolerateattacks(hardwareand

software),todetectmalware

•  Securitybydesign

Tunis

Towards provable security

16/01/2020 - 12

•  cryptographicprimitives-mathematicalproofs,usetheoremprovingandprogramverificationtoachievecomputercheckedproofs.

•  automatedverificationtoolstoanalyzetheprotocolspecificationsandfindvulnerabilitiesintheprotocollogic

•  producingverifiedimplementations.

CryptoVerifEasyCryptCertiCrypt

https://mitls.org/

Tunis

Comprehension,Detection,Resistance(CIDRE)Needstoidentify-  thetargetsofthismalware(aparticularend-user,acompany,

anymachineunderaspecificoperatingsystem,etc.)-  theactionsitintendstoperformtoattackthetargets(sensitive

informationleakage,encryptionandransom,etc.),-  thewayitsucceedstobypassthesecuritymechanisms

protectingthetargets,thewayitprotectsitselfagainstmalwaredetectionengines(obfuscation).

•  Methods:reversemaliciouscode,staticanalysis,morphologicalanalysisbasedoncontrolflowgraphcomparison,…

16/01/2020 - 13

Malwareanalysis

Tunis

03

16/01/2020 Tunis 14

Privacy

Privacy

•  Abilityforindividualstocontroltheirpersonaldataanddecidewhattorevealtowhomandunderwhatcondition.

•  Privacyleaksin:Socialnetworks,Geolocationinformation,Webtracking,Smartworld

•  EuropeanGeneralDataProtectionRegulation(GDPR)definespersonaldata,SensitivePersonalInformation,userempowermentandresponsibilityofallstakeholders.

16/01/2020 - 15 Tunis

16/01/2020 -16

Trade-offbetweenprivacyandutility

Tunis

Privatics

ANRBIOPRIVApplicationof«PrivacybyDesign»tobiometricaccesscontrol•  Aformalframeworkbasedonepistemiclogictoexpressdataminimizationrequirementsaspropertiesdefiningforeachstakeholdertheinformationthatsheis(orisnot)allowedtoknowü alanguagetodefineprivacyarchitecturesü alogicforreasoningaboutarchitectures.ü axiomatizationtoprovethatagivenarchitecturemeetstheexpectedprivacyandintegrityrequirements.

•  appliedinparticulartocomparedifferentarchitecturesforbiometricaccesscontrolandtoprovidearationaleforthechoiceofspecificoptions.

16/01/2020 - 17 Tunis

Formal methods and Privacy

•  Dataanonymization:detectprivateinformationthatcanbeinferredaboutindividualsusingtheanonymizeddatawithprior(orbackground)knowledge

•  Differentialprivacy:protectanindividual’sdatawhilepublishingaggregateinformation.Capabilityforuserstoobfuscatetheirpersonaldata,addingnoisebythemselves(COMETE)

•  Empoweringuserswithpersonalclouds:individualizedmanagementandcontroloverone’spersonaldata.Ensuresecurityandextensibility(PETRUS)

•  Privacypreservingprotocolsandcommunicationtechnologies:Homomorphicandfunctionalencryptiontooperateonencrypteddata

16/01/2020 - 18 Tunis

04

16/01/2020 Tunis 19

Transparency

Transparency

TransAlgo:PlateformescientifiquepourledéveloppementdelatransparencedesalgorithmesetdesdonnéesInria,IMT,CNRS,CEA

https://www.transalgo.org/

•  Unalgorithmeesttransparentsil’onpeutfacilementvérifiersa«responsabilité»,parexemple,s’ilouvresoncode,s’ilexpliciteàlafoislaprovenancedesdonnéesqu’ilautilisées,etcellesqu’ilproduit,s’ilexpliquesesrésultats,ouencores’ilpubliedestracesdesescalculs.Notonsquenousconsidéreronsaussilessituationsoùlecoden’estpasouvertcariln’yaaucuneobligationdedivulgationdecelui-ci.

16/01/2020 - 20 Tunis

16/01/2020 - 21 Slidecredit:A.JolyTunis

•  identifyweaknessesofdeeplearningstrategies•  analysewhatkindsofattacksarepossibleandproposemechanisms

forprotectingconvolutionalneuralnetworksagainstadversarialattacks.

•  monitortheinternalactivationsthatflowbetweenthelayersofadeepnetwork

•  maketheoverallprocessmorerobust

16/01/2020 - 22

FormalmethodsandDeep Neural Network

Tunis

HyAIAI Inria Project Lab : design of novel, interpretable approaches for Artificial Intelligence

HyAIAIapproach:•  proposeaprincipledtwo-waycommunicationstrategybetweenahumanuserandsomeblack-boxMLmodel

•  usesymbolicmodelsasa“translationlayer”betweenthehumanandthelearningmodel(calledthesystem)

https://project.inria.fr/hyaiai/

23

Fourchallenges:1.  Systemmustunderstandhumanrequirements:fairness,privacy2.  Humanmustunderstandsystemresponses:multi-instancelocalexplanations3.  Humanmustunderstandtheinnerworkingofthesystem(“debugging”):morerobust

toadversarialattacks4.  Causality,yetanotherdimensionofexplainability:helptounderstandthemodel

16/01/2020 Tunis

05

16/01/2020 Tunis 24

Accountability

Accountability (Respect des règles)

•  Principlewhichrequiresthatorganizationsputinplaceappropriatetechnicalandorganizationalmeasuresandareabletodemonstratetheircompliancewiththeregulation.

•  Exampleandchallengesofblockchain

16/01/2020 - 25 Tunis

•  Ablockchainimplementsasecureelectronicledger:Ø dataregisteredintheledgercannotberemovedormodifiedØ integrityofpasthistoryreliesoncryptographichashfunctions:thehashofthelasttrustedblockcertifiestheintegrityofthewholeledgersinceitsinception.

Ø theroleofcertifyingtheblockchainisdecentralizedØ blockchainisreplicatedbetweenparticipants:allmusthavethesameviewoftheblockchaintoavoidattacks

Yet,theirrealsecurityandleveloftrustneedtobeproperlyassertedwithanalysisbothfromthecryptographyanddistributedsystemscommunities.

16/01/2020 - 26

Blockchain

Tunis

•  Developblockchainlanguagesandvirtualmachinesthatcomedirectlyequippedwithformalanalysisandverificationtools.

•  Generatealltheseautomatically,correct-by-constructionfromaformalspecification.

16/01/2020 - 27

GrigoreRosu'sFormalSystemsLaboratory(FSL)atUIUCandRuntimeVerification(RV)haveusedtheKframeworktobuildandtestamathematicalmodeloftheEthereumVirtualMachine,andtoformallyverifytheaccuracyofsmartcontracts.KFrameworkEnablesVerificationofSmartContractshttps://runtimeverification.com/blog/

BlockchainChallenges

Tunis

TezosisablockchainlaunchedinJune2018,writteninOCamlandsupportingsmartcontracts.Mi-Cho-Coq,aCoqframeworkforverifyingthefunctionalcorrectnessofsmartcontractswritteninsmartcontractlanguageMichelson

•  Low-levelnetworkattacks:athreatassessmentmodelneedstobeestablishedandadaptedtoeachblockchaintechnology.

•  Blockchain-specificnetworkpoliciesmustbedefined,deployedandverifiedautomatically

•  Softwarefailures:asanysoftwaresystem,duetobugs,attacks,undefinedbehavior,andsoon.

•  Blockchainsasbuildingbricksforhigherlevelprotocols:providingaledger,onwhichhigherlevelprogramsandprotocolscanbeimplemented.

•  Privacylackingbydefault:ensureprivacyusingzero-knowledgeproofsandotheradvancedcryptography

- 28

FormalMethodsandBlockchain

16/01/2020 Tunis

06

16/01/2020 Tunis 29

Certification

•  Certificationreferstoevaluationofthelevelofconfidenceinthestrengthofaproduct,system,solution,serviceororganisation

•  ItreliesonØ NormsandstandardsØ GraduationofcybersecuritylevelsØ Audits

SpecialcaseofCyber-PhysicalSystems(CPS)involvedinSCADAapplications,inmedicaldevices,inrobotics,partofInternetofThings(IoT)

16/01/2020 - 30

Certification

Tunis

•  Characteristics:ü autonomyanddynamicity,self-reconfigurableü partofcriticalsystemsü resourceslimited

•  Challengesofbuildingasafe,efficientandsecureCPS:ü  semanticformalisms,programmingparadigms,modelingandverificationtechniquesforhybridsystems.mixingdiscrete-timeandcontinuous-timedynamics.

ü certificationoftheunderlyingsoftwareinfrastructuresandoperatingsystemsü resource-constraineddevices:tocertifypropertiesofquantitativenature(useofresourcessuchastimeandenergy).

ü lightweightcryptography:efficient,certifiedcryptographicprimitivesforresource-constraineddevices

16/01/2020 - 31

FormalmethodsandCyber-PhysicalSystemsScientific strategic plan

Tunis

RIOT

16/01/2020 Tunis 32

RIOT:ThefriendlyOperatingSystemfortheInternetofThings.RIOTpowerstheInternetofThingslikeLinuxpowerstheInternet.RIOTisafree,opensourceoperatingsystemRIOTsupportsmostlow-powerIoTdevicesandmicrocontrollerarchitectures(32-bit,16-bit,8-bit).RIOTaimstoimplementallrelevantopenstandardssupportinganInternetofThingsthatisconnected,secure,durable&privacy-friendly.

07

16/01/2020 Tunis 33

Ethics

Ethicalreasoning:reasoningtakingintoaccountethicalconsiderations(principalsthatgovernaperson’sbehaviourortheconductingofanactivity)

16/01/2020 - 34

Ethics

Tunis

Thetrolleyproblem:shouldyoupullthelevertodiverttherunawaytrolleyontothesidetrack?https://commons.wikimedia.org/w/index.php?curid=67107784

Formal methods and Ethics

Identifyandformaliseanumberofvaluesandconceptsforanethicalreasoning:•  Formaliseethicaldilemna(howtocharacteriseadilemna?)•  Designareasoningframework:

§ Asystemofvalues:judgementmayneedapartialorderonvalues§  Takeintoaccountseveralpointsofview,identifysourcesofsubjectivity

§  Takeintoaccountuncertaintyinrealworld•  Validatethroughexperimentationwithhumanparticipants

16/01/2020 - 35 Tunis

Hierarchyofvalues:•  Safety•  Security•  Privacy•  Transparency•  Accountability•  Certification

16/01/2020 - 36

Wrap-up:EthicsforCyberSystems

Tunis

- 37 - 37

Merci

helene.kirchner@inria.fr

16/01/2020 Tunis