© Programming Research Brief Overview: Company, Software Products & Methods Dr. Evgueni Kolossov, R&D Director Second HiPEAC

© Programming Research

www.programmingresearch.com

Brief Overview: Company, Software Products & Methods

Dr. Evgueni Kolossov, R&D Director

Second HiPEAC Industry Partner ProgramTallinn, 8 October 2013

•ISO 26262 (automotive) up to ASIL level D•IEC 61508 (general industrial) up to SIL 4•EN 50128 (railways) up to SW-SIL 4•IEC 62304 (medical devices) up to level C•IEC 60880 (nuclear power)


PRQA Overview (www.programmingresearch.com)

Code auditsCoding standards developmentTraining - languages, standards,

productsCustom integrations – compiler,

IDE, VCS

UK: Hersham (HQ) US: Boston + San Jose India: Bangalore Ireland: Dublin Netherlands: Zeist Ukraine: Lviv Romania: Bucharest

Distributors: Germany Japan China Korea

Technical:

Products: Locations Partners

Services:

Over 25 years track record with focus on static analysis C / C++, defect prevention, coding standards compliance and effective code reviews

http://www.toyo.co.jp/

http://www.qasystems.de/

http://www.watertek.com/

http://www.hirain.com/

http://www.mdstec.com/main/english/index_eng.asp


Standing in the Software Community

• Committee (BSI Sector) Member

ISO C

• Committee voting Members

ISO C++

• Committee Founding Member

MISRA C (Motor Industry Software Reliability Association)

• Committee Founding Member

MISRA C++


Multiple Stakeholders

Stakeholders extend through the organisation:o Developers & Project Leads, Test operations, QA managers,

Senior VP management, Customers, Suppliers

Many organisations handle this in an ad-hoc manner today...

Project Devs/LeadsIs my project

meeting compliance on each release?

Corporate VPsWhich projects

present the greatest risk?

CustomersHow does this

release compare to the last one?

QA Manager Is our overall software

quality improving over time?


PRQA Global Solution

Build Server

Su

mm

ary

An

aly

sis

re

su

lts

Build Server

PRQA ServerCentral Analysis

Su

pe

rvis

ory

an

d

ma

na

ge

me

nt

co

ntr

ol


PRQA ServerCentral

Analysis



Distributed development /outsourcing

In-house Development

OEM/Customer Summary Information

and Reports

Management Policy Input and control

Lo

cal An

alysis L

ocal A

nalysis


PRQA Enterprise Solution

Build Server

Build Server

Build Server

L

oca

l An

alys

is

L

oca

l An

alys

is

Lo

cal An

alysis

Lo

cal An

alysis

PRQA ServerCentral

Analysis


PRQA Professional Solution

Build ServerBuild Server

Build Server

PRQA ServerCentral

Analysis


Components


Static Analysis Types

Lint-Like• Inexpensive• Limited Analysis

Capability• High False Positive

and False Negative rates

Bug Catchers• Strong on Simulation• Strong on whole program

test verification• Multi-Language support• Often part of Swiss Army

knife solution: bundled with testing tools

• High false negative rates.• Poor Language Usage,

Portability and Preventative analysis

Automatic Code Inspection• Strong on 4 technology types:

pattern-based, simulation, metrics, and comprehension.

• Facilitates code review and pretest checking with code collaboration, sophisticated suppression management and measurement analysis

• Low False Positive and False Negative rates.

• Weak on multi-language support

• Weak on whole program static test


What & How we are Analyzing?

- Pattern-Based Analysis – This is the scanning of source code and checking for patterns that indicate issues correlating (within various degrees of severity) to defects in software. This involves checks for quality characteristics (or lack thereof) in Portability, Style, Language Usage, and Preventative Practices

- Simulation (Deep Data Flow) is a technique in identifying likely Run-time Defects (otherwise known as “Bugs” during execution). It is a form of testing without actually executing the code

- Metrics – It has often been stated that if you can’t measure it, you can’t improve it. For example, since complex code has been highly correlated to buggy code and impacts the testability and maintainability of software, metrics that measure these aspects can provide great insight into the quality of the code and act as an indicator to where concentrated improvement needs to be made.

- Comprehension – Static Analysis of code can also provide many different views upon the actual structure of the code in forms of graphs and diagrams that help in comprehension, helping with architectural decisions using tools that provide meaningful abstractions

Simulation

StructureMetrics

Pattern-Based


Language Misuse

PRQA is industry leader in analysis of defensive coding practices JSF++ (Joint Strike Fighter - Bjarne Stroustrup) MISRA C++ MISRA C HICPP (our company standard from 2003, new version 3 October 2013)

Not all bugs are dataflow or resource usage based Incorrect language usage can result in hard (expensive) to detect bugs

Our software extends the defensive language analysis to provide for language based bug checking:

Calling an implicitly defined member function, where sister function has been explicitly declared.

Heap object of derived type undergoing derived to base conversion without a virtual destructor.


Resource Misuse

Acquired resources are tracked to ensure that they are released.

Checking is not limited to memory create, open, close, fopen, fclose, strdup, dup

Analysis uses Syntax Usage Engine; tracking is performed by inter-function analysis within the translation unit.

Special handling of constructors and destructors allows for checking that resources allocated in a constructor are freed in a destructor.


Deep-flow Dataflow

Dataflow analysis provides a mechanism to detect serious runtime behaviour problems:

Buffer overflows (security) NULL pointer dereference Undefined mathematical operations Use of unset variables much more...

Results are accurate and precise due to in-depth modelling of the language combined with a state of the art Satisfiability Modulo Theories (SMT Yices 2) solver.

(Dutertre, B., de Moura, L.: A fast Linear-Arithmetic solver for DPLL(T). In: Ball, T., Jones, R. B. (eds.) Computer Aided Verification. LNCS, vol. 4144, pp. 81--94. Springer, Heidelberg (2006))


Deep-flow Dataflow (Continue)

Analysis is performed across function boundaries within a translation unit.

Software highlights obvious defects where cause and effect are localized, or where project wide knowledge is required to determine there are no issues.

Inter variable dependencies tracked ensuring low false positives/negatives

Tracking of values referred to by pointers increases both depth of analysis as well as improving modelling through function boundaries.


Screenshots


Screenshots


Screenshots


General Description

First and foremost we must parse the code correctly: Requires a fully functional C and C++ pre-processor and parser.

Initial analysis takes place during parsing where code is checked for conformance to the respective language standard. Both parsers are written to conform to the standards, and deviations from "legal code" is highlighted with a message and in some cases it is then controlled under a configuration option. The default behaviour is to comply to the standard.

The parser builds an internal Abstract Syntax Tree for the source code and then the rest of the analysis takes place.

Most of the analysis in QA C++ (and some in QA C) is performed in the reverse order of the call tree, ie. 'leaf functions' are analysed before their callers. This allows QA C++ to use information about a called function during the analysis of the caller. This is especially important for 'dataflow' and is a core requirement for inter function analysis within the translation unit.


General Description (Continue)

A benefit of using our own parser technology is that we are not limited in the information that can be used for analysis. Macro history, instantiation history etc. that can normally be dropped by the compiler when generating code can be kept around. This aspect will again be used for some new C++ '11 checking, for example of the 'auto' keyword.

A common dataflow engine is used by QA C and QA C++. QA C++ translates C++ constructs into an equivalent C representation and this is then passed into the dataflow engine.

The resulting flow graph is further simplified and SMT solver is then used to search for defects.

As part of analysis, a semantic representation is also produced and this is then checked during Cross Module Analysis (CMA). A significant amount of undefined behaviour goes undetected by most of the available linkers, for example different function declarations etc. CMA performs this checking.


General Description (Continue)

Areas with limited support:• The output format used for the semantic representation is verbose,

and in the case of C++ can result in huge amounts of information being written (and therefore being read). As projects have increased in size, and with libraries such as boost, this is becoming more of an issue. We're working to change the output format to improve this situation.

• Once this change is made we should be able to widen the scope of our existing analysis and provide new richer analysis for the entire program.


What we are looking for?

• Collaboration in areas:– Architectural analysis of our dataflow with the target for

implementation:• Interprocedural Dataflow Analysis• Security Issues Analysis• Multi-threading Issues• Parallel Processing Issues

– Information about compilers new features & switches– Timing Analysis (executable, run-time)– New methods in code parsing, etc.

• Types of Collaboration: different types are available – subject for discussions


[email protected]


Customer Case study

“QA·C is above other tools when it comes to coding standard compliance”Site Software Director


“Since we began using the MISRA C Compliance Module, the quality and consistency of our first generation code has skyrocketed, and our final products have been virtually error-free” Stuart Jobbins, Head of Software Development

Customer case study


Customer Case study

“QA·C++ provides an efficient, robust, fully automated environment to introduce and enforce coding standards”Benjamin Pitzer, Senior Research Engineer


Customer Case study

“With QA·C our prototypes retain much of their integrity, because most defects are caught early... even as our goals evolve and the code changes, our product quality remains consistently high.”Dana Sawyer, Senior Software Designer


Customer Case study

“For us, it’s critical to focus on detecting issues early and tools like QA·C will help there”Samir Kulkarni, Head of Productivity


No FalsePositives

All False Positives

Reportsevery

violation

Fails to report true

violation

Dr Marijn Temmerman from TERA-Labs observed, “On paper all the selected tools claimed to provide comprehensive MISRA C

compliance checking - but the reality was different!”

Independent research on Static Analysis Tools / MISRA Compliance by TERA-Labs

Documents

© Programming Research Brief Overview: Company, Software Products & Methods Dr. Evgueni Kolossov, R&D Director Second HiPEAC