Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
DECOS
Dependable Embedded Components and Systems
Generic Test Bench
Report on Outlining a Certification
Process
Deliverable D 4.4.1
Project DECOS Contract Number 511764
Document Id 4.4-001_1.0r Date 2006-06-30 Deliverable D 4.4.1
Contact Person Henrik Eriksson Organisation SP
Phone +46 31 165716 E-Mail [email protected]
DECOS Generic Test Bench: Outlining a Certification Process Page 2/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
DECOS Generic Test Bench: Outlining a Certification Process Page 3/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
Distribution Table
Name Company Dptmt. copies Email
Jörn Rennhack AIRB [email protected]
Jens Koch AIRB [email protected]
Markus Buhlmann AEV [email protected]
Dietmar Kant AEV [email protected]
Wolfgang Brandstätter AEV [email protected]
Manfred Gruber ARCS [email protected]
Thomas Gruber ARCS [email protected]
Wolfgang Herzner ARCS [email protected]
Angelika Hölbling ARCS [email protected]
Kurt Lamedschwandner ARCS [email protected]
Michael Putz ARCS [email protected]
Christian Reumann ARCS [email protected]
Erwin Schoitsch ARCS [email protected]
Gerald Sonneck ARCS [email protected]
Egbert Althammer ARCS [email protected]
Andras Pataricza BUTE [email protected]
Gyorgy Csertan BUTE [email protected]
Istvan Majzik BUTE [email protected]
Laszlo Gönczy BUTE [email protected]
Fulvio Tagliabo CRF [email protected]
Thierry Lesergent EST [email protected]
Bernard Dion EST [email protected]
Bruno Martin EST [email protected]
Laurent Pouchan EST [email protected]
Knut Hufeld INF [email protected]
Markus Gusenbauer PROF [email protected]
Mario Schüpany PROF [email protected]
Jan Jacobson SP [email protected]
Henrik Eriksson SP [email protected]
Jonny Vinter SP [email protected]
Thomas Albrecht TTT [email protected]
Georg Niedrist TTT [email protected]
Peter Rech TTT [email protected]
Martin Schlager TTT [email protected]
Neeraj Suri TUDA [email protected]
Shariful Islam (Ripon) TUDA [email protected]
Brahim Ayari TUDA [email protected]
Marco Serafini TUDA [email protected]
Dan Dobre TUDA [email protected]
Wilfried Steiner TUVI [email protected]
Astrit Ademaj TUVI [email protected]
Note: Status 2005-12-23
DECOS Generic Test Bench: Outlining a Certification Process Page 4/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
Change History
Version Date Reason for Change Pages
Affected
0.1w 2006-04-25 First working draft: Henrik Eriksson all
0.2w 2006-06-01 Incorporation of comments from SP internal review (Vinter,
Jacobson, and Strandén) all
0.3w 2006-06-20 Incorporation of comments from SP internal review as well
as comments from ARCS (E. Althammer) all
1.0r 2006-06-30
Released version
Incorporation of comments from ARCS (E. Schoitsch and W.
Herzner)
DECOS Generic Test Bench: Outlining a Certification Process Page 5/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
Contents
1 Introduction..........................................................................................................................................7
1.1 Purpose and Scope ...............................................................................................................................7
2 Certification in General .......................................................................................................................8
2.1 Background............................................................................................................................................8 2.2 Certificate...............................................................................................................................................8 2.3 Safety Case ...........................................................................................................................................9 2.4 Certification According to IEC 61508...................................................................................................10 2.5 Certification in Different Application Domains .....................................................................................12 2.5.1 Certification (or Type Approval) in the Automotive Domain ................................................................12 2.5.2 Certification in the Aerospace Domain ................................................................................................15 2.5.3 Certification in the Industrial Control Domain ......................................................................................20
3 DECOS and Certification ..................................................................................................................22
3.1 Simplified Certification .........................................................................................................................22 3.1.1 Basic Connector Unit ...........................................................................................................................22 3.1.2 Safety-Critical Connector Unit .............................................................................................................24 3.1.3 Complex (Non Safety-Critical) Connector Unit ....................................................................................24 3.1.4 Diagnostic Service ...............................................................................................................................24 3.2 Modular Certification of Nodes ............................................................................................................24 3.2.1 Separating Certification of Architectural Services from Certification of Applications ..........................25 3.2.2 Separate Certification of Application Computers and Job Encapsulation ...........................................26 3.2.3 Separate Certification of Distributed Application Subsystems (DAS) .................................................26 3.3 Incremental Certification......................................................................................................................26 3.4 The Safety Cases of the DECOS Project ............................................................................................27
4 DECOS Certification Process...........................................................................................................28
4.1 Consolidation of Certification Requirements from Three Domains .....................................................28 4.2 Other DECOS Processes ....................................................................................................................29 4.3 Suggested Certification Process .........................................................................................................30 4.4 Documentation Support from the Generic Test Bench........................................................................31
Annex 1. Example of a Certificate.................................................................................................................33
5 Abbreviations and Definitions..........................................................................................................34
6 References .........................................................................................................................................35
DECOS Generic Test Bench: Outlining a Certification Process Page 6/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
Figures
Figure 1: Contents of a safety case....................................................................................................................9 Figure 2: Functional safety assessment during all phases of the IEC 61508 lifecycle ....................................11 Figure 3: Processes according to ARP 4754....................................................................................................18 Figure 4 Flow chart for the conformity assessment procedures provided for in the machinery directive ........20 Figure 5: Structure of a DECOS component ....................................................................................................23 Figure 6: Certification of a DECOS node..........................................................................................................25 Figure 7 Incremental certification .....................................................................................................................27 Figure 8 Boundary of generic safety case........................................................................................................27 Figure 9: General view of DECOS workflows...................................................................................................29 Figure 10: The certification process and its relations to the other processes ..................................................30 Figure 11: Hierarchy of documents containing safety evidence.......................................................................31
Tables
Table 1: Regulations and directives relevant for complex electronic platforms ...............................................12
DECOS Generic Test Bench: Outlining a Certification Process Page 7/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
1 Introduction
1.1 Purpose and Scope
The purpose of this document is to outline a certification process for applications based on the DECOS
technology. The document consists of three parts:
Chapter 2 provides some background by discussing certification in general. Different actors and their roles
within the certification process are described as well as the different needs of different domains such as
automotive and aerospace. Safety cases are also briefly described.
Chapter 3 explains how the DECOS architecture facilitates modular and incremental certification. Especially
through the vertical partitioning imposed by the layered architecture and the horizontal partitioning created by
encapsulation of applications and jobs; i.e. encapsulated execution environments and virtual networks.
Chapter 4 outlines a certification process for the development of applications based on the DECOS
technology. The mutual interaction with the development process and the support provided by the Generic
Test Bench are explained.
DECOS Generic Test Bench: Outlining a Certification Process Page 8/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
2 Certification in General
2.1 Background
Certification exists in different kinds ranging from certification of a software product such as an operating
system to certification of a complete aircraft by e.g. EASA (European Aviation Safety Agency). An electronic
platform belongs perhaps more to the former, but it can be used as a part in compound systems and shall
facilitate certification of compound systems such as the latter. Certification usually involves the task to show
conformance to one or several standards, regulations, directives, etc. For simple non safety-relevant
products this can be done by the manufacturer itself but for complex electronic systems, especially those
related to safety, the conformance must be assessed by an accredited and independent, third party test
house in most cases. The following definitions apply:
• Certification (of conformance) is a procedure where a third party confirms (in writing) that an
identified product, process or service meets the prescribed requirements of a specific standard or
regulating document.
• Conformance assessment is to judge to which level a product fulfils the requirements of a
standard.
• Accredited means that an authority or body fulfils the formal requirements to perform specific tests,
measurements, or certifications.
• Third party is a person or body which is recognized to be independent of all involved parties with
respect to the treated subject. (Remark: Besides the third party, the involved parties usually
represents the interests of the supplier “first party” and the purchaser “second party”)
• Independence is important; especially the level of independence of the assessor from the designer.
For a safety-related application which require high integrity (e.g. SIL 3 and 4 in IEC 61508) third
party assessment is highly recommended whereas an independent person is sufficient for SIL 1.
For a test house or institute to be accredited, the requirements of ISO/IEC 17025:2005 “General
requirements for the competence of testing and calibration laboratories” have to be fulfilled. This competence
is assessed by a national accreditation body on an annual basis.
Usually when a safety-related product is certified more than one standard needs to be considered.
Depending on the domain and operation environment, a complete certification could involve several aspects
besides functional safety (e.g. addressed in IEC 61508):
• Resistance to temperature, dust, and moisture (e.g. IEC 60068-X)
• Resistance to mechanical shock and vibration (e.g. IEC 60068-X)
• Susceptibility and emission of EMI (e.g. EMC 89/336/EEG)
• Means for electrical safety (e.g. LVD 73/23/EEG)
• Encapsulation (protection against penetration of objects or water) (e.g. IEC 60529)
As seen from the list usually both international standards and regional (European) regulations have to be
considered.
2.2 Certificate
The certificate is the formal document issued by a certification authority stating that a specific product fulfils
specific regulations or standards under specified conditions. The document, which usually is based on a test
DECOS Generic Test Bench: Outlining a Certification Process Page 9/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
report, also states the holder of the certificate, the intended application of the product, and how long time the
certificate is valid. All certificates are unique and thus have a unique number identifier.
An example of a certificate can be found in Appendix A.
2.3 Safety Case
A safety case is a document to convince a licensing authority that a given product or system is safe enough
to be taken into use.
The purpose of a safety case is usually described by the following quotation:
“A safety case should communicate a clear, comprehensive and defensible argument that a system is
acceptably safe to operate in a particular context.” [Kelly 2003]
The keywords here are argument and context. In a safety case there are claims and there is evidence.
Claims are specific statements about the safety properties of the system. Evidence results from V&V
activities or specific safety mechanisms such as e.g. a back-up mechanism. The arguments are the links
between claims and evidence. This means that the arguments are really important; it is useless to have
evidence which is not related to a claim via arguments.
The context is also important since the safety case and its evidence is usually only valid under certain
restrictions and conditions.
The concept of safety cases is well-established in e.g. the railway domain and according to the standard EN
50129 [EN 50129], safety cases are divided into three classes:
1. Generic product safety case (independent of application)
2. Generic application safety case (for a class of applications)
3. Specific application safety case (for a specific application)
where the former two are, as the name implies, generic and can consequently be re-used for a class of
applications or for independent applications, respectively.
Regardless of category, the content of a safety case is basically the same, see Figure 1 below [EN 50129].
Figure 1: Contents of a safety case
DECOS Generic Test Bench: Outlining a Certification Process Page 10/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
The system definition shall precisely define or reference the system/subsystem/equipment to which the
safety case refers, including version numbers and modification status of all requirements, design, and
application documentation.
The quality management report shall contain the evidence of quality management. Here aspects from e.g.
ISO 9001 shall be covered, e.g. organisational structure, quality planning, as well as handling and storage.
In the safety management report, the evidence of safety management shall be presented. Here
documentary evidence to demonstrate compliance with all elements of the safety management process
throughout the lifecycle shall be provided.
The technical safety report shall contain the evidence of functional and technical safety. The technical
safety report shall explain the technical principles which assure the safety of the design, including (or giving
references to) all supporting evidence (for example, design principles and calculations, test specifications
and results, and safety analyses).
In the related safety cases part, there shall be references to the safety cases of any subsystem or
equipment on which the main safety case depend.
The conclusion shall summarize the evidence presented in the previous parts of the safety case, and argue
that the relevant system/subsystem/equipment is adequately safe, subject to compliance with the specified
application conditions.
As long as the safety-related application conditions of a generic safety case are fulfilled by the more specific
safety case or are carried forward to the more specific safety case, it is not necessary to repeat the generic
approval process for each application.
2.4 Certification According to IEC 61508
The IEC 61508 standard [IEC 61508] states its scope as follows:
"This International Standard covers those aspects to be considered when electrical/electronic/programmable
electronic systems (E/E/PESs) are used to carry out safety functions. A major objective of this standard is to
facilitate the development of application sector international standards by the technical committees
responsible for the application sector. This will allow all the relevant factors associated with the application,
to be fully taken into account and thereby meet the specific needs of the application sector. A dual objective
of this standard is to enable the development of electrical/electronic/programmable electronic (E/E/PE)
safety-related systems where application sector international standards may not exist".
So far, the IEC 61508 has rarely been used for certification of complete applications. The standard has
however been used as basis for certification of parts of systems such as: advanced valves, safety PLCs,
operating systems, and code generators. Correspondingly, the IEC 61508 could also be used for certification
of a distributed computer platform.
In IEC 61508, certification is not explicitly mentioned but functional safety assessments are mandatory. The
functional safety assessment shall be carried out by persons whose level of independence is determined of
the targeted safety integrity level; e.g. for SIL 4 an independent organization is required. The functional
safety assessment shall be carried out after each phase or a number of phases of the safety life cycle, see
Figure 2. If tools are used as part of design or assessment for any safety lifecycle activity, they should
themselves be subject to the functional safety assessment.
The functional safety assessment shall be planned and the plan shall specify:
- Those to undertake the functional safety assessment
- The output from each (if performed for every lifecycle phase) functional safety assessment
- The scope of the functional safety assessment
- The safety bodies involved
DECOS Generic Test Bench: Outlining a Certification Process Page 11/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
- The resources required
- The level of independence of those undertaking the functional safety assessment
- The competence of those undertaking the functional safety assessment with respect to the
application
During the assessment the assessors shall get access to involved personnel as well as proper
documentation (accurate and concise, structured, version, etc.)
If the software is used as an example, the assessor shall during the functional safety assessment consider:
- The consistency and complementary of the chosen methods, languages, and tools for the whole
development cycle
- Whether the developers use methods, languages, and tools they fully understand
- Whether the methods, languages and tools are well-adapted to the specific problems encountered
during development
1 0 11
N O T E 1 A c tiv itie s re la tin g to v e r if ic a t io n , m a n a g e m e n t o f fu n c t io n a l s a fe ty a n d fu n c t io n a l s a fe ty a s s e s s m e n t a re n o t s h o w n fo r re a s o n s o f c la r ity b u t a re re le v e n t to a ll o ve ra ll, E /E /P E S a n d s o ftw a re s a fe ty l ife c yc le p h a s e s .
N O T E 2 T h e p h a s e s re p re s e n te d b y b o x e s 1 0 a n d 11 a re o u ts id e th e s c o p e o f th is s ta n d a rd .
N O T E 3 P a rts 2 a n d 3 d e a l w ith b o x 9 (re a lis a t io n ) b u t th e y a ls o d e a l, w h e re re le va n t, w ith th e p ro g ra m m a b le e le c tro n ic (h a rd w a re a n d s o ftw a re ) a s p e c ts o f b o x e s 1 3 , 1 4 a n d 1 5 .
C o n c e p t1
O ve ra ll s c o p e
d e fin it io n2
H a z a rd a n d r is k a n a lys is3
O ve ra ll s a fe ty
re q u ire m e n ts4
S a fe ty re q u ire m e n ts
a llo c a tio n 5
B a c k to a p p ro p ria te
o v e ra l l s a fe ty l ife c y c le
p h a s e
O ve ra ll s a fe ty va l id a tio n1 3
O ve ra l l o p e ra tio n ,
m a in te n a n c e a n d re p a ir
O ve ra ll m o d ific a tio n a n d re tro fi t1 4 1 5
D e c o m m is s io n in g
o r d is p o s a l1 6
S a fe ty -re la te d
s y s te m s :
E /E /P E S
R e a lis a tio n(s e e E /E /P E S
s a fe ty
l ife c yc le )
9S a fe ty -re la te d
s y s te m s :
o th e r
te c h n o lo g y
R e a lis a t io n
O ve ra l l in s ta l la tio n
a n d c o m m is s io n in g1 2
8
O ve ra ll p la n n in g
O ve ra lI
o p e ra tio n a n d
m a in te n a n c e
p la n n in g
O ve ra l I
in s ta l la tio n a n d
c o m m is s io n in g
p la n n in g
O ve ra ll
s a fe ty
va lid a tio n
p la n n in g
6 7 8
E x te rn a l r is k re d u c tio n fa c i lit ie s
R e a lis a tio n
Functional safety assessment
shall be performed for all lifecycle
phases. See Note 1 below.
Figure 2: Functional safety assessment during all phases of the IEC 61508 lifecycle
DECOS Generic Test Bench: Outlining a Certification Process Page 12/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
At the conclusion of the functional safety assessment a recommendation of acceptance, qualified
acceptance, or rejection shall be given.
2.5 Certification in Different Application Domains
2.5.1 Certification (or Type Approval) in the Automotive Domain
The regulations and directives that are defining the requirements on motor vehicles in Europe are lagging the
tremendous development which is occurring with respect to electronic systems in e.g. cars. Therefore there
are very few requirements on electrical systems when a type approval is sought. For road vehicles, one does
not explicitly talk about certification but rather about type approval. For motor vehicles in Europe, the
European Commission specifies 56 directives ranging from headlights to permissible sound levels of exhaust
systems. These directives are in their turn based on 122 regulations set by the UNECE, United Nations
Economic Commission for Europe. As mentioned already, there are very few of these regulations and
directives that touch upon complex electronic systems; there are only a few regulations and directives which
concern some related information and they are summarized in the table below. For the directives, the original
directive plus its latest amendment are presented.
Table 1: Regulations and directives relevant for complex electronic platforms
Regulation Directive Title (EC)
ECE Reg 10 72/245/EEC
2006/28/EC
Radio interference (electromagnetic compatibility) of vehicles
ECE Reg 13 71/320/EEC
2002/78/EC
Braking devices of certain categories of motor vehicles and their trailers
ECE Reg 48 76/756/EEC
97/28/EC
Installation of lighting and light-signalling devices on motor vehicles and their
trailers
ECE Reg 79 70/311/EEC
1999/7/EC
Steering equipment for motor vehicles and their trailers
ECE Reg 97 74/62/EEC
95/56/EC
Devices to prevent the unauthorized use of motor vehicles
ECE Reg 100 -- Uniform provisions concerning the approval of battery electric vehicles with
regard to specific requirements for the construction and functional safety
(ECE title)
Besides Regulation 10, the regulations have weak explicit coupling to complex electronic systems. However,
in Annex 6 of Regulation 79 [ECE 79] and Annex 18 of Regulation 13 [ECE 13], there are “special
requirements to be applied to the safety aspects of complex electronic vehicle control systems”. Here, there
are several relevant requirements (the numbers are references to a specific clause in the regulation):
Documentation requirements
• The manufacturer shall provide a documentation package which gives access to the basic design of
the system and the means by which it is linked to other vehicle systems or by which it directly
controls output variables. (3.1)
• The functions of the system and the safety concept, as laid down by the manufacturer, shall be
explained. (3.1)
DECOS Generic Test Bench: Outlining a Certification Process Page 13/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
• Documentation shall be brief, yet provide evidence that the design and development has had the
benefit of expertise from all the system fields which are involved. (3.1)
• For periodic inspections, the documentation shall describe how the current operational status of the
system can be checked. (3.1)
• A description shall be provided which gives a simple explanation of all the control functions of the
system and the methods employed to achieve the objectives, including a statement of the
mechanism(s) by which control is exercised. (3.2)
o A list of all input and sensed variables shall be provided and the working range of these
defined. (3.2.1)
o A list of all output variables which are controlled by the system shall be provided and an
indication given, in each case, of whether the control is direct or via another vehicle system.
The range of control exercised on each such variable shall be defined. (3.2.2)
o Limits defining the boundaries of functional operation shall be stated where appropriate for
system performance. (3.2.3)
• A description of system layout and schematic shall be provided (3.3)
o An inventory list of components shall be provided, collating all the units of the system and
mentioning the other vehicle systems which are needed to achieve the control function in
question. (3.3.1)
o An outline schematic showing these units in combination shall be provided with both the
equipment distribution and the interconnections made clear. (3.3.1)
o The function of each unit of the system shall be outlined and the signals linking it with other
units or with other vehicle systems shall be shown. This may be provided by a labelled block
diagram or other schematic, or by a description aided by such a diagram. (3.3.2)
o Interconnections within the system shall be shown by a circuit diagram for the electrical
transmission links, by an optical-fibre diagram for optical links, by a piping diagram for
pneumatic or hydraulic transmission equipment and by a simplified diagrammatic layout for
mechanical linkages. (3.3.3)
o There shall be a clear correspondence between these transmission links and the signals
carried between units. (3.3.4)
o Priorities of signals on multiplexed data paths shall be stated, wherever priority may be an
issue affecting performance or safety as far as the Regulation is concerned. (3.3.4)
o Each unit shall be clearly and unambiguously identifiable (e.g. by marking for hardware and
marking or software output for software content) to provide corresponding hardware and
documentation association. (3.3.5)
o Where functions are combined within a single unit or indeed within a single computer, but
shown in multiple blocks in the block diagram for clarity and ease of explanation, only a
single hardware identification marking shall be used. (3.3.5)
o The manufacturer shall, by the use of this identification, affirm that the equipment supplied
conforms to the corresponding document. (3.3.5)
o The identification defines the hardware and software version and, where the latter changes
such as to alter the function of the unit as far as this Regulation is concerned, the
identification shall also be changed. (3.3.5)
Safety Concept requirements
DECOS Generic Test Bench: Outlining a Certification Process Page 14/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
• The manufacturer shall provide a statement which affirms that the strategy chosen to achieve the
system objectives will not, under non-fault conditions, prejudice the safe operation of systems which
are subject to the prescriptions of this Regulation. (3.4.1)
• In respect of software employed in the system, the outline architecture shall be explained and the
design methods and tools used shall be identified. The manufacturer shall be prepared, if required,
to show some evidence of the means by which they determined the realisation of the system logic,
during the design and development process. (3.4.2)
• The manufacturer shall provide the technical authorities with an explanation of the design provisions
built into the system so as to generate safe operation under fault conditions. Possible design
provisions for failure in the system are for example: (3.4.3)
o Fall-back operation using a partial system
o Change-over to a separate back-up system
o Removal of the high-level function
• In case of a failure, the driver shall be warned for example by warning signal or message display.
When the system is not deactivated by the driver, e.g. by turning the ignition (run) switch to “off”, or
by switching that particular function if a special switch is provided for that purpose, the warning shall
be present as long as the fault condition persists. (3.4.3)
• If the chosen provision selects a partial performance mode of operation under certain fault
conditions, then these conditions shall be stated and the resulting limits of effectiveness defined.
(3.4.3.1)
• If the chosen provision selects a second (back-up) means to realise the vehicle control system
objective, the principles of the change-over mechanism, the logic and level of redundancy and any
built in back-up checking features shall be explained and the resulting limits of back- effectiveness
defined. (3.4.3.2)
• If the chosen provision selects the removal of the higher-level function, all the corresponding output
control signals associated with this function shall be inhibited, and in such a manner as to limit the
transition disturbance. (3.4.3.3)
• The documentation shall be supported, by an analysis which shows, in overall terms, how the
system will behave on the occurrence of any one of those specified faults which will have a bearing
on vehicle control performance or safety. (This may be based on a Failure Mode and Effects
Analysis (FMEA), a Fault-Tree Analysis (FTA) or any similar process appropriate to system safety
considerations) (3.4.4)
• The chosen analytical approach(es) shall be established and maintained by the manufacturer and
shall be made open for inspection by the technical service at the time of the type approval. (3.4.4)
• This documentation shall itemise the parameters being monitored and shall set out, for each fault
condition of the type defined, the warning signal to be given to the driver and/or to service/technical
inspection personnel. (3.4.4.1)
Verification and test requirements
• The functional operation of the system shall be tested as follows (4.1)
o As a means of establishing the normal operational levels, verification of the performance of
the vehicle system under non-fault conditions shall be conducted against the manufacturer’s
basic benchmark specification unless this is subject to a specified performance test as part
of the approval of this or other regulation. (4.1.1)
o The reaction of the system shall, at the discretion of the type approval authority, be checked
under the influence of a failure in any individual unit by applying corresponding output
DECOS Generic Test Bench: Outlining a Certification Process Page 15/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
signals to electrical units or mechanical elements in order to simulate the effects of internal
faults within the unit. (4.1.2)
o The verification results shall correspond with the documented summary of the failure
analysis, to a level of overall effect such that the safety concept and execution are confirmed
as being adequate. (4.1.3)
Many of these requirements resemble those needed for a safety case or for a functional safety assessment
according to IEC 61508.
The automotive industry has considered the IEC 61508 for several years and currently an ISO standard is
being developed (ISO WD 26262 – Road Vehicles: Functional Safety) based on it. The draft is mainly based
on work performed by VDA/FAKRA in Germany but information has also been incorporated from work done
by BMA in France and MISRA in the UK. In the draft version of ISO WD 26262, a functional safety
assessment shall be performed immediately before product release. This functional safety assessment is
clearly inspired of the one in IEC 61508 but have some aspects (e.g. safety concept) which seem similar to
them from the Annexes of Regulations 9 and 13.
In the software area, AUTOSAR, a domain-specific industry standard is developed by an automotive
consortium.
2.5.2 Certification in the Aerospace Domain
There are several important documents which affect certification of complex electronic systems in the
avionics domain:
- ARP 4761 “Guidelines and Methods for Conducting the Safety Assessment Process on Civil
Airborne Systems and Equipment”
- RTCA DO-178B “Software Considerations in Airborne Systems and Equipment Certification”
- RTCA DO-254 “Design Assurance Guidance for Airborne Electronic Hardware”
- RTCA DO-160 “Environmental Conditions and Test Procedures for Airborne Equipment”
However, the most important one and the common basis for certification of electronics systems is ARP 4754
“Certification Considerations for Highly-Integrated or Complex Aircraft Systems” [ARP 4754].
The ARP 4754 standard states its scope as follows:
“This document discusses the certification aspects of highly-integrated or complex systems installed on
aircraft, taking into account the overall aircraft operating environment and functions. The term “highly-
integrated” refers to systems that perform or contribute to multiple aircraft-level functions. The term
“complex” refers to systems whose safety cannot be shown solely by test and whose logic is difficult to
comprehend without the aid of analytical tools.”
Contrary to e.g. IEC 61508, there are no checklists and “shall” or “must” requirements in ARP 4754; instead
another philosophy is used: to as much as possible focus on fundamental issues; certification of all but
idealized systems will require significant engineering judgement by both the certification authority and the
applicant. Therefore, the quality of such judgements is served best by a common understanding of
fundamental principles. Also, other factors such as the variety of potential systems applications and rapid
development of systems engineering support such a philosophy.
The goal of the ARP 4754 certification process is to substantiate that the aircraft and its systems comply with
applicable airworthiness requirements. In highly-integrated or complex systems there is a need to use
development assurance method as part of the evidence supporting system certification. Careful design of the
system architecture and its components may simplify development assurance and ease certification.
A certification plan is mandatory and it defines the product and installation to be certified, outlines the
product development processes to be used for development assurance, and identifies the proposed means
DECOS Generic Test Bench: Outlining a Certification Process Page 16/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
of compliance with regulations. Early coordination of the certification plan with the certification authority is
encouraged to check that the plan is not missing significant details. Before development activities occur, the
applicant should submit a plan for means of compliance to the certification authority. The applicant should
develop a certification summary to describe how it was determined that the system, as installed on the
aircraft, complies with the agreed certification plan. The certification summary should provide an outline of
the results of the activities established in the certification plan.
Below, various required certification documents (data), or evidence if safety case terminology is used, are
listed:
a. Certification Plan
b. Development Plan
c. Architecture and Design
d. Requirements
e. Validation Plan
f. Verification Plan
g. Configuration Management Plan
h. Process Assurance Plan
i. Configuration Index
j. Functional Hazard Assessment
k. Preliminary System Safety Assessment
l. System Safety Assessment
m. Common Cause Analysis
n. Validation Data
o. Verification Data
p. Evidence of Configuration Management
q. Evidence of Process Assurance
r. Certification Summary
Of these data the ones written in bold font represent the minimum set submitted to the authority. The
certification plan has been touched upon already but to be more specific it should include:
1. A functional and operational description of the system and the aircraft on which the system will be
installed. A description of the system elements including hardware and software. This description
should establish the functional, physical, and information relationship between the system and other
aircraft systems and functions.
2. A statement of the relationship of this certification plan to any other relevant system certification
plan(s).
3. A summary of the functional hazard assessment (aircraft hazards, failure conditions, and
classification).
4. A summary of the preliminary system safety assessment (system safety objectives and preliminary
system development assurance levels.)
5. A description of any novel or unique design features that are planned to be used in meeting the
safety objectives.
6. A description of the new technologies or new technology applications to be implemented.
DECOS Generic Test Bench: Outlining a Certification Process Page 17/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
7. The system certification basis including any special conditions.
8. The proposed methods of showing compliance with the certification basis, including an outline of the
anticipated development assurance processes (safety assessment, validation, verification,
configuration management, and process assurance.)
9. A list of the data to be submitted and the data to be retained under configuration control, along with a
description or sample of data formats.
10. The approximate sequence and schedule for certification events.
11. Identification of the personnel or specific organization responsible for certification coordination.
Here, the amount of detail contained in the plan could vary depending on the classification of the associated
aircraft hazard(s).
The system configuration index identifies all of the physical elements that, together, comprise the system.
In addition the configuration index identifies procedures and limitations that are integral to system safety. A
typical system configuration index will include the following information:
• Configuration identification of each system item
• Associated item software
• Interconnection of items
• Required interfaces with other systems
• Safety-related operational or maintenance procedures and limitations
In ARP 4754, safety integrity levels (SILs) are not mentioned. Instead, there are system development
assurance levels ranging from A to E, where level A corresponds to systems having catastrophic failure
conditions and E to systems with conditions having “no safety effect”. These assurance levels determine the
level of rigor of system safety assessment, requirement validation, and implementation verification.
The safety assessment process is central for generating evidence for certification. The safety assessment
process runs in parallel with the system development process and they have mutual interaction. The
relationships between the safety assessment activities themselves and safety assessment activities and the
system development process are shown in Figure 3 below.
DECOS Generic Test Bench: Outlining a Certification Process Page 18/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
Aircraft Level
Requirements
Aircraft Level
FHA
Development of
System
Architecture
Allocation of
Requirements to
Hardware and
Softrware
Allocation of
Aircraft
Functions to
Systems
SSAs System Implementation
PSSAs
System-level
FHA Sections
CCAs
Certification
Safety Assessment Process System Development Process
Functions
Functions
Implementation
Requirements
Separation
Aircraft
System
Architecture
System
Requirements
Architectural
Failure
Conditions
& Effects
Failure Condition
Effects
Classification
Safety Objectives
Item Requirements
Safety Objectives
Analyses Required
Item Requirements
Separation &
VerificationResults Physical System
Functional
Interactions
Failure Condition
Effects
Classification
Safety Requirements
Figure 3: Processes according to ARP 4754
The different safety assessment activities are:
• Functional Hazard Assessment (FHA) - examines aircraft and system functions to identify potential
functional failures and classifies the hazards associated with specific failure conditions. The FHA is
developed early in the development process and is updated as new functions or fault conditions are
identified.
DECOS Generic Test Bench: Outlining a Certification Process Page 19/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
• Preliminary System Safety Assessment (PSSA) - establishes specific system and item safety
requirements and provides preliminary indication that the anticipated system architecture can meet
those safety requirements. The PSSA is updated throughout the system development process.
• System Safety Assessment (SSA) - collects, analyzes, and documents verification that the system,
as implemented, meets the system safety requirements established by the FHA and the PSSA.
• Common Cause Analysis (CCA) - establishes and validates physical and functional separation and
isolation requirements between systems and verifies that these requirements have been met.
Besides the safety assessment process, there are two other processes running in parallel with the system
development process: requirement validation and implementation verification. The requirement validation
is guided by a validation plan where for each step the requirements are checked for correctness and
completeness. The validation also focuses on justifying and stating assumptions on which the development
is based. Analyses, models, and tests should be used to validate the requirements which should be
traceable to parent requirements, design decisions, or standards.
Validation is guided by a validation plan throughout the development process. The validation plan should
include:
• The methods to be used
• The data to gathered or generated
• What should be recorded
• How the status of validation will be maintained, or managed, when changes are made to
requirements.
• Roles and responsibilities associated with the validation.
• A schedule of key validation activities.
The implementation verification is guided by a verification plan where for each step it is confirmed that the
intended functions have been correctly implemented and the requirements have been satisfied. Inspections
and reviews, analysis, and tests should be used to verify that the implemented system fulfils its functional
design requirements. Checklists are used for inspections and reviews verify that the system meets its
requirements. Testing is used to show that the system performs its intended functions as well as not
performing unintended functions.
Verification is guided by a verification plan throughout the development process. The verification plan should
include:
• Roles and responsibilities associated with conducting the verification activities.
• A description of the degree of independence of the design and verification activities.
• Application of verification method(s).
• Data to be produced.
• Sequence of dependent activities.
• A schedule of key verification activities.
Finally, there is one configuration management process as well as process assurance activities. The
configuration management process deals with technical and administrative control of the configuration of:
system requirements, system parts, and certification data and their changes.
The objectives of the process assurance activities are:
1. To ensure the necessary plans are in place or developed, and then maintained for all aspects of
system development.
DECOS Generic Test Bench: Outlining a Certification Process Page 20/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
2. To ensure development activities and processes are conducted in accordance with those plans.
3. To ensure evidence is available to show the implementation of the plans.
2.5.3 Certification in the Industrial Control Domain
Besides the automotive and the aerospace applications, a complex electronic platform can also be used in
products for advanced industrial control systems.
Safety of machinery is regulated in the European Machinery Directive (98/37/EC). The directive describes
the essential health and safety requirements for all machines put on the European market. Most of the
requirements relate to risks independent of the control system. Noise, radiation and mechanical stability are
examples of the addressed risks. Risks related to faults in the control system are also included in the scope
of the machinery directive. Starting, stopping and emergency stopping are explicitly mentioned.
The manufacturer of the machinery (or components for use in machines) must declare the conformance with
the essential requirements of the machinery directive. This is made by writing the “declaration of conformity”
and by affixing of the CE mark to the machine. The declaration of conformity is made by the manufacturer,
not by any independent test house or certification organisation. The actions of the manufacturer may be
regarded as a kind of “self-certification”.
Potential high-risk machines are listed in the fourth annex of the machinery directive. The safety of such
machines may be assessed by a “notified body” of any of the member states. The notified body may then
issue an EC type approval certificate. But the CE marking and the declaration of conformance with the
machinery directive is still the responsibility of the manufacturer.
Figure 4 Flow chart for the conformity assessment procedures provided for in the machinery directive
The possible roads leading to a CE mark for a machinery manufacturer is shown in Figure 4. First of all, it
has to be determined if the machine belongs to those listed in Annex IV of the directive. If it does but is not
meeting harmonized standards, an EC type examination by a notified body is necessary. If it has been
designed according to standards, three options apply: either one can, to be on the safe side, let a notified
bode perform a type examination, or one can request the notified body to audit the technical file, or one can
DECOS Generic Test Bench: Outlining a Certification Process Page 21/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
simply submit the technical file to the notified body for filing. If the machine does not belong to those listed in
Annex IV, the manufacturer prepares the technical file and files it itself.
Machine control systems are addressed by the European standard EN 954 “Safety of machinery – Safety-
related parts of control systems”. The standard divides the control systems into five categories dependent on
fault tolerance. Validation is recognised as one of the steps of the development life cycle, but certification is
not addressed. Part 2 of the standard is targeted on validation of control systems.
The international standard IEC 62061 “Safety of machinery – Functional safety of electric/electronic/program-
mable electronic control systems” may also be used for machine control systems. This standard is a sector-
specific interpretation of the standard IEC 61508 for the machinery sector. Validation is briefly mentioned, but
certification is not dealt explicitly with by the standard. For proof of conformance with the standard, the
general procedures of IEC 61508 would apply.
Programmable logic controllers (PLCs), sensor systems and bus communication systems may be regarded
as components of a machine control system. Certificates issued by independent certification organisations
exist for some of these components. The certificates typically state that the component is suitable for use in a
safety-related function of a certain safety integrity level (SIL). Designers of machine control systems rely on
these certificates when selecting the components for the machine. The machine control system is designed
without re-validating the components.
DECOS Generic Test Bench: Outlining a Certification Process Page 22/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
3 DECOS and Certification
To ensure the success of an integrated architecture for safety-related applications, the architecture must
have built in properties that allows for modular certification [Rushby 2001]. If not, the whole system needs to
be recertified when a part of it is modified or updated. As a consequence the DECOS architecture and
technology are being developed with simplified certification in mind [Kopetz 2004]. DECOS components are
vertically and horizontally structured to support modular certification. There is a strict separation of the
architectural services and the application as well as a separation of DASs (Distributed Application
Subsystems) and jobs at the component level. This separation of certification efforts allows for the
construction of independent safety arguments for different DASs on one hand and the architecture itself on
the other hand [Bishop 1998].
3.1 Simplified Certification
Certification is a significant cost factor in the development of ultra-dependable systems, e.g., in the avionic
domain [Hayhurst 1999]. Today, between 60% and 70% of the cost of an avionics component is verification
and certification cost [Langley 2003]. Depending on the level of criticality, certification according to DO-178B
[RTCA 178B] increases the cost of developing software by 300–500% [Atkinson 2002]. Consequently, there
is a need for architectures that are designed for validation [Johnson 1992]. Design for validation occurs by
devising a complete and accurate reliability model, by avoiding design faults, and by minimizing parameters
that have to be measured.
Another key element for controlling certification cost is architectural support for modular certification. Modular
certification [Rushby 2001] is a certification strategy that promises a massive reduction in certification cost
through modularization and reuse of certification arguments. The DECOS architecture offers modular
certification by separating the certification of architectural services from applications and by supporting
independent safety arguments for different DASs.
1. Separating certification of architectural services from certification of applications. The clear
interfaces between the platform and applications, provided via the platform interface are a
prerequisite for the separation of the certification of architectural services from the certification of
applications. The certified architectural services of the integrated architecture establish a baseline
safety argument for the certification of the overall system [Nicholson 2000].
2. Separating certification of different DASs. The integrated architecture allows the independent
certification of different DASs, instead of considering the system as an indivisible whole in the
certification process. The safety argument for each DAS is provided to the integrator by the suppliers
along with the compiled application code of the jobs in the corresponding DAS. In order to construct
the safety argument for the overall system, the system integrator combines the safety arguments of
the independently developed DASs and acquires additional evidence, such as results of a formal
verification of the architectural services. The decomposition of the overall system into encapsulated
DASs with different criticality levels reduces the overall certification efforts and allows focusing on
the most critical parts. Furthermore, the separate certification of DASs is beneficial, if functionality is
reused in different systems. In this case, the safety argument for the functionality needs to be
constructed only once.
3.1.1 Basic Connector Unit
Effective partitioning between safety-critical and non safety-critical subsystems is a prerequisite for the
independent certification of the safety-critical subsystem and leads to a noteworthy reduction of the
DECOS Generic Test Bench: Outlining a Certification Process Page 23/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
certification effort of a mixed-criticality node. This partitioning is performed by the Basic Connector Unit, see
Figure 5 below. If credible evidence for the ability of the Basic Connector Unit to prevent interference in the
access of network resources (e.g., in case of a failure of a job in the non safety-critical subsystem) is
available, then the non safety-critical subsystem need not be certified to the same level as required for the
safety-critical subsystem. Since the reasoning about the Basic Connector Unit plays a major role in the
certification process, it is of utmost importance to limit the complexity of the Basic Connector Unit, i.e. the
mental effort for understanding its operation. Therefore, the Basic Connector Unit aims at a minimal
functionality for the allocation of network resources.
Furthermore, the Basic Connector Unit enforces uniformity of inner and outer port types by imposing the
restriction of state messages on all ports. The self-contained nature and idempotence of state messages
relieves the designer from the need to devise flow control mechanisms for ensuring exactly-once processing
guarantees and thus state synchronization. Since applications are often only interested in the most recent
value of a real-time object, state messages permit the overwriting of old state values by more recent state
values. The encapsulation service of the Basic Connector Unit statically allocates component resources (i.e.
memory and processor) to the safety-critical and the non safety-critical subsystem, respectively. Such a
statically defined allocation facilitates certification (for details of the structure of a DECOS node see
deliverable D 1.4.3 [DECOS D1.4.3])
Figure 5: Structure of a DECOS component
Controlling the complexity of the certification process is also the reason for constraining the safety-critical
software to the access of Basic Connector Units. Each state message port of a Basic Connector Unit
provides a temporal firewall interface [Kopetz 1997], which is fully specified at the operational level. The
syntactic structure of the data items in the temporal firewall and the global points in time when the data items
in the temporal firewall are updated by the communication system are a priori specified.
DECOS Generic Test Bench: Outlining a Certification Process Page 24/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
3.1.2 Safety-Critical Connector Unit
The Safety-Critical Connector Unit (Figure 5) allocates network resources to the jobs of the safety-critical
subsystem and realizes the safety-critical high-level services such as the virtual network service and the
fault-tolerance service. In order to reduce certification efforts (as well as application complexity), fault-
tolerance mechanisms are ideally provided by the architecture and exploited transparently to the application.
As for the Basic Connector Unit the resources are statically allocated.
In analogy to the Basic Connector Unit, simplicity of the Safety-Critical Connector Unit is of major concern in
order to control certification efforts. For this reason, all inner ports are state message ports and the control
flow is always directed towards the connector unit. For the safety-critical DASs, there is a minimal platform
interface functionality that is required for providing the specified services. Such a minimal functionality helps
to control certification efforts. Additionally, in order to further support certification, the overhead of the Safety-
Critical Connector Unit with respect to diagnosis is reduced to a minimum.
3.1.3 Complex (Non Safety-Critical) Connector Unit
The Complex Connector Unit (Figure 5) performs the allocation of network resources for the non safety-
critical subsystems of a component. The Complex Connector Unit does not directly access the
communication controller, but builds on top of a Basic Connector Unit. This way, the Complex Connector
Unit is not involved in the fault isolation and error containment between the safety-critical and non safety-
critical subsystems of a component, as this separation is performed by an underlying Basic Connector Unit.
Therefore, the Complex Connector Unit and the non safety-critical subsystems of a component need not be
certified to the highest criticality levels and the Complex Connector Unit can provide increased functionality
at the cost of increased complexity.
3.1.4 Diagnostic Service
The task of the diagnostic dissemination service is the forwarding of diagnostic information for a subsequent
analysis. The diagnostic dissemination service exploits the event-triggered communication service and thus
provides an elementary interface. The information producing subsystem acts according to the information
push paradigm [DeLine 1999] and can perform its information dissemination function without depending on
any control signals from the information consuming subsystem. The elementary interface ensures that no
back-pressure flow control can affect non-faulty DASs in case of an error. This is especially important for the
certification of safety-critical DAS. Here, in case of a bidirectional control flow, the certification process would
also require the certification of the diagnosis subsystem.
3.2 Modular Certification of Nodes
The DECOS architecture is as mentioned developed to support modular certification of nodes, which is a key
element for restoring to the integrated architecture the certification benefits of a federated approach. Each
node consists of distinct elements (Basic Connector Unit, Safety-Critical Connector Unit, Complex Connector
Unit, application computers, and communication controller) with precisely specified ports in between. The
ability to construct independent safety arguments for these elements within a node helps in reducing the
overall certification efforts to a manageable level and permits the reuse of safety arguments across different
systems. As shown in Figure 6 below, the modular certification of a node proceeds along two axes. The
safety argument for the architectural services is separated from the application logic. In addition, DASs that
are allocated to different application computers can be separately certified.
DECOS Generic Test Bench: Outlining a Certification Process Page 25/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
Figure 6: Certification of a DECOS node
It should be noted that the physical separation of application computers and secondary connector units
(SCU) (i.e. Safety-Critical Connector Unit or Complex Connector Unit, respectively) is not required, as long
as the demanded fault protection is achieved. So, the interconnection protocol between application computer
and its SCU can be a local network, but also a dual-ported RAM or even a shared memory.
3.2.1 Separating Certification of Architectural Services from Certification of Applications
Certification efforts are significantly reduced by separating the safety arguments relating to the architectural
services provided by the integrated architecture from the certification of application jobs that build on top of
this architecture. The certified core services of the integrated architecture establish a baseline safety
argument for the certification of the overall system [Nicholson 2000].
In DECOS, certification occurs at three levels: On a first level, the DAS-independent basic architectural
services and Basic Connector Units are certified. In particular, the safety argument for the Basic Connector
Unit must provably demonstrate the correct handling of messages at the inner ports towards Safety-Critical
Connector Units independently from the behaviour of the Complex Connector Unit at its respective inner
ports. On the second level, the high-level architectural services for the safety-critical component subsystem
are certifiable without taking into account the actual application logic of jobs. Finally, the certification of the
application logic of jobs occurs on the third level. For this third level, the certification of the underlying
architectural services serves as the baseline for the overall safety argument.
DECOS Generic Test Bench: Outlining a Certification Process Page 26/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
3.2.2 Separate Certification of Application Computers and Job Encapsulation
A DECOS node can contain multiple application computers, each of which can host jobs of one or more
DASs. Since application computers interact only via underlying connector units, the certification of an
application computer can occur independently from the other elements of the node, i.e. only based on the
application computer itself, the I/O subsystems of the application computer, and the inner port specification
of the underlying Safety-Critical Connector Unit.
The modular certification of application computers also enables the certification to different criticality levels.
For example, the certification of one application computer to level B [RTCA 178; ARP 4754] does not
preclude the certification of other application computers to a higher criticality class (i.e. level A).
Nevertheless, the Safety-Critical Connector Unit and Basic Connector Unit must always be certified to the
highest criticality of the application.
In particular, it is assured by design that non safety-critical elements do not have any effect on the safety-
critical elements. The dedicated application computers for the non safety-critical and safety-critical
subsystems of a node in combination with the temporal and spatial partitioning performed by the underlying
Basic Connector Unit make it not necessary to certify the non safety-critical application computers and the
Complex Connector Units. Due to the higher functionality, the support for the event-triggered control
paradigm, and the resulting higher level of complexity in these elements, a certification of these elements
would be intractable with state-of-the-art technology.
If a certified job encapsulation is available on the application computer, which guarantees job separation in
both space (memory protection) and time (no processing time stealing), the latter can be shared among
several jobs of different DASs. This supports an important DECOS goal, namely better utilisation of hardware
resources (and implemented by the EEE, encapsulated execution environment).
3.2.3 Separate Certification of Distributed Application Subsystems (DAS)
DASs consist of jobs, their software units indivisible with respect to distribution, where each job has a certain
criticality. Their assignment to the respective subsystems (i.e. the safety-critical and complex), as well as
their encapsulated execution on DECOS nodes allows for certifying DASs independently from each other, as
long as they do not exchange information (by means of virtual gateways, see [DECOS D2.2.3]; for instance
to share sensor data among several DASs). Still, if information is exchanged among DASs, it occurs under
usage of a certified architectural service, and is restricted to a well defined set of information, which allows
for restricting evaluation of fault propagation in the value domain to precisely this information.
3.3 Incremental Certification
Modular certification requires error containment and encapsulation services from the integrated architecture.
These services also permit incremental certification [Nicholson 2000], i.e. the maintaining of a safety
argument for the remaining part of the system when a DAS is modified or added. Incremental certification
requires the ability to integrate new DASs without the need to re-certify the whole system, see Figure 7. In
particular, modifications to non safety-critical DASs do not involve a recertification of safety-critical DASs.
Incremental certification is very important in e.g. the automotive domain where the sheer number of options
and variants makes the design extremely complex and consequently also certification. For example, one
automotive platform is used for several variants (sedan, hatchback, cab, etc) with different engine choices,
e.g. petrol or diesel. Then the customer can select from a multitude of options ranging from electronically
DECOS Generic Test Bench: Outlining a Certification Process Page 27/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
adjustable driver seats to navigation systems. All these options and variants shall be possible without
compromising the safety and thus keeping the certification valid.
Figure 7 Incremental certification
3.4 The Safety Cases of the DECOS Project
At least four safety cases are expected for an application based on the DECOS technology: one generic for
the DECOS core services (which are considered pre-validated by previous projects in context of DECOS and
the Generic Test Bench), one generic for the DECOS nodes, one generic for the DECOS high-level services,
and one specific for each application. So far only one generic safety case has been established in the
DECOS project: the generic safety case for the safety-critical part of a DECOS node. This safety case is
based on EN 50129 and has been thoroughly described in another document [DECOS GSC] and its
boundary is shown in Figure 8.
Communication Controller
Basic Connector Unit Communication
Network Interface
Safety- Crit. Connector Unit Complex Connector Unit
Application Computer
Job Job Job
Application Computer
Job Job Job
Platform Interface
Core Services
Appl. Prog. Interface
Allocation Layer
VN, Gateways, Diagnosis
VN, Gateways, Diagnosis
Symbols:
Push Pull
Time Triggered
Bus medium
Safety Critical Subsystem Non Safety Critical Subsystem
Figure 8 Boundary of generic safety case
DECOS Generic Test Bench: Outlining a Certification Process Page 28/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
4 DECOS Certification Process
4.1 Consolidation of Certification Requirements from Three Domains
As described in Section 2 the different domains DECOS is targeting have different views and levels of
maturity with respect to certifying safety-related applications based on computer systems (complex hardware
and software).
Among the three domains (industrial control, automotive, and aerospace), the aerospace domain has a lot of
experience and well-established standards for certification of complex electronic systems. These kinds of
systems have more recently been introduced in automotive applications and consequently there are fewer
regulations and standards dealing with this issue. However, as presented earlier, there are Appendices in
the EU regulations touching on the subject as well as the on-going work with the automotive derivative of
IEC 61508: ISO WD 26262. Concerning certification of industrial or process control systems they are
regulated by IEC 61508 derivatives: IEC 61511 for process control, IEC 61131 for PLCs, and IEC 62061 for
machine control. These kinds of applications (systems) are rarely certified since they are installed in a plant
of the system developer itself and thus not in a need for a certification mark to assure its safety and quality.
On the other hand, many components of these systems, e.g. safety PLCs, valves, operating systems, are
certified according to IEC 61508. Obviously, for the component manufacturer the certification is used as a
sales pitch towards the system integrator.
The certification process must be tailored for almost every system or application; there are different
applicable standards and regulations and in addition the operational environments are different. With this in
mind, it is hard to define a universal certification process that fits all. There are however some steps which
seems to be common among the three investigated domains. They are:
1. There shall be a plan for the certification (or functional safety assessment) which outlines assurance
processes and means of compliance to the relevant standards and regulations
2. An early connection with the certification authority shall be established to coordinate the certification
plan.
3. Documentation, a safety case, shall be put together containing:
- A description of the system, its functionality, and its intended operational environment. The
system, its parts and interfaces can be described with schematics and block diagrams for
hardware and e.g. models based on UML for the software.
- The safety concepts used to provide a fail-safe operation system under fault conditions should
be described. If degraded modes of operation are possible for certain fault conditions these
conditions should be specified and the corresponding level of operational degradation.
- The validation plan(s) and the evidential results from the validation activities of that plan:
reviews, analyses, and tests; the activities shall consider both fault and non-fault conditions.
- The verification plan(s) and the evidential results from the verification activities of that plan:
reviews, analyses, and tests; the activities shall consider both fault and non-fault conditions.
- A plan for configuration management
4. The content of the documentation shall be concurrently gathered during development and
verification and validation.
The basic idea of the DECOS-related certification process is to separate the generic parts of the
certification process from the application-specific ones, where in many cases domain-specific standards
have to be fulfilled (although following the generic principles of IEC 61508).
DECOS Generic Test Bench: Outlining a Certification Process Page 29/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
4.2 Other DECOS Processes
In DECOS there are already a few workflows defined [DECOS D1.1.3] and to be able to outline a certification
process they need to be considered. A general view of the already defined workflows in DECOS is presented
in Figure 9 below. From left to right there are three interdependent and cooperating workflows (separation
visualized by two vertical grey lines):
1. Architectural design (HW-SW integration)
2. Software development
3. Verification and validation (Test Bench)
platform spec.information
behaviouralmodeling
HW-SW
integration
req.
capture
HW-SW integration
PIMcreation
implementation
deploymentPSM
DECOS comp code library
executables
SW development
job modelSCADE/Matlab
PIM
requirements
additionalrequirements
job code library
req.definition
creation of v-
plan
Definition of
V&V activities
Test Bench
Execution of
V&V activities
Figure 9: General view of DECOS workflows
The first step in the architectural designs is to represent the functionality, performance, and dependability
requirements of the subsystems in a platform-independent model (PIM). Based on a model of the cluster, its
nodes, and their capabilities, the PIM is transformed into a platform-specific model (PSM) which also is the
output from this workflow.
In the software development workflow, the software structure described in the PIM is imported into SCADE
and the actual behaviour designed and implemented. Then the certified code generator of SCADE is used to
get source code which during deployment is assembled together with e.g. middleware and architectural
service code modules to obtain executables.
In parallel with these two flows, there is the verification and validation workflow supported by the Generic
Test Bench. Guided by validation plans, the safety is assessed, requirements are validated, and
implementations verified throughout all steps of the design and development workflows. The verification and
validation activities of the DECOS prototype Test Bench span from reviews via analyses and formal
verification to testing and fault injection.
DECOS Generic Test Bench: Outlining a Certification Process Page 30/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
4.3 Suggested Certification Process
The goal of this report is to outline a certification process, not to deliver a complete and untouchable one.
The reason is that the other DECOS processes (development and design and Test Bench) are not yet
completely stable and also the possibility of getting input from the application subprojects is necessary.
Based on the consolidated certification requirements and the already existing processes (development &
design and verification & validation) in the DECOS project the following tentative certification process is
suggested, see Figure 10 below.
Development of
concept
Hazard & risk
analysis
Draft certification plan
------------------------------
Establish dialog with
certification authotity
------------------------------
Draft safety case
Development & Design
Process
Verification & Validation
Process
Certification
Process
Integration of
- HW/SW
- System
Design &
implementation of
- Software
- Hardware
Development of
- System architecture
- Subsystems
Requirements
analysis
Integrity/assurance
assessment
Common cause
failure analysis
Verification &
Validation of
- Hardware
- Software
- Integration
-System
Express safety
arguments in terms of
- Safety requirements
- Integrity requirements
- Assurance
requirements
-------------------------------
Determine strategies to
satisfy these
requirements and
techniques to provide
safety evidence
------------------------------
If necessary, refine
safety arguments
Extract and collect
safety evidence and
allocate them to safety
arguments
-------------------------------
Submit the safety case
to the certification
authority
Certification
Concept
Requirements
Components
HW & SW reqs
System
Requirements
Requirements
Requirements
Evidence
Safety caseSystem description and safety concepts
Figure 10: The certification process and its relations to the other processes
The certification process runs in parallel with the other two processes and shall be started as early as
possible to avoid making design decisions that have to be remade later on when the certification authority
has complaints. Note that, compared with Section 4.2, the architectural design and software development
DECOS Generic Test Bench: Outlining a Certification Process Page 31/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
processes have been merged into one to improve the readability. Also, in Figure 10 the processes start
earlier and are presented on a more abstract level compared with the workflows in Figure 9.
4.4 Documentation Support from the Generic Test Bench
Besides the simplified certification provided by the DECOS architecture itself, the Generic Test Bench also
has some features that support the certification process [DECOS D4.1.1; DECOS D4.1.2; DECOS D4.1.3].
The Test Bench framework is implemented using the tool DOORS which among its features has a simple
build-in Microsoft Word document generator. Validation plans and checklists are represented as DOORS
modules and each object in a module represents a V&V activity or a specific question to answer during e.g. a
review. When a V&V activity is executed and all necessary input and output data are available; a V&V
activity report template document is generated for that specific activity. The document is given a unique
name (time stamped) and some already existing information is automatically filled in (method, responsible,
etc.). At the same time, the reference to the report is updated in the validation plan module. When the activity
is finished, the status (pass, indecisive, fail) in the validation plan module is updated as well. A V&V activity
which has been finished with a positive result represents evidence which should be included or referenced in
the safety case. When all activities of the validation plan have been executed or whenever the v-plan
responsible person chooses, it is possible to generate a validation report. The validation report summarizes
which activities that have been executed and with what result; i.e. it is possible to get the status of the
progress of validation activities. For smaller systems or components, this validation report itself could be
used as basis for certification. However, for a large system it is likely that a complete safety case has to be
submitted to satisfy the certification authorities. The validation report can also be used by a third party
evaluator if such is needed (e.g. for ultra-dependable systems). This set of documents forms a hierarchy of
safety evidence but it is important to remember that other evidence is needed as well (safety concepts,
process assurance, etc) and last but not least there shall be arguments motivating why the evidence is
useful.
Refe
rences to
Refe
rences to
Basis
for
Figure 11: Hierarchy of documents containing safety evidence
DECOS Generic Test Bench: Outlining a Certification Process Page 32/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
The hierarchy of documents relevant for certification is shown in Figure 11. For each document a responsible
person or organization are required. Starting from the bottom, there are the V&V activity reports which are
produced by the V&V activity responsible. A V&V activity report template is automatically generated at the
end of the execution of a V&V activity. These V&V activity reports are referenced in the validation report and
the overall results of the V&V activities are presented. For small systems of lower criticality this report can be
used as a basis for certification. For more critical systems, third party evaluation (assessment) is needed and
the report of the third party assessor is also used as a basis for certification. Finally for large systems of high
criticality, it is common to use a safety case which is more formal and more rigorous than a validation report,
however, both the validation report and the third party assessment report contributes to the safety case.
DECOS Generic Test Bench: Outlining a Certification Process Page 33/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
Annex 1. Example of a Certificate
CERTIFICATE no. 12 34 56
Product TTTech TTP DECOS node
Holder/Issued for/Manufacturer
TTTech Computertechnik AG, Vienna, Austria
Product name
TTP DECOS node v 1.0
Product description
The TTP DECOS node is an electronic unit which functions as one node in a DECOS cluster. The TTP DECOS node
consists of four PCBs (TTP-Powerlink FPGA, Powerlink TC1796, Powerlink Evaluation board V 2.9, and MFM
Physical layer).
Certificate
The above described product fulfils the requirements set out in standards: IEC 61508:1-7 and DECOS v 1.0.
Marking
Each product that conforms in all respects with the original item certified may display the text “DECOS compliant”.
Validity
This certificate is valid not later than 1st of July 2012
Vienna, 1st of July, 2007
DECOS Test Institute
Certification Group
Kurt Gödel Alan Turing
Manager, Certification Technical Officer
DECOS Generic Test Bench: Outlining a Certification Process Page 34/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
5 Abbreviations and Definitions .
CCA Common Cause Analysis
COTS Commercial off-the-shelf
DAS Distributed Application Subsystem
EMC Electromagnetic Compatibility
EMI Electromagnetic Interference
FHA Functional Hazard Assessment
FMEA Failure Mode and Effects Analysis
FTA Fault Tree Analysis
LVD Low Voltage Directive
PIM Platform Independent Model
PSM Platform Specific Model
PSSA Preliminary System Safety Assessment
SIL Safety Integrity Level
SSA System Safety Assessment
V&V Verification and Validation
DECOS Generic Test Bench: Outlining a Certification Process Page 35/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
6 References
[Atkinson 2002] Atkinson, R., “COTS Tools Reduce the Cost of Embedded Software Certification”, COTS Journal, 2002
[ARP 4754] SAE ARP 4754 Certification Considerations for Highly-Integrated or Complex Aircraft Systems
[Bishop 1998] Bishop, P.G. and Bloomfield, R.E., “A Methodology for Safety Case Development”, In Proceedings of the Safety-Critical Systems Symposium, 1998
[DECOS D1.1.3] DECOS Design Methodology and Spec. Model DECOS Deliverable 1.1.3, 2005,
DECOS_4.4-001_1.0r_Certification-Process.doc
[DECOS D1.4.3] DECOS PIL Structure Design and API Specification, DECOS Deliverable 1.4.3,
2006, DECOS_1.4-006_1.0r_PIL-Design-And-API.doc
[DECOS D2.2.3] DECOS Virtual Communication Links and Gateways – Implementation of Design
Tools and Middleware Services, DECOS Deliverable 2.2.3, 2005,
DECOS_4.4-001_1.0r_Certification-Process.doc.doc
[DECOS D4.1.3] DECOS Report on First Basic Implementation, DECOS Deliverable 4.1.3, 2005,
DECOS_4.1-007_1.0r_TB_Basic-Implementation.doc
[DECOS D4.1.2] DECOS Test Bench Design and Specification, DECOS Deliverable 4.1.2, 2005, DECOS_4.1-003_1.0r_Test-bench_Design.doc
[DECOS D4.1.1] DECOS Requirements for Generic Test Bench, DECOS Deliverable 4.1.1, 2004,
DECOS_4.1-001_1.0r_SP4-req.doc
[DECOS GSC] DECOS Generic Safety Case for DECOS Nodes DECOS Document, 2005,
DECOS_4.1-004_1.0r_Generic_Safety_Case.doc
[DeLine 1999] DeLine, R., Resolving Packaging Mismatch, PhD Thesis, 1999
[ECE 13] ECE Regulation 13 - Braking devices of certain categories of motor vehicles and their trailers
[ECE 79] ECE Regulation 79 - Steering equipment for motor vehicles and their trailers
[EN 50129] EN 50129 Railway Applications – Safety Related Electronic Systems for Signalling
[Hayhurst 1999] Hayhurst et al., “Streamlining Software Aspects of Certification”, NASA Tech Report, 1999
[IEC 61508] IEC 61508:1-7 Functional Safety of Electrical/Electronic/Programmable Electronic Safety-Related Systems
[Johnson 1992] Johnson, S.C. and Butler, R.W., “Design for Validation”, IEEE Aerospace and Electronic Systems Magazine, 1992
[Kelly 2003] Kelly, T., “A Systematic Approach to Safety Case Management”, In Proceedings of the SAE Conference, 2003
[Kopetz 1997] Kopetz, H. and Nossal, R., “Temporal Firewalls in Large Distributed Realtime Systems”, In Proceedings of the IEEE Workshop on Future Trends in Distributed Computing, 1997
[Kopetz 2004] Kopetz, H., et al., “From a Federated to an Integrated Architecture for Dependable Real-Time Embedded Systems”, 2004
[Langely 2003] NASA Formal methods site
[Nicholson 2000] Nicholson, M. et al., “Generating and Maintaining a Safety Argument for Integrated Modular Systems”, In Proceedings of the 5
th Australian Workshop on Safety
Critical Systems and Software, 2000
[RTCA 178B] RTCA DO-178B – Software Considerations in Airborne Systems and Equipment Certification
DECOS Generic Test Bench: Outlining a Certification Process Page 36/36
Version: 1.0 Status: released DECOS_deliv_Certification_Process.doc
[Rushby 2001] Rushby, J., “Modular Certification”, SRI Technical Report, 2001