Profile Development Process Jack McFadden EMC Engineer Jack.McFadden@ets-lindgren.com 512.779.1523

PDP Presentation

Profile Development Process Presentation

7-23-2012

Objective

The  presentation’s  purpose  is  to  show  a  systematic  method  to  developed  Test  Project  (profile, script, program and etc.) while adhering to Validation/Verification Requirements. The demonstration will used profile and/or script throughout the presentation.

PDP Presentation

Profile Development Process Presentation

10 Jan 2013

PDP Presentation

Certification: The National Association of Radio and Telecommunications Engineers, Inc (NARTE) certified EMC Engineer, Member # EMC 003170-NE Certification: The National Association of Radio and Telecommunications Engineers, Inc (NARTE) certified EMC technician, Member # EMC 001003-NT

http://www.narte.org/ Certification: American Software Testing Qualifications Board, Inc. (ASTQB), Certified Tester Foundation Level (CTFL), Certification # 11-CTFL-00484-USA http://www.astqb.org/sections/list-of-certified-testers.php

My Background

Classical Software Development Approach

PDP Presentation

Classical Software Development Approach

PDP Presentation

Classical Software Development Approach

PDP Presentation

Houston, we found a problem

Classical Software Development Approach

PDP Presentation

Houston, we found a problem

Corrective Action

PDP Presentation

Many modern day quality systems methods are based from the Deming Cycle

Another Common Software Development Approach

PDP Presentation

Revised V-Cycle; provides an avenue for feedback to be applied into the design process

System Design Phase System Integration Phase

PDP Presentation

Item Description

Requirements Tolerance

EUT Active Shall not exceed levels shown in the RE102 Radiated Emission Limit Plot.

±3 dB

Ambient -6 dB below EUT Active Limits, the RE102 Radiated Emission Limit Plot. Results are to be included within the test report if Active Mode results exceed Limits.

Stub Radiator

Verify antenna (signal) path’s integrity at each of the antenna’s highest frequency, approximately.

System Verification

Verify signal path using a calibrated signal at low, med and high frequencies for the active monopole and at the high frequency of the antennas. Rod antenna system verification use a 10 ρf capacitor within the signal path as described within the MIL-STD-461F.

PDP - Requirements The Requirements answer the question; why are you testing. It could be voluntary standard such as an original equipment manufacturer’s specification or a regulatory requirement, like the EMC Directive. It lists the severity level, calibration process, system check, and the objective of the test. The requirements feed the entire profile development process. Using the requirements, you start understanding the test purpose and this assists in creating the test boundaries. This is the foundation of the profile. The table and graph provide an example of MIL-STD-461F, Radiated Emission RE102 test requirements.

0

10

20

30

40

50

60

70

80

1.00E+04 1.00E+05 1.00E+06 1.00E+07 1.00E+08 1.00E+09 1.00E+10 1.00E+11

Field

Stren

gth (d

BuV/

m)

Frequency (Hz)

EUT Active Limit

SV Neg. Tolerance

SV Pos. Tolerance

SV Target/Ambient Limit

PDP Presentation

Test Condition I.D. Test Condition Requirement Test Standard MIL-STD-461F Test Type Radiated Emissions Measurement Instrument EMC/EMI Receiver/Analyzer Detector Used Peak Frequency Accuracy 2% Sweep Method Single Sweep (or Multiple) Measurement Tolerance ±3 dB Resolution Bandwidth

Frequency Range Min. Single Sweep

Duration in seconds Min. Number of Data Points

Start Stop

1 kHz 10 kHz 150 kHz 4.2 280

10 kHz 150 kHz 30 MHz 89.55 5970

100 kHz 30 MHz 1 GHz 291 19400

1 MHz 1 GHz 18 GHz 510 34000

Video Bandwidth Maximum available (3 MHz) Antenna Related Requirements

Antenna Beamwidth Half beamwidth coverage vs. Equipment Under Test width

Antenna Height 120 cm from enclosure floor

Antenna Type

Active monopole 104 cm Biconical

Double Ridge Horn 69 x 94.5 cm opening 24.2 x 13.6 cm opening

PDP – Test Conditions ETS-Lindgren calls this element Test Conditions. The Test Conditions answer the questions of when and how. The test conditions are test parameters necessary in order to meet the test requirements. They are the frequency range, frequency step size, frequency dwell duration, modulation, antenna polarization, number of sides to test and etc. They are typically found within the test standard(s). An example of test conditions is shown in the following table.

PDP Presentation

PDP – Test Specimen Requirements The Test specimen requirements answer the questions of where, who, what and sometimes how. The test specimen requirements are the test specimen’s: modes of operation, monitoring method, acceptance criteria, susceptibility criteria, and the specimen’s lines/cables to test, pre-functional test and post-functional test. The test specimen requirements are found in multiple sources: product specification, test standard, governing (regulatory) standards and etc. The Test Specimen Requirements may not be appropriate for the developing a particular test profile. If profile developer is creating a generic profile, there isn’t a set test specimen and therefore, there are no specific test specimen requirements.

Test Specimen Operational Requirements – what does it need to work? Utility (Service) Current Draw Phase(s) Supply Voltage(s) Wattage Control & I/O Signals Analog Digital Safety (Interlocks)

Test Specimen Functional Requirements – what does it do? Operational Modes, typically found in product specification, tolerance

Test Specimen Requirements – where is the possible noncompliance signals? Sources of emanations Look at critical signals

Test Specimen Requirements – How will it be monitored? Monitoring method

will it be supplied it operational requirements Simulated/Actual do you know when it is compliant Acceptance Criteria

do you know when it is noncompliant Susceptibility Criteria

PDP Presentation

PDP – Operator Control Level There are different operator levels and they can be placed into three categories. Those categories are experts, average/nominal and the novice. The profile developer must consider what the operator level will be that is using the test profile. The profile control must be appropriate to the probable worse case operator level. If your operator is an experienced EMC technician or test engineer the profile control can be minimized. This operator has sufficient knowledge and experience to be given greater control over the test profile. If the operator is a novice, then strict controls should be in place to assure proper operation of the profile without proper controls, the validation of the profiles could be called into question.

Operator Control Level Expert Minimum Operation and Control over Profile

Nominal Medium Operation and Control over Profile Novice Rigid Operation and Control over Profile

PDP Presentation

PDP – Test Level Likewise, the profile developer needs to understand the level of testing. Is the profile to be used for Regulatory (compliance) or Research and Development (pre-compliance)? If the profile is to be used for pre-compliance then the profile developer needs to create the profile to maximize its benefit while reducing its duration. The goal for pre-compliance is generating good data without sacrificing a lot of time. One recommendation is reducing the system check or calibration. It is not a good idea to eliminate the system check completely but reducing the number of points will give you a reasonably confidence the data is valid. For example, if the standard system check requires measurements at the low, mid and high frequency range of an antenna for compliance testing then the pre-compliance system check can be reduced to the one point, typical worse case point. This should provide a reasonable confidence the data collected is within the nominal values. Please note; this reduced compliance/pre-compliance system check assumes the satisfactory completion of the system check. The operator and their required control level and technical skill level should also be considered. Increased checks prior to testing could provide great benefit to the novice operator. If the test profile is to be used for Compliance testing then the profile developer needs to consider the view point of the approval agent. What information is required for compliance test report? What evidence needs to be acquired and recorded to prove the test was performed in accordance to the test standard? Should there be specific points (quality checks) where operator intervention is mandated for the profile to continue its operation? For example, if the test profile requires a positioner to move an antenna from one polarization to another polarization is it unreasonable for the operator to be queried with a prompt to ensure the current polarization is the desired polarization after the polarization change occurs.

Test Control Level Precompliance Reduced level of control over Profile, Compliance Rigid level of control over Profile

PDP Presentation

System Design Phase System Integration Phase

√

√

√

√

?

How are we going to validate/verify (V/V) the test profile

PDP Presentation

How are we going to validate/verify (V/V) the test profile

Please note – V/V is not debugging. Debugging is a normal process within the design cycle.

PDP Presentation

How are we going to validate/verify the test profile

Module Testing – Static Review and Profile subroutine testing with simulator Integration Testing – Profile testing using simulator (PS) System Testing - Profile testing using simulator (PS) and Profile testing using actual equipment but necessary the production equipment (PE) Acceptance Test – Profile Testing using actual production equipment (PE)

PDP Presentation

How are we going to validate/verify the test profile

System Checks with known sources can be used to validate/verify the test profile Ambient Results can be used when you know the ambient condition or create a known ambient conditions or known sources then validate/verify the test profile. EUT Active using known sources can be used to validate/verify the test profile

Module Testing – Static Review and Profile subroutine testing Integration Testing – Profile testing using simulator (PS) System Testing - Profile testing using simulator (PS) and Profile testing using actual equipment but necessary the production equipment (PE) Acceptance Test – Profile Testing using actual production equipment (PE)

PDP Presentation

System Design Phase System Integration Phase

√

√

√

√

Design/Coding Efforts

√

√

√

√

PDP Presentation

Item I.D. No. Risk Description Item I.D. No. Risk Description 1 Incorrect frequency range 2 Incorrect resolution bandwidth 3 Incorrect sweep time 4 Incorrect number of data points/steps 5 Incorrect video bandwidth 6 Incorrect Detector 7 Missing/incorrect data plots 8 Incorrect equations 9 Incorrect/missing transducer factors 10 Incorrect/missing data arrays 11 Incorrect switch settings 12 Incorrect limit 13 Incorrect instrument drivers 14 Incorrect instrument addressed 15 Incorrect preselector settings 16 Incorrect preamplifier settings

PDP – Risk Analysis The information gathered up to this point needs to be assessed. The profile developer knows: what, why, where, when, who and how to test. The next step is to determine what/where the greatest probability for test profile errors is likely to occur. The intent is to reduce the probability for error. This Error Reduction is performed by understanding where errors are likely to occur, generate a plan for handling potential errors and implementing the plan. The following table is an example of a MIL-STD-461F, Radiated Emissions RE102 risk assessment.

PDP Presentation

Item I.D. No. Description Likelihood of Occurrence Severity / Impact Score 1 Incorrect frequency range Medium High High 2 Incorrect resolution bandwidth Medium High High 3 Incorrect sweep time Medium High High 4 Incorrect number of data points/steps Medium Low Medium 5 Incorrect video bandwidth Medium Low Medium 6 Incorrect Detector Low High Medium 7 Missing/incorrect data plots Low High Medium 8 Incorrect equations Medium High High 9 Incorrect/missing transducer factors Medium High High 10 Incorrect/missing data arrays Medium High High 11 Incorrect switch settings Low High Medium 12 Incorrect limit Low High Medium 13 Incorrect instrument drivers Low High Medium 14 Incorrect instrument addressed Low High Medium 15 Incorrect preselector settings Low High Medium 16 Incorrect preamplifier settings Low High Medium

After you identified the risks; the analysis begins. You weigh the risk versus two categories: Severity, and Probability of Occurrence. The following table lists the results of the prior RE102 risk description list. Any items scoring medium or high should be addressed by a corrective action to reduce the risk of error.

PDP Presentation

Item I.D. No.

Description Likelihood of Occurrence

Severity / Impact

Score Correction Acton

1 Incorrect frequency range Medium High High SR & PS 2 Incorrect resolution bandwidth Medium High High SR & PS 3 Incorrect sweep time Medium High High SR & PS 4 Incorrect number of data points/steps Medium Low Medium SR & PS 5 Incorrect video bandwidth Medium Low Medium SR & PS 6 Incorrect Detector Low High Medium SR 7 Missing/incorrect data plots Low High Medium SR & PS 8 Incorrect equations Medium High High SR & PS 9 Incorrect/missing transducer factors Medium High High SR, PS & PE 10 Incorrect/missing data arrays Medium High High SR & PS 11 Incorrect switch settings Low High Medium SR & PE 12 Incorrect limit Low High Medium SR & PS 13 Incorrect instrument drivers Low High Medium SR & PE 14 Incorrect instrument addressed Low High Medium SR & PE 15 Incorrect preselector settings Low High Medium SR & PE 16 Incorrect preamplifier settings Low High Medium SR & PE

Legend: SR Static Review PS Performing Profile operation using simulator PE Performing Profile operation using actual equipment (acceptance testing)

PDP – Corrective Action The corrective action is performed in three Verification/Validation (V/V) steps: profile static review, profile operation using simulation and performing the test profile using actual instruments. These steps are recorded. The information is compiled into the V/V Report. The following table is an example of the Emissions Risk Assessment with the corrective action types identified.

PDP Presentation

Item I.D. No. Description Likelihood of

Occurrence Severity /

Impact Score Action Items

1 Incorrect frequency range Medium High High SR & PS 2 Incorrect step size Medium High High SR & PS 3 Incorrect dwell duration Medium High High SR & PS 4 Incorrect number of amplitude target Medium Low Medium SR & PS 5 Exceeding measurement tolerances Medium Low Medium SR & PS 6 Over driving the amplifier Low High Medium SR & PS 7 Missing/incorrect data plots Low High Medium SR & PS

8 Incorrect equations Medium High High SR & PS 9 Incorrect/missing transducer factors Medium High High SR & PE 10 Incorrect/missing data arrays Medium High High SR & PS 11 Incorrect switch settings Low High Medium SR & PE 12 Incorrect limit Low High Medium SR & PS 13 Incorrect instrument drivers Low High Medium SR & PS

14 Incorrect instrument addressed Low High Medium SR & PE Legend:

SR Static Review PS Performing Profile operation using simulator PE Performing Profile operation using actual equipment (acceptance testing)

Immunity/Susceptibility Risk Analysis Table

PDP Presentation

Profile Multiple Variable Methods Risk Analysis

35  tab’s  &   Total variables = 209

PDP Presentation

Item I.D. No.

TILE Profile Method Description

Profile Complexity Number of Variables vs. Method Risk

Score I/S Actions I/S√ Action (Calibration) Calc.

Data Handling Events

Number of Switching

Events

Total variables

1

Profile with discrete I/S actions for each lines under test (qty 4), EUT profile monitoring 0 channels and EUT Operational Mode (qty 1)

836 (Actions = 4)

71 25 0 0 932 Higher

2

Profile with single I/S actions for each lines under test (qty 4), EUT profile monitoring 0 channels and EUT Operational Mode (qty 1)

209 (Action = 1)

71 9 36 0 325 Lower

TILE Immunity/Susceptibility Profile Methods Risk Analysis using variable quantity

PDP Presentation

TILE Profile Multiple Variable Methods Risk Analysis

13  tab’s  &   Total variables = 62

15  tab’s  &   Total variables = 79

PDP Presentation

Item I.D. No.

TILE Profile Method Description

Profile Complexity Number of Variables vs. Method Risk

Score Quick Scan Actions Measurement

Actions) Calc.

Data Handling Events

Number of Switching

Events

Total variables

1

Profile with discrete Quick Scan & Measurement actions for each lines under test (qty 4), EUT profile monitoring 0 channels and EUT Operational Mode (qty 1)

248 (Actions = 4)

237 24 0 12 521 Higher

2

Profile with single Quick Scan & Measurement actions for each lines under test (qty 4), EUT profile monitoring 0 channels and EUT Operational Mode (qty 1)

62 (Action = 1)

79 6 18 3 168 Lower

TILE Emission Profile Methods Risk Analysis using variable quantity

PDP Presentation

Key Take aways Create and use Program Development Process The information is also available at the following links:

http://www.astqb.org/ http://www.computer.org/portal/web/swebok http://www.sqe.com/ https://support.ets-lindgren.com/Pages/Default.aspx

PDP Presentation

PDP Presentation

Nomenclature Title Rev. Date

n/a Software Validation in Accredited Laboratories, A Practical Guide, Gregory D. Gogates, Fasor Inc. ://ftp.fasor.com/pub/iso25/validation/adequate_for_use.pdf

n/a 07 June 2010

n/a Description of the SWEBOK Knowledge Area Software Engineering Process (Version 0.9), Khaled El Emam, NRC, Canada

0.9 2001

n/a Software Training and Consulting (SQE Training) Testing, Development, Management Requirements and Security

V4.1 2004-2011

MIL-STD-2155 Military Standard - Failure Reporting, Analysis and Corrective Action System (FRACAS)

(AS) 24 Jul 1985

TPDP TILE Profile Development Process, J. McFadden, ETS-Lindgren n/a 06 Aug 2012

References

QUESTIONS? Jack McFadden EMC Engineer Jack.McFadden@ets-lindgren.com 512.779.1523

PDP Presentation