FIT for Embedded System Test Development

PRESENTER : Jingwen Chen

Main topicsAgile methodology from the desktop world XP Inspired embedded system life-cycleAdapting FIT into embedded system developmentWhat is FIT/FitNesseEmbeddedFitNesse testing frameworkMatLab/C++ FitNesse testing frameworkIssues with FIT in embedded design Conclusion

What we want to avoid ? System defectsSoftware errors cost $59 billion / yearBaxter Healthcare has to recall all its 256,000 Colleague Volumetric Infusion Pumps because the hardware / software problems that already cause 3 deaths and 6 serious injuries.We want to find an appropriate and efficient defect free development process for embedded devices

Agile methodology from the desktop business worldMinimize risk by developing software in short iterations, ech iteration is like a miniature software project of its own

Extreme Programming (XP) http://www.extremeprogramming.org/map/project.html

Agile methodology from the desktop business worldExplore design space Test Driven Development (TDD) Improved defect reductionEnhanced communication between the customers and developers Extremes of knowledge within a multi-disciplinary team Customer: product end-users and researchersDeveloper: Engineers

XP inspired (XPI) product lifecycle for biomedical embedded systemStage 1 : XPI Product envisagement Customers together with developers produce executable system specificationSystem acceptance tests Stage 2 : XPI Proof of conceptAlgorithms developed in MATLAB and validated by the acceptance tests written in first stageAdditional unit test in MATLAB using MUnit

XP inspired (XPI) product lifecycle for biomedical embedded systemStage 3 : XPI Prototyping of initial productForming the initial product using host machine simulation / embedded systemSystem moved into the C/C++ production language and validated via tests written in stage 1 and stage 2Additional embedded unit tests added for new environment.Stage 4 : XPI Full production of systemThe production of the full system on real embedded system target board Satisfy the acceptance tests and unit tests developed

Build an appropriate Agile development environment for embedded systemA mechanism to allow customers without embedded system knowledge to form a part of the development team

FitNesse/FIT is one possible solution

What is FIT/FitNesse ?A software testing tool designed for customers with limited IT knowledgeA wiki-based system for creating, organizing, and running FIT test tablesFit test tables are easy to be read and written by customer

An example of FIT test table

Starting from a simple temperature measurement embedded systemTest software function: CalculateTemperature ( ) running on the target board convert pulse width to actual temperature

Target Platform Hardware LayerAnalog DeviceADSP-BF533 processorSystem Software LayerApplication Layer CalculateTemperature( )VDK real-time operating systemInput device: TMP03 thermal sensor

Voltage pulse:

Time LowTime High

Stage 1. XPI Product envisagement

Customers together with the developers write the Fit test table

Stage 2. XP Inspired Proof of concepts

MatLabFitNesse testing frameworkDeveloping the function in MatLab validated by the fit test in stage 1 Fixture forTest tableFitNesseWeb serverWiki editor and storageMATLAB ENGINEMATLAB API RunnerFitServer

Stage 3. XPI initial production systemMigrate the MatLab code into embedded system language validated via tests for stage 1 and stage 2 using host machine simulation HOST MACHINEEmbeddedFitNesse Fixture forTest table RunnerFitServerC++ Simulation EnvironmentFitNesseWeb serverWiki editor and storageMATLAB ENGINEMATLAB API

Stage 4. XPI full production system

Running the function on real embedded target platform and satisfying the original test in stage 1 HOST MACHINEEmbeddedFitNesse Fixture forTest tableDSP Interface classS.M.I.L.E TARGET PLATFORMMATLAB ENGINEMATLAB APIFitNesseWeb serverWiki editor and storageVDSP APIVDSP Environment RunnerFitServerC++ Simulation Environment

Some tests we have done for embedded hardware:An error on the ADSP-BF533 processor was found by undergraduate students taking our embedded systems course.They were using TDD to develop the register setting function SetCoreTimer ( ) running on the embedded target board.

Issues with the full EmbeddedFitNesse testing framework HOST MACHINEEmbeddedFitNesse Fixture forTest tableFitNesseWeb serverWiki editor and storageLong communication path !S.M.I.L.E TARGET PLATFORMDSP Interface classVDSP APIVDSP Environment RunnerFitServerC++ Simulation EnvironmentMATLAB ENGINEMATLAB API

FitNesse in embedded design

How to use FIT tables to specify an embedded system properly ? Embedded system acceptance tests includes:system operationsexternal and internal stimulus responsesinterrupt latencysystem reliability Extend the fit test table to handle diverse test cases.timing diagramssignal comparison figuresEfficiency and reliability concernsMake the EmbeddedFitNesse testing frame work more efficiently

Is FitNesse a good solution for embedded system Agile development ?

ConclusionXPI Lifecycle embedded system developmentEmbeddedFitNesse testing frameworkMatLab/C++ FitNesse testing frameworkIssues with FIT in embedded design

Thank you !