Click here to load reader

1 Benoit Baudry – 2000 : Masters degree at the Univ. de Rennes 1 in – june 2003 : PhD thesis, « Testable assembly and component validation » with Yves

  • View
    213

  • Download
    0

Embed Size (px)

Text of 1 Benoit Baudry – 2000 : Masters degree at the Univ. de Rennes 1 in – june 2003 : PhD...

  • Slide 1
  • 1 Benoit Baudry 2000 : Masters degree at the Univ. de Rennes 1 in june 2003 : PhD thesis, Testable assembly and component validation with Yves Le Traon and Jean-Marc Jzquel in the Triskell group at the Univ. de Rennes 1 Next : Postdoc position at the CEA-Saclay on MDA
  • Slide 2
  • 2 Test in the Triskell group Triskell : Model Driven Engineering for Component Based Software (http://www.irisa.fr/triskell/welcome.htm) UML-based OO testing (Yves Le Traon) : Test order, integration strategies Vu Le Hanh. -- test et modle UML : stratgie, plan et synthse de test. -- PhD thesis Test generation from the requirements (Clmentine Nebut) UML-based test generation Alain Le Guennec. -- Gnie Logiciel et Mthodes Formelles avec UML Spcification, Validation et Gnration de tests. -- PhD thesis PhD starting on MDA and testing (Franck Fleurey)
  • Slide 3
  • Testable assembly and component validation Benoit Baudry Triskell group, IRISA, Rennes, France Jean-Marc Jzquel Yves Le Traon
  • Slide 4
  • 4 Complex software systems are built with components as the unit for reuse Trustable components Techniques to assemble components
  • Slide 5
  • 5 Trustable component Specification Implementation V & V : Test cases set Trust based on consistency executables contracts
  • Slide 6
  • 6 Components assembly Impact of design by contract Impact of coupling on a testability factor
  • Slide 7
  • 7 Context Software components Object oriented design and analysis Specific structures in OO programs Need to adapt testing techniques R. V. Binder, "Testing Object-Oriented Systems: Models, Patterns and Tools". Addison-Wesley 1999.
  • Slide 8
  • 8 Summary Related work Software testing Mutation analysis Automatic test cases generation Design by Contract and testing Component testability
  • Slide 9
  • 9 Software testing Two types : Structural based on implementation Functional based on specification of functionalities Objectives Examine or execute a program searching for errors Possibly : robustness performances safety properties
  • Slide 10
  • 10 Le test de logiciel Generation Test data Execution Test case Oracle Diagnostic Test criterion Stop correct correct verified verified
  • Slide 11
  • 11 Mutation analysis R. DeMillo, R. Lipton and F. Sayward, "Hints on Test Data Selection : Help For The Practicing Programmer". IEEE Computer 11(4): 34 - 41 April 1978. Technique that aims at validating the quality of a test cases set Errors injected in the program under test Compute the proportion of errors detected by the test cases
  • Slide 12
  • 12 Mutation analysis Several error types: mutation operators Operators defined from analysis of errors sets observed during development J. Offutt, A. Lee, G. Rothermel, R. H. Untch and C. Zapf, "An Experimental Determination of Sufficient Mutant Operators". ACM Transactions on Software Engineering and Methodology 5(2): 99 - 118 April 1996. Recent work [Ma02, Alexander02] propose OO specific operators
  • Slide 13
  • 13 Mutation analysis Faulty program : mutant Test cases detect mutants Test cases kill mutants Mutation score Proportion of killed mutants quality of test cases Two oracles Traces difference Executable contracts
  • Slide 14
  • 14 Mutation analysis P Mutants generation Mutant 1 Mutant 2 Mutant 3 Mutant 4 Mutant 5 Mutant 6 TC Executio n Killed mutant Diagnosis Alive mutant Incomplete specification Automatique Manuel Optimiser Equivalent mutant Deleted from the set of mutants Add contracts Insufficient test cases
  • Slide 15
  • 15 Automatic test cases generation and optimisation
  • Slide 16
  • 16 Automatic test cases optimisation Average mutation score easy to reach By hand or random generation Unit testing Class testing in an OO context Component testing Classes assembly with a main interface class
  • Slide 17
  • 17 Genetic algorithm Build an initial population Compute the fitness value for each individual Mutation score Reproduction Crossover Mutate one or several individuals Boucle gntique Several stop criteria: good mutation score, number of generations test_set is do time.set_hour(10) time.set_minute(7) time.set_second(2) time.set(10,7,5) end test_hour_12 is do time.set_hour(0) end test_is_am_pm is do time.set_hour(12) time.set_hour(1) time.set_hour(13) time.set_hour(0) end test_hour is do time.set_hour(0) time.set_hour(1) end test_out is do time.set(02,02,02) end test3 is do time.set(0,0,0) end test_comparaison is local time1 : P_TIME do !!time1 time.set(15,59,59) time1.set(14,59,59) time1.set(16,00,00) end test_hour_12 is do time.set_hour(0) end test_out is do time.set(02,02,02) end test_set is do time.set_hour(10) time.set_minute(7) time.set_second(2) time.set(10,7,5) end test_hour_12 is do time.set_hour(0) end test_is_am_pm is do time.set_hour(12) time.set_hour(1) time.set_hour(13) time.set_hour(0) end test_hour is do time.set_hour(0) time.set_hour(1) end test_out is do time.set(02,02,02) end test3 is do time.set(0,0,0) end test_set is do time.set_hour(10) time.set_minute(7) time.set_second(2) time.set(10,7,5) end test_out is do time.set(02,02,02) end test3 is do time.set(0,0,0) end test_hour is do time.set_hour(0) time.set_hour(1) end test_hour_12 is do time.set_hour(0) end test_is_am_pm is do time.set_hour(12) time.set_hour(1) time.set_hour(13) time.set_hour(0) end test_hour is do time.set_hour(0) time.set_hour(1) end test_out is do time.set(02,02,02) end test3 is do time.set(0,0,0) end test_hour is do time.set_hour(0) time.set_hour(1) end test_out is do time.set(02,02,02) end test3 is do time.set(0,0,10) end
  • Slide 18
  • 18 Genetic algorithm Tools development (Java, C#) For mutation analysis: JMutator, NMutator Framework for the genetic algorithm Test driver Experiments Several classes or components
  • Slide 19
  • 19 C# case study
  • Slide 20
  • 20 Genetic algorithm Fixed size for the test cases set Crossover not much useful Reproduction not efficient to keep memory
  • Slide 21
  • 21 Bacteriologic algorithm Rosenzweig, Species diversity in space and time, CUP,1995 Delete : The notion of individual Crossover operation Introduc : The notion of bacterium a test case A memory set of good bacteria
  • Slide 22
  • 22 Bacteriologic algorithm Build an initial medium Compute the fitness value for each bacterium Mutation score Memorise the best ones Mutation Several stop criteria: good mutation score, number of generations Memory test_out is do time.set(15,59,59) end test_hour is do time.set_hour(0) time.set_hour(1) end test3 is do time.set(0,0,0) end test_set is do time.set_hour(10) time.set_minute(7) time.set_second(2) time.set(10,7,5) end Bacteriologic medium test_out is do time.set(15,59,59) end test_hour is do time.set_hour(0) time.set_hour(1) end test_out is do time.set(02,02,02) end test3 is do time.set(0,0,0) end test_out is do time.set(15,59,59) end test_out is do time.set(15,60,59) end
  • Slide 23
  • 23 Bacteriologic algorithm Tools development (Java, C#) Experiments Several case studies Tuning / validating the model
  • Slide 24
  • 24 C# case study
  • Slide 25
  • 25 Results Original algorithm for automatic test cases generation Tools development Work in progress with new programs and new fitness functions
  • Slide 26
  • 26 Design by contract for robustness and diagnosability
  • Slide 27
  • 27 Design-by-contract A design method for OO components (B. Meyer) Boolean assertions: pre et post conditions for each method invariants for global properties A broken contract indicates the presence of a bug: Precondition violated a client has broken the contrat Postcondition violated an error in the method
  • Slide 28
  • 28 Two quality criteria Robustness Ability for a component to detect a faulty internal state Diagnosability effort for the localization of a fault and the preciseness allowed by a test strategy on a given system, knowing there is a fault
  • Slide 29
  • 29 Robustness A Local robustnessability of contracts to detect errors Combination is better than addition Global robustness B C A contracts Det(A,C)
  • Slide 30
  • 30 Robustness Global robustness for a components assembly depends on: Local robustness of components The Det(i,j) probability a component i detects errors in j Mutation analysis test cases set with a 100% mutation score local robustness mutation score of contracts Det(i,j) probability mutation score for contracts in i with mutants of component j
  • Slide 31
  • 31 Robustness
  • Slide 32
  • 32 Diagnosis scope Classical softwareDiagnosability Disgnosis scope Software designed by contracts Exception handling
  • Slide 33
  • 33 Diagnosability 00,20,40,60,81 Contracts/assertions density Diagnosability 0 100 200 300 400 500 600 700 800 900 1000 0.2 0.4 0.6 0.8 Contracts /assertions efficiency
  • Slide 34
  • 34 Results Qualitative study of design-by-contract Summary Adding contracts, even if they are weak, improves the components quality Efficiency of contracts has more impact than the density
  • Slide 35
  • 35 Testability anti-patterns in

Search related