Upload
others
View
12
Download
2
Embed Size (px)
Citation preview
BIOPRESENTATION
International Conference OnSoftware Testing Analysis & Review
November 15-19, 2004Anaheim, CA USA
T14
November 18, 2004 1:30 PM
AUTOMATED DATABASE
TESTING WITH NUNIT
Alan CorwinProcess Builder, Inc.
Alan Corwin Alan Corwin has been a software developer, trainer, and course designer for nearly thirty years. He is president and founder of Process Builder, CIO of OptionBots, and teaches High Performance Data Integration in the University of Washington’s Advanced Web Development program.
2
Planned?Isn’t test-driven development planned?Yes. Although planning is often decried as a waste of time by the XP community, simply writing your tests before you write your code is a form of planning. It’s the next level of planning that few XPersare willing to take – planning to do a series of standard tests for an entire application.
3
Test GoalsTest the Generic Database RequirementsProvide a Framework for More Complex TestsSupport Rapid Development (Daily Deliverables!)Support Test-Driven Development Allow Tests to be run both by developers and testers.Test engineers and developers should be able to quickly and easily add test cases as needed.Eliminate unnecessary communication cycles.
4
Test Design FeaturesTests are fully automated.Tests are black box (ignorant of the underlying data structure)Tests are infinitely expandable.Tests remain valid even when the implementation changes.
6
The Test ProcessDeveloper Builds the Test DataSet ClassesDeveloper Creates the Test Data FileThe Test Data Files are placed in a common directory.Developer constructs and executes the basic test classes.Test Engineers develop test cases and add them to the test file.Tests are continually added.Each time the product is changed or a test case is added, all of the tests are rerun.
Developers run all tests when the code changes.Test engineers run all tests when new test cases are added.
8
What is the Purpose of the test DataSet?
To hold the values necessary to robustly test the Create, Read, Update, and Delete functions.To provide a strongly typed DataSet to increase reliability and reduce coding.To simply the generation and initial population of the test data file.
9
Why do you need a DataSet?First, you don’t “need” a DataSet. They are just a good idea.DataSets reduce the amount of code required to construct the tests.DataSets increase the readability of code.DataSets eliminate many kinds of errors in the test code itself.
10
Create the Test DataSetEach DataSet is based on an underlying table, view, or stored procedure.Start the DataSet Wizard by adding a new DataSetto your test project.Drag the underlying object to the designer.Switch to the XML view.Modify the primary key field.Duplicate the data fields.Add “Update” to the end of each new field name.Save.
Demo Here!
11
What Just Happened?The format for the test data file was defined.An XSD (XML schema definition file) was just created.A Strongly Typed DataSet class was generated (3000 or more lines of highly reliable code) – requires that Generate DataSet be turned on (it is by default).
12
Generate the Data FileHere’s where XML Spy comes in handy.Open the Schema file you just created in XML SpyChoose the ‘Generate Sample XML File’ command from the DTD/Schema menu.When the dialog appears, select your options including the number of rows to generate, and click OK.Save the file in your public TestData directory as an XML file.Edit the data rows as needed.
Demo Here!
13
The Test Data FileXML Format Means
Editable with a wide variety of tools (including Notepad)Built to a known standard supported by every software vendorEasy to transfer to any other data formatEasy to read and understand..NET-ready
For proper testing, you need both a green bar data file and a red bar data file.
14
NUnit BasicsWhat is NUnit?NUnit LimitationsObtaining and Installing the ProductConstructing A Simple TestRunning Tests
15
What Is NUnit?A Public Domain Test Harness for .NET
A Set of Classes to Facilitate Test Development and ExecutionA Command Line InterfaceA GUI to Run the Tests from Windows
A Tool For Developer-Centered TestingDeveloped as JUnit, Ported to .NET, then Rewritten in C#
16
NUnit LimitationsYou can’t test exception propagation easily.
Hard Code TestsRed Bar Tests
Can’t debug when you run from the UI(This was a much longer list until version 2.1
which corrected most of the problems.)
Unlike user interfaces, database and business objects should throw rather than handle exceptions.
17
Obtaining and Installing NUnithttp://www.nunit.orgGo to the Downloads PageRun the Executable
(That’s all there is to installing the program. Note, however that each project only knows about the test harness when it contains a reference to the NUnit libraries.)
18
Constructing a Simple TestAdding the Harness ReferenceCreating a Test ClassAdding Test Fixture DecorationsWrite the Test Logic
19
Adding the Harness ReferenceRight-click the References icon for the test project in the Solution Explorer and select Add Reference.When the Open dialogue appears, navigate to the NUnit Installation directory.Select all of the DLLs and click Open.(I’m not sure they are all needed, but the documentation does not specify which are needed.)
20
Creating a Test ClassAdd a new class to the project.Import the NUnit Framework
Imports NUnit.Framework (VB.NET)using NUnit.Framework; (C#)
Add a TestFixture Decoration to the class declaration.
<TestFixture()> (VB -- must be on the same line as the class declaration)[TestFixture] (C# -- Must be on the declaration line or the line before)
21
Write the Test LogicGet the Test DataFor Each Test Case
Create the RecordVerify the RecordUpdate the RecordVerify the RecordDelete the RecordVerify the Deletion
The test logic is often in a different class than the test execution.
That is the approach that we use because it makes debugging both the test and the application easier.
23
Process Change ResultsAs a result of adding NUnit and the basic database test process demonstrated here, we were able to shrink delivery times significantly while testing our database applications hundreds of times more thoroughly than we could before. To our credit and embarrassment, our prototypes are now more robust and bug-free than our finished applications used to be. (And our finished applications were always more robust than the majority of what you see.)
24
Current Deliverables for One Programmer for One Four Hour Work Period
One DataSet ClassOne Command Helper ClassFive Stored ProceduresOne Business Object Class for Individual ObjectsOne Business Object Class for the DataSet
One Test Class for Each Business Object ClassOne Test Data File containing at least 3 test casesA Data Entry form for the individual object class.A List form for the DataSetClass
25
SummaryNUnit is free, flexible, and easy to use.XML test data files are easy to create and use.NUnit supports a simple, standard approach to database testing that addresses the needs of both the test team and the development team.The result of this approach is an automated test suite that creates significant gains in quality and productivity.
26
Technical AppendixTest LogicSet TestsAdditional TestsStandardizing Test ClassesResourcesAutomation of Test Automation
27
Test Logic: Get the DataThree Details Make getting the Data Easy
Our test data is in an XML file.We have a strongly typed DataSet class that matches that XML file.Microsoft provides the ReadXML method.
VBDim myTestData As New MyTestDataSetName()myTestData.ReadXML(fullPathOfTestFile)
C#MyTestDataSetName myTestData = New MyTestDataSetName();myTestData.ReadXML(fullPathOfTestFile);
28
Test Logic: Get Each Test Case
A Strongly Typed DataSet consists of Strongly Typed DataRows. VBFor Each rowInstance _
As BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow _in _testSet.Tables[0].Rows
_testRow = rowInstance ‘Assign to class variable for common use.Next rowInstance
C#foreach (BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow rowInstance in _testSet.Tables[0].Rows)
{_testRow = rowInstance;} // assign to a class variable for common use.
These Row classes were generated when the Test Data DataSet was created.
29
Test Logic: The CRUD TestsThe Logic of your CRUD tests will depend on the structure of your application, but the set of tests will always have to be done within the test data loop. VBFor Each rowInstance _
As BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow _in _testSet.Tables[0].Rows
_testRow = rowInstance ‘Assign to class variable for common use.RunCrudTestSuite()
Next rowInstance
C#foreach (BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow rowInstance in _testSet.Tables[0].Rows){
_testRow = rowInstance; // assign to a class variable for common use. RunCrudTestSuite();
}
A better design would pass the test row to the CRUD test routine..
30
Our Basic CRUD Testsprivate void RunCrudTestSuite(){
int recordID = theNewID();TestInitialValues(recordID);UpdateRecord(recordID);TestUpdateValues(recordID);DeleteRecord(recordID);TestDeletion (recordID);
}
The recordID ties the tests to the same database object.
31
theNewID()protected override int theNewID(){
SampleBusinessObject testObject = new SampleBusinessObject();testObject.Name = _testRow.name;testObject.OptionBotID = _testRow.optionBotID;testObject.Save();Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred.");return testObject.ID;
}
Access to the database is allowed only through the business objects.A new business object is created.The properies are set from the test data row.The business object is saved.A check is made to see that no exceptions occurred.The id of the new record is returned.
Note the Assertion. This is an NUnit Assertion.
32
TestInitialValues()protected override void TestInitialValues(int idOfObject){
SampleBusinessObject testObject = new SampleBusinessObject(idOfObject);Assert.AreEqual(testObject.Name, _testRow.name, “Wrong name.");Assert.AreEqual(testObject.OptionBotID, _testRow.optionBotID, “Wrong OptionBot ID!");Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred.");
}
Tests are very simple.You put a value into the field.You check to make sure that the same value was recorded and retrieved.
33
UpdateRecord()protected override void UpdateRecord(int idOfObject){
SampleBusinessObject testObject = new SampleBusinessObject(idOfObject);testObject.Name = _testRow.nameUpdate;testObject.OptionBotID = _testRow.optionBotIDUpdate;testObject.Save();Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred.");return testObject.ID;
}
This is very similar to creating the record except that we start with an existing object.Note that we are populating the business object with the update values from the test data row.
34
TestUpdateValues()protected override void TestUpdateValues(int idOfObject){
SampleBusinessObject testObject = new SampleBusinessObject(idOfObject);Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred.");Assert.AreEqual(testObject.Name, _testRow.nameUpdate, “Wrong name.");Assert.AreEqual(testObject.OptionBotID, _testRow.optionBotIDUpdate, “Wrong OptionBot
ID!");}
Update test is almost identical to the initial values test.Note that we are testing against the update values.
35
DeleteRecord()
protected override void DeleteRecord(int idOfObject){
SampleBusinessObject testObject = new SampleBusinessObject(idOfObject);testObject.Delete();
}
DeleteRecord is very simple. The logic for the deletion is in the business object so our test code simply calls the objects Delete Method.
36
TestDeletion()protected override void TestDeletion(int idOfObject){
SampleBusinessObject testObject = new SampleBusinessObject(idOfObject);Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred.");Assert.AreEqual(testObject.IsNew, true, “The object was not deleted.");
}
Update test is almost identical to the initial values test.Note that we are testing against the update values.
37
The Test DecorationNote that although we had to add a TestFixture decoration to the class declaration, none of the individual tests have had any additional decorations. These cannot be run from the NUnit GUI. That is intentional; the CRUD regression tests are designed to be run as a unit.Add a test in your Test Fixture class to run the whole set and add the declaration as shown on the next screen.
38
Test Decoration ExampleThe test decoration allows the NUnit GUI to recognize a method as a test.
[Test]public void TestBackSpreadParameterSets(){
SampleObjectTest _thisTest = new SampleObjectTest(@"H:\TestData\OptionBot\SampleObjects.xml");_thisTest.RunAllTests();
}
In VB.NET, the correct decoration is ‘<Test()>’.
39
Test Initialization ExampleNUnit allows you to specify a set of actions that are performed when the Test Fixture class is initialized.
41
Additional TestsAre Driven by Customer Requirements Are added at the specification of the test engineer.Are added by the developer.Additional Test Example
42
Standardizing the Test ClassesTest Class Interfaces are used to standardize the language used for the test methods.Abstract Test Classes capture the common elements of test classes. The latest versions of our test classes are currently available at http://www.ProcessBuilder.com for the following languages
VB.NETC#
43
NUnit ResourcesNUnit Home Page, http://www.nunit.orgNUnit Resources http://www.testdriven.com/modules/news/