Embed Size (px)
DESCRIPTION
Testing: the final frontier. These are the voyages of Software Craftsmanship. Its continuing mission: to explore strange new worlds, to seek out new code and new patterns, to boldly go where no one has gone before. Looking to take your testing to new worlds? Want to verify your code meetings your user's requirements? Come learn about behavior and property testing in .Net. We'll look at how SpecFlow can get everyone on the same page and how to find more edge cases with property testing using FsCheck.
Citation preview
T E S T I N G T H E N E X T G E N E R AT I O N
M I K E H A R R I S
fscheck
specflow
is testing oftraits
specflow
is testing ofproperties
fscheck
B E I N G
B E I N G
B E I N G T H I N G S
B E I N G T H I N G S
A S M A R T I N H E I D E G G E R S P E A K S O F T H E M I N
“ T H E O R I G I N O F T H E W O R K O F A R T "
I N T E R P R E TAT I O N S O F T H I N G S
• substances with properties or as bearers of traits
• sense perceptions
• formed stuff
S U B S TA N C E S W I T H P R O P E R T I E S O R A S B E A R E R S O F T R A I T S
fscheck
specflow
is testing oftraits
specflow
is testing ofproperties
fscheck
S U B S TA N C E S W I T H P R O P E R T I E S O R A S B E A R E R S O F T R A I T S
Testing of Traits
is testing oftraits
specflow
I N E E D A F I Z Z B U Z Z T R A N S L AT O R
C H E C K P O I N T 1 ) N O B M I ' D I V I S I B L E P O N G 3 F I Z Z J I C H E G H ' O H . N O B M I ' D I V I S I B L E P O N G 5 B U Z Z J I C H E G H ' O H . L AT L H S O ' M E H J I C H E G H ‘ O H . !2 )⎕ I O← 0 ( L , ' F I Z Z ' ' B U Z Z ' ' F I Z Z B U Z Z ' ) [ ¯ 1 +( L×W = 0 ) + W← ( 1 0 0×~ 0 = W ) + W←⊃ + / 1 2× 0 = 3 5 |
⊂ L← 1 +⍳ 1 0 0 ]
S A M E PA G E !
C H E C K P O I N T 2 - > 2 3 - > F I Z Z 4 - > 4 5 - > B U Z Z 6 - > F I Z Z
M A K E I T S O .
F I Z Z B U Z Z
FizzBuzz(1 ) => “1” FizzBuzz(2 ) => “2” FizzBuzz(3 ) => “Fizz” FizzBuzz(4 ) => “4” FizzBuzz(5 ) => “Buzz” FizzBuzz(6 ) => “Fizz” … FizzBuzz(15) => “FizzBuzz”
namespace FizzBuzz { public class FizzBuzzer { public string Translate(int value) { var result = string.Empty; if (value % 3 == 0) result += "Fizz"; if (value % 5 == 0) result += "Buzz"; return string.IsNullOrEmpty(result) ? value.ToString() : result; } } }
T E S T I N G T R A I T S
T H E N U N I T W AY
[TestCase( 2, Result = "2")] [TestCase( 3, Result = "Fizz")] [TestCase( 4, Result = "4")] [TestCase( 5, Result = "Buzz")] [TestCase( 6, Result = "Fizz")] [TestCase(10, Result = "Buzz")] [TestCase(15, Result = "FizzBuzz")] [TestCase(45, Result = "FizzBuzz")] public string FizzBuzz_Tests(int value) { return fizzbuzzer.Translate(value); }
S P E C F L O W W AY
Feature: FizzBuzzerSpec In order to be able to communicate with the FizzBuzzons As a Starfleet Captain I want to be able to translate numbers to their FizzBuzz value
Scenario: 2 must translate to the string 2 Given a FizzBuzzer And the value of '2' When translate is invoked Then the result should be '2'
Scenario: 3 must translate to the string Fizz Given a FizzBuzzer And the value of '3' When translate is invoked Then the result should be 'Fizz'
Scenario: 5 must translate to the string Buzz Given a FizzBuzzer And the value of '5' When translate is invoked Then the result should be 'Buzz'
Scenario: 15 must translate to the string FizzBuzz Given a FizzBuzzer And the value of '15' When translate is invoked Then the result should be 'FizzBuzz'
Feature: FizzBuzzerSpec In order to be able to communicate with the FizzBuzzons As a Starfleet Captain I want to be able to translate numbers to their FizzBuzz value Scenario: 2 must translate to the string 2 Given a FizzBuzzer And the value of '2' When translate is invoked Then the result should be '2' Scenario: 3 must translate to the string Fizz Given a FizzBuzzer And the value of '3' When translate is invoked Then the result should be 'Fizz' Scenario: 5 must translate to the string Buzz Given a FizzBuzzer And the value of '5' When translate is invoked Then the result should be 'Buzz' Scenario: 15 must translate to the string FizzBuzz Given a FizzBuzzer And the value of '15' When translate is invoked Then the result should be 'FizzBuzz'
Feature: FizzBuzzerSpec In order to be able to communicate with the FizzBuzzons As a Starfleet Captain I want to be able to translate numbers to their FizzBuzz value
Scenario: 2 must translate to the string 2 Given a FizzBuzzer And the value of '2' When translate is invoked Then the result should be '2'
using FizzBuzz; using TechTalk.SpecFlow; using NUnit.Framework; namespace FizzBuzzSpec { [Binding] public class FizzBuzzerSpecSteps { [Given(@"a FizzBuzzer")] public void GivenAFizzBuzzer() { ScenarioContext.Current.Add("FizzBuzzer", new FizzBuzzer()); } [Given(@"the value of '(.*)'")] public void GivenTheValueOf(int value) { ScenarioContext.Current.Add("value", value); } [When(@"translate is invoked")] public void WhenTranslateIsInvoked() { var fizzbuzzer = ScenarioContext.Current.Get<FizzBuzzer>("FizzBuzzer"); var value = ScenarioContext.Current.Get<int>("value"); ScenarioContext.Current.Add("actual", fizzbuzzer.Translate(value)); } [Then(@"the result should be '(.*)'")] public void ThenTheResultShouldBe(string expected) { var actual = ScenarioContext.Current.Get<string>("actual"); Assert.That(expected, Is.EqualTo(actual)); } } }
Given a FizzBuzzer !
[Given(@"a FizzBuzzer")] public void GivenAFizzBuzzer() { ScenarioContext.Current.Add( "FizzBuzzer", new FizzBuzzer()); }
And the value of '2' !
[Given(@"the value of '(.*)'")] public void GivenTheValueOf(int value) { ScenarioContext.Current.Add("value", value); }
When translate is invoked [When(@"translate is invoked")] public void WhenTranslateIsInvoked() { var fizzbuzzer = ScenarioContext .Current.Get<FizzBuzzer>("FizzBuzzer"); var value = ScenarioContext .Current.Get<int>("value"); ScenarioContext .Current.Add( "actual", fizzbuzzer.Translate(value)); }
Then the result should be '2' [Then(@"the result should be '(.*)'")] public void ThenTheResultShouldBe( string expected) { var actual = ScenarioContext .Current.Get<string>("actual"); Assert.That(expected, Is.EqualTo(actual)); }
Scenario: 2 must translate to the string 2 Given a FizzBuzzer And the value of '2' When translate is invoked Then the result should be '2'
Scenario: 5 must translate to the string Buzz Given a FizzBuzzer And the value of '5' When translate is invoked Then the result should be 'Buzz'
Given a FizzBuzzer !
[Given(@"a FizzBuzzer")] public void GivenAFizzBuzzer() { ScenarioContext.Current.Add( "FizzBuzzer", new FizzBuzzer()); }
And the value of '5' !
[Given(@"the value of '(.*)'")] public void GivenTheValueOf(int value) { ScenarioContext.Current.Add("value", value); }
When translate is invoked [When(@"translate is invoked")] public void WhenTranslateIsInvoked() { var fizzbuzzer = ScenarioContext .Current.Get<FizzBuzzer>("FizzBuzzer"); var value = ScenarioContext .Current.Get<int>("value"); ScenarioContext .Current.Add( "actual", fizzbuzzer.Translate(value)); }
Then the result should be 'Buzz' [Then(@"the result should be '(.*)'")] public void ThenTheResultShouldBe( string expected) { var actual = ScenarioContext .Current.Get<string>("actual"); Assert.That(expected, Is.EqualTo(actual)); }
Scenario: 2 must translate to the string 2 Given a FizzBuzzer And the value of '2' When translate is invoked Then the result should be '2'
[NUnit.Framework.TestAttribute()] [NUnit.Framework.DescriptionAttribute("2 must translate to the string 2")] public virtual void _2MustTranslateToTheString2() { TechTalk.SpecFlow.ScenarioInfo scenarioInfo = new TechTalk.SpecFlow.ScenarioInfo( "2 must translate to the string 2", ((string[])(null))); #line 6 this.ScenarioSetup(scenarioInfo); #line 7 testRunner.Given("a FizzBuzzer", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Given "); #line 8 testRunner.And("the value of \'2\'", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "And "); #line 9 testRunner.When("translate is invoked", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "When "); #line 10 testRunner.Then("the result should be \'2\'", ((string)(null)), ((TechTalk.SpecFlow.Table)(null)), "Then "); #line hidden this.ScenarioCleanup(); }
G O O D , B U T N O T G R E AT.
Examples: | value | expected value | | 2 | 2 | | 4 | 4 | | 3 | Fizz | | 9 | Fizz | | 5 | Buzz | | 25 | Buzz | | 15 | FizzBuzz | | 30 | FizzBuzz |
Scenario Outline: FizzBuzz translation Given a FizzBuzzer And the value of '<value>' When translate is invoked Then the result should be '<expected value>' Examples: | value | expected value | | 2 | 2 | | 4 | 4 | | 3 | Fizz | | 9 | Fizz | | 5 | Buzz | | 25 | Buzz | | 15 | FizzBuzz | | 30 | FizzBuzz |
[Given(@"a FizzBuzzer")] public void GivenAFizzBuzzer() { ScenarioContext.Current.Add( "FizzBuzzer", new FizzBuzzer()); }
[Given(@"the value of '(.*)'")] public void GivenTheValueOf(int value) { ScenarioContext.Current.Add("value", value); }
[When(@"translate is invoked")] public void WhenTranslateIsInvoked() { var fizzbuzzer = ScenarioContext .Current.Get<FizzBuzzer>("FizzBuzzer"); var value = ScenarioContext .Current.Get<int>("value"); ScenarioContext .Current.Add( "actual", fizzbuzzer.Translate(value)); }
[Then(@"the result should be '(.*)'")] public void ThenTheResultShouldBe( string expected) { var actual = ScenarioContext .Current.Get<string>("actual"); Assert.That(expected, Is.EqualTo(actual)); }
Examples: | value | expected value | | 2 | 2 | | 4 | 4 | | 3 | Fizz | | 9 | Fizz | | 5 | Buzz | | 25 | Buzz | | 15 | FizzBuzz | | 30 | FizzBuzz |
[TestCase( 2, Result = "2")] [TestCase( 3, Result = "Fizz")] [TestCase( 4, Result = "4")] [TestCase( 5, Result = "Buzz")] [TestCase( 6, Result = "Fizz")] [TestCase(10, Result = "Buzz")] [TestCase(15, Result = "FizzBuzz")] [TestCase(45, Result = "FizzBuzz")] public string FizzBuzz_Tests(int value) { return fizzbuzzer.Translate(value); }
Examples: | value | expected value | | 2 | 2 | | 4 | 4 | | 3 | Fizz | | 9 | Fizz | | 5 | Buzz | | 25 | Buzz | | 15 | FizzBuzz | | 30 | FizzBuzz |
G R E AT.
"FizzBuzzSpec\FizzBuzzSpec\FizzBuzzerSpecFeature\_15MustTranslateToTheStringFizzBuzz","Passed","00:00:00.183","(local)" Given a FizzBuzzer -> done: FizzBuzzerSpecSteps.GivenAFizzBuzzer() (0.0s) And the value of '15' -> done: FizzBuzzerSpecSteps.GivenTheValueOf(15) (0.0s) When translate is invoked -> done: FizzBuzzerSpecSteps.WhenTranslateIsInvoked() (0.0s) Then the result should be 'FizzBuzz' -> done: FizzBuzzerSpecSteps.ThenTheResultShouldBe("FizzBuzz") (0.0s)
D O E S T H I S R E A L LY W O R K ? ! ?
W E C O U L D H AV E T H E S Y S T E M G E N E R AT E D ATA F O R U S
T O I L L U S T R AT E T H E A D H E R E N C E O F P R O P E R T I E S
M A K E I T S O .
Testing of Properties
is testing ofproperties
fscheck
T E S T I N G P R O P E R T I E S
T H E N U N I T W AY
[Test] public void Divisible_By_3_Is_Fizz( [Range(3, 300, 3)] int value) { var removeBuzz = (value % 5 == 0) ? 3 : value; Assert.That(FizzBuzz(removeBuzz), Is.EqualTo("Fizz")); }
[Test] public void Divisible_By_5_Is_Buzz( [Random(1, 10000, 100)] int value) { var removeFizz = (value % 3 == 0) ? 5 : value * 5; Assert.That(FizzBuzz(removeFizz), Is.EqualTo("Buzz")); }
F S C H E C K W AY
[TestCase] public void Number_Divisible_By_3_Will_Return_Fizz() { Spec.ForAny<int>(value => _fizzBuzzer.Translate(value).Equals("Fizz")) .When(value => value % 3 == 0 && value % 5 != 0) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
[TestCase] public void Number_Divisible_By_5_Will_Return_Buzz() { Spec.ForAny<int>(value => _fizzBuzzer.Translate(value).Equals("Buzz")) .When(value => value % 3 != 0 && value % 5 == 0) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
[TestCase] public void Number_Divisible_By_15_Returns_FizzBuzz() { Spec.ForAny<int>(value => _fizzBuzzer.Translate(value).Equals("FizzBuzz")) .Or(value => value % 15 != 0) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
H M M M M
W E C O U L D C R E AT E A M O D E L T O D E M O N S T R AT E T H E
R O B U S T N E S S O F T H E S O L U T I O N
M A K E I T S O .
T E S T I N G A G A I N S T A M O D E L
F S C H E C K W AY
class FizzBuzzModel { public static string Translate(int value) { if (value % 15 == 0) return "FizzBuzz"; if (value % 3 == 0) return "Fizz"; if (value % 5 == 0) return "Buzz"; return value.ToString(CultureInfo.InvariantCulture); } } [TestCase] public void Will_Match_Model() { Spec.ForAny<int>(value => FizzBuzzModel.Translate(value).Equals(_fizzBuzzer.Translate(value))) .QuickCheckThrowOnFailure(); }
class FizzBuzzModel { public static string Translate(int value) { if (value % 15 == 0) return "FizzBuzz"; if (value % 3 == 0) return "Fizz"; if (value % 5 == 0) return "Buzz"; return value.ToString(CultureInfo.InvariantCulture); } }
[TestCase] public void Will_Match_Model() { Spec.ForAny<int>(value => FizzBuzzModel.Translate(value) .Equals(_fizzBuzzer.Translate(value))) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
H M M M M
W E C O U L D U S E A S Y S T E M T O G E N E R AT E
D E G E N E R AT I V E T E S T D ATA T O D I S P L AY C O R R E C T N E S S
M A K E I T S O .
T E S T I N G N E G AT I O N
F S C H E C K W AY
[TestCase] public void Numbers_Not_Divisible_By_3_Do_Not_Fizz() { var noThrees = from number in Any.OfType<int>() where number%3 != 0 select number; Spec.For(noThrees, x => _fizzBuzzer.Translate(x) .Contains("Fizz") == false) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
[TestCase] public void Numbers_Not_Divisible_By_5_Do_Not_Buzz() { var noFives = from number in Any.OfType<int>() where number%5 != 0 select number; Spec.For(noFives, x => _fizzBuzzer.Translate(x) .Contains("Buzz") == false) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
H M M M M
[TestCase] public void Numbers_Divisible_By_15_Must_FizzBuzz() { var onlyFifteens = from number in Any.OfType<int>() where number%15 == 0 select number; Spec.For(onlyFifteens, x => _fizzBuzzer.Translate(x) .Equals("FizzBuzz")) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
H M M M M
[TestCase] public void Classify_Values() { Spec.ForAny<int>(x => true) .Classify( x => _fizzBuzzer.Translate(x).Equals("Fizz"), "Fizz") .Classify( x => _fizzBuzzer.Translate(x).Equals("Buzz"), "Buzz") .Classify(x => _fizzBuzzer.Translate(x).Equals("FizzBuzz"), "FizzBuzz") .Classify(x => _fizzBuzzer.Translate(x) != string.Empty, "Other") .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests. 55% Other. 22% Fizz, Other. 13% FizzBuzz, Other. 9% Buzz, Other.
H M M M M
[TestCase] public void Collect_Values_For_Negatives() { Spec.ForAny<int>(x => true) .When(x => x < 0) .Collect(x => _fizzBuzzer.Translate(x)) .QuickCheckThrowOnFailure(); }
Ok, passed 100 tests.
G R E AT !
T H A N K Y O U
M I K E H A R R I S @ M I K E M K H H T T P : / / C O M P - P H I L . B L O G S P O T. C O M /
