Embed Size (px)
Citation preview
Dror Helper
[email protected] | @dhelper | http://blog.drorhelper.com
Unit testing patterns for concurrent code
Examples: https://github.com/dhelper/ConcurrentUnitTesting
• Senior consultant @CodeValue
• Developing software (professionally) since 2002
• Mocking code since 2008
• Test Driven Developer
• Blogger: http://blog.drorhelper.com
About.ME
The free lunch is over!
• Multi-core CPUs are the new standard
• New(er) language constructs
• New(ish) languages
We live in a concurrent world!
[Test]
public void AddTest()
{
var cut = new Calculator();
var result = cut.Add(2, 3);
Assert.AreEqual(5, result);
}
Meanwhile in the unit testing “world”
Several actions at the same time
Hard to follow code path
Non deterministic execution
The dark art of concurrent code
• Trustworthy
– Consistent results
– Only fail due to bug or requirement change
• Maintainable
– Robust
– Easy to refactor
– Simple to update
• Readable
Good unit tests must be:
× Inconsistent results
× Untraceable fail
× Long running tests
× Test freeze
Concurrency test “smells”
public void Start()
{
_worker = new Thread(() => {
while (_isAlive) {
Thread.Sleep(1000);
var msg = _messageProvider.GetNextMessage();
//Do stuff
LastMessage = msg;
}
});
_worker.Start();
}
How can we test this method
[TestMethod]
public void ArrivingMessagePublishedTest()
{
var fakeMessageProvider = A.Fake<IMessageProvider>();
A.CallTo(() => fakeMessageProvider.GetNextMessage()).Returns("Hello!");
var server = new Server(fakeMessageProvider);
server.Start();
Thread.Sleep(2000);
Assert.AreEqual("Hello!", server.LastMessage);
}
Testing start take #1
× Time based - fail/pass inconsistently
× Test runs for too long
× Hard to investigate failures
Test smell - “Sleep” in test
“In concurrent programming if something can happen, then sooner or later it will, probably at the most inconvenient moment”Paul Butcher – Seven concurrency models in seven weeks
[TestMethod]
public async Task ArrivingMessagePublishedTest()
{
var fakeMessageProvider = A.Fake<IMessageProvider>();
A.CallTo(() => fakeMessageProvider.GetNextMessage()).Returns("Hello!");
var server = new Server(fakeMessageProvider);
server.Start();
await Task.Delay(2000);
Assert.AreEqual("Hello!", server.LastMessage);
}
Testing start take #2
Solution: avoid concurrent code!
We extract all the logic from the hard-to-test component into a component that is testable via synchronous tests. http://xunitpatterns.com/Humble%20Object.html
Pattern #1 – humble object pattern
Code under Test
Start
Humble object
Async
Perform action
Perform Action
Assert Result
Production Code
public void Start()
{
_worker = new Thread(() => {
while (_isAlive) {
Thread.Sleep(1000);
var msg = _messageProvider.GetNextMessage();
//Do stuff
LastMessage = msg;
}
});
_worker.Start();
}
public void Start()
{
_worker = new Thread(() => {
while (_isAlive) {
Thread.Sleep(1000);
_messageHandler.HandleNextMessage();
}
});
_worker.Start();
}
[TestMethod]
public void ArrivingMessagePublishedTest()
{
var fakeMessageProvider = A.Fake<IMessageProvider>();
A.CallTo(() => fakeMessageProvider.GetNextMessage()).Returns("Hello!");
var messageHandler = new MessageHandler(fakeMessageProvider);
messageHandler.HandleNextMessage();
Assert.AreEqual("Hello!", messageHandler.LastMessage);
}
And finally – the test
public class MessageManager
{
private IMesseageQueue _messeageQueue;
public void CreateMessage(string message)
{
// Here Be Code!
_messeageQueue.Enqueue(message);
}
}
public class MessageClient
{
private IMesseageQueue _messeageQueue;
public string LastMessage { get; set; }
private void OnMessage(object sender, EventArgs e)
{
// Here Be Code!
LastMessage = e.Message;
}
}
Concurrency as part of program flow
Test before – Test After
Start
Code under test
Async
Logic2
Assert results
Logic1
Start Logic1 Assert ResultsFake
Start Logic2 Assert ResultsFake
[TestMethod]
public void AddNewMessageProcessedMessageInQueue()
{
var messeageQueue = new AsyncMesseageQueue();
var manager = new MessageManager(messeageQueue);
manager.CreateNewMessage("a new message");
Assert.AreEqual(1, messeageQueue.Count);
}
Testing Flow part 1
[TestMethod]
public void QueueRaisedNewMessageEventClientProcessEvent()
{
var messeageQueue = new AsyncMesseageQueue();
var client = new MessageClient(messeageQueue);
client.OnMessage(null, new MessageEventArgs("A new message"));
Assert.AreEqual("A new message", client.LastMessage);
}
Testing Flow part 2
The best possible solution
No concurrency == no problems
× Do not test some of the code
× Not applicable in every scenario
Avoid concurrency Patterns
public class ClassWithTimer
{
private Timer _timer;
public ClassWithTimer(Timer timer)
{
_timer = timer;
_timer.Elapsed += OnTimerElapsed;
_timer.Start();
}
private void OnTimerElapsed(object sender, ElapsedEventArgs e)
{
SomethingImportantHappened = true;
}
public bool SomethingImportantHappened { get; private set; }
}
How can we test this class?
[TestMethod]
public void ThisIsABadTest()
{
var timer = new Timer(1);
var cut = new ClassWithTimer(timer);
Thread.Sleep(100);
Assert.IsTrue(cut.SomethingImportantHappened);
}
Not a good idea
Also seen with a very small number (or zero)
Usually done when need to wait for next tick/timeout
× Time based == fragile/inconsistent test
× Hard to investigate failures
× Usually comes with Thread.Sleep
Set timeout/interval to 1
Fake & Sync
Test
Code under Test
Fake
Logic
Assert Results
[TestMethod, Isolated]
public void ThisIsAGoodTest()
{
var fakeTimer = Isolate.Fake.Instance<Timer>();
var cut = new ClassWithTimer(fakeTimer);
var fakeEventArgs = Isolate.Fake.Instance<ElapsedEventArgs>();
Isolate.Invoke.Event(
() => fakeTimer.Elapsed += null, this, fakeEventArgs);
Assert.IsTrue(cut.SomethingImportantHappened);
}
Using Typemock Isolator to fake Timer
• Mocking tool limitation (example: inheritance based)
• Programming language attributes
• Special cases (example: MSCorlib)
Solution – wrap the unfakeable
× Problem – it requires code change
Unfortunately not everything can be faked
public class ThreadPoolWrapper
{
public static void QueueUserWorkItem(WaitCallback callback)
{
ThreadPool.QueueUserWorkItem(callback);
}
}
Wrapping Threadpool
public class ClassWithWrappedThreadpool
{
public void RunInThread()
{
ThreadPool.QueueUserWorkItem(_ =>
{
SomethingImportantHappened = true;
});
}
public bool SomethingImportantHappened { get; private set; }
}
Wrapping Threadpool
ThreadPoolWrapper.QueueUserWorkItem(_ =>
[TestClass]
public class WorkingWithThreadpool
{
[TestMethod, Isolated]
public void UsingWrapperTest()
{
Isolate.WhenCalled(() => ThreadPoolWrapper.QueueUserWorkItem(null))
.DoInstead(ctx => ((WaitCallback)ctx.Parameters[0]).Invoke(null));
var cut = new ClassWithWrappedThreadpool();
cut.RunInThread();
Assert.IsTrue(cut.SomethingImportantHappened);
}
}
Testing with new ThreadpoolWrapper
How can we test that an
asynchronous operation
never happens?
Test Code Under Test
Is Test
Async
Deterministic Assert Results
Run in sync
Another day – another class to testpublic void Start()
{
_cancellationTokenSource = new CancellationTokenSource();
Task.Run(() =>
{
var message = _messageBus.GetNextMessage();
if(message == null)
return;
// Do work
if (OnNewMessage != null)
{
OnNewMessage(this, EventArgs.Empty);
}
}, _cancellationTokenSource.Token);
}
Switching code to “test mode”
• Dependency injection
• Preprocessor directives
• Pass delegate
• Other
• Fake & Sync
• Async code sync test
Run in single thread patterns
When you have to run a concurrent test in a predictable way
Synchronized run patterns
The Signaled pattern
Start
Code under test
Run
Fake object
Signal
Call
Wait Assert result
public void DiffcultCalcAsync(int a, int b)
{
Task.Run(() =>
{
Result = a + b;
_otherClass.DoSomething(Result);
});
}
Using the Signaled pattern
[TestMethod]
public void TestUsingSignal()
{
var waitHandle = new ManualResetEventSlim(false);
var fakeOtherClass = A.Fake<IOtherClass>();
A.CallTo(() => fakeOtherClass.DoSomething(A<int>._)).Invokes(waitHandle.Set);
var cut = new ClassWithAsyncOperation(fakeOtherClass);
cut.DiffcultCalcAsync(2, 3);
var wasCalled = waitHandle.Wait(10000);
Assert.IsTrue(wasCalled, "OtherClass.DoSomething was never called");
Assert.AreEqual(5, cut.Result);
}
Using the Signaled pattern
Busy Assert
Start
Code under test
Run
Assert or Timeout
[TestMethod]
public void DifficultCalculationTest()
{
var cut = new ClassWithAsyncOperation();
cut.RunAsync(2, 3);
AssertHelper.BusyAssert(() => cut.Result == 5, 50, 100, "fail!");
}
Busy assertion
× Harder to investigate failures
× Cannot test that a call was not made
Test runs for too long but only when it fails
Use if other patterns are not applicable
Synchronized test patterns
Because concurrency (and async) needs to be tested as well
Unit testing for concurrency issues
Test for async
Start Method under test
Fake object
wait
Assert results
Test for Deadlock
Start
Code under test
Execute
ThreadCall
Thread
Assert resultWaitWait
[TestMethod, Timeout(5000)]
public void CheckForDeadlock()
{
var fakeDependency1 = A.Fake<IDependency>();
var fakeDependency2 = A.Fake<IDependency>();
var waitHandle = new ManualResetEvent(false);
var cut = new ClassWithDeadlock(fakeDependency1, fakeDependency2);
A.CallTo(() => fakeDependency1.Call()).Invokes(() => waitHandle.WaitOne());
A.CallTo(() => fakeDependency2.Call()).Invokes(() => waitHandle.WaitOne());
var t1 = RunInOtherThread(() => cut.Call1Then2());
var t2 = RunInOtherThread(() => cut.Call2Then1());
waitHandle.Set();
t1.Join();
t2.Join();
}
T1
T2
L1
L2
Avoid Concurrency
Humble object
Test before –test after
Run in single thread
Fake & Sync
Async in production -sync in test
Synchronize test
The Signaled pattern
Busy assertion
Concurrency tests
Test for Deadlock
Test for non blocking
Concurrent unit testing patterns
It is possible to test concurrent code
Avoid concurrency
Run in single thread
Synchronize test
Conclusion
Dror Helper
C: 972.50.7668543
B: blog.drorhelper.com
w: www.codevalue.net
