Embed Size (px)
Citation preview
PARALLEL & ASYNCPROCESSING USING TPL DATAFLOW
Petru Rebeja
AGENDA
• What is Dataflow?
• When to use it?
• How to use it?
• Q&A
THE BIG PICTURE
CLR Thread Pool
Tasks
PLINQ Parallel Loops
Concurrent Collections
Dataflow
DATAFLOW BENEFITS
• Effortless use of multi-threading
• Performance boost via painless optimization
• Development focus is on the ‘what’ rather than ‘how’
DATAFLOW USAGES
High throughput, low-latency scenarios
Robotics
Manufacturing
Imaging Biology
Oil & Gas
Finance
PROGRAMMING MODEL
• Actor-based programming
• In-process message passing
• Components (blocks) for creating data processing pipelines
ARCHITECTURE
IDataflowBlock
ISourceBlock<TOutput> ITargetBlock<TInput>
IPropagatorBlock<Tinput,Toutput>
COMPOSITION
Source
Target
Propagator
OptionalTransform
BUFFERING BLOCKS
BufferBlock<T>
BroadcastBlock<T>
WriteOnceBlock<T>
EXECUTION BLOCKS
ActionBlock<T>
TransformBlock<T,V>
TransformManyBlock<T,V>
GROUPING BLOCKS
BatchBlock<T>
JoinBlock<T1,T2,…>
BatchedJoinBlock<T1,T2>
BEHAVIOR CONFIGURATION OPTIONS
• BufferBlock<T>
• BroadcastBlock<T>
• WriteOnceBlock<T>
DataflowBlockOptions
• ActionBlock<T>
• TransformBlock<TIn, TOut>
• TransformManyBlock<TIn, TOut>
ExecutionDataflowBlockOptions
• BatchBlock<T>
• JoinBlock<T1, T2[, T3]>
• BatchedJoinBlock<T1, T2>
GroupingDataflowBlockOptions
COMPLETION & CANCELLATION
• To know when a block completes await block.Completion
or add a continuation task to it
• To propagate completion from source to target, set
DataflowLinkOptions.PropagateCompletion when
linking
• Set DataflowBlockOptions.CancellationToken to
enable cancellation
ERROR HANDLING
• If the exception does not affect the integrity of the
pipeline – use a try/catch inside the block
• Otherwise, handle errors outside of the pipeline by
• Adding a continuation to block.Completion
• Propagating errors through the pipeline
DEALING WITH CONCURRENCY
• Rule of thumb: avoid shared state whenever possible.
• Use ConcurrentExclusiveSchedulerPair to perform
updates on shared state
• Be aware of the caveats with
ConcurrentExclusiveSchedulerPair
CREATING CUSTOM BLOCKS
The easy way:
DataflowBlock.Encapsulate<TInput, TOutput>(
target, source)
CREATING CUSTOM BLOCKS
The hard(core) way:
class CustomBlock:
IPropagatorBlock<TInput, TOutput>
{
}
CREATING CUSTOM BLOCKS
Either way you choose, don’t forget to:
• Propagate completion
• Pool for cancellation
REFERENCES & FURTHER READING
Dataflow (Task Parallel Library) http://msdn.microsoft.com/en-us/library/hh228603(v=vs.110).aspx
Stephen Toub
TPL Dataflow Tourhttp://channel9.msdn.com/posts/TPL-Dataflow-Tour
Joseph Albahari
The Future of .NET Parallel Programming
http://channel9.msdn.com/events/TechEd/Australia/Tech-Ed-Australia-2011/DEV308
Stephen Toub
Inside TPL Dataflowhttp://channel9.msdn.com/Shows/Going+Deep/Stephen-Toub-Inside-TPL-Dataflow
Alexey Kursov
Pipeline TPL Dataflow Usage exampleshttps://www.youtube.com/watch?v=AI9KxgDF43khttps://www.youtube.com/watch?v=AI9KxgDF43k
Richard Blewett, Andrew Clymer
Pro Asynchronous Programming with .NET
APRESS 2013ISBN: 978-1430259206
AKKA.NET http://getakka.net/
QUESTIONS?
