Click here to load reader

Scaling NServiceBus NSBCon NY by Jimmy Bogard

  • View

  • Download

Embed Size (px)


The Daily AS/400 Dump Processing bulk data

Text of Scaling NServiceBus NSBCon NY by Jimmy Bogard

2. The Daily AS/400DumpProcessing bulk data 3. Characteristics 1/day dump of entire SKUcatalog Flat file 1.4M rows daily and growing Fill in details for supportwebsite 4. Consuming FilesDataImporter Deadlock DBRollbackWhat made it through? 5. Consuming FilesDataParserProcessorDBCommitCommit1. ProcessFile2. ProcessLine 6. Streaming Bulk Copy 7. Streaming Bulk Copy 8. Data WarehouseExportsExporting Bulk Data 9. Producing FilesId Name Changed1 Walter Y2 Cora N3 Alejandro N4 Carlton Y5 Janice NFileExporterSELECT NameWHERE Changed = YKiera YUPDATE PeopleSET Changed = NWere missing a change 10. Producing FilesId Name BatchId1 Walter NULL2 Cora 13 Alejandro 14 Carlton NULL5 Janice 2FileExporterUPDATE PeopleSET BatchId = 3WHERE BatchId IS NULL33SELECT Id, NameFROM People= 3Kiera NULL1. Mark records toexport2. Export marked records 11. Search DatabaseEnrichmentRouting slips 12. Characteristics Enriching search data from varioussources Some sources are required, some not Some dependent on previous steps Initiated with large batch ofinitial messages 13. Option #1: ObserverPublisherSagaPriceCatalogContentOptionsSaga Creation 14. Option #1: ObserverPublisherSagaPriceCatalogContentOptionsQueue Contention 15. Option #1: ObserverPublisherSagaPriceCatalogContentOptionsSaga ContentionPriceCatalogContentOptions 16. Option #2: ControllerPublisherSagaPriceCatalogContentOptionsSlooooooow 17. Option #2: ControllerPublisherSagaPriceCatalogContentOptionsComplicated and slowPriceCatalogContentOptions 18. McDonalds SagasSandwich Drinks Coffee ShakesRegister1) Place OrderCustomerFries2) Order Accepted3) Item CompletedOrder #14) Order CompletedTray Contention 19. Subways SagasRegister3) Pack Meal1) Place OrderCustomer2) Prep MealPrepPack4) Order Completed 20. Routing SlipHow do we route a message consecutively through aseries of processing steps when the sequence of stepsis not known at design-time and may vary for eachmessage?Attach a Routing Slip to each message, specifying thesequence of processing steps. Wrap each component witha special message router that reads the RoutingSlip and routes the message to the next component inthe list. 21. Option #3: Routing SlipInitiatorPriceCatalogContentOptions 22. TransactionProcessingDitching DTC and exactly-oncemessaging 23. Characteristics Processing daily sales transactionsfor loyalty reward system Highly cyclical load Longest observed finish time 10hours New load 4-5X larger on a daily basis 24. Measuring performance Current rate: 5 msgs/sec Target rate: 80-90 msgs/sec Measured max: 18 msgs/sec 25. Hardware Upgrade SSD All The Things Up the cores Up the RAM Upgrade the network Result: doubled throughput 26. Theoretical maxesTest Transactions Elapsed Time ThroughputNo-op On 75 sec 133/secDB On 378 sec 26/secDB + Message On 544 sec 18/secNo-op Off 29 sec 345/secDB Off 51 sec 196/secDB + Message Off 109 sec 92/sec 27. 2PC in messagingMyServiceDB2. Save1. Receive3. Send 28. Idempotent handlersTxnServiceDB2. Already recorded?3. Record txn1. Store Txn 29. Record intentTxnServiceDB2. Already recorded?3. Record txn4. Issue coupon1. Store Txn 30. Decoupling downstreameffectsCouponServiceDB2. Send1. Find unsent message3. Mark as sent 31. Caveats No first-level retries No second-level retries Pipeline fixes all these things 32. Result 8 hours -> 10 minutes 180 msgs/sec 33. Takeaways Embrace real-world metaphors Optimize when needed, not before Tons of material out there Clemens Vasters More features,more cost.