Upload
shohre-mansouri
View
111
Download
0
Embed Size (px)
Citation preview
Stories from the trenchesIt’s just a bloody one way synchronisation between two systems. How hard can it be?
Melbourne Alt.NetMercury Project
23 February 2016
Shohre Mansouri1
how the battle started
2
3
First Attack
4
Exsiting wcf methods for useShipmentId[] GetUpdatedShipments(UpdateAtFrom, UpdateAtTo)
Shipment GetShipment(ShipmentId)
5
Use “UpdateAt” as means of getting the updates
Every 2 minutes:GetUpdatedShipmentsCommand(From, DateTime.UtcNow)Handler sends: GetShipmentCommand(ShipmentId)
ShipmentId UpdateAt
Shipment1 12:00:01
Shipment2 12:00:02
Shipment3 12:00:02
Shipment4 12:00:03
Shipment5 12:00:04
6
Use “UpdateAt” as means of getting the updates
Every 2 minutes:GetUpdatedShipmentsCommand(From, DateTime.UtcNow)Every 1 minute:GetUpdatedShipmentsFromDatabaseCommand() -> GetShipment(ShipmentId)
ShipmentId UpdateAt
Shipment1 12:00:01
Shipment1 12:00:02
Shipment2 12:00:02
Shipment1 12:00:03
Shipment1 12:00:04
7
No Unbounded sets
Every 2 minutes:GetUpdatedShipmentsFromWcfCommand(From, DateTime.UtcNow, PageSize)Every 1 minute:GetUpdatedShipmentsFromDatabaseCommand(PageSize)
ShipmentId UpdateAt
Shipment1 12:00:01
Shipment1 12:00:02
Shipment2 12:00:02
Shipment1 12:00:03
Shipment1 12:00:04
8
Sample
UpdatedAt => From && UpdatedAt < To
ShipmentId UpdateAt
1 Shipment3 12:00:01 x2 Shipment1 12:00:02 x x3 Shipment2 12:00:02 x x4 Shipment1 12:06:04 X
5 Shipment2 12:06:06 X X
6 Shipment3 12:08:00 X
From To (Now) PageSize
12:00:00 12:02:00 3
12:00:02 12:04:00 3
12:04:00 12:06:00 3
12:06:00 12:08:00 3
12:08:00 12:10:00 3
12:10:00 12:12:00 3
9
Sample ShipmentId UpdateAt
1 Shipment3 12:00:01 x x x X2 Shipment1 12:00:01 x x x X3 Shipment2 12:00:01 x x x X4 Shipment1 12:00:01
5 Shipment2 12:00:01
6 Shipment3 12:06:06
From To (Now) PageSize
12:00:00 12:02:00 3
12:00:01 12:04:00 3
12:00:01 12:06:00 3
12:00:01 12:08:00 3
12:00:01 12:10:00 3
UpdatedAt > From && UpdatedAt < To
10
There was a bug!
11
This other WCF methodGetUpdatedShipments(FromVersion, PageSize)
Version is a sequential integer assigned to each update!
12
Use Version as means of getting the updates
Every 2 minutes:GetUpdatedShipmentsCommand(StartVersion, PageSize)Every 1 minute:GetUpdateShipmentFromDatabaseCommand(PageSize)
ShipmentId Version
Shipment1 100
Shipment1 104
Shipment2 106
Shipment1 111
Shipment1 112
13
Shipments have filesConsistency of the shipments and document!
Saga: For handling long-running business processes
“NServiceBus employs event-driven architectural principles to bake fault-tolerance and scalability into these processes.”
14
Added Blob storage And saga
15
DTC
is o
n!
Shipments have documentspublic class GetShipmentSagaData : IContainSagaData{
public virtual Guid Id { get; set; }public virtual Guid UpdatedShipmentCorrelationId { get; set; }public virtual string Originator { get; set; }public virtual string OriginalMessageId { get; set; }public virtual string ShipmentId { get; set; }
// The list of all documents that need to be uploaded:public virtual IList<DocumentKey> ShipmentDocumentKeys { get; set; }
// The documents that have been uploaded (successfully, or unsucessfully)public virtual IList<CompletedShipmentDocument> CompletedDocumentsOnSaga { get; set; }
public virtual ShipmentSagaData ShipmentData { get; set; }
}16
Keeping up-to-Date Vs InitialisingGetUpdatedShipmentsFromWcfCommand(version: 110, pageSize: 10)
GetUpdatedShipmentsFromWcfCommand(130, 10)
GetUpdatedShipmentsFromWcfCommand(146, 10)
GetUpdatedShipmentsFromWcfCommand(250, 10)
GetUpdatedShipmentsFromWcfCommand(295, 10)
GetUpdatedShipmentsFromWcfCommand(300, 10)
2 minutes later: GetUpdatedShipmentsFromWcfCommand(300, 10)
4 minutes later: GetUpdatedShipmentsFromWcfCommand(300, 10) 17
Keeping up-to-Date Vs Initialising
New -> Polling -> Complete and persist the next New
Version Polled Status Timestamp
100 Complete 4 minutes ago
112 Complete 2 minutes ago
112 New 1 second ago
18
Messages were lost!Azure service bus is not
Distributed transactional!
19
Stuck polling sessionsVersion Polled Status Timestamp
100 Complete Last night - 4 minutes
112 Complete Last night - 2 minutes
112 Polling Last night
20
OutBoxAdd another graph here!
21
NService Bus OutBoxNow we need an Outbox!
22
Code of outboxBusConfiguration busConfiguration = new BusConfiguration();
busConfiguration.EnableOutbox();
<appSettings> <add key="NServiceBus/Outbox" value="true" /></appSettings>
23
OutBox CaveatsBoth the business data and deduplication data need to share the same database.
So we moved NSB Saga to SQL server!
24
Finaly!
25
DTC
is o
ff!
Stuck polling sessionsThis time with messages in the error queue...
Version Polled Status Timestamp
100 Complete Last night - 4 minutes
112 Complete Last night - 2 minutes
112 Polling Last night
26
Got rid of statusVersion Polled Timestamp
100 4 minutes ago
112 2 minutes ago
112 Now - 1 second
27
We lost the battle but won the war
28
Only one thing is leftInitialising!
But I know it is going to be bloody easy.
29
Questions?
30