Building occasionally connected applications using event sourcing

© 2014 Aviva Solutions 10 april 2023

Occassionally Connected ApplicationsWith Event Sourcing

Dennis Doomen

[email protected]

http://caliburnmicro.codeplex.com/

© 2014 Aviva Solutions 10 april 2023Dennis Doomen

About Me• Principal Consultant• 17 years in IT• C++ origins but C# since 2001• Specialties

• .NET• Architecture• Scrum/Kanban/XP• ALM

• Speaker• Public initiatives

• C# Coding Guidelines• Fluent Assertions

• Internet• www.dennisdoomen.net• DZone MVB• @ddoomen

http://csharpguidelines.codeplex.com/

http://fluentassertions.codeplex.com/



http://www.dennisdoomen.net/

http://agile.dzone.com/users/dennisdoomen

http://agile.dzone.com/users/dennisdoomen

http://twitter.com/

http://twitter.com/


Contents• What we tried to solve• Our options• What we choose for• Design Challenges• Scalability Challenges• Architectural Challenges• Concurrency Challenges• Future?


What we tried to solve


Central Farm

Seaborn Servers

Decentralized Servers

Mobile Clients

Intermittent Satellite

Connection

High-Latency

LAN

3G (w/o attachments)

WLAN (attachments)

Only synchronize what is relevant!

>100

XCopy RDBMS


What we had

Command Service

Commands

Domain Model

Repositories

Domain Services

SQL / Oracle

Service Agents

Web Application

Backoffice Systems

Command HandlersQuery Processor

Repositories

Projections

Query Handlers

Patterns & Principles• Domain Driven Design• CQRS• Repository Abstraction• TDD/BDD• SOLID

Framework & Libraries• NHibernate• Autofac• ASP.NET WebForms / MVC• SQLite

Queries


Our options1. SQL Server Replication2. Command Synchronization3. Aggregate Serialization4. RavenDB 5. Event Sourcing


1. SQL Server Replication

Automatic reconnectsProven technology

Automatic schema updatesDevelopment can be outsourced

Synchronization over HTTP

Only SQL Server (Express)Filtering is limited

Affects client infrastructureNo real-world examplesUnsure about limitationsRequires SQL expertize.

Difficult to diagnose issues during development


2. Command Synchronization

Supports any databaseNo special deployments requirements

Potentially unresolvable out-of-sync issue

Requires bypassing business rulesCommand != actual changeLimited filtering possibilities

Custom serializationSeparate queue of commands


3. Aggregate Serialization

Supports any databaseNo special deployments requirementsDoesn’t interfere with business rules

No out-of-sync issues

Requires specialized code per aggregate

Needs an enforced order of synchronization.

Payload is not optimizedRequires a lot of querying and change

tracking


4. RavenDB

Reduces complexitySmart reconnects

Synchronization built-inArchiving built-inBuilt-in cachingVery scalable

Improves productivityImproves query performance

Lucene based

Acceptance by enterprise clientsNew querying (map/reduce) concepts

Limited track record


Web Application

Command Service

Query Processor

ChangeUserEmailHandler

User Aggregate

Domain UOW

UserProjector

Projections UOW

Query Handler

Read DB Write DB

Submit ChangeUserEmailCommandExecute query

Get<TAggregateRoot>(key, version)

LINQ, HQL, SQL

Commits

Invoke method

DAO

Load(events)

Apply

Get changes

Dispatcher Commit

Handle(UserEmailChangedEvent)

Submit changes

Event Sourcing


5. Event Sourcing

Transparent to clientsWorks with any SQL/No-SQL DB

Can improve performanceAdvance conflict resolution

Natural unit of synchronizationCan be very scalable

Lots of new conceptsBig impact on domain model

Synchronization must be custom-builtLots of unforeseen challenges


What we chose for


The Revised Architecture

Command Service

Commands

Domain Model

Domain Repository

Domain Services

Event Store

Service Agents

Web Application

Backoffice Systems

Command Handlers

Query Store

Query Processor

Projection Repositories

Queries

Projectors

Events

DispatcherEvents

Projections

Query Handler


Advantages• Optimized projections• Projections = cache -> rebuild anytime• Reporting node• Intrinsic auditing• Supports temporal projections


Changes

1

2

3

6297

6298

6298

Blobs

Query Data

Documents

Groups

Users

Objects

Folders

User Interface

Changes Queries

Attachments,Images, etcChanges

1

2

3

6299

6300

6301

Blobs

Query Data

Documents

Groups

Users

Objects

Folders

User Interface

ChangesQueries

Attachments,Images, etc

Primary Node Laptop / Secondary Node

Check-out / Check-in

WIFI/LAN/3G

WIFI/LAN

Just changes ownership of an aggregate and

forces synchronization

Each aggregate is owned by a node to prevent functional

conflictsCan be rebuild from the changes table at any point

in time

Optimized for querying

SOAP/WS-*

SOAP/WS-*

HTTPS-REST

E.g. UserPasswordChanged, DocumentSigned,

TitleChanged, RoleRevoked

Contains a subset of the master

Synchronization Overview


Design Challenges


NEventStore vs Lokad.CQRS vs NCQRS

User Aggregate

EventStoreDataMapper

ChangeEmail()

NEventStore 3

Commits

User State

Apply(EmailChangedEvent)

GetAggregateRoot(key)

Domain Unit Of Work

Caller

Load(key)

Open Stream(guid)

new User()

Load(events)

OnEventApplied()

SubmitChangesWrite Into Stream Per Aggregate


Immediate vs Eventual Consistency

Projections Commits

EventStoreDataMapper

NEventStore 3

Projector

NHibernate

Dispatched Event

Submit Changes

Transactional Boundary Transactional Boundary

Queries


Event Versioning

Event V1 Event V2

Some Event

Combined Event

Other Event

Combined Event

Separate Event

Separate Event


Other Challenges• No direct AR dependencies• Granularity of events• Domain Events as facts, not triggers


Scalability Issues


Scalability Limitations• Commits table grows and gets slower• No partitioning of global data• Single point of failure• Upgrades take too much time


Changes

1

2

3

6297

6298

6299

Blobs

Query Data

Documents

Groups

Users

Objects

Folders

Changes

1

2

3

6299

6300

6301

Blobs

Query Data

Documents

Groups

Users

Objects

Folders

Version 1.0 Version 3.0

Schema 1.0

Pass 1= no down time

Out-of-place migration

Schema 3.0

Pass 2= down time

Setup

Upconverts commits, blobs, etc


Multi-Tenant Scaling

Integration NodeReporting

Server

Node(owns tenants 1 & 2)



Node (failover for tenants 5 & 6)


Node(owns ref. data &

node config)

Node (uses tenants 1 & 2, owns tenant 11)

Node(uses tenants 1 & 11)



Connects to back-office systems Specific reporting

representations of the data

Owns global data and ownership configuration

Decentralized node


Architectural Issues


Limiting Constraints• Recovery in web farm• Selective rebuilding of projections• No asynchronous dispatching• Relational database is bottleneck during load

balancing• Network overhead web->database


Web Site Web Site

Load Balancer

RDBMS

Projections

RDBMS

Commits

Worker Thread

Worker Thread

?

Application Tier

Database Tier

First Attempt


Web Site Web Site

Load Balancer

RDBMS

Projections

RDBMS

Commits

Worker Thread

Worker Thread

RDBMS

Projections

Application Tier

Database Tier

Second Attempt


Web Site Web Site

Load Balancer

RavenDB

Projections

RDBMS

Commits

Worker Thread

Worker Thread

RavenDB

Projections

Application Tier

Database Tier

Final Attempt


RavenDBAdvantages• No migrations needed• No more column length problems• Faster• Less SQL Server load• More scalable• Dynamic Indexing • Keyboard/faceted search w/

Lucene• Concious decision on

asynchronicity

Disadvantages• New technology, new concepts• No LINQ• Async indexes• Requires rewrite of all projectors and queries


Front-End Server

RavenDB

Projections

Commits

Application Tier

Database Tier

Web Site WCF Services

Job Scheduler

Worker Thread ???

?


Front-End Server

RavenDB

Projections

Commits

Application Tier

Database Tier

Web Site

WCF Services

Job SchedulerWorker Thread

/queries/{query}


Web Site

RavenDB

EventStore

Component RepositoryMain Repository

QueryHost Raven

QueryHost

QueryHost Client

Module Module

OWIN Startup

Projectors Projectors

QueriesQueries

Query Handlers

Query Handlers HTTP (network or memory)

Thread Pool

Query

Commit

/queries/{query}

QueryHost RavenDB

Owin

WebAPI 2

NEventStore

Katana

AutofacDurable Commit Dispatcher

Querying Controller

Query Host Settings

RavenSession Checkpoint Store

HTTP-based QueryProcessor Interfaces


Concurrency Issues


StreamId: User-Dedo, Rev: 5

Primary Node Laptop / Secondary Node

Event Merging



UserCreatedEvent

GrantedRoleEvent

PhoneNumberAddedEvent

GrantedRoleEvent

PasswordChangedEvent



UserCreatedEvent

GrantedRoleEvent

PhoneNumberAddedEvent



RoleRevokedEvent

Last Sync Point


GrantedRoleEvent

Time




RoleRevokedEvent

StreamId: User-Dedo



Future?


Opportunities• In-memory projections• Gossip-based detection• ATOM feeds


Reading Material• NEventStore on GitHub• Domain Events by Udi Dahan• Event Versioning by Rinat Abdullin• Effective Aggregate Design by Vaughn Vernon• NHibernate vs Entity Framework in DDD by me…• DomainDrivenDesign.org• RAFT, a distributed consensus protocol• Cedar by Damian Hickey

https://github.com/NEventStore/NEventStore

http://www.udidahan.com/2009/06/14/domain-events-salvation/

http://abdullin.com/post/event-sourcing-versioning/

http://dddcommunity.org/library/vernon_2011/

http://www.dennisdoomen.net/2013/03/entity-framework-56-vs-nhibernate-3.html




http://domaindrivendesign.org/

http://thesecretlivesofdata.com/raft/

https://github.com/damianh/Cedar


[email protected]

Twitter@ddoomen

Blogwww.dennisdoomen.net

Technology

Building occasionally connected applications using event sourcing