The CQRS EventStore

Using and Abusing

Europe Virtual ALT.NET, 25 October 2011

Jonathan Oliver

About Me—Jonathan Oliver

Founder: SmartyStreets (smartystreets.com)

Founder, co-host: Distributed Podcast

Committer, NServiceBus & Autofac

Project owner, CQRS EventStore

SmartyStreets

Finances the EventStore project

Started in 2006 (as Qualified Address)

All your street address are belong to us

My personal guinea pig

Fully geo-distributed

Prerequisites

Messaging

SOA

DDD

CQRS (strict definition)

Event Sourcing

CQRS Architecture

My CQRS Journey

Hand-crafted SQL and stored procedures

Canonical data model

Application integration via shared database

Generated DAL

My CQRS Journey (continued)

Temporal database design

Messaging

Breakthrough—Putting DBs in their place

Greg Young—InfoQ and DevTeach Videos

Event Sourcing

Capturing business intent

Rebuild state from events

Rewind, pause, fast-forward

Build alternate models

Immutable data is crazy scalable

Complete audit

Not Another Framework!

It’s not a framework—I hate frameworks

It doesn’t permeate your code

You can swap it out and/or build your own

Nothing existed at the time—March 2009

Implementation reflects the developer

Challenges and Considerations Goal: technology is an implementation

detail

Transactions typically exist in SQL land

Lots of interesting alternatives appearing

Momentum around NoSQL increasing

Two-phase commit virtually non-existent

Getting to the Correct Model

Absorbing transactions

Cross-resource transactional consistency

Commits

Exposing a robust, easy-to-use model

EventStore Model

Event Stream

Event Store

Persistence Engine

Durable Storage

“ISession”

“ISessionFactory”

“NHibernate Magic”

Disk, DB, Memory, etc.

Serializer

(Snapshotting and some advanced capabilities have been ommitted)

App / Service Layer“Repository”

Dispatcher / Bus

Pipeline Hooks

EventStore

commit

commit

Typical Workflow

Message Handler

EventStore

Disk

commit ack

Async Dispatch Scheduler

Transaction #1

•Optimistic Concurrency• Idempotency/De-duplication

Transaction #2

Transaction #3

Dispatch Scheduler

Messaging Technology

Wire Protocol

EventStore

Disk

Mark as dispatched

EventStore and Streams

Streams vs. Aggregates

Usage Example

Implementation

Pipeline Hooks

Pipes and filters

Filter out, modify, up-convert

Select, Precommit, and Postcommit

Implementation

Dispatchers

Synchronous vs. Asynchronous

Any bus, broker, or messaging technology

Any serialization mechanism

Production example

Implementation

Serialization

XML, JSON/BSON, Protocol Buffers, Thrift, MessagePack, Avro

Serialization pipeline

Implementation

SQL Storage Implementation

The primary storage engine

Easy to understand

Great management tools

Proven, mature implementation

10+ flavors (SQL Server, MySQL, SQLite, etc.)

Code

Raven Storage Implementation

Easy alternative for .NET

Great commercial support

Great management tools

Responsive community + Oren/Ayende

Gotcha: 2PC and repeatable reads

Mongo Storage Implementation

Highly robust engine

Established as a primary NoSQL engine

Lots of management tools

Code

Upcoming Implementations

CouchDB

Amazon

Azure

Oracle

Redis

Cassandra

.NET System.IO APIs

Wireup

Completely optional

Avoiding additional dependencies

Can use StructureMap, Autofac, Ninject, etc.

CommonDomain

Spike code that worked too well

Simple, opinionated “DDDD” framework

Purposefully minimalistic

“Essence” to be merged into EventStore

Thank You

Blog: blog.jonathanoliver.com

Twitter jonathan_oliver

GitHub github.com/joliver