ABDW17-Lightning Talks track-Real-Time Insight Into a Billion Emails

Real-time Insight into a Billion EmailsSteve Verlander | CTO

APEX BIG DATA WORLD 2017

The Challenge…Typical marketing campaign of 10 to 20 million messagesBetween 50 to 100 campaigns a month for over 1 Billion emails

Campaign segmentation required using historical dataA/B Testing for subject lines and content effectivenessUp to date reporting and analyticsHigh performance and throughputScalability on demandTrack live feedback including opens, clicks, and bounces

The Old Technology…

Oracle RAC cluster databasesMySQL Databases clustered and shardedDistributed SQL Server databasesNoSQL Solutions like MongoDBMap/Reduce using Hadoop

The Limitations…Relational database systems couldn’t scale to handle the high volume of events

Expensive to make fault tolerant, often requiring hardware that never gets used

Difficult and costly to maintain

Scalable solutions like Map/Reduce only provided delayed results

The Solution…Apache Apex and DataTorrent RTS

Can handle thousands of transactions per second

Very low latency and built in fault tolerance

Easy to deploy on premise or to the cloud

Scales effortlessly

The Demo…Simulation of thousands of Email Open Events per second

Real-time analysis of campaign effectiveness

Easy to use visualization tools to see your data in real-time

Steve Verlander | steve@nicea.com

Questions?