Reporting from the Trenches: Intuit & Cassandra

Reporting from the Trenches – How Intuit Uses Cassandra Effectively to Improve Customer Experiences

Rekha Joshi, Staff EngineerIntuit, Inc.

Thank you for joining. We will begin shortly.

Webinar Housekeeping

All attendees placed on mute

Input questions at any timeusing the online interface

Speaker Bio

O’Reilly Certified Apache Cassandra Architect

Rekha JoshiStaff Engineer at Intuit

1 About Intuit

2 Use Case: Personalized A/B Testing 3 Database Requirements

4 Cassandra: Intuit NoSQL Standard

5 Using Cassandra Effectively

Intuit On Mission

Intuit Data Platforms

50M+manage all of the data

complex compliancePublic and private cloud

customers to handle6+

petabytes of data

45M+ Customers

Manage all of the data 6+ Petabytes of data

Complex compliance

Use Case: Personalized A/B Testing

Opinion-vs-Opinion Wars

Huge Investment

Angry Customer

Experiment, experiment, experiment!

Let Data Be The Decision Maker!

No Personalized A/B Testing?

With Personalized A/B Testing!!

Use Case: Personalized A/B Testing

To Continuously Improve User Experience, Data Is Better Than Guess!

Personalized A/B Testing Platform

User Assignment

Personalization Service

Segmentation Filters and Sampling

Personalization Engine

Analytics

Set up and administration

Profile Store

User Actions

A/B Testing Service

Deployment

Monitoring

Alerting

Amazon CloudJenkinsCoopr ChefCloudformationECS/Docker

CloudwatchSplunkGraphiteGrafanaLogstashPrometheusNew Relic

SensuNew Relic AlertsHipchatPagerDuty

Database Requirements

• High Data Security• No Data Loss• No Downtime• Linear Scalability• Tunable Consistency• Performance Under Workloads

All This Data!!!!!

Can I Lift This Alone?

Need for Speed

Cassandra, Who?

Cassandra is a Java based NoSQL, linearly scalable, best in class tunable performance, fault tolerant, distributed, masterless, time series database.

Cassandra: The Hybrid Kid has the Edge!

DynamoDB(Amazon)

Big Table(Google)

Cassandra

Inherits data distribution Inherits data model

Masterless ArchitectureLinear Scalability Tunable Consistency/Performance

ApplicationQuery Access Patterns

influencing influencing

Cassandra and DataStax Enterprise

Advanced Security

Integrated Analytics (Spark)

Advanced Tools

24/7 Support

A Truly Successful Software

• Solves A Real Need• Is A Building Block for Platforms• Becomes Open Source• Gets Commercial Backing• Tools Ecosystem Builds Around It• Establishes Strong Users Base• Companies in Critical Domains use It!!

Database Options

Intuit and Cassandra

Cassandra = Intuit Technology Standard of Choice for NoSQL Distributed Database

High Data SecurityNo Data LossNo Downtime

Linear ScalabilityTunable ConsistencyOther NoSQL variants

Performance Under Workloads

Did You Use Cassandra Effectively?

Garbage Collection Issue

New objects created at faster rate, than they are GC’ed Can causes STOP-THE-WORLD GC pauses! •Configure Heap size, MAX_HEAP_SIZE•Set up GC logging CASSANDRA_HEAP_DIR•Configure CMS GC/G1GC•Automated Heap Dump•Upgrade System

Cassandra is a Java based NoSQL linearly scalable, fault tolerant, distributed time series database.

Clock Issue

Ensure when you move setups/do upgrades, the ntp server is set correctly

Cassandra is a NoSQL linearly scalable, fault tolerant, distributed time series database.

Understand the Node Ring

Repeat after me: Cassandra is a Java based NoSQL linearly scalable, best in class tunable performance, fault tolerant, distributed, masterless, time series database.

Nodetool statusNodetool ringNodetool infoNodetool cfstatsNodetool tpstats

What If A Node Goes Down?

ReplicationConsistencyNodetool repairNodetool decommissionNodetool snapshots

Cassandra is a NoSQL linearly scalable, fault tolerant, distributed, masterless time series database.

Tuning The Application

Cassandra is a Java based NoSQL linearly scalable, best in class tunable performance, fault tolerant, distributed, masterless, time series database.

Refactor data modelRevisit the usage access patternsParanoid Monitoring

Tuning For Reads

• Caching Layer – Key Cache/Row Cache• SSTable Compactions Frequency

• Multiple SSTable inefficient

Cassandra is a Java based NoSQL linearly scalable, best in class tunable performance, fault tolerant, distributed time series database.

Tuning For Writes

Cassandra is a Java based NoSQL linearly scalable, best in class tunable performance, fault tolerant, distributed time series database.

• Memtable – Fast Writes• CommitLog – Separate Dedicated Disk

Tuning the System

EXT4 Filesystem System Memory, CPU, DiskParanoid Monitoring

Cassandra is a NoSQL linearly scalable, fault tolerant, distributed, masterless time series database.

Little Talked Aspect Of The Pareto Principle!

Heavy Lifting? Easy!

Thank you!

Input questions at any timeusing the online interface

https://www.linkedin.com/in/rekhajoshmhttps://twitter.com/rekhajoshm

Reporting from the Trenches: Intuit & Cassandra

Technology

Intuit Case Analysis

Intuit websites

Intuit QuickBooks Enterprise Solutions 11http-download.intuit.com/http.intuit/CMO/proadvisor... · Intuit® QuickBooks® Enterprise Solutions 11.0 White Paper September 2010 Intuit,

Intuit Report

Intuit GoPayment•U

Intuit brochure

We are Intuit

Intuit CEO Shareholder Meeting

Intuit final ppt

290072 REL Intuit Enrollmen

Running Cassandra on Amazon’s ECS - Meetupfiles.meetup.com/7439192/Cassandra-ECS.pdf · • Cassandra • ECS • Cassandra on Docker best practices • Cassandra on ECS. Motivation

Apache Cassandra in Action - O'Reilly Mediaassets.en.oreilly.com/1/event/55/Apache Cassandra in Action... · Apache Cassandra in Action. Why Cassandra? ... Cassandra in production

intuit .k 1Ah

Intuit Mobile Custom Views

Intuit business-pinterest-slideshare

Intuit Partner Platform Overview

Chicago Cassandra - Cassandra from Python

Intuit demandforce

Intuit SBDC Toolkithttp-download.intuit.com/http.intuit/CMO/intuit/press_room/vpk_quickbooks_ss/intuit...Internet-based Version Available g Manage.a.business.from.anywhere..QuickBooks:

Vijay anand intuit.34664759