Terracotta Java Scalability - Stateless Versus Stateful Apps

Copyright Terracotta 2007

Stateful Applications in a Stateless World

How to write stateful web applications that scale like stateless ones

Open Source Clustering for Java


The Innocent Web Application

  Within every innocent web application lies a sleeping monster

  There comes a time when every successful web application outgrows its single machine architecture

  Whether for high-availability or scalability or both, the adult web application must live on more than one application server

  That's when the latent beast strikes...



The State Monster



The State Monster

  Conversational state across requests

  Application state: caches, indexes, etc.



The Stateless Convention

  Push state out of your application server to achieve "statelessness"

  Application server is for computation only

  Any application server can handle any request

  "...with a little ingenuity, [we have created a] very fast application that is much more resilient to application restarts"



The Stateless Convention: Push State to the Database



The Stateless Convention: Push State to the Database

  DB becomes a bottleneck

  Not optimized for object data



The Stateless Convention: Push State to Peers



The Stateless Convention: Push State to Peers

  Push state everywhere gives HA, but doesn't scale

  Push conversational state to buddy can't survive buddy pair death or full cluster restart. Doesn't help for application state.



The Stateless Convention: Push State to Client

  Highly scalable for conversation state UNLESS state is large

  Doesn't help with application state



Stateless Convention: The Real Problem

"With a little ingenuity," many of these problems can be solved on a per-application basis.

The real problem with the "stateless" convention is...



The Stateless Convention: Eats Developer Brains



The Stateless Convention: Eats Developer Brains

  Pushing state out of app server doesn't get rid of state. There is no such thing as "stateless"

  Serialization/externalization:   Full-object graph serialization   breaks object identity   corrupts data and programming

model

  "with a little ingenuity..."-- seems fun/easy to try, but ends up taking over the development effort



Changing the Assumptions: JVM-level Clustering



Performance + Reliability + Simplicity

  10X throughput over conventional APIs   All Reads from Cache (implicit locality)   All Writes are Deltas-only   Statistics and Heuristics (greedy locks)

  Scale out the Terracotta Server   Simple form of Active / active available today   V1.0 GA this year

  Looks like Java to me (code like your mom used to make)   Normal access patterns: no check-out before view and check-in on commit   Code and test at a unit level without infrastructure intruding on app logic   Threads on multiple JVMs look like threads on the same JVM   PUFIELD, GETFIELD, synchronized(), wait(), notify() work as expected



Demo: Session Clustering



Demo: CoordinationWithQueues (from our Apress book)



QueueReader

private static final class QueueReader implements Runnable { private final NamedQueue myQueue;

public QueueReader(NamedQueue queue) { this.myQueue = queue; }

public void run() { while (true) { try { System.out.println(new Date() + ": message from the " + myQueue.getName()

+ " queue: " + myQueue.take()); } catch (InterruptedException e) { // A real application would do something interesting with this // exception. e.printStackTrace(); } }}



Queue Writer

//...private void run() { while (true) { try { String mood = MOODS[random.nextInt(MOODS.length)]; myQueue.add(myColor + " says, \"I'm feeling " + mood + ".\""); Thread.sleep(1000); } catch (InterruptedException e) { // A real application would do something with this exception e.printStackTrace(); } }}// ...



CoordinationWithQueues Config File ... <dso> <instrumented-classes> <include> <class-expression> org.terracotta.book.coordination.queues.CoordinationWithQueues </class-expression> </include> <include> <class-expression> org.terracotta.book.coordination.queues.CoordinationWithQueues$NamedQueue </class-expression> </include> </instrumented-classes> <roots> <root> <field-name>org.terracotta.book.coordination.queues.CoordinationWithQueues.QUEUES</field-name> </root> <root> <field-name> org.terracotta.book.coordination.queues.CoordinationWithQueues.sharedCounter</field-name> </root> </roots> <locks> <autolock> <method-expression> void org.terracotta.book.coordination.queues.CoordinationWithQueues.main(java.lang.String[]) </method-expression> <lock-level>write</lock-level> </autolock> </locks> </dso>...


Case Studies



Fast, Simple, Scalable

[T]hanks to Terracotta and your post about the message bus you did we have stripped all J2EE out of our architecture making it much simpler for service developers, much faster, and much more scalable. What has us so excited about this approach is exactly what you said about developer productivity - that is Right On!

--http://blog.markturansky.com/archives/77

“



Large Publisher Gets Caught Down the Path with Oracle

Scaling Out or Up?



Breaking the Pattern without Leaving “Load-Balanced” World

• $1.5 Million DB & HW savings • Doubled business • More than halved database load



Coming Soon: Examinator Reference Application

  Online test proctoring application   Popular stack

  Spring MVC, Webflow   Sitemesh   Freemarker   Spring Security   Tomcat/Jetty   JPA   Hibernate   MySQL   Maven   Junit

  Fully tested & tuned   Example architecture &

components   Example project layout and tools

integration (build/test/debug/deploy)

  Fully documented best practices and recommended SDLC processes



Gnip—Internet Data Aggregation and Delivery Service







DB

JMS JMS

Initial Architechture   JMS Queuing   Backing Database

Liabilities   Many moving parts   Many technologies   Increased developer specialization   Lower core-competency density




Terracotta Architecture   Natural, in-memory data structures   Durable, TC-backed objects

S3 Backup

EC2 Node 1: TC Client EC2 Node 2: TC Client

EC2 Node 3: TC ServerPeriodic

snapshot of

Terracotta

object store for

backup




Performance   50,000 messages/second   3-4 EC2 nodes

S3 Backup

EC2 Node 1: TC Client EC2 Node 2: TC Client

EC2 Node 3: TC ServerPeriodic

snapshot of

Terracotta

object store for

backup

Benefits   Fewer technologies, increased

core competency   App built w/ simple Java

constructs   Clustering and redundancy “just

works”   Excellent monitoring/debugging

tools   “I can’t believe you give them

away for free…”



Common Use Patterns



Community Resources

  http://www.terracotta.org/

  http://forums.terracotta.org/

  Mailing lists:   [email protected]   [email protected]

  IRC:   #terracotta channel on freenode

  Subversion   http://svn.terracotta.org/svn/tc   http://svn.terracotta.org/svn/forge

Technology

Terracotta Java Scalability - Stateless Versus Stateful Apps