of 12/12
ARCHITECTING EXTREMELY LARGE SCALE WEB APPLICATIONS A MUST read for every architect ABSTRACT An overview of the much needed vicissitude in the architectural thought transformation from monolithic to microservices architecture. PRASHANTH B PANDURANGA Director of Technology

Architecting extremelylargescalewebapplications

  • View

  • Download

Embed Size (px)

Text of Architecting extremelylargescalewebapplications


New York times auto scaled to 500,000 users, HipChat has about 1.2 billion messages/documents stored, Sales force deals with 1,300,000,000 daily transactions with over 24,000 database transactions per second and over 22PB of raw SAN storage capacity, CinchCast has over 50 million page views a month, Pinterest has over 18 million visitors with a 10X growth rate, Amazon has over 55 million active customer accounts, Flickr has over 4 billion queries per day, Netflix has 48 million members with over 50,000 requests per second and the list goes on. . Thanks to HighScalability, where the above statistics are derived from, you get a picture of the websites and the scale that I am referring to! I had covered the SAAS requirements in my previous blog. . In this blog I shall provide a birds eye view of the technologies used by a few large scale websites and their impact on the multi-tier web application architecture. Reference: HighScalability has some awesome architecture blogs, I have derived the technology stack from multitude of articles hosted by highscalability and a few from Netflix blogshttp://highscalability.com/blog/category/exampleTechnology Stack

This section is not meant to be an extensive overview of each of the below websites and their technology stack, but rather provides an overview of the variety of tools used and the logical layers that form the architecture. NOTE: The below mentioned tools used by the companies are point in time and may have been changed.

Figure 1 NewYorkTimes Technology Stack

Figure 2 HipChat Technology stack

Figure 3 Salesforce Technology stack

Figure 4 Netflix Technology stack

I have elaborated on SAAS requirements and general architectural requirements in my previous blog.https://prashanthpanduranga.wordpress.com/2015/03/25/inevitability-of-multi-tenancy-saas-in-product-engineering/The requirements which apply the most to large scale web applications: Performance: There are lot of statistics published relating to performance implications of web applications. One such statistic, Users abandon website even if there is a 2 second delay during a transaction. Large scale web applications obviously have very high performance requirements. A very large percentage of applications gets redesigned primarily for a better user experience. Average Page Load Time as a fraction of Server and client time, Network time, Page views, Bounce Rate, Percentage Exit, Average Redirection Time, Average Domain Lookup Time, Average Server Connection Time, Average Server Response Time, Average Page Download Time, Average content load time, Average session time, DNS resolution time, TCP connection time, Time to first byte, Full page object load time, Requests per second, error rates, Peak response time, Uptime, CPU utilization, Memory Utilization are just a few metrics to look forAvailability: Business continuity is of utmost importance. Various availability techniques can be applied on every layer. AlwaysOn Failover Cluster Instances, AlwaysOn Availability Groups, Database mirroring, Log shipping, Redundancy models - Active-Active, Active-Passive, Redundancy Methods - Hot Standby, Warm Standby, Cold Standby, and the measurement of the same expressed as mean time to failure, mean time to repair, Availability as a measure of MTTF / (MTTF + MTTR), Eliminating single points of failure, Accelerating fault detection, isolation and resolution, hot spares, warm spares, cold spares, clustering, RAID, redundancy, Heart beats, watermarking resources, check pointing, Watch dogs and moreMonitoring and Diagnostics: 26 front end proxy serves. Double that in backend app servers [Atlassian HipChat], 15,000+ hardware systems [Salesforce], 100 hardware nodes in production [CinchCast], 180 Web Engines + 240 API Engines, 88 MySQL DBs (cc2.8xlarge) + 1 slave each, 110 Redis Instances, 200 Memcache Instances, 4 Redis Task Manager + 80 Task Processors, Sharded Solr [Pinterest], 1000+ supported devices [Netflix]Imagine maintaining those, when there are innumerable servers involved, failure of system components is common, needless to say, to take appropriate timely actions monitoring and diagnostics plays a very important role. Scalability: Capability of supporting and optimizing resource utilization on increasing workloads on various dimensions such as memory, cores, data structures, throughout and more. Goes without doubt that the application need to scale, in order for the application to perform well, and without automating it, the application cannot stay inexpensive. Automation: Identifying failures and automating re-provisioning of those components/servers is extremely importantArchitecture has significantly emerged from a monolithic architecture to microservices architecture.

Figure 5 Minimalistic layers

Lets take a look at the logical segregation of layers in current large scale web applications.

In the following table we look at some of the technologies used by high scale websites and compare them to other similar tools/technologiesNote: The technologies covered by the above websites was used as the benchmark for comparison. Comments are provided only where I wanted to provide additional details. Tools/software

Usage/DescriptionTOP few similar tools/software to considerComments

RabbitMQMessage broker systemMessage oriented middlewareQueuing softwareESBActiveMQ, Amazon SQS, HornetQ, HiveMQ, JMS, Kafka, ZeroMQ, MSMQ, NServiceBus, Azure Service Bus, OpenMQ, Redis, Storm, Akka, Apache Camel and Spring, OFM, Fuse ESB, WebsphereMQ, Windows Service Bus, BizTalk, WSO2, Mule, Talend ESB, Gearman, JBoss, ServiceMix, OpenESB, Apache QPIDLook out for AMQP compliance Some of the tools referred arent Message brokers but are used in conjunction to perform the sameOther AMQP PIKAShovel

KinesisReal time data processingKafka, Storm

TornadoWeb ServerHTTP ServerNginx, Apache, IIS, lighttpd, haproxy, varnish, glassfish, Jetty, Geronimo, Tomcat, Some are even used as reverse proxy, proxies,

CassandraDistributed database management systemMongodb, aerospike, accumulo, azure table storage, bigtable, couchbase, couchdb, dynamodb, datastax, ElasticSearch, Greenplum, Vertica, HBase, InfiniDb, InnoDB, MariaDB, neo4j, Netezza, TeraData, RedShift, Riak, RavenDB, Solr, Spark, VoltDB, With over 200 dbs, its difficult to list all: Checkout the below linkhttps://prashanthpanduranga.wordpress.com/2013/12/23/why-nosql-ok-but-why-so-many/Some of them are dataware housing Solutions, While some are data processing engines

HanaIn-Memory databaseGemFire, Hekaton, Aerospike, BigMemory, DataBlitz, EhCache, eXtremeDB, FuelDB, HazelCast, MonetDB, Coherence, VoltDBAny Key value store can be used for the same, some of the enterprises have experimented using NoSQL store used as a cache and unstructured db solution

LinuxOperating SystemSUSE, FreeBSD, Solaris, Debian/Ubuntu, Windows Server, Mac OS X, RHELOnly Server OS included in the list

SockJSWeb Socket like objectWeb socket, socket.io, atmosphere, SignalR, Alchemy, FleckCouple of them listed are open source

LibevEvent LoopLibEvent, Asio, Nginx, epoll

FabrikVisual programming IDEAlso known for Content construction Kits (CCK)IDE: Visual Studio, Eclipse, NetBeans, AptanaCCKS: Seblod, K2, chronoform, Zoo, Breezingforms, Cobalt, FlexiContent

JavaProgramming languageC, C++, Python, C#, PHP, Javascript, Ruby, R, Matlab, Objective-C, Visual Basic, Perl, Swift, Scala, Shell, GO, LISP, SAS, F#, Groovy, LuaSome of them listed are web programming languages adiitionals: HTML, SQL, Haskell

TwistedEvent driven network programming frameworkTornado, Django, Asyncio,

AWSCloud providerAzure, Rackspace, CenturyLink, Salesforce, Engineyard, Google, OpenStack, SAP, CloudBees, CumuLogic, Eucalyptus, Gigaspaces, Mulesoft, Parallels, Pivotal, puppet Labs, Ravello, Rightscale, SoftwareAG, Xively, AT & T, Cisco, Comcast, EMC, GoGrid, CSC, HP, IBM smartcloud, Joyent, The list includes infrastructure, platform, storage and security cloud providers

LuceneTest search Engine LibraryAzure Search, Autonomy, Solr, GSA, Attivio, DTSearch, elasticSearch, endeca, FAST, MarkLogic, Nutch, Sphinx, Sketchy, ScumblrA few NOSQL databases have been used for the same, This list does not include all the NOSQL databases that could be used

Adobe AirCross-platform runtimeCordova(Phonegap), Appcelerator, Qt, Sencha, cocos2d-x, Xamarin, ionic, Kony, mono, xcodeThe ones listed here are cross platform as well as mobile development platforms.

SensuMonitoring FrameworkZabbix, Nagios, icinga, monit, Riemann, statsd, graphite, zenoss, collectd, munin, cacti, new Relic, ganglia, splunk, sentry, dynatrace, datadog, skylight, zenoss, observium, spiceworks, solarwinds, fiddler, wireshark, httpwatch, firebug, soapUI, OpManagerThe list includes some of the: Infrastructure monitoringSearching, monitoring and analysing, Network monitoringScalable distributed monitoring system

PagerDutyNeoLoadIncident management systemand performance testing and monitoringOpsGenie, VictorOps, xmatters, pingdom, Gomez, webpagetest, monitis, uptrends, keynote, OpsView, Apache JMeter, LoadRunner, WebLOAD, Appvance, NeoLoad, LoadUI, WAPT, Loadster, LoadImpact, Soasta, Rational Performance Tester, Testing Anywhere,OpenSTA,QEngine (ManageEngine), Loadstorm, CloudTest, Httperf, SilkPerformer, BlazeMeter, Visual Studio Test Suite, Also includes web site monitoring Cloud based quality testingPerformance monitoring

ChefIT AutomationPuppet, ansible, salt, docker, Jenkins, Capistrano, saltstackConfiguration management, SCCM

memCacheDistributed memory object cachingApc, memcached, dynacache, ehcache, xcache, key value based NOSQL databases are also used

RazorPhysical and virtual hardware provisioning solutionAxemblr, Cobbler, JuJU, SaltCloud, Dell Crowbar, Ansible, CFEngine, Chef

PerforceVersion Management and Content collaborationGit, SVN, TFS, bitbucket, ClearCase, Subversion

PytheasITIL assets management softwareRemedy (BMC), Assyst (Axios), FrontRange, EasyVista, Hornbill, HP Service Manager, SmartCloud Control Desk (IBM), ServiceNowIT incident management, IT problem management, IT change management, IT release governance, IT user self-service, IT request management, IT knowledge management, IT service support analytics and reporting, IT SLA management Ref: Gartner

ZUULService that provides dynamic routing, monitoring, resiliency and securityNginx, lightpd, Netscaler, HAProxy, Radware, CoyotePoint, Barracuda, Kemp, Varnish, Avast, Norton, Kaspersky, Mcafee, AVG, Avast, Bitdefender, F5, PaloAlto, Cisco ASA, Cisco ACE, Foundary, Juniper SSG, MS TMGCan be firewall, router, web load balancing server, proxy Server etc.

Feign Java http client binderRetrofit, JAX-RS, web socket, Jersey, CXF, Apache HC

Includes transport libraries

HiveQuerying and managing large datasets residing in distributed storageImpala, BigSQL, HAWQ

AWS ELBElastic Load balancingNginx, HAProxy, Route53, Azure Traffic Manager, F5 Port-bound servers, sticky sessions, TCP session reassignment, automatic unfail, slow start, SynGuard, dynamic feedback protocol, NAT, maximum connection, Round Robin, Least Connections, Weighted Round Robin, Weighted Least Connections, Fastest ResponseLayer 4 and Layer 7 load balancing Cloud Load balancing features: Dedicated (static) IP address,SSL terminationMultiple protocols, Advanced access control, Connection logging, Advanced algorithmic routing, Session persistence,Connection throttling, Node management, High availabilityContent caching, Persistent connections, Gzip compression,Regionalized load balancers

gZipApplication used for file compression and decompressionhttpZip,deflate, 7zip, bzip2, zlib

AkamaiContent delivery networkAzure CDN, Cloudfront, Torbit, Incapsula, Cotendo, Fastly

HTML 5 frameworksJavascript Frameworkshttps://www.facebook.com/notes/prashanth-panduranga/frameworks/10152107517972934

OpenStackOpen Source Cloud computing platformOpenStack currently has the following features: Compute(Nova), Object Storate (Swift), Block Storage (Cinder), Networking (Neutron), Dashboard (Horizon), Identity Service (Keystone), Image Service (Glance), Telemetry (Ceilometer), Orchestration (Heat), Database (Trove), Bare Metal Provisioning (Ironic), Multiple tenant cloud messaging (Zaqar), Elastic Map Reduce (Sahara)

HadoopDistributed storage and distributed processing of very large data sets on computer clusters

AegisthusBulk Data Pipeline out of Cassandra

Eureka Eureka is a REST (Representational State Transfer) based service that is primarily used in the AWS cloud for locating services for the purpose of load balancing and failover of middle-tier servers

GenieFederated Job Execution Engine

ClojureDynamic programming language that targets the Java Virtual Machine

PigPenMap-Reduce for Clojure

Governator Governator is a library of extensions and utilities that enhance Google Guice to provide: classpath scanning and automatic binding, lifecycle management, configuration to field mapping, field validation and parallelized object warmup

InvisoVisualize Hadoop performance

RibbonRibbonis a Inter Process Communication (remote procedure calls) library with built in software load balancers

HystrixHystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable

SuroDistributed data pipeline

AminatorA tool for creating EBS AMIs

LipstickPig Visualization framework

ZenoIn-Memory Data Propagation Framework

BleskLightweight client for pushing notifications to web based applications/sites

TurbineTurbine is a tool for aggregating streams of Server-Sent Event (SSE) JSON data into a single stream. The targeted use case is metrics streams from instances in an SOA being aggregated for dashboards

PriamCo-Process for backup/recovery, Token Management, and Centralized Configuration management for Cassandra

WorkflowableWorkflowable is a Ruby gem that allows adding flexible workflow functionality to Ruby on Rails Applications

s3mperS3mper is a library that provides an additional layer of consistency checking on top of Amazon's S3 index through use of a consistent, secondary index

AstyanaxJava Client for Apache Cassandra

DenominatorDenominator is a portable Java library for manipulating DNS clouds. Denominator has pluggable back-ends, including AWS Route53, Neustar Ultra, DynECT, Rackspace Cloud DNS, OpenStack Designate, and a mock for testing

GCVizGarbage Collector Visualization framework

CuratorThe Curator Framework is a high-level API that greatly simplifies using ZooKeeper. It adds many features that build on ZooKeeper and handles the complexity of managing connections to the ZooKeeper cluster and retrying operations

StaashA language-agnostic as well as storage-agnostic web interface for storing data into persistent storage systems, the metadata layer abstracts a lot of storage details and the pattern automation APIs take care of automating common data access patterns

EddaEdda is a Service to track changes in cloud deployments

BrutalAn asyc centered chat bot framework for python programmers written using the twisted framework

CassJMeterJMeter plugin to run cassandra tests

GlistenGroovy library for building JVM applications with Amazon Simple Workflow (SWF)

PigPlatform for analyzing large data sets

SparkEngine for big data processing, with built-in modules for streaming, SQL, machine learning and graph processing

Karyon Framework and a library for a cloud ready web service. Blueprint for the services. It contains Bootstrapping, Libraries and Lifecycle Management, Runtime Insights and Diagnostics, Pluggable Web Resources, Cloud-Ready hooks

EBSElastic Block store, persistent block level storage volume

CurlerA Gearmanworkerwhich cURLs to do work

archaius, , Library for configuration management API

ZooKeeperZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services

Parallel processing - Explicit and Implicit parallelism, batch parallelism, asynchronous programming, segregating layers, distributing workloads, Load balancing, multi- tenancy, scaling out on all layers, sharding, partitioning, CAP preference, reads, writes, statelessness, logging and telemetry, automating, SOA adoption, caching, throttling, distributing requests across multiple zones, effective usage of CDNs, Auto provisioning, Auto scaling, compression, queuing, workload distribution, batch processing, designing system with fault tolerance, redundancy, Consistency, Availability, Partition Tolerance, event processing, web sockets, cloud computing, fog computing, Grid Computing, Client side workload distribution, In-Memory processing, Proxies, No single points of failure. Resilience to failure, Graceful degradation, Recoverability from failure, design for failure, Database Transactions, Client side transactions, two-phase commit, Auto-commit, Partition Everything, DB operations ordering, Considerations for Eventual consistency, Functional Segmentation, Application Pools, Prevention of session state, Async Everywhere, Index, Structured Indexes, text indexes, entity indexes, Fuzzy match indexes, pre-aggregated indexes, pre-calculated indexes, embedded value indexes, join indexes, link indexes, De-Normalized Indexes (all kinds) are all important considerations for a highly successful and scalable website. Rest assured if you have considered all the above factors in your architecture you are on your way to create a scalable one. Do let me know if you have questions regarding any particular subject and I will be glad to write up on the same. .