Twitter Copes with Web2.0rhea

The Twits are catching up…

Alex CarianISM 158

May 6, 2010

From TheRegister.co.uk:

“Twitter: It's the end of the sysadmin as we know it”

• “"This is a whole new world… For the longest time, people ran large data systems on a kind of ad hoc basis. We're in a world now where so many people are depending on the real-time web. A system administrator is not just a system administrator anymore.” 1 John Adams, Ops Engineer

- Ad hoc solution: designed for a specific problem or task, cannot be adapted to other purposes 3

The Real-Time Web?

-Initially, the core of Twitter’s code ran:- Ruby on Rails (application framework)- Apache/Mongrel (web servers)

“You have to use analytics. You have to grab data. You have to look

at where a site is trending and where things are going so you can scale...

If you don't start doing this work early, if you don't start collecting this

early, you will fail.” - John Adams, Twitter Ops Engineer

Twitter’s Aches and Pains

- Despite the benefits of the RoR environment, Twitter experiences outages, most notably during Steve Jobs’ Keynote address at Macworld ‘08 3

-A developer, in his spare time, ports a section of the website code into the Scala programming language

- Allows for object-oriented and functional programming paradigms- Task that originally took 2.5 hrs now takes…

20 seconds!

Functional Bits

-Twitter maintains Ruby on Rails for building “user facing features” and MySQL for database management1

- Now entrusts back end infrastructure to Unicorn, a RoR application server

- Grocery Store model- Still open source- Still based on Mongrel (the former app server)

Solipsism 2.0

-Twitter saw a 1358% increase in traffic in 2009 (comScore)- Utilizes a monitoring system known as ‘Ganglia’

- open source program grown out of UCB and NSF grants- Tracks 15,000 points of site performance, including grid load, cluster load, # of hosts up/down

TWITPOCALYPSE?!

The “Fail Whale”

- Many suspected Twitter’s service would come unwound after reaching the Unique Tweet ID limit (2^31 = 4,294,967,295)

-Twitter’s ops team mined site data to predict when the limit would be reached- Allowed the team to implement necessary changes to avoid complications- Current number of Tweets posted: ~13,475,000,000 5

Another Word on Sysadmin 2.0…

“"Another discovery that we made when trying to increase the scale of Twitter was that disk is the new tape," he said. "With any sort of social networking operation - juggling followers, sending mail, etc. - disk is extremely slow.” 1 -John Adams

- Hard drive? That’s so 2009…. Or 1959…

-Twitter now uses memcached- Data and objects cached in RAM to reduce the number of times an external data source (such as a database or API) must be read. 4

- Don’t overdo it.

Twitter’s Open Back End

- Company uses open sourced technology in most operations- RoR, Unicorn, Ganglia,

- Developed several tools at home- Kestrel, message queue server- FlockDB, follower database- Both now released as open source

Will Twitter maintain its open source track record?

Sources Cited

1. Metz, Cade. “Twitter: It's the end of the sysadmin as we know it”. The Register, May 4, 2010. <http://www.theregister.co.uk/2010/05/04/twitter_and_the_end_of_the_sysadmin_as_we_know_it/>

2. Metz, Cade. “Twitter jilts Ruby for Scala” The Register, April 1, 2009. http://www.theregister.co.uk/2009/04/01/twitter_on_scala/

3. Riley, Duncan. “Twitter Fails Macworld Keynote Test” TechCrunch, Jan 2008. <http://techcrunch.com/2008/01/15/twitter-fails-macworld-keynote-test/>

3. “Ad hoc”. Wikipedia4. “memcached”. Wikipedia5. “Twitter Twitpocalypse Status”. Accessed May 6, 2010.

<http://www.twitpocalypse.com>

Sources Cited

1. Metz, Cade. “Twitter: It's the end of the sysadmin as we know it”. The Register, May 4, 2010. <http://www.theregister.co.uk/2010/05/04/twitter_and_the_end_of_the_sysadmin_as_we_know_it/>

2. Metz, Cade. “Twitter jilts Ruby for Scala” The Register, April 1, 2009. http://www.theregister.co.uk/2009/04/01/twitter_on_scala/

3. Riley, Duncan. “Twitter Fails Macworld Keynote Test” TechCrunch, Jan 2008. <http://techcrunch.com/2008/01/15/twitter-fails-macworld-keynote-test/>

3. “Ad hoc”. Wikipedia4. “memcached”. Wikipedia5. “Twitter Twitpocalypse Status”. Accessed May 6, 2010.

<http://www.twitpocalypse.com>