How MongoDB is Being Used in China - Case Studies

Copyright © 2013 ChinaNetCloud Server Management � Cloud computing � Audit & Optimization

Running the World’s Internet Servers

MongoDB Chinese Use Cases By Steve Mushero CEO of ChinaNetCloud

Running the World’s Internet Servers www.ChinaNetCloud.com

About Steve

Founder, CEO & CTO of ChinaNetCloud •  Before that, CTO of Tudou (土豆网)

8 years in China •  Lived in Silicon Valley, Seattle, and New York

25 years experience as a CTO •  Experience in development and operations

•  Decades of DBA, performance, security experience

•  Remembers flat file, network, IMS databases

2


About ChinaNetCloud

§  Founded in Shanghai in 2008 •  By Silicon Valley Technology Guys

§  Vision: Run All the World's Internet Servers •  It’s out job to run the Internet

§  We manage servers and systems •  For Chinese Internet and Game Companies

§  Thousands of Servers - Hundreds of Customers •  We've seen systems of every type and size

•  We are hiring – Linux Engineers / System Admins

3


Databases

Hadoop & things

Mostly PHP & MySQL

Some MongoDB

Less PostgreSQL

A bit of Oracle

Chinese Internet Situation


Startups & Applications

Chinese Internet Situation II

Still growing fast

Startups move fast

Value flexibility (schema-free companies)

Resume-building helps adoption

Lots of mobile apps in 2012/2013

Lots of new startups

E-Commerce FINALLY growing

But lots of dating, SNS, games, etc.


1

Sharding for large data sets

Flexible-Structure Document Database

2

3

4

5

6

Non-relational Database, networks, analysis

High-Performance Database

Replication with multi-Master Write

Basic datastore to replace RDBMS

MongoDB General Use Cases


Major Challenges

7

Stability

Locking

Tuning

Competition from Redis, MySQL

In China, lack of skilled engineers

In China, Hadoop suddenly sexy


Cases

8

Mobile Gaming

SNS Travel

IPTV Company

Baby Care Site

Job & Resume Site


Big Game Company

q  Runs several mobile games

q  Each in several countries

q Most on several platforms

9

Why §  Big data for logs, Users to a lot of things §  Unstructured logs

Experience §  Good §  User log analysis and data §  Core data simple key-value, Redis/MySQL

Pro/Con

Interesting

Problems §  Early on stability, locking §  Now runs well

§  Lazy write to DB, first in Memcache §  Works well for write-heavy systems

§  Also for Game status data, all write

§  High-speed, schema-free §  Slow to restart, reload data


SNS Travel Site / App

q  Lots of data

q  Lots of pictures

q  Complex travel

10

Why §  Schema free §  High Performance

Experience §  Good §  Core data system, relatively unusual

Pro/Con

Interesting

Problems §  Version 1.6 crashed and lost all data §  Now runs well with 2.0

§  Core data system §  Core user relationships & travel plans §  Backup every hour via Mongo Dump for safety

§  Schema-free §  Need experienced people when there are problems


IPTV Company

11

Why §  Complements MySQL §  For unstructured & business-related data

Experience §  Good with 2.0 now §  Lock time ratios high for a long time, 75%+ §  User log analysis and related data

Pro/Con

Interesting

Problems §  1.x Stability issues §  Lack of experts to help them

§  Find high performance IO powerful in 2.0 §  Load only 100 requests/second, multi-instance §  Mostly 32GB RAM, medium-sized

q  Lots of users

q  Lots of data

q  High uptime

§  Complex analysis easy §  Lack of skills


Baby Care Site

12

Why §  Performance to 50M PV/day §  Lots of data (~100GB already) §  Sharding if needed

Experience §  Good so far §  Not really scaled yet §  Data growing fast (comments / BBS)

Pro/Con

Interesting

Problems §  Locks, server seems normal §  Have to restart – ongoing issue

§  Lots of text / comments §  No MySQL or other backing DB

§  Schema Free data §  Closely matches business model

q  Fast

q  Large data

q  Flexible


Job & Hiring Site

13

Why §  Very fast development / limited time §  Flexibility as needs / app changing §  Popular with staff, improve resumes

Experience §  Mostly 64 GB RAM §  Medium load – 500-1500 requests/sec §  Random Global Locks suddenly in 2.0

Pro/Con

Interesting

Problems §  Locks, server seems normal

§  Have to restart – ongoing issue §  Lack experience on tuning §  RAM overflow on Master

§  Using sharding §  Using replication to balance load §  Also tracking logs, analyis

§  Schema Free data §  Good for storing OO objects in DB §  Closely matches business model §  Stability Issue / Manageability

q  Complex data

q  High concurrency

q  Constant change


Summary

§  MongoDB is new in China

§  MongoDB is sexy in China

§  Lots of interest

§  Well-suited for many startups §  Especially for flexibility and speed

§  Some challenges §  Stability §  2.x much better

§  Lack of talent §  More events !

§  Bright future in China

14


? Questions