MySQL Databases at RightMedia

8/13/2019 MySQL Databases at RightMedia

1/17

MySQL Databases at RightMedia


2/17

-2 - Yahoo! Confidential

Intro To RightMedia

RightMedia is an advertising exchange

Publishers, Advertisers, Networks compete and co-operate

Rules and links between networks, publishers and advertiserslead to auctions for every ad call for fairness.

Over 7 Billion impressions per day!

Almost 8,000 ad servers in SP1,SK1,AC4,CH1,JP1

At peak, servers capable of processing 100Kimpressions/second

With pixels, cookie requests, etc, 250K calls/second


3/17


Data Cycle

Adserver data is aggregated and flushed every 10 minutes

Multiple aggregation layers per colo

Multiple data pipelines (similar to Data Highway)

Reporting/Impression Log data stored in MySQL (rpt_prod

and logdb) Customers use hourly reporting data to make decisions on

campaigns and spending (trf_prod)

Cache generation cycle every 2 hours

Traffic cache Predict cache

Budget cache


4/17


Data Cycle (Cont.)

Caches are pushed to each colo and distributed to adservers.

Cycle starts again


5/17


MySQL Databases @ RightMedia

Rpt_prodReporting Data Warehouse

51 database servers : 16 masters, 32 slaves (3 for archive)

Data sharded over 16 partitions

Middle ware server ReportWare converts XML requests to SQLand returns data in XML.

Reportware aggregates the 16 result set queries into one

DB size is 6.5TB

All Dell PE 2950s 32GB memory, 2xQuad CPU, 6x146GbSAS drives

Current schema very flatall tables are similar to materializedviews

New version (AC4)Star/Snowflake schema


6/17


MySQL Databases @ RightMedia (Cont.)

Trf_prodHeart of RightMedia

7 database servers: 1 master 6 slaves per colo

Provides operational store for YieldManager UI, API, CreativeTester, Analytics, Predict, Budget processes.

Very high transaction rate : nearly Half a billion per day

Relatively small but important

Downtime leads to larger failures

Ideal candidate for MySQL Cluster (Fault-tolerant system)


7/17-7 - Yahoo! Confidential


Predict Databases

Data store for adserver learning

Prediction engine uses this data to lower CPM and maximizerevenue

Key system for LZ1.1, LZ2, LZ3




Budget Databases


9/17

MySQL ArchitectureFeatures and Components



The Big Picture



FeaturesMySQL vs Oracle

Features/Functionality MySQL Oracle

Strenths Price/Performance

Great performce

Capable of running largeOLTPs and VLDBs

Database Server Instance Instance stores global memoryin mysqld background process

Instance has numerousbackground processes.

System SGA shared by procs.

Physical Storage Made up of database schemas.

Each storage engine storesinformation differently.

Tablespaces used for systemmetadata.

Partitioning Free, in Version 5.1

Limited functionality

$$$

Lots of options

Replication Free, relatively simple setup.Great horizontal scalability

$$$More complex, but lots ofoptions.

Backup/Recovery No online backup

Home-grown/3rdparty

RMAN

Stored Procedures Very basic Advanced features



Query Cache

Global memory area used to cache queries and result sets.

SQL query (SELECT) and result stored in hash table

Query is checked before parsing and optimization and resultset returned immediately

WICKED FAST PERFORMANCE!!

Excellent for static tables or tables with low change rate

Any INSERT,UPDATE,DELETE or any other DML operation ona table will invalidate all cache entries for that table.



Storage Engine Architecture

Most flexible (and coolest) part of MySQL server.

Allows the freedom of choice to mix and match storageengines within the same database and server instance.

Storage engines are pluggable and can be dynamicallyloaded into the server instance.

Storage engine API allows developers to enhance andcreate own storage engines.

The MySQL community offers different pluggable engines fordifferent business solutions.



MySQL Storage Engines

MyISAM

Default storage engine for MySQL

Disk Based storage - BTREE

Table level locking

No transactional support Very low memory footprint

Full Text searching capability

Index cache onlyOS buffers for data cache

Extremely high performance Prone to corruption on server crash



MySQL Storage Engines (Cont.)

InnoDB

ACID Compliant

Transaction/XA support

Row level locking

Clustered index Both data and index cached

Recovery can be time consumingcan very slow

Not scalable (Can be fixed with patches)


16/17


17/17

- 17 - Yahoo! Confidential

MySQL Storage Engines (Cont.)

Archive

Highly compressed data files

Useful for storage of historical data

Only allowed operations are SELECT and INSERT

No indexes MySQL 5.1 now allows index on AUTO_INCREMENT columns

Documents

MySQL Databases at RightMedia