Migrating One of the Most Popular eCommerce Platforms to MongoDBMongoDB Munich 2013, October 14th

Aron Spohr, fashion4home GmbH, Berlin

aron.spohr@fashionforhome.de

Will it blend?

What is Magento?

● Open eCommerce platform● Serves all primary features of a Webshop● Written in PHP● Works on top of MySQL out of the box● Extensible architecture● Runs on over 180,000 sites● eBay owned since 2011

The Shopping Cart

● Quote● Items● Addresses

● one Model for each● one MySQL-Table for each

Data Structure

quote_id discount_amount grand_total

560 100.00 299.00

561 0.00 1028.40

item_id quote_id product_id qty price

100 560 1257 1 39.90

101 560 1349 2 140.10

address_id quote_id city street type

388 560 Munich Hauptstr. 33a shipping

389 560 Berlin Zahlstrasse 12 billing

390 561 Hamburg Geheimweg 3 shipping, billing

quote

quote_item

quote_address

as a developer in// Loading a quote from database$quote = Mage::getModel(‘sales/quote’)->load(560);

// Loading a filtered collection of quote items from database$items = $quote->getItemCollection();$items->addFieldToFilter(‘product_id’, 1257);$items->load();

SELECT * FROM sales_quote WHERE quote_id=560;

SELECT * FROM sales_quote_item WHERE quote_id=560 AND product_id=1257;

every Model has ● a Resource Model to load/save one record from/to DB● a Collection Model to load multiple records from DB

Persistence in

Model Resource Model

DBCollection Model

Model Model Model

App

licat

ion

function load($object, $id) {

$stmt = “SELECT * FROM “ . $this->getTableName() . ” WHERE “ . $this->getIdFieldname() . ”=$id”;

$data = $sqlAdapter->fetchRow( $stmt );

$object->setData( $data );

}

Resource Model basics of

function load() {

$stmt = “SELECT * FROM “ . $this->getTableName() . ” WHERE “ . $this->renderFilters();

foreach($sqlAdapter->fetchRows( $stmt ) as $row) {$object = new Object();$object->setData($data);$this->addItem($object);

}}

Collection Model basics of

Why should we change that?

● You don’t have to● It always depends on your application

Reasons we had:● Have more/other scalability options● Make it easier to work with raw data● Be more flexible with your data schema● Learn more about the software you are using

Let’s get started

● not change the way Magento works● a very simple, self-explainable data schema● make it easy to migrate old data● not lose any existing features

Data Structure - mongoDB{

quote_id: 560, discount_amount: 100.00, grand_total: 299,

item: [{item_id: 100, product_id: 1257, qty: 1, price: 39.9},{item_id: 101, product_id: 1349, qty: 2, price: 140.10}

],address: [

{address_id: 388, city: ‘Munich’, street: ‘Hauptstr. 33a’, ...},{address_id: 389, city: ‘Berlin’, street: ‘Zahlstrasse 12’, ...}

]

}

SELECT * FROM quote_item;

db.quote.find( {}, { ‘item’: 1 } );

● { ‘item’:[ {item_id: 100, product_id: 1257, qty: 1 },{item_id: 101, product_id: 1349, qty: 2 } ] }

● { ‘item’:[ {item_id: 102, product_id: 4421, qty: 1 } ] }● { ‘item’:[ {item_id: 103, product_id: 2301, qty: 1 },

{item_id: 104, product_id: 5511, qty: 1 } ] } ● ...

Do we still have a table?

a Tool to simplify our work with mongoDB...

Loading a collection of thingsloadDataCollection(‘/quote/item’, array());

● { item_id: 100, product_id: 1257, qty: 1 }● { item_id: 101, product_id: 1349, qty: 2 }● { item_id: 102, product_id: 4421, qty: 1 }● ...

db.quote.find( {}, { ‘item’: 1 } );

● { ‘item’:[ {item_id: 100, product_id: 1257, qty: 1},{item_id: 101, product_id: 1349, qty: 2} ] }

● { ‘item’:[ {item_id: 102, product_id: 4421, qty: 1} ] }● ...

The path to our data

● Tells us where to find the data● Is very similar to a table name

/item/quoteName of Collection

Name of Array in document

We rewire all Table names in

quote

quote_item

quote_address

/quote

/quote/item

/quote/address

$rows = $newAdapter->loadDataCollection($this->getTableName(),$this->renderFilters() );

foreach($rows as $row) {$object = new Object();$object->setData($data);$this->addItem($object);

}

The new Collection Model

loadDataCollection(‘/quote/item’, array(product_id => 1257));

● { item_id: 100, product_id: 1257, qty: 1 }● { item_id: 342, product_id: 1257, qty: 2 }● ...

db.quote.find( { ‘item.product_id’: 1257 }, { ‘item’: 1 } );

● { ‘item’:[ {item_id: 100, product_id: 1257, qty: 1} ] }● { ‘item’:[ {item_id: 342, product_id: 1257, qty: 2} ] }● ...

Loading a collection of things (filtered)

as a developer in

// loading a filtered collection of quote items from database$items = Mage::getModel(‘sales/quote_item’)->getCollection();

$items->addFieldToFilter(‘quote_id’, 560); $items->addFieldToFilter(‘product_id’, 1257);

$items->load();

Loading a collection of things (filtered)

loadDataCollection( ‘/quote/item’, array( ‘quote_id’ => 560, ‘product_id’ => 1257) );

db.quote.find( { ‘item.quote_id’: 560, ‘item.product_id’: 1257 }, { ‘item’: 1 } );

- no result

- no result

● This is not a relational database anymore● Quote Items may not have a quote_id in our schema

Loading a collection of things (filtered)

loadDataCollection( ‘/quote{quote_id:560}/item’, array(product_id=> 1257));

● { item_id: 100, product_id: 1257, qty: 1 }

db.quote.find( {‘quote_id’: 560, ‘item.product_id’: 1257}, { ‘item’: 1 } );

● { ‘item’:[ {item_id: 100, product_id: 1257, qty: 1} ] }

On-the-fly Tablename completion

/quote{quote_id:$quote_id}/item

getTablename()

/quote{quote_id:560}/item

completePath(filters, properties)

as a developer in

// load a single item from db, change qty, save it$item = Mage::getModel(‘sales/quote_item’)->load(101);$item->setQty(2);$item->save();

// add a product to cart$item = Mage::getModel(‘sales/quote_item’);$item->setQuoteId(560)->setProductId(1566)->setQty(1);$item->save();

Loading a single record

loadData( ‘/quote{quote_id:560}/item’, ‘item_id’, 100);findOne({ ‘quote_id’: 560, ‘item.item_id’: 100}, {‘item.$’: 1});

loadData( ‘/quote/item’, ‘item_id’, 100);loadData( ‘/quote{quote_id:$quote_id}/item’, ‘item_id’, 100);findOne({ ‘item.item_id’: 100}, {‘item.$’: 1});

Result for all three{ item_id: 100, product_id: 1257, qty: 1 }

Saving a single record

InsertingsaveData( ‘/quote{quote_id:560}/item’,

array(‘item_id’ => 732, ‘product_id’ => 1257, ‘qty’ => 1));db.quote.update( { quote_id: 560 }, { $push : {‘item’ =>

{ item_id: 732, product_id: 1257, qty: 1 }} });

UpdatingsaveData( ‘/quote/item’, array(‘item_id’ => 732, ‘qty’ => 2));saveData( ‘/quote{quote_id:$quote_id}/item’, ...);db.quote.update( { item.item_id: 732 },

{ $set : { item.$.qty: 2 } } );

function load($object, $id) {

$data = $adapter->loadData($this->getTablename(),$this->getIdFieldname(),$id);

$object->setData( $data );

}

The new resource model

function save($object) {

$id = $this->getNewId();$data = $adapter->saveData(

$this->getTablename(),$this->getIdFieldname(),$id,$object->getData());

$object->setId($id);}

The new resource model

create a simple application to● load using the old resource model● save using the new resource model

Migration of old data in

Model

Old Resource Model DB

App

licat

ion

New Resource Model

Thanks a lot

Firefly Glow Cube

Dining Table Campagna