Long story short, a true tale about an hopeless DevTeam

CMF: a pain in the F

David Funaro & Alessandro NadalinMay, 14 2011

let us introduce

The Problem™

There comes the boss< there's a new project

There comes the boss

< seems easy, let's use Joomla!

Don't open that door!

The 80/20 dilemma

CMS

80% is cool at all

but as

the requirements grows

(you always knew that)

the project becomes a mess

There comes the boss< can I haz fotonic menthal scanner in the website

There comes the boss

< can't do boss Joomla! is not meant...

There comes the boss< STFU NOOB DO THAT OR BLA BLA BLA

what if

There comes the boss

< seems complex,no Joomla this time,

let's use a framework!

There comes the boss< STFU NOOB WE SHOULD GO LIVE NEXT WEEK BLA BLA

The 20/80 dilemma

Framework

20% is cool at all

the 80%is reallyfunny

but has a cost

Time To Market

again, what if

There comes the boss

< seems complex,no Joomla this time,

when should it go live?

There comes the boss< salesguy just told the client tomorrow

There comes the boss

< ok, let's use a CMF

unfortunately, there is no stableCMF in the PHP ecosystem

so it has a cost

initial development

investment=

know how (team) + quality (framework) +

velocity (that customers pretend)

=

CMF

"Our" CMF

should be able to manage a site editing pages withdifferent elements.

Easily, like a CMS.

Target: End User/Site admin

Target: Developer

should be able to plug the CMF with NO PAIN,using the kick-ass framework at the bottom of the CMF.

2things

Pages

Pages

main structure of a web site organized as a treea way to express our contentsusually composed by a series of containers ( that we'll refer to as boxes )

Boxes

Boxes

The way to separate single elements of a page

Loosely coupled HMVC structures

Entities

Boxes {Bridge}

Pages

Boxes let entities be shown inside webpages

Boxes

We dispose of different types of boxes:

CommonDSL

Common Box

Simple Input field TextareaVideoImageFile Richtext

CompositeInput field + imageinput field + image + richetextextarea + video

what about the DSL?

Domain Specific Language

DSL = ! (Website 1 Website 2 )

Event

News Users

Books Review

Page

Widget

Event

News

UsersCars

Contract

Widget

Page

Example: Identify DSL

Cars

Contract

NewsBooks

Review

Page

Widget

DSL

Web Elements = CMF Entities

back to the CMF

CMF Stack

Framework: symfony, for exampleCMF entities: Site, Pages, Menus, Users ...DSL entities: cars, contracts (for Hertz)Common Boxes : Textarea, image, video, ...Dsl boxes: Cars image gallery

Common Boxes + DSL Boxes=

The main tool for the developer to build a page with the CMF

Example : page & box composition

The Page-Box Binding Storage

Data can change its structure fastly, according to the domain model.

We need a technology supporting this fast change.

Since we have N box type, to compose a page we shouldlook for a few tables.

We need a technology supporting this kind of lookups.

so,How will we

manage the data?

RDBMS, of course

because different data always have

the same need

one size fits all

We tend to think at the way we should store data

not how we're gonna use it

and we think that the

shit-it-always-happens migration

will be easy

also if that column is useful for one and one only row

among milions

so we know we're gonna facea few well-known problems...

Data Mapping

Data Structure RDBMS

id 10

name tom

parent_id 19

we have to find a way to re-organize the data-structre to insert our tree inside a table.But is really different! Tree find a node in O(nlogn)

Data Mapping

Data Structure RDBMS

is better than before, but isn't the best way

Data Mapping

Data Structure RDBMS

... and NOW??????

?

The Alien

BUT you always find

something that as a different Behaviour

spend more time to normalize your models

Unpredictable Tomorrow

User Story evolve continous

Worse than the Alien

EAVformerly known as:

normalization WTF?!?!?!

or:how to do 70k joins to retrieve an entity of you model

Overnormalized

user_id | mail_iduser_id | phone_id

user_id | address_id

user_id | mail | phone | address

Large schema is broken into smaller ones

Overnormalized

user_id | address_id

user_id | phone_id

user_id | mail_id

...

...

...

...

...

LEFT JOIN

LEFT JOIN

LEFT JOIN

Overnormalized

Users ListLEFT JOIN

LEFT JOIN

LEFT JOIN

LEFT JOINLEFT JOIN

LEFT JOINLEFT JOIN

Overnormalized

LEFT JOIN

LEFT JOIN

LEFT JOINLEFT JOIN

LEFT JOIN

LEFT JOIN

LEFT JOINLEFT JOINLEFT JOIN

LEFT JOINLEFT JOIN

LEFT JOIN

LEFT JOIN

LEFT JOINLEFT JOIN

LEFT JOIN

LEFT JOINLEFT JOIN

What if you need to show only a single primary address?

Users List

back to the CMF

Bind Page with Boxes

Foreing Key -> TextareaBox

Bind Page with Boxes

Foreing Key -> ?

AND NOWAND NOW ???

Mantain the RI#1

Not Mantain the RI#2

Maintain or not Maintain the RI; that is the

question

we have to choose

have you ever thought

"there should beanother way!"

?

Can someone help us?

NoSQLlets try

world

Written in: C/C++ Main point: Blazing fastProtocol: Telnet-likeDisk-backed in-memory database, but since 2.0, it can swap to disk. Master-slave replication Simple keys and values, but complex operations like ZREVRANGEBYSCORE INCRValues can be set to expire (as in a cache)

$ redis-cli set mykey "my binary safe value" => OK $ redis-cli get mykey => my binary safe value

Best used: rapidly changing data with a foreseeable database size (should fit mostly in memory)

For example: analyticsreal-time data collectioncachingMQlog

http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

very fast

Object != key -> value

very fast

Object != key -> value

Not reliable

very fast

Can I haz otha nozql?

> j = { name : 'david', surname : 'funaro' };

{ "name" : "david", "surname" : "funaro" }

> db.things.save(t);

> db.things.find();

{ "_id" : ObjectId("4c2209f9f3924d31102bd84a"), "name" : "mongo" }

Written in: C++ Main point: Retains some friendly properties of SQLProtocol: Custom, binary (BSON) Master/slave replication speaks JavaScriptBuilt-in shardingPerformance over features After crash, it needs to repair tables

Best used:you need dynamic queriesyou prefer to define indexes, not map/reduce functionsyou need good performance on a big DB

For example: all things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back

http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

Document Oriented

Document Oriented

Schema Free

Document Oriented

Schema Free

Navigate Object

Can I haz otha nozql?

Written in: Erlang Main point: DB consistency, ease of useProtocol: HTTP, some say RESTBi-directional replication => master-masterconflict detection ( 409 Conflict )MVCC: write operations do not block reads Previous versions of documents are available Crash-only (reliable) design Needs compacting from time to time Views: embedded map/reducejQuery library included

curl -d '{"keys":["bar"]}' -X POST http://host:port/foo/_all_docs?include_docs=true

Best used: accumulating occasionally-changing datapre-defined queriesversioning is king

For example: CRM/CMS systemsAPI

http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

Reliable

Document Oriented

Reliable

Document Oriented

Schema Free

Reliable

Document Oriented

Schema Free

Navigate Object

Reliable

Can I haz otha nozql?

Why do we so hardlyneed a GraphDBlike OrientDB?

"GraphDBs don't avoid relations but they embrace them in a way that they are not a computational problem anymore, by making them explicit instead of implicit through joins."

Claudio Martella( http://blog.acaro.org/entry/somebody-is-going-to-hate-me-nosparql )

Easy joins

select from users where city.country.continent = 'Oceania'

Same link semantics,different objects

select links from menu >> 28:1, 17:5

Same link semantics,different objects

select from [28:1, 17:5]

28:1

ExternalLinkurl: http://www.google.com

17:5

ProductLinkid: 4

28:1

ExternalLinkurl: http://www.google.com

17:5

ProductLinkid: 4

$link->render()

Written in: JavaMain point: GraphProtocol: Binary or HTTPACIDDisk-persisted or in-memory,Cluster150k inserts per secondSchema-less, *-full, *-mixedSpeaks JSONIntegrated GUIDocumentalSQL commands

curl -X PUT '127.0.0.1:2480/document/demo/1:2' --data "{'@rid': '1:2'}"

Reliable

Graph - Documental

Reliable

Graph - Documental

Schema-*

Reliable

Graph - Documental

Schema-*

Reliable

Easy traversal

Graph - Documental

Schema-*

Reliable

Easy traversal

SQL-friendly

Graph - Documental

Schema-*

Reliable

Easy traversal

SQL-friendly

UberFast ( binary protocol )

Graph - Documental

Schema-*

Reliable

Easy traversal

SQL-friendly

UberFast ( binary protocol )

Universal ( HTTP protocol )

is so cool

but has a problem...

ver 1.0

rc1

but has a couple problems...

JAVA

but somebody

started writing the binary-protocol binding

https://github.com/AntonTerekhov/OrientDB-PHP( beta, 28 April 2010 )

and others

wrote the HTTP one

https://github.com/odino/Orient

$driver = new Orient\Http\Client\Curl();$orient = new Orient\Foundation\Binding( $driver, '127.0.0.1', '2480', 'admin', 'admin', 'demo');

$response = $orient->query("select from fellas where any() traverse(0,-1) ( @rid = 1:1 )");

$output = json_decode($response->getBody());foreach ($output->result as $friend){ echo $friend->name;

// other fun...}

you can start playing with a pure

GraphDB now

High-level PHP Component?

No

but we know that ;)

that's why in DNSEE we will build a basic

Object Graph Mapperfor OrientDB

...because...

hey, you remember about our CMF?

codename: ConGoW

codename: ConGoW

contents gone wild

( don't ask why )

ConGoW

soon the real kickstart

ConGoW

soon the real kickstart10 devs at the starting point

ConGoW

soon the real kickstart10 devs at the starting pointOSS on GitHub

ConGoW

soon the real kickstart10 devs at the starting pointOSS on GitHubStack

Symfony2

ConGoW

soon the real kickstart10 devs at the starting pointOSS on GitHubStack

Symfony2Orient

ConGoW

soon the real kickstart10 devs at the starting pointOSS on GitHubStack

Symfony2OrientDoctrine2

"My only friend, the end"

"My only friend, the end"

David Funaro@ingdavidinodavidfunaro.com

"My only friend, the end"

David Funaro@ingdavidinodavidfunaro.com

Alessandro Nadalin@_odino_

odino.org

"My only friend, the end"

David Funaro@ingdavidinodavidfunaro.com

Alessandro Nadalin@_odino_

odino.org

"My only friend, the end"

David Funaro@ingdavidinohttp://davidfunaro.com

Alessandro Nadalin@_odino_

http://odino.org

http://joind.in/talk/view/3008

Creditshttp://www.flickr.com/photos/pagedooley/5313217918/sizes/l/in/photostream/

http://www.flickr.com/photos/zoetnet/5520594473/sizes/l/in/photostream/http://www.flickr.com/photos/mhxbhd/3962410821/sizes/o/in/photostream/

http://www.flickr.com/photos/labyrinthx/1955594336/sizes/z/in/photostream/http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

http://www.flickr.com/photos/salim/19426192/sizes/o/in/photostream/http://www.flickr.com/photos/mybluevan/3780363712/sizes/l/in/photostream/

http://farm4.static.flickr.com/3064/3086258014_f1925639e0.jpg