You Might Also Like: Implementing User Recommendations in Rails

You May Also Be Interested In:Implemen'ng User Recommenda'ons in Rails

Ma#hew Deiterstheagiledeveloper.com

Thursday, June 10, 2010

http://theagiledeveloper.com

http://theagiledeveloper.com

theagiledeveloper.com @mdeiters





How do they do that?



think about our data in new ways



Understand the tools to do something interesBng



Use those tools in your Rails applicaBon



"Everything should be made as simple as possible, but not simpler."

-‐ Albert Einstein





CreaBve RevoluBonthe Golden Age of Adver;sing



Their vision of consuming life had liIle to do with actual experience of American consumers



An Academic approach

“Adver;sers followed the wishes of conserva;ve corporate clients, who wanted the safe, “scien;fc” adver;sing



Technology Changed&

Approach Changed



In the late 40’s, only .05% of Americans had a TV

End of 50’s, sBll only about 50%

5 years later in 1962, 90% of Americans had a TV



Adver;sing was comfortable “Male dominated” “academic” approach

Audience SegmentaBon



SimilariBes in Web Development Today



NoSQLDynamic Languages



We’ve been doing CRUD for 30 yearsboring for us and the user



Our CreaBve RevoluBonHow can we change our approach



Amazon is reportedly making ~25% of sales on personalized suggesBons

Net sales are at $7.13 billion in the first quarter Jan-‐Mar 2010 -‐ roughly 2 billion in 3 months

75%

25%

RecommendaBons = Money



#1 & #2 – StumbleUpon's rank among social media traffic sources in the US.

118% -‐ the growth in acBve users since 2009.

Almost 10 Million registered users

400 – average number of Bmes a user stumbles per month.

RecommendaBons = Traffic



Discovering the relaBonships in your data

Modeling the relaBonships

Using graphs in your Rails App



Precious

Dick Cheney

President Obama ”The greatest thing about Facebook, is that you can quote something and totally make up the source.” - George Washington

John Adams lol...so true



Precious

Dick Cheney

President Obama ”The greatest thing about Facebook, is that you can quote something and totally make up the source.” - George Washington

John Adams lol...so true



http://www.hackdiary.com/2010/02/10/algorithmic-recruitment-with-github/Thursday, June 10, 2010

http://www.hackdiary.com/2010/02/10/algorithmic-recruitment-with-github/



Social Networking

Content

Website AnalyBcs

PredicBve Analysis








RelaBonal Databases != worksSQL is set based



Lady GagaKevin Bacon



Lady GagaKevin Bacon



Example: Finding 2 degrees away for a user in SQL

-‐ 100 people executes in 0.01 sec



-‐ 60K over an hour



SQL Smell

Courtsey of http://techportal.ibuildings.com/2009/09/07/graphs-in-the-database-sql-meets-social-networks/

WITH RECURSIVE transitive_closure(a, b, distance, path_string) AS( SELECT a, b, 1 AS distance, a || '.' || b || '.' AS path_string, b AS direct_connection FROM edges2 WHERE a = 1 -- set the starting node

UNION ALL

SELECT tc.a, e.b, tc.distance + 1, tc.path_string || e.b || '.' AS path_string, tc.direct_connection FROM edges2 AS e JOIN transitive_closure AS tc ON e.a = tc.b WHERE tc.path_string NOT LIKE '%' || e.b || '.%' AND tc.distance < 3)

SELECT * FROM transitive_closure--WHERE b=3 -- set the target node

ORDER BY a,b,distance

SELECT b FROM (

WITH RECURSIVE transitive_closure(a, b, distance, path_string) AS( SELECT a, b, 1 AS distance, a || '.' || b || '.' AS path_string FROM edges2 WHERE a = 1 -- set the starting node

UNION ALL

SELECT tc.a, e.b, tc.distance + 1, tc.path_string || e.b || '.' AS path_string FROM edges2 AS e JOIN transitive_closure AS tc ON e.a = tc.b WHERE tc.path_string NOT LIKE '%' || e.b || '.%' AND tc.distance = 0)

SELECT b FROM transitive_closureUNION ALL

(WITH RECURSIVE transitive_closure(a, b, distance, path_string) AS( SELECT a, b, 1 AS distance, a || '.' || b || '.' AS path_string FROM edges2 WHERE a = 4 -- set the target node

UNION ALL

SELECT tc.a, e.b, tc.distance + 1, tc.path_string || e.b || '.' AS path_string FROM edges2 AS e JOIN transitive_closure AS tc ON e.a = tc.b WHERE tc.path_string NOT LIKE '%' || e.b || '.%' AND tc.distance = 0)

SELECT b FROM transitive_closure)) AS immediate_connectionsGROUP BY b

HAVING COUNT(b) > 1;

2 degrees more then 2 degrees


http://techportal.ibuildings.com/2009/09/07/graphs-in-the-database-sql-meets-social-networks/



First Last

1

2

3

Luke Skywalker

Darth Vader

Princess Leah

How is Luke connected to Princess Leah?



First Last

1

2

3

Luke Skywalker

Darth Vader

Princess Leah




First Last

1

2

3

Luke Skywalker

Darth Vader

Princess Leah




First Last

1 Luke Skywalker

GraphsRela;onships are a First Class Ci;zen just like the data

2 Darth Vader

3 Princess Leah

Father

Father



First Last

1 Luke Skywalker

Rows Nodes

2 Darth Vader

3 Princess Leah

Father

Father

Node

Point

Actor

Vertex



First Last

1 Luke Skywalker

Edges

2 Darth Vader

3 Princess Leah

Father

Father

Edge

Rela;onship

Arc

Link



Less complex, 100% natural

Luke

PrincessDarth








In Memory Ruby GraphGreat for small staBc datasets or ad hoc querying



Rails App

RGL

RGL

Rails App

BackroundRB

RGL

or



# http://github.com/fmeyer/rgl

require 'rubygems'require 'rgl/adjacency'require 'rgl/dot'

graph = RGL::DirectedAdjacencyGraph.newgraph.add_edge 'mary', 'john'graph.add_edge 'mary', 'henery'graph.add_edge 'john', 'frank'graph.add_edge 'frank', 'henery'

# Use DOT to visualize this graph: # http://graphviz.org/Download_macos.phpgraph.write_to_graphic_file('jpg') `open graph.jpg`


http://graphviz.org/Download_macos.php










20,000 nodes and 1 million edges = 300 MB



Persistence & Dynamic Data



Neo4j



Neo4j.rb

Neo4j

Rails App

Neo4j.rbLucene



class Person include Neo4j::NodeMixin property :name has_n :friendsend

Neo4j.startNeo4j::Transaction.run do andreas = Person.new :name => 'andreas' john = Person.new :name => 'John', :age => 30 andreas.friends << johnend



Neo4jr-‐socialMore REST, Less Java

Rails App

Neo4jr-social

Neo4j

Memcache

RDBMS

SolrLucene



Neo4jr-‐socialThe SOLR of Graphs (uses Neo4jr-‐simple)

#> sudo gem install neo4jr-social

#> start-neo4jr-social



No JRuby Required

Packaged as self-‐contained jeIy webserver

Deployable WAR

Focused on SNA

Basic Built in Querying

Extensible ~/.neo4jr-‐social



Facebook’s Friend SuggesBonsDemo

Johnathan



LinkedIn’s Degrees of SeparaBonDemo

Johnathan



http://wiki.github.com/tinkerpop/gremlin/





SQLAssembly Required

Graphs (Neo4j)Works out of the Box


AllSimplePaths & ShortestPath(Who do we follow on Twiber in common)



AllSimplePaths & ShortestPath(Who do we follow on Twiber in common)


Dijkstra

score: 20 score: 5

score: 5

score: 5 score: 5

score: 5

score: 10

score: 10

(Who do we follow on Twiber in common -‐ on steroids)


Closeness Centrality(Who has the most followers on Twiber)


Betweenness Centrality

http://www.stoweboyd.com/message/its-betweenness-that-matters-not-your-eigenvalue-the-dark-ma.html

(Who has more influen;al people following them on Twiber)





http://www.hackdiary.com/2010/02/10/algorithmic-recruitment-with-github/Thursday, June 10, 2010



Eigenvector Centrality(PageRank)



Performance



Sparse vs Dense




Person

started_on => 'Jan 1 2010'finished_on => 'Jan 7 2010'hours => 40type => 'worked_on'

Project

Metadata


Person

started_on => 1265639038finished_on => 1266243897hours => 40type => 'worked_on_reorg'

Project

Metadata


Breadth or Depth



What Next...



Inspired to think about your data in new ways



GraphsUnderstand the tools to do something

interes;ng with it



Use these tools in your Rails applicaBon


NoSQL AugmentaBon

Application

Memcache

RDBMS

Neo4j

Solr

Lucene



CreaBve RevoluBon



QuesBons?



Links

•h#p://gist.github.com/431000

•h#p://github.com/andreasronge/neo4j

•h#p://github.com/mdeiters/neo4jr-‐simple

•h#p://github.com/mdeiters/neo4jr-‐social

•h#p://www.hackdiary.com/2010/02/10/algorithmic-‐recruitment-‐with-‐github/

•h#p://techportal.ibuildings.com/2009/09/07/graphs-‐in-‐the-‐database-‐sql-‐meets-‐social-‐networks/


http://gist.github.com/431000

http://gist.github.com/431000

http://github.com/andreasronge/neo4j




http://github.com/mdeiters/neo4jr-simple






http://github.com/mdeiters/neo4jr-social

http://github.com/mdeiters/neo4jr-social





