10
Where is my tweet? Henok Mengistu Insight Data Engineering Fellow Silicon Valley, Summer 2016

Insight dataengineering henok_rehearsaldemo

Embed Size (px)

Citation preview

Page 1: Insight dataengineering henok_rehearsaldemo

Where is my tweet?Henok Mengistu

Insight Data Engineering Fellow

Silicon Valley, Summer 2016

Page 2: Insight dataengineering henok_rehearsaldemo

Motivation

Page 3: Insight dataengineering henok_rehearsaldemo

Motivation

But, this number doesn't show how the tweet spreads-out?

Page 4: Insight dataengineering henok_rehearsaldemo

But, a re-tweet graph could show

Page 5: Insight dataengineering henok_rehearsaldemo

A Demo

http://52.33.140.25/http://www.whereismytweet.online/

Page 6: Insight dataengineering henok_rehearsaldemo

Under the hood

Page 7: Insight dataengineering henok_rehearsaldemo

Engineering Challenges

● Stitching the different components ● Re-tweets could arrive out of order

– Spark can't sort across a data stream

– The driver node should collect and sort re-tweets

Page 8: Insight dataengineering henok_rehearsaldemo

● I am Henok– Originally, from Ethiopia

– Currently, a PhD student at the University of Wyoming

● Working on Evolutionary Computation● I was also working as a Teaching assistant

– I like soccer, but not skiing

Page 9: Insight dataengineering henok_rehearsaldemo

Thank you!

Page 10: Insight dataengineering henok_rehearsaldemo

Queries

● On the re-tweet graph

– who are my audiences? ● Geographically, social groups

– Betweenness centrality ● Who is relevant to spread out my tweet?● Identify influential followers