Computing Professional Identity for the Economic Graph

Computing Professional Identity for the Economic Graph

Strata, New York – October 16th 2014

©2013 LinkedIn Corporation. All Rights Reserved.

Agenda

1 Introduction

2 LinkedIn’s Vision

3 Computing Professional Identity

4 Selected Topics

5 Summary

6 Final Words

1 Introduction



4 Selected Topics

5 Summary

6 Final Words

4

About me


Vitaly Gordon

5

About me


6

About me


7

About me


8

About me


9

About me


10

About me


@bigdatasc /in/vitalygordon

11

What’s in it for you?


12



1. You will learn about how LinkedIn takes a massive vision and breaks it down to small data problems

13




2. You will learn about how hard cleaning data can be

14




2. You will learn about how hard cleaning data can be

3. You will learn why LinkedIn needs endorsements

15

What’s in it for me?


16



17



@bigdatasc

1 Introduction



4 Selected Topics

5 Summary

6 Final Words

19©2013 LinkedIn Corporation. All Rights Reserved.

Create economic opportunity for every professional in the world



• CEO• Chief Executive Officer• CEO and Founder• CEO & Co-founder• President and CEO• Owner


• IBM• International Business Machines• International Bus. Machines• IBM Research• IBM T.J. Watson Research Center• IBM Canada• IBM India


• UCLA• University of California, Los Angeles• UC Los Angeles• The Anderson School of Management


1 Introduction



4 Selected Topics

5 Summary

6 Final Words

©2013 LinkedIn Corporation. All Rights Reserved.26

Why Do We Need Identity Standardization?







Text Based Solution

Applies acronym expansion (e.g. vp -> vice president) Applies abbreviation expansion (e.g. sr. -> senior) Select the most common standard titles Selects standard sub strings (e.g. software engineer and tech lead

in search -> [software engineer, tech lead])


Problems with a Text Based Approach

Senior Software Engineer

Software Engineer



Software Engineer Software Developer Programmer



Architect





38

Profile Inferred Skills Endorsements Skill Vectors

39


Data Mining Hadoop Machine Learning Java Algorithms Python MapReduce Data Science0%

5%

10%

15%

20%

25%

30%

40


Data Mining Hadoop Machine Learning Java Algorithms Python MapReduce Data Science0%

5%

10%

15%

20%

25%

30%

http://www.slideshare.net/s_shah/strata-endorsements

41

Ontology Creation

42

Ontology Creation

43

Classification

1 Introduction



4 Selected Topics

5 Summary

6 Final Words


Normalization

46

Normalization


Clustering

48

Clustering

• Each topic is a distribution over words• Each document is a mixture of corpus-wide topics• Each word is drawn from one of those topics

49

Anomaly Detection

50

Anomaly Detection

51

Anomaly Detection

http://www.slideshare.net/tdunning/strata-2014-anomaly-detection

1 Introduction



4 Selected Topics

5 Summary

6 Final Words

60

Summary


1. User generated content from 300M members, creates 300M problems

61

Summary



2. Data cleaning is so much more than filtering out empty values

62

Summary



2. Data cleaning is so much more than filtering out empty values

3. Try to be creative and work around difficult language problems

1 Introduction



4 Selected Topics

5 Summary

6 Final Words

We’re building the next big thing

We’re building the next big thingJoin Us

We’re building the next big thingJoin Us!

We’re building the next big thingJoin Us!

DJ Patil Gary Flake Beau Cronin

@bigdatasc /in/vitalygordon

Data & Analytics

Computing Professional Identity for the Economic Graph