29
How to easily analyse mountains of data and not pay a penny Michael Conroy Senior Analyst, Tempero @mickyconroy Please tweet #smwdatageeks Jonathan Stray Project Lead, Overview @jonathanstray

How To Analyse Data

Embed Size (px)

DESCRIPTION

Tempero and the Associated Press introduce “Overview”, a free, open source platform for extracting meaning from the torrent of social media data faced by, well, pretty much everyone in social media these days. Created by the Associated Press, Overview helps journalists find stories in large amounts of data, by cleaning, visualizing and interactively exploring large document and data sets. Turns out, the data mining techniques employed by Overview are also perfect for uncovering nuggets of insight from tweets and other types of social media data, too. Hosted by Tempero Senior Analyst Mick Conroy and Overview Project Lead Jonathan Stray, this demonstration will outline a clever, practical approach for turning data into useful insight – easily, cheaply and quickly. A great introduction to simple data mining for data journalists or analysts alike.

Citation preview

Page 1: How To Analyse Data

How to easily analyse mountains of data and not pay a penny

Michael ConroySenior Analyst, Tempero@mickyconroy

Please tweet #smwdatageeks

Jonathan StrayProject Lead, Overview@jonathanstray

Page 2: How To Analyse Data

So who are we then?

Mick ConroySnr Social Media AnalystTempero

Jonathan StrayProject Lead, OverviewAssociated Press

Page 3: How To Analyse Data

Tempero’s Clients

Page 4: How To Analyse Data

We’re all drowning in text!

Page 5: How To Analyse Data

These only go so far...

Page 6: How To Analyse Data

Please tweet #smwldn #smwdatageeks

• Journalists!

• How do you deal with WikiLeaks?

• What about Anonymous?

• The problems are very similar

It’s not just “us”…

Page 7: How To Analyse Data

Introducing Overview

overview.ap.org

Page 8: How To Analyse Data

Jonathan StrayOverview Project Lead

An overview of Overview

Page 9: How To Analyse Data

Please tweet #smwldn #smwdatageeks

• Understanding “how many” is easy

• Understanding WHAT is being said – conversation analysis - is HARD

Overview and Social?

Page 10: How To Analyse Data

Humans and Algorithms…

Page 11: How To Analyse Data

Exploring social data with Overview

Live Demo

Page 12: How To Analyse Data

An introduction to drones

Page 13: How To Analyse Data

An introduction to drones

Page 14: How To Analyse Data

Please tweet #smwldn #smwdatageeks

1. Acquire

2. Parse/Filter

3. Mine

4. Represent

5. Refine

6. Interact

7 stages of data analysis

Page 15: How To Analyse Data

1. Acquire

Page 16: How To Analyse Data

1. Acquire

Page 17: How To Analyse Data

2. Parse and Filter

interaction.link interaction.contenthttp://twitter.com/Proud_Libtard/statuse ... RT @Mikoshoes17: majority of the deaths ...http://twitter.com/FOXXXI/statuses/24734 ... nicki minaj on american idol, even more ...http://twitter.com/TFitz62/statuses/2473 ... @BarackObama U say ur foreign policy is ...http://twitter.com/NoDronesCanada/status ... Sept 28 we find out which arm company lo ...http://twitter.com/ownersmag/statuses/24 ... So GOP solutions to all foreign policy p ...http://twitter.com/AymenAhmad1/statuses/ ... @KanwalZaidi @SenRehmanMalik The stubbor ...http://twitter.com/KCruickshank1/statuse ... RT @OwenJones84: Other than propping up ...http://twitter.com/svdaveo/statuses/2473 ... RT @BuzzFeedAndrew: RT @BuzzFeedPol: Wha ...http://twitter.com/PacketknifeToo/status ... Marcus Aurelius: CIA Scrambles to Rush S ...http://twitter.com/RSIClub/statuses/2473 ... Ellip6: from military drones to the Worl ...http://twitter.com/Gertsevoa/statuses/24 ... Parrot Ar.Drone 2.0 - полетееел ...http://twitter.com/NooseCorpSE/statuses/ ... RT @OwenJones84: Other than propping up ...

text urlRT @Mikoshoes17: majority of the deaths ... http://twitter.com/Proud_Libtard/statuse ...nicki minaj on american idol, even more ... http://twitter.com/FOXXXI/statuses/24734 ...@BarackObama U say ur foreign policy is ... http://twitter.com/TFitz62/statuses/2473 ...Sept 28 we find out which arm company lo ... http://twitter.com/NoDronesCanada/status ...So GOP solutions to all foreign policy p ... http://twitter.com/ownersmag/statuses/24 ...@KanwalZaidi @SenRehmanMalik The stubbor ... http://twitter.com/AymenAhmad1/statuses/ ...RT @OwenJones84: Other than propping up ... http://twitter.com/KCruickshank1/statuse ...RT @BuzzFeedAndrew: RT @BuzzFeedPol: Wha ... http://twitter.com/svdaveo/statuses/2473 ...Marcus Aurelius: CIA Scrambles to Rush S ... http://twitter.com/PacketknifeToo/status ...Ellip6: from military drones to the Worl ... http://twitter.com/RSIClub/statuses/2473 ...Parrot Ar.Drone 2.0 - полетееел ... http://twitter.com/Gertsevoa/statuses/24 ...RT @OwenJones84: Other than propping up ... http://twitter.com/NooseCorpSE/statuses/ ...

Page 18: How To Analyse Data

3. Mine!

Page 19: How To Analyse Data

4. Represent

Pakistan D

rone St

rike

Parrot A

R Dro

ne

Surve

illance

Bradford

Protest

Haqqani

Imra

n Khan

Afganistan D

rone St

rike

US Dro

ne Enquiry

Robotic Horse

Australia

n Miss

ions

Yemen Dro

ne War

Apple Rejects App

Debates about m

orality

Israeli d

rone enters

Lebanon airs

pace

Black O

ps 2

Drone H

acking

Win a dro

ne

Drone Jo

urnalis

m

Police U

se

Immigra

tion Control

Citizen Su

rveilla

nce

Conserva

tion

BBC discusse

s priv

acy

Airspace

safety

conce

rns

Alan Sugar m

isspells

"dro

ves"

Drone as i

nsult

0

50

100

150

200

250

300

350

Page 20: How To Analyse Data

5. Refine

Page 21: How To Analyse Data

5. Refine

Page 22: How To Analyse Data

5. Refine

Page 23: How To Analyse Data

Please tweet #smwldn #smwdatageeks

• Add methods for people to manipulate the data, or control what features are visible

• More importantly...it empowers people to explore your data!

• Also good for showing off

6. Make it interactive

Page 24: How To Analyse Data

Please tweet #smwldn #smwdatageeks

• Do something with the data

• If you don’t, you’ve just made expensive graphs

• What do you know now that you didn’t know before?

Make decisions

Page 25: How To Analyse Data

Please tweet #smwldn #smwdatageeks

• Look through the rest of Datasift’s data

• Make ever more beautiful (and useful!) visualisations

What else could you do?

Page 26: How To Analyse Data

Or get in touch with us!

Page 27: How To Analyse Data

We work with everything

Page 28: How To Analyse Data

London’s Social Media Map

Page 29: How To Analyse Data

Thanks for not falling asleep. Questions?

Michael ConroySenior Analyst, Tempero@mickyconroy

Please tweet #smwdatageeks

Jonathan StrayProject Lead, Overview@jonathanstray