David De Roure

Social Machines ofScience and Scholarship

1. The End of the Article

2. How digital research is done today

3. Social Objects

4. Social Machines

http://www.scilogs.com/eresearch/pages-of-history/

A revolutionary idea…Open Science!

http://ww

w.scilogs.com

/eresearch/pages-of-history/D

avid

De

Ro

ure

1. It was no longer possible to include the evidence in the paper – container failure!

“A PDF exploded today when a scientist tried to paste in the twitter firehose…”

2. It was no longer possible to reconstruct a scientific experiment based on a paper alone

3. Writing for increasingly specialist audiences restricted essential multidisciplinary re-use

Grand Challenge Areas:• Energy• Living with Environmental Change• Global Uncertainties• Lifelong Health and Wellbeing• Digital Economy• Nanoscience• Food Security• Connected Communities• Resilient Economy

Today’s research challenges do not respect traditional disciplinary boundaries

4. Research records needed to be readable by computer to support automation and curation

A computationally-enabled sense-making network of expertise, data, models and narratives.

5. Single authorship gave way to casts of thousands Presented by

David De RoureTechnical Adviser

Kevin PageSoftware designerDon CruickshankMusical DirectorIchiro Fujinaga

Philosophy Consultantand Catering

J. Stephen Downie

6. Quality control models scaled poorly with the increasing volume

7. Alternative reporting necessary for compliance with regulations

8. Research funders frustrated by inefficiencies in scholarly communication

An investment is only worthwhile if• Outputs are discoverable• Outputs are reusable• Outputs accrue value

1. The End of the Article

2. How digital research is done today

3. Social Objects

4. Social Machines

Chr

istin

e B

orgm

an

More people

Mor

e m

achi

nes

This is a Fourth Quadrant Talk

Big DataBig Compute

Conventional Computation

The Future!

SocialNetworking

e-infrastructure

onlineR&D

The Fourth Quadrant

F i r s t

Bio

Ess

ays,

, 26

(1):

99–

105,

Jan

uary

200

4

http://research.microsoft.com/en-us/collaboration/fourthparadigm/

The Problem

signal

understanding

http://www.bodleian.ox.ac.uk/bodley/library/special/projects/whats-the-score

Digital Music Collections

Student-sourced ground truth

Community Software

Linked Data Repositories

Supercomputer

23,000 hours ofrecorded music

Music InformationRetrieval Community

SALAMI

Ashley Burgoyne

salami.music.mcgill.ca

Jordan B. L. Smith, J. Ashley Burgoyne, Ichiro Fujinaga, David De Roure, and J. Stephen Downie. 2011. Design and creation of a large-scale database of structural annotations. In Proceedings of the International Society for Music Information Retrieval Conference, Miami, FL, 555–60

class structure

Ontology models properties from musicological domain• Independent of Music Information Retrieval research and

signal processing foundations• Maintains an accurate and complete description of

relationships that link them

Segment Ontology

Ben Fields, Kevin Page, David De Roure and Tim Crawford (2011) "The Segment Ontology: Bridging Music-Generic and Domain-Specific" in 3rd International Workshop on Advances in Music Information Research (AdMIRe 2011) held in conjunction with IEEE International Conference on Multimedia and Expo (ICME), Barcelona, July 2011

MIREX TASKSAudio Artist Identification Audio Onset Detection

Audio Beat Tracking Audio Tag Classification

Audio Chord Detection Audio Tempo Extraction

Audio Classical Composer ID Multiple F0 Estimation

Audio Cover Song Identification Multiple F0 Note Detection

Audio Drum Detection Query-by-Singing/Humming

Audio Genre Classification Query-by-Tapping

Audio Key Finding Score Following

Audio Melody Extraction Symbolic Genre Classification

Audio Mood Classification Symbolic Key Finding

Audio Music Similarity Symbolic Melodic Similarity ww

w.m

usic

-ir.o

rg/m

irex

Downie, J. Stephen, Andreas F. Ehmann, Mert Bay and M. Cameron Jones. (2010). The Music Information Retrieval Evaluation eXchange: Some Observations and Insights. Advances in Music Information Retrieval Vol. 274, pp. 93-115

Music Information Retrieval Evaluation eXchange

seasr.org/meandreMeandre

Stephen Downie

Web as lens

Web as artefact

Web as

infrastructure

Web Observatorieshttp://www.w3.org/community/webobservatory/

Scientists

TalkForum

ImageClassification

data reduction

Citizen Scientists

Information Circuits

Community Software

Execution

Digital MusicCommunity annotation

Linked Data Repositories

Workflows

Generate Paper

Conference

Notifications and automatic re-runs

Machines are users too

Autonomic

Curation

Self-repair

New research?

1. The End of the Article

2. How digital research is done today

3. Social Objects

4. Social Machines

The challenge is to foster the co-constituted socio-technical system on the right i.e. a computationally-enabled sense-making network of expertise, data, models and narratives.

Big data elephant versus sense-making network?

Iain Buchan

data

method

KnowledgeInfrastructureKnowledge

Objects

Descriptivelayer

Observatories

An

no

tati

on

http://www.myexperiment.org/

Reusable. The key tenet of Research Objects is to support the sharing and reuse of data, methods and processes. Repurposeable. Reuse may also involve the reuse of constituent parts of the Research Object. Repeatable. There should be sufficient information in a Research Object to be able to repeat the study, perhaps years later. Reproducible. A third party can start with (some of) the same inputs and methods and see if a prior result can be confirmed.

Replayable. Studies might involve single investigations that happen in milliseconds or protracted processes that take years.Referenceable. If research objects are to augment or replace traditional publication methods, then they must be referenceable or citeable.Revealable. Third parties must be able to audit the steps performed in the research in order to be convinced of the validity of results.Respectful. Explicit representations of the provenance, lineage and flow of intellectual property.

The R dimensions

Replacing the Paper: The Twelve Rs of the e-Research Record” on http://blogs.nature.com/eresearch/

used

wasGeneratedBy

wasStartedAt

"2012-06-21"

Metagenome

Sample

wasAssociatedWith

Workflow server

wasInformedBy

wasStartedBy

Workflow run

wasGeneratedBy

Results

Sequencing

wasAssociatedWith

Alice

hadPlan

Workflow definition

hadRole

Lab technician

Resultshttps://w3id.org/bundleStian Soiland-Reyes

Research Object Bundle

Research Objects

ComputationalResearch Objects

WorkflowsPacks O

AIO

RE

W3C PRO

V

Join the W3C Community Group www.w3.org/community/rosc

www.researchobject.org

1. The End of the Article

2. How digital research is done today

3. Social Objects

4. Social Machines

Nigel Shadbolt et al

Real life is and must be full of all kinds of social constraint – the very processes from which society arises. Computers can help if we use them to create abstract social machines on the Web: processes in which the people do the creative work and the machine does the administration… The stage is set for an evolutionary growth of new social engines. Berners-Lee, Weaving the Web, 1999

The Order of Social Machines

Some Social Machines

myExperiment is a Social Machineprotected by the reCAPTCHA Social Machine

“The myExperiment social machine protected by the reCAPTCHA social machine was attacked by the spam social machine so we built a temporary social machine to delete accounts using people, scripts and a blacklisting social machine then evolved the myExp social machine into a new social machine…”

Whither the Social Machine?

Kevin Page

Whither the Social Machine?

Kevin Page

Whither the Social Machine?

Kevin Page

What to observe? Logs Analytics Data findings

e.g. Success rate of transcription

Social sciences Qualitative study Motivation

Individual andgroup

Mixed methods Differences in

technique and scale Unlikely to be an simple

transferable metric

Kevin Page

Trajectories... distinguished by purpose

Trajectories through Social Machines https://sites.google.com/site/bwebobs13/

Kevin Page

Cat De Rourehttp://botornot.net

https://support.twitter.com/entries/18311-the-twitter-rules

Scholarly Machines Ecosystem

Scholarly Machines Ecosystem

Physical World(people and devices)

Building a Social Machine

Design andComposition

Participation andData supply

Model of social interaction

Virtual World(Network of social interactions)

Dave Robertson

An informal definition of a digital library is a managed collection of information, with associated services, where the information is stored in digital formats and accessible over a network

Bush, Vannevar (July 1945). "As We May Think". The Atlantic.

Correspondence Chess

http://commons.wikimedia.org/wiki/File:Postcard-for-correspondence-chess.jpg

1. The End of the ArticleStill necessary but no longer sufficient

2. How digital research is done todayNew methods, automation and more to come

3. Social ObjectsWhy papers work so well, and new artefacts are emerging

4. Social MachinesThis community knows how to help design Social Machines… and a Social Machines ecosystem

In Conclusion

• Where’s the critical reflection in the new paradigm?• Am I guilty of data fundamentalism?• User-centric, but not discussed User Experience• Object conflation – maybe computers need different

social objects?• Is this Taylorization of research?• Are we burning a paradigm into the infrastructure?

Critical thinking

david.deroure@oerc.ox.ac.uk

www.oerc.ox.ac.uk/people/dder

www.scilogs.com/eresearch

@dder

Slide credits: Christine Borgman, Iain Buchan, Ichiro Fujinaga, Kevin Page, Stephen Downie, Jun Zhao, Stian Soiland-Reyes, Nigel Shadbolt, Dave RobertsonThanks to the SOCIAM and SALAMI teams, to Carole Goble and colleagues in myExperiment, Wf4Ever, myGrid and FORCE11, to friends and colleagues in GSLIS and to students and colleagues at the DH@Ox Summer School 2013

SOCIAM: The Theory and Practice of Social Machines is funded by the UK Engineering and Physical Sciences Research Council (EPSRC) under grant number EPJ017728/1 and comprises the Universities of Southampton, Oxford and Edinburgh. See sociam.org.

Research also supported in part by Wf4Ever (FP7-ICT ICT-2009.4 project 270192),e-Research South (EPSRC EP/F05811X/1), Digital Social Research (ESRC RES-149-34-0001-A), Smart Society (FP7-ICT ICT-2011.9.10 project 600854).

http://www.slideshare.net/davidderoure/social-machines-of-science-and-scholarship

Research Objects http://www.researchobject.org/Social Machines http://sociam.org/ myExperimenthttp://www.myexperiment.orgWf4ever http://www.wf4ever-project.org Web Science Trust http://webscience.org/ FORCE11 http://www.force11.orgSALAMI http://salami.music.mcgill.ca/ MIREX http://www.music-ir.org/mirex/ Zooniverse https://www.zooniverse.org/ DPRMA http://dprma.oerc.ox.ac.uk/ W3C Community Groups:ROSC http://www.w3.org/community/rosc/Web Observatory http://www.w3.org/community/webobservatory