iSpot Analysed: Participatory Learning and Reputation

iSpot Analysed: Participatory Learning & ReputationLAK11, Banff, 28 February 2011Doug Clow and Elpida Makriyannis,

• Millions interested in nature• They watch nature TV programmes

• Vast wealth of• OER on nature• more formal learning

• How can we help people to learn?

• Lower barriers to identification• Open to all• Provide identification checking

through a social network

= social networking for natural history

• Use thrill of observing nature & sense of achievement when you identify it

• BUT identification is difficult

Take a photo

iSpot ecosystem

underpinning theory

Fairy Rings of Participation (Makriyannis & De Liddo, 2010)

general analytics

Visits to iSpot, from Google Analytics

Observations posted to iSpot, by month posted, from iSpot database

Observations posted by month observed, from iSpot database

reputation and learning

• ‘Reputation’ as proxy measure of learning• not (just) social approval

• Assessment hugely important in learning• Expensive to provide • Very hard to provide in informal learning contexts

reputation analytics

Invertebrates: Observations per user, ordered by number of observations

it’s not a power law

Invertebrates: observations per user, ordered by number of observations, log-log plot, showing power law (dotted line) with exponent of -1.3

The Learning Analytics

Cycle

• Who’s learning anything?• ‘Reputation’ as proxy measure of learning• Main feedback cycle is of ‘reputation’ to the users• What can we see in the pattern of feedback?

Invertebrates: log plot of reputation received, ranked by reputation received, showing clear discontinuity at reputation < 1.0.

Invertebrates: reputation received, log-log plot, first 618 users (reputation score 1.0), showing power law (dotted line) with an exponent of -1.4.

it’s not a power law

Invertebrates – reputation received for users with reputation < 1.0, showing logarithmic curve fit (solid line) giving y = -0.197ln(x) + 0.8317, R2 = 0.98. NB Not log plot.

it’s not a power law(but it might be logarithmic)

Invertebrates: Reputation given ordered by reputation given, log-log plot, showing power law (dotted line) with exponent of -3.6.

it’s not a power law

Agreements received against agreements given for Invertebrates, log-log plot, showing fitted power law (dotted line) with exponent 0.57 and R2 = 0.47.

Reputation received against reputation given for Invertebrates, log-log plot, showing fitted power law (dotted line) with exponent 0.345 and R2 = 0.62.

it’s not a power law

what have we learned?

• Observations and reputation received (learning) are highly unequally distributed – ‘fat tailed’

• Reputation given is even more highly unequal• experts are having an amplified effect

• Any correlation between• agreements given and received• reputation given and received

is weak, highly nonlinear, and distinct

• They’re not power laws

iSpot reputation• informal learning context• feedback is direct to other learners

• not mediated by analysts or faculty

• participation pattern typical of social software• highly unequal effect of expert opinion on reputation

• effective informal learning assessment by social networking

future work

• Adapt reputation system to other domains• More sophisticated fitting• Social network analysis• Identifying learning (e.g. reputation vs formal course)• More qualitative research

• iSpot Team: Jonathan Silvertown, Doug Clow, Richard Greenwood, Richard Lovelock, Mike Dodd, Martin Harvey, Donal O’Donnell, Jenny Worthington, Marion Edwards, Jon Rosewell, Janice Ansine, iSpot Mentors

• Photos: Mike Dodd, Jonathan Silvertown, Martin Harvey

d.j.clow@open.ac.uk@dougclowhttp://dougclow.wordpress.com