From Archives to Climate Science: a story of pirates, old weather, serendipity, data rescue, lovelorn sailors, perseverance, community values, and clearing the fog… Paula Goodale Research Associate, Information School The Secret Life of a Weather Datum @PaulaGoodale, @lifeofdata with thanks to Jo Bates (iSchool), Joan Arthur (Old Weather)
1. From Archives to Climate Science: a story of pirates, old
weather, serendipity, data rescue, lovelorn sailors, perseverance,
community values, and clearing the fog Paula Goodale Research
Associate, Information School The Secret Life of a Weather Datum
@PaulaGoodale, @lifeofdata with thanks to Jo Bates (iSchool), Joan
Arthur (Old Weather)
2. The Secret Life of a Weather Datum
www.lifeofdata.org.uk
3. Romilly Close Pi weather station designer (Aerospace
Engineering UG student) Fred Sonnenwald - Pi weather station
project co-supervisor (Civil Engineering) Sophie Rutter Usability
testing (Information School) Jo Bates, Information School,
University of Sheffield Paula Goodale, Information School,
University of Sheffield YuWei Lin, University for the Creative Arts
Dave Mee Tandot/The Garden #lifeofdata
4. The Secret Life of Data? Big data, open data, managing data,
data = oil .Lots of talk about data! Critical Data Studies Dalton
and Thatcher (2014): in both its production and interpretation, all
data big included is always the result of contingent and contested
social practices that afford and obfuscate specific understandings
of the world How to capture the complex socio-cultural shaping of
(big) data infrastructures? Where to begin? Where to end? How to
talk to people about it?
5. The Secret Life of a Weather Datum Research Questions 1.
What is the journey that weather data produced by the UKs Met
Office takes from its production through to its collation and
re-use as big weather data in different contexts? 2. What
socio-cultural values and practices are articulated in the
transformation of this data on its journey from production through
to various contexts of collation, distribution and re-use, and how
do these socio-cultural values and practices themselves transform
as they interact with the data over the course of its journey? 3.
What institutional policies and practices, and government policies
and legislation, shape the distribution and licensing of weather
data for re-use in different contexts? 4. How can the complexity of
the socio-cultural dynamics shaping the production, collation,
distribution and re-use of big weather data be communicated to a
wider audience?
6. Methods Case studies and participants Semi-structured
interviews and Observations Documentary evidence and Digital
ethnography Thematic analysis Data production Weston Park weather
station Met Office Amateur observers Old Weather Policy maker
Climate Science Met Office Hadley Centre Climatic Research Unit,
UEA Intergovernmental Panel on Climate Change (IPCC) Met Office
projects ACRE and Old Weather Financial Markets Weather Market Data
supplier [Financial markets] Policy maker Citizen Science Amateur
observers Old Weather / Archives Met Office projects - WOW and Old
Weather
7. Methods Case studies and participants Semi-structured
interviews and Observations Documentary evidence and Digital
ethnography Thematic analysis Towards ethnography Site visits
Multiple interviews Observation Field notes Photographs Video
Interviews - core themes Own role, data practices Motivations and
challenges Relationships with other people and organisations
External environment e.g. economic and policy issues Tailored to
individuals Observations working environment Communal and personal
space Branding and artefacts Personal interactions Technology
8. Methods Case studies and participants Semi-structured
interviews and Observations Documentary evidence and Digital
ethnography Thematic analysis Extending coverage and filling in
gaps Official records Access restrictions Discourse Community
Documentary evidence Reports Policy documents Research papers Web
sites Corporate publications Digital ethnography Twitter IPCC
conference Online forums Citizen scientists
9. Methods Case studies and participants Semi-structured
interviews and Observations Documentary evidence and Digital
ethnography Thematic analysis Coding Manual 2-3 coders Coding
themes Attitudes, values and beliefs Valuable activities and
practices Social relationships The data journey Working papers One
per case Analysis by coding themes By organisation Cross-case
analysis Web site Research papers
10. lifeofdata.org.uk #lifeofdata
11. Old Weather Case Study: Data collection 1x interview ACRE
project 1x interview Met Office climate scientist 1x interview
Climate historian 2x interviews Old Weather members Content /
discourse analysis of Old Weather forum Content analysis of
documents and web pages
12. Old Weather: Climate Science Challenges To improve climate
models you need more data, over longer time periods, from more
locations I usually say to people if youve got a weather
observation, if its south of the equator, well take it. Right. If
its north of the equator before about 1950 well take it. If its
north of the equator, outside 1950, and its not in North America,
Europe Japan, or the North Atlantic were probably still interested.
[MO_03]
13. Pirates ACRE project Atmospheric Reconstructions of the
Earth A loose network of like-minded climate scientists engaged in
data recovery projects I often think about you know, the
organisations work a bit like the Royal Navy, we work like Pirates
of the Caribbean. [MO_06] Its very difficult to get funding for
data recovery because people want scientific results. A lot of the
funding bodies or people who make these decisions dont realise that
unless youve got the data, youre not going to do any science, but
they dont want to fund the recovery. As soon as you say data
rescue, data recovery, they switch off. [CRU03]
14. A Mission Challenge: knowing whats available and where its
located I compiled an inventory of every major Royal Navy vessel
from 1800 to 1950, where it was, what its movements were, where its
logbook is, and I also compiled reports for them, which are
available, concerning logbooks generally, where they are in the
archives, what they consist of, what some of the problems are in
using them, this sort of thing. [CRU_03]
15. Serendipity My favourite research method is serendipity
[CRU_03]
16. When it says miscellaneous in my view its because its got
lots of stuff with numbers on and they didnt know what to do with
it. [CRU_03]
17. Data Rescue and Recovery Challenge knowing the data Well,
my expertise is finding the data, knowing how it was collected,
knowing some of the problems of the, what Ill do is Ill look
through the logbooks, and Ive got a lot of experience, Ive looked
at thousands of them, and I anticipate the problems theyre going to
have with the data. And sometimes I say, Youre going to have this
problem. This is how you overcome it. [CRU_03]
18. Data Rescue and Recovery Challenge - records may not be
kept in ideal conditions it shows the stuff in archive is just
piled up and its all mouldering. I didnt actually go there but it
was--, its horrendous. Things are just falling to pieces and its
just piled up on the shelves and its all Its irreplaceable.
[CRU03]
19. Digitisation and Preservation and the archivist looks at
them and says, How good condition is this record? You know, Can we
just photograph it, or do we need to repair it first? Is it going
to fall to pieces on us? Does it need curating? So they do the
curating as necessary, and then we try and photograph it, and that
requires a bit of human expertise because the quality of the ink
after 150 years is not always great. Okay. But usually we get
fairly legible photographs out of it. [MO_03]
20. Old Weather: Transcribing the logs Process Select a ship
Transcribe the log book Weather data Notes Each one transcribed at
least 3 times Cross-check for errors Data sent to Met Office for
processing Challenges Hand-writing! Place names Can be dull /
repetitive
21. There are some things computers cannot do well From 4 to 6
light breezes from the S x W and cloudy - latter part light rain
Air 80. Water 82
22. TWYS Type What You See Typed records are a rare treat!
23. Lovelorn Sailors Not just numbers inventories, events,
stories Attachments formed to historical people A rich source of
community engagement Edited notes are captured for future reference
Sharing with historians www.naval-history.net
24. Our interest in our subject, our own humanity for our
scribes and for each other. Produces a powerful community, a
family. [Joan Arthur, Old Weather]
25. Community I think the sense of contributing to something
that I care about, but also definitely the forum. The forum is
massively important. [OW_02] One of the other threads is Signs of
Old Weather Addiction [OW1] Helping with transcriptions Sharing
stories, fun and games Personal interactions, keeping up morale
Recognition and information from the scientists is also valued
26. Motivations I think it is, yeah, you know it's the weather,
it's the history, and it's the forum I think for me are the sort of
the three key important things that have sort of kept me interested
in it really. [OW_02] I think initially it was because of the whole
thing to do with climate change. Because obviously, I've got a
concern for the planet, you know, and the way that we treat it and
so on. And so actually doing something that was in some way going
to contribute to that kind of research attracted me. [OW_01]
29. Data Processing Transcribed data is sent to the Met Office
Transformed into a standard IMMA data format Checked for errors
Uploaded to ICOADS data centre for access by climate scientists
across the world Primarily used in reanalysis projects Reanalysis
is a scientific method for developing a comprehensive record of how
weather and climate are changing over time. In it, observations and
a numerical model that simulates one or more aspects of the Earth
system are combined objectively to generate a synthesized estimate
of the state of the system. A reanalysis typically extends over
several decades or longer, and covers the entire globe from the
Earths surface to well above the stratosphere.
30. Old Weather Data in Climate Science Video: National
Maritime Museum, www.oldweather.org
http://player.vimeo.com/video/15388983
31. My job is fog removal But the yellow region that's where
because we did our citizen science Old Weather Project, and we got
new weather observations, there now isn't as much fog as there used
to be. So the yellow area, I call that the glow of discovery.
That's where there isn't any fog any longer. [MO3] Video:
https://vimeo.com/99917882
32. Other Archival Data Climatic weather stations - Weston
Park, Museums Sheffield - More than 130 years of weather data -
Collected and preserved by generations of museum curators -
Archived by the Met Office and by the Museum - Used in climate
science - Available for research - Public good data and weather
station belongs to the people of Sheffield
33. Other Archival Crowdsourcing Projects Transcribe Bentham
BBC World Service archive TNA Operation War Diary
http://helpinghistory.com/ Crowd-sourcing 101
34. References Dalton, C. & Thatcher, J. (2014). What Does
a Critical Data Studies Look Like and Why Do We Care? Seven Points
for a Critical Approach to Big Data. Society & Space [Online].
Available:
http://societyandspace.com/material/commentaries/craig-dalton-and-jim-
thatcher-what-does-a-critical-data-studies-look-like-and-why-do-we-care-seven-points-
for-a-critical-approach-to-big-data/ Eveleigh, A. M. M., Jennett,
C., Blandford, A., Brohan, P., Cox, A. L. (2014). Designing for
dabblers and deterring drop-outs in citizen science. CHI'14:
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems. ( pp.2985-2994). New York, NY, USA: ACM Eveleigh, A.
(2014). Crowding out the archivist? Locating crowdsourcingwithin
the broader landscape of participatory archives. In Ridge, M.
(Ed.).Crowdsourcing our cultural heritage ( ). Ashgate. Eveleigh,
A., Jennett, C., Lynn, S., Cox, A. L. (2013). "I want to be a
captain! I want to be a captain!": Gamification in the Old Weather
citizen science project.Gamification'13: Proceedings of the First
International Conference on Gameful Design, Research, and
Applications. ( pp.79-82). New York, NY, USA: ACM Press Stebbins,
R. A. (1996). Volunteering: A serious leisure perspective.
Nonprofit and Voluntary Sector Quarterly, 25(2), 211-224.
35. Web sites Life of Data: http://www.lifeofdata.org.uk/ Life
of Data research data archive:
https://archive.org/search.php?query=secret%20life%20of%20
a%20weather%20datum Old Weather: http://www.oldweather.org/ Old
Weather Forum: http://forum.oldweather.org/ 130 years of Weston
Park Weather Station:
http://www.museums-sheffield.org.uk/blog/2012/9/130-years-
of-weston-park-weather-station
36. Thanks! Paula Goodale @PaulaGoodale,
[email protected] The Secret Life of a Weather Datum
@lifeofdata, www.lifeofdata.org.uk