Content Working Group

2015 NDSA Web Archiving Survey Report HighlightsNicholas Taylor (@nullhandle)

Web Archiving Service Manager

Stanford University Libraries

Archives 2016: Web Archiving Roundtable

August 3, 2016

Content Working Group

NDSA Web Archiving Survey Working Group

Jefferson BaileyInternet Archive / Archive-It

Edward McCainUniversity of Missouri

Abbey PotterLibrary of Congress

Abbie GrotkeLibrary of Congress

Christie MoffattNational Library of Medicine

Nicholas TaylorStanford University Libraries

Content Working Group

NDSA Web Archiving survey background

2011

• 78 respondents

• program info

• tools/services

• access

• policies

2013

• 92 respondents

• program info• staff, metrics, skills,

content concerns

• tools/services

• access/discovery• new options

• policies• embargo, social

media, robots.txt, resources

2015

• 106 respondents

• program info• areas of progress,

collaboration interests + barriers

• tools/services• replication targets

• access/discovery• researchers

• policies

Content Working Group

Respondent Characteristics

“Lego People” by Scoobay under CC BY-NC-SA 2.0

Content Working Group

increasing proportion of universities

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

2011 2013 2015

Content Working Group

SAA WebArch RT tops group affiliations

group 2011 2013 2015

8% 7% 8%

31% 33% 39%

45% 50%

Content Working Group

growth in staffing at the middle

.25 FTE56%.5 FTE

25%

.75 FTE0%

1 FTE5%

1 to 3 FTE7%

3+ FTE7%

2013

.25 FTE58%

.5 FTE13%.75 FTE

5%

1 FTE5%

1 to 3 FTE13%

3+ FTE6%

2015

Content Working Group

Maturity and Progress

“Apple Mouse Evolution” by raneko under CC BY 2.0

Content Working Group

shift from pilot to production (all responses)

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

Planning Pilot Production Discontinued

2011 2013 2015

Content Working Group

shift from pilot to production (longitudinal responses)

0

2

4

6

8

10

12

14

16

18

2011 2013 2015

Planning Pilot Production

Content Working Group

increased perceptions of progress

Significant progress

29%

Some progress

43%

About the same15%

Slightly worse

off1%

Much worse

off2%

Other10%

2013

Significant progress

49%

Some progress

22%

About the same19%

Slightly worse off

0%

Much worse off

0%

Other10%

2015

Content Working Group

perceived progress on data capture, appraisal, vision

40.00%

17.65%

34.12%

5.88%

45.88%

37.65%

58.82%

22.35%

10.59%12.94%

24.71%

16.47%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Content Working Group

low perceived progress on access, metadata, QA

15.85%

28.05%24.39% 24.39%

12.20%9.76%

12.20%

36.59%

21.95%

29.27%

37.80%

52.44%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

Content Working Group

Archiving Focus

“Ant Farm Media Van v.08 (Time Capsule) in Bellewether at Southern Exposure” by Steve Rhodes under CC BY-NC-SA 2.0

Content Working Group

more archiving of own content

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Own content 3rd-party content Both Other

2011 2013 2015

Content Working Group

access, cost, quality are top valued metrics

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Access + use Cost Data volume Institutionalbuy-in

Loss Quality RiskManagement

Other

2013 2015

Content Working Group

tools, appraisal, QA are top valued skills

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

2013 2015

Content Working Group

collaboration interest in QA, capture, metadata

40.91%

52.27%

31.82%

27.27%

51.14%

64.77%

46.59%

3.41%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Riskmanagement

Captureoptimization

Collectiondevelopment

APIs Metadatastandards

QA Tooldevelopment

Other

Content Working Group

“Photocopier” by Joriel "Joz" Jimenez under CC BY-NC-ND 2.0

Tools and Services

Content Working Group

more using multiple archiving solutions

25.40%

60.32%

14.29%

19.51%

63.41%

15.85%

4.81%

63.29%

30.38%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Local External Both

2011 2013 2015

Content Working Group

flat data transfer from service provider

19.15%

80.85%

20.29%

79.71%

20.27%

79.73%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

Transfer data Do not transfer data

2011 2013 2015

Content Working Group

transferring data locally + to other services

58.82%

47.06%

5.88%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Local External Both

Content Working Group

trust provider + building locally for data transfer

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

Trust data captureservice provider

Building localinfrastructure

No place tostore/maintain it

Not sure what we’d do with it once we got it

Other

2011 2013 2015

Content Working Group

Archiving Policies

“Handle With Care” by ServInt under CC BY-NC-ND 2.0

Content Working Group

more conditional handling of robots.txt

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

Always Never Sometimes/it depends Don't know

2011 2013 2015

Content Working Group

policies of other orgs key for policy-making

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

ARL Code ofBest Practices

OaklandArchive Policy

Section 108Study Group

Legal counsel Statutoryauthority

Policies ofother orgs

PreviousNDSA surveys

Other

2013 2015

Content Working Group

Landscape Summary

“Mt Baldy from Box Springs Mountain wi Theodolite” by signal mirror under CC BY 2.0

Content Working Group

overall picture

• generally positive perceptions of progress

• more moves to production

• growing proportion of universities

• growing proportion of focus on own content

• staffing largely remains fractional w/ minor growth

• majority use external service but hints of hybrid approaches

• more comfort w/ conditional policy approaches

Content Working Group

recurring themes: progress

• data capture• highest perceived progress

• 2nd highest collaboration interest

• appraisal• 2nd highest perceived progress

• 2nd highest valued skill

Content Working Group

recurring themes: opportunities

• quality assurance• highest collaboration interest

• 3rd highest desired skill

• 3rd lowest perceived progress

• access• lowest perceived progress

• tied for highest valued metric

• metadata• 2nd lowest perceived progress

• 3rd highest collaboration interest

Content Working Group

Nicholas Taylor@nullhandle

“Thank You” by vistamommy under CC BY 2.0