Human Rights Archives and Documentation, CHRDR Conference 4-6 October 2007

Issues in Human Issues in Human RightsRights

Web ArchivingWeb ArchivingRobert WolvenRobert Wolven

Columbia University LibrariesColumbia University Libraries

Libraries have a mission to build, Libraries have a mission to build, organize, and preserve coherent organize, and preserve coherent collections for researchcollections for research

There’s a great deal of human rights-There’s a great deal of human rights-related content on the webrelated content on the web

Much of it is not currently collected Much of it is not currently collected by librariesby libraries

Something should be done about thatSomething should be done about that

A great deal of content exists only A great deal of content exists only onlineonline

There’s a high risk that some will There’s a high risk that some will disappeardisappear

Libraries and archives are custodians Libraries and archives are custodians of our cultural heritageof our cultural heritage

Libraries and archives should lead in Libraries and archives should lead in preserving “at risk” contentpreserving “at risk” content

Web Archiving as Web Archiving as PreservationPreservation

Small footprint in organizationSmall footprint in organization

The Hoover effectThe Hoover effect

Haphazard library collectionsHaphazard library collections

Ineffective accessIneffective access

A Lot of ContentA Lot of Content

Much of it is not … Much of it is not … collectedcollected

Refugees InternationalRefugees International 40 documents on web site40 documents on web site

0 in Columbia collections0 in Columbia collections

10 listed in OCLC10 listed in OCLC 1 held by more than 2 libraries1 held by more than 2 libraries No library holds more than 3No library holds more than 3

Web Archiving IssuesWeb Archiving Issues

Ways and MeansWays and Means Selection policiesSelection policies Permissions – and ObligationsPermissions – and Obligations Organization and IntegrationOrganization and Integration Presentation and UsesPresentation and Uses Sharing the Costs and BenefitsSharing the Costs and Benefits Organizational TransformationOrganizational Transformation

Center for Research Libraries’ Center for Research Libraries’

Political Communications Web Archive Political Communications Web Archive ProjectProject

Project website: Project website: http://www.crl.edu/content/PolitWeb.hthttp://www.crl.edu/content/PolitWeb.htmm

Final report: Final report: http://www.crl.edu/PDF/PCWAFinalReport.pdfhttp://www.crl.edu/PDF/PCWAFinalReport.pdf

Web Archiving ToolsWeb Archiving Tools

Archive-It (Internet Archive) Archive-It (Internet Archive) http://www.archive-it.orghttp://www.archive-it.org

PANDAS (National Library of PANDAS (National Library of Australia)Australia) http://http://pandora.nla.gov.au/pandas.htmlpandora.nla.gov.au/pandas.html

OCLC Digital ArchiveOCLC Digital Archive http://www.oclc.org/digitalarchive/defaulhttp://www.oclc.org/digitalarchive/defaul

t.htmt.htm

““I only want to download I only want to download text/html and nothing else. text/html and nothing else.

Can I do it?”Can I do it?” You can … add a filter that excludes You can … add a filter that excludes

all filters that end in other than 'html|all filters that end in other than 'html|htm', etc., or, if you want to instead htm', etc., or, if you want to instead look at document mimetypes, you can look at document mimetypes, you can Add a ContentTypeRegExpFilter filter Add a ContentTypeRegExpFilter filter as a midfetch filter to the http as a midfetch filter to the http fetcher.fetcher.

[From Heritrix FAQ [From Heritrix FAQ http://crawler.archive.org/faq.html#user-heritrix]http://crawler.archive.org/faq.html#user-heritrix]

Policy Policy Technology Technology

Full web site or selected content?Full web site or selected content?

Preserve relationships, “look and Preserve relationships, “look and feel”?feel”?

All file types?All file types?

How often?How often?

From the European Union Agency for Fundamental Rights website

“CorrigendumChanges have been made to the report on

"Muslims in the European Union – Discrimination and Islamophobia" after it was printed.

Following pages are replacing the Annex page in the EN and FR version of the report. PDF”

Selection by Type of Selection by Type of AgencyAgency

GovernmentalGovernmental

InternationalInternational

AcademicAcademic

EducationalEducational

Selection by FocusSelection by Focus

Global, regional, localGlobal, regional, local

Ethnicity, religion, gender, ageEthnicity, religion, gender, age

Legal, medical, economicLegal, medical, economic

Crisis-drivenCrisis-driven

Selection by ContentSelection by Content

Fixed documents:Fixed documents: Case studiesCase studies Position papersPosition papers Topical reportsTopical reports Press releasesPress releases Bulletins, newslettersBulletins, newsletters Activity reportsActivity reports

Selecting by ContentSelecting by Content

Non-textual (image, sound, video)Non-textual (image, sound, video)

Ephemeral, dynamic contentEphemeral, dynamic content

Redundant (?) content:Redundant (?) content: Languages, formatsLanguages, formats Republished or unique?Republished or unique?

Rights and ObligationsRights and Obligations

Permissions: ask or assumePermissions: ask or assume Rights:Rights:

Dark archiveDark archive Closed archiveClosed archive Conditional exposureConditional exposure

Obligations:Obligations: Parallel (mirror) accessParallel (mirror) access Free, reliable accessFree, reliable access Perpetual access Perpetual access

Organization and Organization and IntegrationIntegration

… … or, now that we have it, how do we or, now that we have it, how do we know what we’ve got? know what we’ve got?

How do other agencies know what’s How do other agencies know what’s been done?been done?

How do researchers find it?How do researchers find it?

“From 1 March the European Monitoring Centre on Racism and Xenophobia (EUMC) became

the EU Agency for Fundamental Rights (FRA). The content on the website is being gradually transformed to reflect the scope, activities and

products of the new Agency.”

Integrating AccessIntegrating Access

Through Authority ControlThrough Authority Control

Through controlled vocabularyThrough controlled vocabulary

Through seriesThrough series

Integrating CollectionsIntegrating Collections

With print – in the catalogWith print – in the catalog

With archives – in finding aidsWith archives – in finding aids

With digital collections – in …With digital collections – in …

UseUse

Internal organization and navigationInternal organization and navigation

IndexingIndexing

Analytical toolsAnalytical tools

Citation: pedigree and persistent Citation: pedigree and persistent linkslinks

Sharing Costs and Sharing Costs and BenefitsBenefits

Centralized CollaborationCentralized Collaboration

Distributed CollaborationDistributed Collaboration

Disclosure (at what level of detail)Disclosure (at what level of detail)

Exposure (to the web; OAI-PMH)Exposure (to the web; OAI-PMH)

Transformative ActionTransformative Action

Concept of “collecting”Concept of “collecting”

Modes of selectionModes of selection

Bridging communities of practiceBridging communities of practice

““Where do you Where do you stop?”stop?”