An Introduction to AtoM, Archivematica, and Artefactual Systems

Dan Gillean and Justin SimpsonHigher Education Archive Programme Network

MeetingLondon, UK– December 2, 2016

An Introduction to AtoM, Archivematica, and

Artefactual Systems

Who are we?

Justin SimpsonDirector, Archivematica Technical

Services

Dan Gillean, MAS, MLIS

AtoM Program Manager

Who is Artefactual?

Artefactual Systems is dedicated to producing standards-based open source web applications for digital preservation, description, and access. We are the lead developers of AtoM (for archival description and access) and Archivematica (for managing digital preservation workflows).

We are archivists, librarians, system administrators, and developers that are passionate about open-source and standards-based systems. We are also one of the most hard-working and respected technical crews in the archives and library community. We put out and support several Archivematica and AtoM releases a year and deliver technical services to a growing list of satisfied clients.

Our Team_

Our Development Priorities

Open SourceStandards-based

Generalized for broad use

Our Development Approach• Standards-based / follows best

practices• Open source / Creative Commons• Multilingual support whenever

possible• Generalize specific use cases• Include all features in public

release• Iterative development via multiple

contributions over subsequent releases

Community-based Development

Our Business Model

The “Bounty Model”

of Open Source Business

• Maintain our applications under A-GPLv3 licenses

• Provide and maintain the documentation under CC-BY-SA licenses

• Produce additional resources (e.g. videos, presentations, webinars) whenever possible

• Participate actively in the user forums

• Offer additional paid services• Always include development in

public project

Our Services• Hosting• Remote technical

maintenance• Consultation• Data migration• Training• Application theming• Software development

https://www.artefactual.com/services/

Our Applications

https://www.accesstomemory.org

Web based application for standards-based description and access in a multilingual, multi-repository environment

Web based application to support digital preservation workflows based on the OAIS reference model

https://www.archivematica.org

Our Applications

https://www.accesstomemory.org

Web based application for standards-based description and access in a multilingual, multi-repository environment

Web based application to support digital preservation workflows based on the OAIS reference model

https://www.archivematica.org

Your

Meet AtoM (hello world!)

Web based: Once installed, you only need a modern web browser and an internet connection to connect to the application from anywhere

Standards-based: AtoM implements existing national and international archival content and metadata transmission standards and best practices, so your data is standards compliant and interoperable

Multilingual: Supports translations of both content and user interface elements, can be used as a multilingual catalogue

Multi-repository: Can be used as a portal site, content aggregator, or union catalogue to provide access to content from multiple institutions

What is AtoM?AtoM stands for

Access to Memory

It is a web-based, open source application for standards-based archival description and access in a multilingual, multi-repository environment.

Web-based: platform

independentBrowser-based user interface. • Anyone with access to a browser (e.g.,

Chrome, Internet Explorer, Firefox, Safari etc.) has access to all the features and functionality of the AtoM application.

Platform independent application. • The application runs on a web server that

can be installed and run on many platforms.

Open source: free as in freedom

AtoM is built with open source tools (NGINX, MySQL, Symfony, Elasticsearch), rather than proprietary (i.e., closed-source) ones.The underlying software code of AtoM is open source.

Free to useFree to modifyFree to share

Documentation is freely availablePublic User Forum for troubleshooting and community-buildingWiki with community resources, example users, etc.YouTube video tutorials and webinar recordings

Open source: free as in freedom

Standards-based description

User-friendly content standard edit templates

Templates: ISAD(G), DACS, RAD, DC, MODS, ISAAR-CPF, ISDIAH, ISDF

Multi-lingual supportTranslate

Content and user interface elements

Multi-repository support per-institution

themingUpload logos and banners, change background colors via user interface

Archivematica IntegrationDIP upload to AtoM

Maintain link between Archivematica AIP and uploaded DIP objects in AtoM

2014

2008

2009 2010 2011 2012 2013

0.X-BETA 1.0-BETA

FIRST NON-BETA

RELEASE1.1 2.01.31.2

TRILLIUM THEME

AtoM’S DEVELOPMENT

2019

2013

2016 2017 2018

1.x 2.22.0 2.42.3JOB

SCHEDULER

2.0.1 2.1.1

20152014

2.1

2.2.1

2012

1.3.1

CLIPBOARD

AtoM 2.4• Full bulk import / export via the

user interface• Search index improvements• Authority records and

repositories on the Clipboard

AtoM’S DEVELOPMENT

AtoM

in

stal

latio

ns

1 pin per locale; only up to v2.1

You can find many more examples of AtoM sites from around the world on our wiki Users page. If you don’t see your institution’s AtoM installation listed here and would like to, please send us an email and we will be happy to add it!

https://wiki.accesstomem

ory.org/Com

munity/Users

Meet Archivematica (hello

world!)

What is Archivematica?Archivematica is a

web- and standards-based, open-source application which

allows your institution to

preserve long-term access to

trustworthy, authentic and reliable digital

content.

Standards basedOpen source

CustomizableIntegrated w 3rd party systemsActive community

2014

2008

2009 2010 2011 2012 2013

2007: UNESCO REPORT

0.1-ALPHA

DASHBOARD

INTRODUCED0.7

1.0RELEAS

ED!0.9

0.8

Bradley, K., Lei, J., Blackall, C. Towards An Open SourceArchival Repository and Preservation System (2007)

Planning and development begin. Initial Funding via UNESCO MotW Subcommittee, IMF Archives, City of Vancouver Archives

0.6-ALPHA

February 2010

May 2010

February 2011

February 2012

PREMISin

METS

0.10April 2013

August 2012 STORAGE

SERVICE 0.2January 2014

Archivematica’s development

Standards-based

ISO 14721 (OAIS)

PREMIS METS BagIT

PREMIS in METS XML

Packaged according to BagIt specifications

Virus scan, normalization report, extraction log, etc

For browsing in Archivematica

Original + normalized objects, submission docs, original metadata included at SIP creation

Archivematica AIP structure

<mets:amdSec> <mets:techMD>      PREMIS: OBJECT <mets:rightsMD>      PREMIS: RIGHTS <mets:digiprovMD>       PREMIS: EVENT <mets:digiprovMD>      PREMIS: AGENT

METS SECTIONS<metsHdr>  METS header

<dmdSec>  Descriptive metadata

<amdSec> Administrative metadata

<fileSec>  File section<structMap>  Structural Map

PREMIS in METS

Micro-services

Handshakes

Partnerships

Gears – Joe DeSousa. https://www.flickr.com/photos/mustangjoe/22711070429

Metal Handshake – Grey Geezer. https://commons.wikimedia.org/wiki/File:Metal_Handshake.jpg

Hands Passing Baton - tableatny, https://www.flickr.com/photos/53370644@N06/4976497160

Do one thing well… Inside and Out

Micro-services Architecture

Micro-services Architecture

Bulk_extractorClam AVElasticsearchExifToolFfmpegFido

FITSGearmangzipImagemagickInkscapeJHOVE

md5deepMediaInfoNFS-commonPython-lxmlSiegfriedunar

…and more…

Micro-services: Tools Used

Handshakes:

Integration not duplication

Handshakes:AtoMArchivesSpaceArchivist’s ToolkitArkivumBinder

CONTENTdmDataverseDSpaceDuraCloudFedora

HPTrimHydraIslandoraLOCKSSOpenStack

Still from the film Color Blind - Photograph by Pui Shan Chan February 2009. https://en.wikipedia.org/wiki/File:Black_%26_White_Handshake_-_Still_from_the_film_Colour_Blind_(2009).JPG

Partnerships

Building Community Services Together

archivesDIRECT• Partnership with

DuraSpace• U.S. Based• Launched August 2014• Secure storage and

monitoring via DuraCloud

• Artefactual provides AM technical support

http://archivesdirect.org/

Perpetua• Partnership with

Arkivum• U.K. Based• Launched July 2016• Secure storage and

monitoring via Arkivum• Artefactual provides AM

technical support

http://arkivum.com/perpetua/

ArchivesCANADA

Digital Preservation Service

• Partnership with The Canadian Council of Archives (CCA)

• Canada Based• Launched September

2016• Artefactual provides AM

technical support, storage, monitoring

http://archivescanada.ca/ACDPS

Some examples of how AtoM and Archivematica are used by our global communities

AtoM and Archivematica Use

Cases

AtoM Use CasesMedium to Large Deployment Model (Artefactual):

• ES and MySQL deployed on separate server• Internal read/write and testing

webservers• Separate public read-only front-

end web server – content proxied via firewall and replication script

AtoM Use Cases

http://search-bcarchives.royalbcmuseum.bc.ca/

BC Archives – Royal BC Museum

AtoM Use Cases

https://archivescanada.accesstomemory.ca/

ArchivesCanada – Canadian Portal

AtoM Use Cases

https://beatoninstitute.com/

Beaton Institute

AtoM Use Cases

https://beatoninstitute.com/

Beaton Institute

https://beatoninstitute.com/

AtoM Use Cases Beaton Institute

https://beatoninstitute.com/

Archivematica Use Cases

http://atom.archives.sfu.ca/

Simon Fraser UniversityBuilding a Digital Repository with Archivematica and AtoM

Archivematica Use Cases

http://www.sfu.ca/archives/digital-repository.html

Simon Fraser UniversityBuilding a Digital Repository with Archivematica and AtoM

Archivematica Use Cases

http://www.sfu.ca/archives/digital-repository/dr-overview.html

Simon Fraser UniversityBuilding a Digital Repository with Archivematica and AtoM

Archivematica Use Cases

http://www.sfu.ca/content/dam/sfu/archives/DigitalPreservation/DRArchitecture.png

Simon Fraser UniversityBuilding a Digital Repository with Archivematica and AtoM

Archivematica Use Cases

http://www.sfu.ca/content/dam/sfu/archives/DigitalPreservation/DRArchitecture.png

Simon Fraser UniversityBuilding a Digital Repository with Archivematica and AtoM

= Archivematica pipelines= AM Storage Service= SFU AtoM

Archivematica Use Cases Simon Fraser UniversityBuilding a Digital Repository with Archivematica and AtoM

Archivematica Use Cases Simon Fraser UniversityBuilding a Digital Repository with Archivematica and AtoM

http://www.sfu.ca/content/dam/sfu/archives/DigitalPreservation/FormatPolicyRegistry.pdf

Questions?

info@artefactual.com

RESOURCESAtoM homepage: https://www.accesstomemory.org

AtoM demo: http://demo.accesstomemory.org

Roadmap: https://wiki.accesstomemory.org/Releases/Roadmap

Documentation: https://www.accesstomemory.org/docs/latest/User forum: https://groups.google.com/forum/#!forum/ica-atom-users

RESOURCESAM homepage: https://www.archivematica.org

AM demo: http://sandbox.archivematica.org

Wiki: https://wiki.archivematica.org

Documentation: https://www.archivematica.org/docs/

User Forum: https://groups.google.com/forum/#!forum/archivematica