The National Archives cloud storage and digital preservation

Cloud Storage and Digital Preservation

Neil Beagrie (Charles Beagrie Ltd) Paul Miller and Andrew Charlesworth

Emma Markiewicz (TNA)

TNA Webinar : 13 May 2014

TNA Cloud Storage Guidance

Neil Beagrie : Charles Beagrie Ltd

Paul Miller : Cloud of Data

Andrew Charlesworth : University of Bristol

Daphne Charles : Charles Beagrie Ltd

The Consultancy Team

Webinar Overview 1. Why now?

2. The Guidance and Case Studies

3. Cloud Services and Storage

4. Legal Issues

5. Digital Preservation and Cloud Storage

6. Conclusions – potential, challenges, outcomes

Why Cloud Storage and DP (now)? • Cloud First policy 2013

– Mandated for central government, strongly recommended to rest of public sector

• Digital preservation as a strategic interest – Increased digital content– cloud as component of preservation

solutions • Emerging specialist cloud preservation services

– Increased utility of established generic services. • Pilot activities in sector and opportunities for shared learning • More options now - not just Public Cloud • Financial pressures and interest in shared services

Guidance Outline • Introduction

– Cloud storage overview, digital preservation security, legal, costs.

• Step by Step Guide – Business case, service options and providers, procurement.

• Future Developments • Current Best Practice • Further advice and guidance - annotated bibliography

– Relevant studies & standards + 5 Case Studies (linked documents)

• Legal Issues tables

Case Studies • Archives and Records Council Wales Digital Preservation

Consortium [Microsoft Azure/Archivematica]

• Dorset History Centre [Preservica Cloud edition]

• Parliamentary Archives [Public Cloud/Preservica Enterprise edition]

• Tate Gallery [Archivum OSCAR]

• University of Oxford [private cloud]

• King’s College London (DuraSpace)

Future Updates • Second Edition of Guidance to disseminate case study and

provider offering updates (Spring 2015) • Second Webinar focusing on updates (Spring 2015) • Second Edition of DPC Digital Preservation Handbook

(planned 2015-2016) • After 2015 the guidance will be integrated into the

Handbook alongside its wider treatment of digital preservation issues

• Subscribe to the digital preservation and NRA-Archives announcement email lists on JiscMail, and the TNA blog via its RSS feed to keep up to date with announcements

Cloud Services • Cloud service definition (NIST 2011)

– On-demand self-service. Users can access resources automatically as needed.

– Broad network access. Resources are accessed over the network using standard tools.

– Resource pooling. Resources are shared between users according to demand.

– Rapid elasticity. Users can easily provision or release resources as needed.

– Measured service. Use of resources is metered, and users are charged on that basis.

• “Cloud services” can have all, most, (or none!)

Cloud Storage • Cloud encompasses many different things

– Not all of them are relevant today

• Agility is (usually) the biggest benefit – Not cost (but can make savings or activity practical) – ‘Agility’ and ‘Long-term storage’ ?

• There’s a lot of hype – But some of it is true

• Cloud is not the answer to every IT problem.

It’s not just Amazon…

It’s not only in America…

Building Blocks for re-sellers…

Here be lawyers… • Risk assessment

– Legal requirements relating to management, preservation and storage of data (maintenance, data security, audit)

– Legal requirements arising from obligations to or from third parties (IPRs, data protection)

– Risks/issues relating to external service providers generally or Cloud Service technologies or services specifically

• Allocation of responsibility and risk – Development of appropriate Contractual and Service Level

Agreements to address elements of the risk assessment

Digital Preservation “..the series of managed activities necessary to ensure continued access to digital materials for as long as necessary… beyond the limits of media failure or technological [and organisational] change”

Record Lifecycles, Cloud and DP • Increasing volume (and

diversity) of digital record • Digital preservation still

often small-scale or tba • Most archives small or small

part of larger ones • Specialist digital activities

(and support for them) needed

• Cloud may help with some of them (now and in future)

Service Providers • Generalists

– e.g. Amazon, Rackspace, et al

• Specialist/Niche service providers (relevant to sector) – e.g. Preservica (Tessella), Arkivum (A-Stor and OSCAR),

Archive-It (Internet Archive), DuraCloud (DuraSpace – US only currently), Archivematica (trial cloud deployment in Wales)

Specialist Providers • Native (generalist) cloud providers will require additional

effort – From you, your contractors, or a third party like Duracloud or

Preservica • Security policies, data encryption, preservation tools, etc • Retention procedures • Integration with local management and access tools • Design for failure!

• None of this is impossible – Emerging priorities and practice? – snapshot in guidance and

case studies

Certification and Audit • Cloud Security

– ISO 27001:2013, Information security management systems — Requirements.

– Business Impact Levels (IL) – EU’s Agency for Network & Information Security (ENISA) 2013, Cloud

Security Incident Reporting

• Digital Repositories – ISO 16363: 2012, Audit and certification of trustworthy digital

repositories – Data Seal of Approval – DIN 2012, DIN 31644 Criteria for Trusted Digital Repositories

Conclusions

The Potential • Cloud services can provide professionally managed digital

storage and integrity checking • Archives can add tools, etc. tailored for digital preservation

requirements via specialist vendors • There can be potential cost savings but key benefit may be

enabling activity for smaller archives • The flexibility of the cloud allows relatively rapid and low-

cost testing and piloting of providers • There is much greater flexibility and more options in

deployment of cloud services (public, private, hybrid, community) than before

The Challenges • The Cloud is designed for flexibility and rapid change.

Archives however are long-term • Cloud can be cheaper, but it requires you to think

differently about the way budgets are managed • As in any form of outsourcing, exercise due diligence in

assessing and controlling the risks. You need to ensure legal requirements will be met

• Explicit provision must be made for pre-defined exit strategies should you need to move to another provider (or their viability changes)

Outcomes • There is much that can be learnt from archives who

have already piloted or moved to use of cloud storage

• Several archives have integrated cloud storage into their digital preservation activities

• We profile case studies in the Guidance

Further information or Follow-up

Consultancy team contact email: info@beagrie.com TNA Guidance on Cloud Storage and Preservation will be published on TNA website shortly

Questions?