Policy on digital records preservation in the NSW public sector

Cassie Findlay

Senior Project Officer, Government Recordkeeping

Key concepts

• Digital records: evidence of business in the form of emails, web pages, Word documents, scanned documents, spreadsheets etc.

• Preservation: Process and operations involved in ensuring the technical and intellectual survival of authentic records over time (AS ISO 15489 Part 1 Clause 3.14)

• Migration: involves a set of organised tasks designed to periodically transfer records from one hardware or software configuration to another, or from one generation of technology to another, while maintaining the records’ authenticity, integrity, reliability and useability’.

The preservation challenge of digital records• Many digital formats are proprietary • Even widely accepted formats cannot be read after a

few generations of technological change.• A 2005 survey showed that 38% of public offices had

some technology dependent records that were either not accessible or accessible only with some difficulty.

• State Records and public offices have a shared responsibility to preserve accessible and authentic digital records of government business.

Background

• Renewed strategic focus on digital recordkeeping and archiving + Proposed project to build State’s first digital archive

• Need for a policy basis for both public offices’ and State Records’ approaches to the preservation of digital records

Policy development timeline

• Feb-March 2007: Research into Australasian and international approaches and frameworks

• May 2007: Workshops with State Records staff• July 2007: Draft policy released for public comment

to NSW public sector and archival community• Sept-Oct 2007: Revising policy based on comments

received • Dec 2007: Policy to be submitted to State Records’

Board• 2008: Follow up with updated guidance on digital

records preservation

Policy scope

• Preservation activities undertaken by public offices on their own records eg. migrating records in an EDRMS forward; copying digital records to a different storage medium; taking inactive records ‘offline’

And

• Preservation activities that may be taken in the future by State Records in order to preserve digital State archives

Policy informed by..

• Key threats to digital records• Government’s and State Records’ requirements• Available tools and techniques for preservation

Key threats to digital records

• Loss of authenticity if they are not adequately controlled and protected

• Becoming unreadable as software applications and hardware change

• Physical deterioration of media

Loss of authenticity

• If records are not kept in official recordkeeping systems they can be vulnerable to tampering or accidental loss

• There should be an unbroken ‘chain of custody’• If a record is not linked to the business it relates to it

loses its meaning• Records that are not authentic cannot be relied upon

for legal and other purposes

Technology obsolescence• Records are dependent on the software used to

create and maintain them • Usually software companies eg Microsoft control the

software• Systems are decommissioned and organisations

purchase new software not compatible with the old• Storage media changes:

Physical deterioration of media

• Threats to digital records media include:– Magnetic fields– Temperature & humidity– Dust– Inappropriate housing or shelving– Disaster

Government / State Records requirements

• Legal requirements:– State Records Act 1998– Evidence Act 1995– Electronic Transactions Act 2000

• Size and complexity of the NSW public sector• Technologies public offices are using now• The need to preserve both State archives and

temporary value records• The maturity and ‘fit’ of available preservation

techniques

Available tools and techniques

• Digital preservation has advanced rapidly in recent years (NAA, PROV already operational)

• A collaborative effort from libraries, archives, universities and others (ADRI, APSR)

• Tools & techniques we have focused on:– OAIS model– Open standards eg ODF– Tools for normalisation and management of digital

preservation processes– Emerging techniques eg emulation to be monitored

Principle 1: Digital State records should be migrated forward as technologies change

• Monitor records to detect risk of obsolescence• Plan and manage migration projects• Migrate long term value / archival records into stable

formats• Where no migration paths are available contact State

Records to discuss alternative preservation techniques

• State Records proposes to migrate archival records to open standard formats plus XML using Xena

Principle 2: The content and essential characteristics of digital State records must remain unchanged throughout the preservation process

• Testing should be employed to check that the content and essential characteristics are not compromised by the preservation process

• The responsible public office needs to define what ‘content and essential characteristics’ means for its own records

• State Records will require this information when accepting transfers of digital State archives

Principle 3: Digital State records must be preserved in context

• Information needed to understand and use records should be linked to otherwise associated with them throughout the preservation process

• The digital records preservation process itself must be recorded

Principle 4: Digital State records must be secure and tracked throughout the preservation process.

• The preserver should implement security measures to ensure that the records being preserved are not compromised during any preservation process

• It must be possible to demonstrate an unbroken chain of custody throughout the preservation process.

Principle 5: Digital records preservation programs should be flexible

• State Records will maintain digital State archives in bitstream in addition to archival formats

• The preserver should seek to based preservation approaches on the use of non-proprietary technologies to avoid loss of control because of changed commercial arrangements

Responsibilities

• Chief executives: – ensure all records that require technology to be read or

understood remain accessible for as long as they are required (State Records Act 1998, section 14)

• Nominated senior officers: – address the preservation of digital records in organisational

policy and procedure– Ensure essential characteristics are defined prior to

preservation processes

• State Records:– Determine arrangements for the preservation and

management of digital State archives– Provide public offices with information and advice on digital

records preservation

For more information

• Contact us:– Ph: (02) 9673 1788– Email: srecords@records.nsw.gov.au– WWW: www.records.nsw.gov.au