Final Report for Defra Project SP08003

Improving soil data use across government

Paul Newell Price, Chris Procter, Steven AnthonyADAS UK Ltd.

Alex Sinclair and Audrey LitterickSAC Commercial Ltd.

Crawford JordanDepartment for Agriculture and Rural Development for Northern Ireland

February 2006

EXECUTIVE SUMMARYThe main aims of this project were to provide detailed information on the soil data used by Government Departments in the UK, and to investigate the potential for providing soil dataset metadata for integration into Defra’s SPIRE cataloguing tool.

The report sets out the results of interviews with the users of soil information within Government Departments and Agencies across the UK and of a survey of existing soil dataset attributes and metadata standards held by soil data providers.

The interviews have revealed that some UK Departments and Agencies (or contractors on their behalf) use soil datasets as an integral part of their work and on a regular basis. These include Defra Water Quality Division, Countryside Council for Wales, Environment Agency, Scottish Natural Heritage and Northern Ireland Department for Agriculture and Rural Development. However, there are many that only use one or two datasets or restrict their use to the general information provided by MAGIC (Multi-Agency Geographic Information for the Countryside). The used datasets are often those that have been funded by the Department or Agency in question (e.g. European Wildlife Division, English Nature in the case of the Environment Change Network).

There are a number of drivers that are increasing the requirement for soil data, including the Soil Action Plan for England, the Water Framework Directive and the Habitats Directive. However, most Departments and Agencies have not investigated the possibility of gaining access to additional soil datasets that may be useful to them.

Due to the recent increased profile of soil and the need for strategies to protect soil, many UK Government Departments and Agencies anticipate a greater need to access soil data in the future.

At present, many departments are not aware of what soil datasets are available. However, this project in itself has increased awareness of existing soil information. The integration of soil metadata into SPIRE (the Defra Spatial Information Repository) should further improve the ability of Departments and Agencies to assess which soil datasets are fit for their purposes.

In order to facilitate ease of access to information, and as an extension to this project, a metadata database will be produced to enable the attribute metadata collected under the project to be queried. The tool will store the data in a relational fashion that will allow the user to interrogate the whole database using simple, pre-defined free text and fixed queries. Users will be able to search the database by a number of attributes to determine which datasets the attribute of interest resides in and whether a particular dataset will meet their requirements in terms of spatial resolution, frequency of sampling and other factors.

We anticipate that populating the SPIRE cataloguing tool will be an incremental process. The number of datasets within SPIRE and the level of detail will increase as new datasets and new metadata on existing datasets become available. It should also be possible to include information on the sub-sets of existing datasets as this is produced.

2

TABLE OF CONTENTS

1 INTRODUCTION............................................................................................................................ 7

2 PROJECT OBJECTIVES...............................................................................................................8

3 APPROACHES.............................................................................................................................. 83.1 CURRENT SOIL DATA USAGE..........................................................................................................83.2 FUTURE REQUIREMENTS...............................................................................................................9

4 SOIL DATA USE.......................................................................................................................... 114.1 ENGLAND AND WALES – SOIL DATA USE AND FUTURE REQUIREMENTS..........................................11

4.1.1 3.2.1 England and Wales - overlaps in current holdings and potential for data sharing....144.1.2 England and Wales – additional soil and complementary datasets..................................15

4.2 SCOTLAND - SOIL DATA USE AND FUTURE REQUIREMENTS............................................................174.2.1 Scotland - overlaps in current holdings and potential for data sharing..............................194.2.2 Scotland – additional soils and complementary datasets..................................................19

4.3 NORTHERN IRELAND - SOIL DATA USE AND FUTURE REQUIREMENTS..............................................204.3.1 Northern Ireland - overlaps in current holdings and potential for data sharing..................244.3.2 Northern Ireland complementary datasets........................................................................24

5 LICENSING AND ACCESS ISSUES...........................................................................................245.1 DEFRA....................................................................................................................................... 245.2 OTHER DEPARTMENTS AND AGENCIES IN ENGLAND AND WALES...................................................255.3 SCOTLAND LICENSING AND ACCESS.............................................................................................255.4 NORTHERN IRELAND LICENSING AND ACCESS...............................................................................26

6 COMPLEMENTARY DATASETS................................................................................................26

7 DESCRIPTIVE SOIL METADATA...............................................................................................277.1 SPIRE METADATA......................................................................................................................297.2 SURVEY OF EXISTING SOIL DATASET ATTRIBUTES..........................................................................30

8 CONCLUSIONS........................................................................................................................... 32

TABLES

Table 1. Government Departments and Key Contacts..........................................................10

Table 2a. England and Wales Departments and Agencies responses to the question of whether they have access to soil datasets.............................................................................12

Table 2b. England and Wales Departments and Agencies that have access and would like access to soil datasets............................................................................................................13

Table 3. England and Wales Government Departments and Agencies access to complementary datasets.........................................................................................................16

Table 4. Scotland Government Departments and Agencies responses to the question of whether they have access to soil datasets.............................................................................17

Table 5. NI Departments (NI executive - those consulted are shown in bold).......................20

Table 6. Northern Ireland Departments and Agencies responses to the question of whether they have access to the main GB soil datasets......................................................................21

Table 7. Northern Ireland Departments and Agencies responses to the question of whether they have access to NI specific soil datasets.........................................................................22

3

Table 8. Northern Ireland Government Departments and Agencies access to complementary datasets..................................................................................................................................23

Table 9. Defra RDS Climate data – Met Office data at 5 km interval....................................27

Table 10. Derived data at 1 kilometre interval (ESRI shapefiles). Interpolated from the 5 km climate variables.....................................................................................................................27

Table 11. Derived data at 50 metre interval (ESRI grid). Interpolated from the 5 km climate variables using OS panorama as the source for altitude........................................................27

Table 12. Scotland Departments and Agencies access to complementary datasets.............28

Table 13. SPIRE descriptive metadata mandatory and optional elements............................29

Table 14. DARDNI extract from the data collection exercise...Error! Bookmark not defined.

APPENDICES

Appendix A - Contact Details

Contact details of all the people consulted within the UK Government Departments and Agencies approached through the course of this project.

Appendix B - Current and future soil data use within UK Government Departments and Agencies

Provides specific information on the current and desired use of soil datasets within each Government Department and Agency. The Appendix also includes a list of the Regulations and Strategies that drive the need for soil data within each Department and a list of soil-related projects where appropriate. These are provided as part of the business justification for requiring soil data. Where licensing is an issue, this was also presented for each Department.

Appendix C - Function Matrices for UK Government Departments/Agencies

Provides information on Government Departments/Agencies perception as to what soil data they hold, what they have access to and what they would like to access. It also matches specific soil datasets to the soil functions that the dataset is used to address.

Appendix D - Defra, CCW, SNIFFER, DARD and EHS-NI Projects that have used soil data

Provides an additional inventory of soil-related projects where a greater amount of information was provided than could be included in Appendix B.

4

ACKNOWLEDGEMENTSWe would like to thank the following for their help with the gathering of information for this project:

England and Wales

David Richardson, Defra, Flood ManagementDr Zoë Davies, Defra, Sustainable Farming and Food Science Division Mrs Julie Holloway, Defra, National Land & Grazing Management TeamRoger Unwin, Defra, Technical Advisory UnitHeather McBeth, Government Office South West, Food and Rural Affairs Team Nick Blakey, Defra, Waste Strategy DivisionDr Soheila Amin-Hanjani, Defra, Head of BranchTim Williamson, Defra, Air & Environmental QualityHelen Pontier, Defra, European Wildlife DivisionAngus Huntley, Department for TransportJoby Blume, Department for Trade and IndustryCarol Hrynkiewicz, Office of the Deputy Prime MinisterPeter Bide, Minerals and Waste Planning DivisionNick Duncan (deceased), N A Duncan & AssociatesJane Morris, Environment Agency, Policy Manager - SoilJonathan Burney, English NatureGeorge Hinton, English NatureSebastian Payne, English Heritage, Research and StandardsJen Heathcote, English HeritageAlice Froggatt, English Heritage, Corporate GIS Co-ordinatorColin Pritchard, Countryside Agency, Landscape Access and Recreational DivisionDylan Ll. Williams, Cyngor Cefn Gwlad Cymru / Countryside Council for WalesIan Rugg, Welsh Assembly Government, Department for Environment Planning & CountrysideSimon Price, Head of Environment Policy Keith Maddison, Ministry of Defence – Environment Support TeamMark Hill, Ministry of Defence – Construction Support TeamT.R.E.Thompson, Cranfield University, National Soil Resources Institute, Timothy S. Farewell, Cranfield University, National Soil Resources Institute, Dr. Jean Alexander, British Geological Survey Head, Intellectual Property Rights Section Dr Helaina IJ Black, ECOPRO, Head of Soil Ecology Group

Scotland

Dr Antje Branding and Dr Geeta Wonnacott, Scottish Executive Air, Climate and Engineering Division, Air, Climate and Engineering DivisionDr Patricia Bruneau, SNH Soil Adviser Mark Aitken, Scottish Environment Protection Agency, Fiona McTaggart and Vanessa Kind, SNIFFERDr Helen McKay, Forestry Commission

Northern Ireland

Hugh McGinn, Regional Development ; Water ServiceJim Hayes, Department of Culture Arts and LeisureSeamus McErlean, Department of Agriculture and Rural DevelopmentJim Sutherland, DARDDr Crawford Jordan, Department of Agriculture and Rural Development

5

Myles Patton, Agricultural and Food EconomicsArchie K. Murchie, Applied Plant Science DivisionBrian Solan, HPTODarrell Abernethy, Department of Agriculture and Rural DevelopmentMr. Derek Reay, Geological Survey of Northern IrelandHelen Murphy, EHS: Built Heritage, Protecting Historic MonumentsWendy Mckinley, Environment & Heritage Service: Water Management Unit Doe NiCatherine Porter (2nd: Damian McFerran, CEDaR Manager), CEDaREugene Murray, Environment & Heritage Service - Natural Heritage DirectorateRichard Cole, Rivers Agency Maurice A McCoy, Department of Agriculture and Rural Development, N Ireland

6

1 INTRODUCTION

Action 13 of the First Soil Action Plan for England states:

Defra will work with other Government Departments and Agencies (including in the Devolved Administrations), the National Soil Resources Institute at Cranfield University (as co-owners of key datasets) and other soil users, to develop and provide better access to information on soil.

The first milestone of this action was an internal review of current arrangements for Government access to soil information. A questionnaire-based survey established a baseline of information on the types of soil data licensed by Government Departments and Agencies. However, insufficient information was obtained to allow a strategy to be developed for improving the awareness of soil information and to facilitate data sharing. Specifically, there was not a common and higher level description of the data that could effectively communicate fitness for purpose of the datasets that are available in a variety of formats and different spatial scales.

Defra identified the need to improve soil data use across Government and to develop a cataloguing tool for integration into SPIRE (Spatial Information Repository) to improve awareness and enable easier access to these data. SPIRE is a Government initiative to serve core spatial datasets to Defra and partners from a central repository, with an expected fully functional delivery date of March 2007.

What is SPIRE?

“The SPIRE programme has a vision to implement the Defra Geographic Information [GI] Strategy, aiming to improve the quality and accessibility of geographic information leading to the wider use of GI within the Defra family, to enable better performance towards strategic outcomes and to support evidence based policy making and delivery”

The SPIRE Programme Vision, 2005

There are a number tasks that relate to the improved use of soil data by UK Government Departments and Agencies. The first task is to assess the extent to which soil data are being used within Government Departments and determine what soil issues and functions are being addressed by this use. This information can then be used to assess which datasets may be useful to Departments in the future. It is also necessary to obtain detailed information on the soil datasets themselves. This is best obtained, where possible, from the soil data providers, along with information on complementary datasets. By linking soil information with complementary datasets, users should be in a better position to quantify the varied and important functions provided by soil. It is then necessary to determine the extent to which available soil metadata can be integrated into SPIRE to enable users to determine the fitness for purpose of soil datasets. This involved an assessment of whether useful soil metadata exists and whether it is in a suitable form for integration into SPIRE. Key issues are first of all whether the degree of detail required is available, and secondly achieving the best trade-off between simplification and preservation of the detail required by expert users of the data.

This current project aims to address these tasks.

7

2 PROJECT OBJECTIVES

The specific project objectives were to:

i. Obtain detailed information on the soil data used within UK Government Departments and Agencies or licensed to contractors on their behalf;

ii. Identify the current and future requirements of the UK Government Departments and Agencies for soil data and present their business justification for these requirements;

iii. Identify the extent of overlaps between current holdings to clarify the extent to which information needs from within Government can already be met if there were an appropriate mechanism for information sharing;

iv. Provide details on complementary datasets potentially available to fulfil additional requirements and fill gaps in data availability;

v. Investigate the potential of developing descriptive soil metadata for integration into SPIRE through the process of a survey of existing soil dataset attributes and metadata standards.

3 APPROACHES

3.1 Current soil data usageA total of 30 Government Departments and Agencies across England, Wales, Scotland and Northern Ireland were interviewed over the course of the project. Table 1 presents the Departments and Agencies and the principal representatives interviewed. Contact details for each of the representatives are set out in Appendix A.

In England and Wales, Government Departments and Agencies were asked whether they use or have access to the following soil datasets: LandIS - includes the National Soil Map (NATMAP), soil series attribute and function

values (SOILSERIES), the National Soils Inventory (NSI), the Hydrology of Soil Types (HOST) and other soil data sets

Soilscapes (data available from Cranfield, but maps available on MAGIC) Representative Soil Sampling Scheme (RSSS) Countryside Survey (CS) Environment Change Network (ECN) G-BASE Muliti-Agency Geographic Information for the Countryside (MAGIC) Agricultural Land Classification (ALC) Spatial Environmental Information System for Modelling the Impact of Chemicals

(SEISMIC) National Soils Inventory (NSI) Hydrology of Soil Types (HOST) Forestry Commission Soil Classification (FCSC)

In Scotland, the following additional Scottish datasets were included in the survey:

SSD - Scottish Soils Database held by Macaulay Institute (part of NSI) LCA - Land Capability for Agriculture, held by Macaulay Institute LCF - Land Capability for Forestry HMSD - Heavy Metals Sensitivities Database SSFIS - Scottish Soil Fertility Information Service, held by SAC SLC - Scottish Land Cover (vegetation) database, held by Macaulay Institute POP - Persistent Organic Pollutants database, held by SEPA

8

RSE - Risk of Soil Erosion Maps (derived dataset based on Scottish Soils Database, part of NSI, held by Macaulay Institute)

SFP - Survey of Fertiliser Practice (dataset held by SEERAD for Scotland) CLD - Critical Loads Database, held by CEH WMLR - Waste Management Licensing Regulations (database of soil analytical results

from enforcement actions, held by SEPA) SSR - Sewage Sludge Regulations (database of soil analytical results from enforcement

actions, held by SEPA) ESC - Ecological site classification SBBI - Scottish Blanket Bog Inventory NISAD - Soil Attributes Database Northern Ireland GLASOD - UNEP and International Soil Reference and Information Centre world map,

the status of human-induced soil degradation LIDAR 2000 – remote sensed vegetation and land cover data

In Northern Ireland, the following specific Northern Irish datasets were included:

DARD* Soil Classification Maps 1:50K DARD Soil Classification Maps 1:250K DARD Geochemical Maps DARD 5K 1995 Inventory DARD 1K 1995 DARD 5K 2005 DARD RSSS 2005 DARD Soil Carbon Agricultural Land Classification (ALC) - NI Hydrology of Soil Type (HOST) Classification - NI G-BASE-NI TELLUS Environmental Change Network (ECN) – NI

* Department of Agriculture and Rural Development

3.2 Future requirementsTwenty-five people from 18 Government Departments and Agencies were interviewed through the course of the project. In some cases, both a soil specialist and a GIS specialist were interviewed together. However, for the majority of Departments, only one representative was available and any gaps in the required information were gathered from Departmental colleagues in the days following the meeting.

Specific information on soil data use and access for each Department Agency and Defra Division interviewed is provided in sections B.1 to B.19 of Appendix B.

9

Table 1. Government Departments and Key Contacts.Government Department / Agency Contact Name Delivered ByDefra* Alan Waller ADAS - Defra Flood Management Division (FMD) David Richardson ADAS - Defra Livestock Science (LS) Zoe Davies ADAS - Defra Rural Development Service (RDS) - National

Land & Grazing Management Team (NLGMT)Julie Holloway ADAS

- Defra RDS – Technical Advisory Unit (TAU) Roger Unwin ADAS - Defra Waste Strategy (WS) Nick Blakey ADAS - Defra Water Quality Division (WQD) Soheila Amin-Hanjani ADAS - Defra Air & Environmental Quality (AEQ) Tim Williamson ADAS - Defra European Wildlife Division (EWD) Helen Pontier ADASDepartment for Transport (DfT) Angus Huntley ADASDepartment of Trade and Industry – Regional Observatories

Joby Blume ADAS

Office of the Deputy Prime Minister (ODPM) Carol Hrynkiewicz ADASPeter Bide

Environment Agency (EA) Jane Morris ADASEnglish Nature (EN) Jonathan Burney ADAS

George HintonEnglish Heritage (EH) Jen Heathcote ADAS

Alice FroggattCountryside Agency (CA) Colin Pritchard ADASCountryside Commission for Wales (CCW) Dylan Williams ADASWelsh Assembly (WAG) Ian Rugg ADASHighways Agency (HA) Simon Price ADASMinistry of Defence (MOD) – Environment Support Team Keith Maddison ADASMOD – Construction Support Team (CST) Mark HillEnvironment & Heritage Service Wendy McKinley DARDNIDepartment of Environment (NI) - Planning Service Eamon Lenaghan DARDNIDepartment of Agriculture and Rural Development (DARDNI)

Crawford Jordan DARDNI

DARDNI – Rivers Agency Ricky Cole DARDNIDepartment of Regional Development (NI) Hugh McGinn DARDNIDepartment of Enterprise Trade and Industry – Geological Survey

Derek Reay DARDNI

Scottish Environment Protection Agency (SEPA) Mark Aitken SACForestry Commission (FC) Helen McKay SACScotland and Northern Ireland Forum for Environmental Research (SNIFFER)

Fiona McTaggart and Vanessa Kind

SAC

Scottish Natural Heritage Patricia Bruneau SACThe Scottish Executive Environment and Rural Affairs Department (SEERAD)

Antje Branding and Geeta Wonnacott

SAC

10

4 SOIL DATA USE

4.1 England and Wales – Soil data use and future requirementsTables 2a and 2b provide an overview of the responses from Government Departments and Agencies in England and Wales, in relation to their use of soil data. Table 2a provides detail on all the responses, while Table 2b presents the soil datasets that are either accessed or desired by each Department/Agency. The responses indicate the Department’s perception as to whether they have access to a dataset and whether they would like access in the future. Not all the answers are correct [i.e. in some cases a Department has stated that they do not have access to a dataset, when in fact they could have access if they demanded it), but they do indicate the current level of awareness.

Defra Livestock Science (LS), Defra Waste Strategy Division (WSD) and the Department for Transport (DfT) provided the least information. Defra LS collect their own soil data at the field and plot scale, but do not envisage using national soil information in the future. Like many other Departments, Defra WSD and the DfT do not use soil information directly, but commission scientific work to consultants who will use the data on their behalf. Defra WSD do anticipate a greater need for soil information in the future. In the case of the DfT, contractors working on new developments usually refer to the 1:250,000 soil maps and bulletins of the Soil Survey of England and Wales (as was). An ALC survey is also commissioned for assessing agricultural impacts (see section B.17 in Appendix B).

In Table 2a, some of the answers indicate a lack of awareness of some of the datasets or ability to access. For example, the Environment Agency (section B.11, Appendix B) were unaware of ECN, but (as one of the joint sponsors of the dataset) should have access to the derived data. Also, all the Defra Divisions have access to the RSSS dataset. However, the following Divisions stated that they did not have access, but would like it in the future:

RDS National Land and Grazing Management Team Air and Environmental Quality European Wildlife Division

Defra Livestock Science (LS), Defra Waste Strategy Division (WSD) and the Department for Transport (DfT) provided the least information. Defra LS collect their own soil data at the field and plot scale, but do not envisage using national soil information in the future. Like many other Departments, Defra WSD and the DfT do not use soil information directly, but commission scientific work to consultants who will use the data on their behalf. Defra WSD do anticipate a greater need for soil information in the future. In the case of the DfT, contractors working on new developments usually refer to the 1:250,000 soil maps and bulletins of the Soil Survey of England and Wales (as was). An ALC survey is also commissioned for assessing agricultural impacts (see section B.17 in Appendix B).

11

Table 2a. England and Wales Departments and Agencies responses to the question of whether they have access to soil datasets.

Dataset Defra FMD

Defra LS

Defra RDS

NLGM

Defra RDS TAU

Defra WSD

Defra WQD

Defra AEQ

Defra EWD

DfT ODPM EA EN EH CA CCW WAG HA MOD CST

MOD EST

LandIS 1 4 1 1 4 1 1 4 4 3 1 3 3 3 3 1 3 4 4 & 3Soilscapes 2 4 2 3 4 1 2 4 4 1 3 2 2 4 3 1 3 4 1RSSS 1 4 3 1 4 1 3 3 4 4 1 3 2 4 3 1 4 4 4 & 3CS 1 4 1 1 4 1 1 1 4 3 1 3 3 4 & 2 1 1 4 4 4 & 3ECN 2 4 4 1 4 2 1 1 4 4 4 3 3 3 1 1 4 4 4 & 3G-BASE 2 4 3 3 4 2 3 3 4 3 2 3 2 or 3 4 4 & 3  4 3 3 3TELLUS 2 4 2 4 4 2 4 3 4 4 2 N/A N/A N/A N/A N/A N/A 4 4MAGIC 1 4 2 1 4 1 2 2 4 1 1 1 2 1 N/A N/A 1 4 1ALC 1 4 1 1 4 1 2 2 4 4 1 2 2 or 3 3 1 1 1 4 4 & 3SEISMIC 2 4 2 4 4 2 1 4 4 4 4 2 2 3 3 4 & 3 2 4 4 & 3NSI 2 4 3 1 4 1 1 4 4 4 4 3 3 3 3 1 3 4 4HOST 1 4 3 1 4 1 3 4 4 3 1 3 3 3 3 4 & 3 3 4 4 & 3FCSC 2 4 3 1 4 2 2 3 4 4 2 3 2 3 3 2 2 4 4 & 3SSD 1LECBW 1SHS                       1              

Footnotes: The numbers correspond to the following answers: 2 or 3 = NO, NOT NEEDED FOR THE MAJORITY OF USERS, BUT

1 = YES, HAS ACCESSCOULD BE USEFUL FOR THE FEW PEOPLE WORKING ON ANCIENT TECHNOLOGIES

2 = NO, DON'T NEED3 = NO, BUT WOULD LIKE ACCESS SSD = Scottish Soils Database4 = NO, AND ARE UNAWARE OF IT SHS = Soil and Herbage Survey4 & 3 = NO, WAS UNAWARE OF IT, BUT WOULD LIKE ACCESS LECBW = Long-term Ecological Change in British Woodlands

12

Table 2b. England and Wales Departments and Agencies that have access and would like access to soil datasets.

Dataset Defra FMD

Defra LS

Defra RDS

NLGM

Defra RDS TAU

Defra WSD

Defra WQD

Defra AEQ

Defra EWD

DfT ODPM EA EN EH CA CCW WAG HA MOD CST

MOD EST

LandIS 1 1 1 1 1 3 1 3 3 3 3 1 3 3Soilscapes 3 1 1 3 3 1 3 1RSSS 1 3 1 1 3 3 1 3 3 1 3CS 1 1 1 1 1 1 3 1 3 3 1 1 3ECN 1 1 1 3 3 3 1 1 3G-BASE 3 3 3 3 3 3 3 3   3 3 3TELLUS 3 MAGIC 1 1 1 1 1 1 1 1 1ALC 1 1 1 1 1 3 3 1 1 1 3SEISMIC 1 3 3 3 3NSI 3 1 1 1 3 3 3 3 1 3 HOST 1 3 1 1 3 3 1 3 3 3 3 3 3 3FCSC 3 1 3 3 3 3 3SSD 1LECBW 1SHS                       1              

Footnotes: The numbers correspond to the following answers:1 = YES, HAS ACCESS3 = NO, BUT WOULD LIKE ACCESS

13

In Table 2a, some of the answers indicate a lack of awareness of some of the datasets or ability to access. For example, the Environment Agency (section B.11, Appendix B) were unaware of ECN, but (as one of the joint sponsors of the dataset) should have access to the derived data. Also, all the Defra Divisions have access to the RSSS dataset. However, the following Divisions stated that they did not have access, but would like it in the future:

RDS National Land and Grazing Management Team Air and Environmental Quality European Wildlife Division

Overall, only five of the 8 interviewed Defra Divisions were aware that they have access to the LandIS package of datasets. Of the other Departments and Agencies interviewed, only the Welsh Assembly stated that it has access. The Environment Agency are a specific case in that they do not have access to the LandIS package, but they do have a licence to use a soil associations and soil series (look up) package until 2007. This “1 km soil series %” dataset was derived from the national soil map 1 km dataset, before the LandIS datasets NATMAP1000 and SOILSERIES were released.

There was a lot of interest in gaining access to LandIS from the ODPM, English Nature, English Heritage, Countryside Agency, Countryside Council for Wales, the Highways Agency and MOD. In many cases, the need for access is driven by regulation such as the Habitats Directive (EN, CCW, HA, MOD) and Water Framework Directive. However, as soils move up the environmental agenda, soil information is also required in environmental statements to complement other spatial data.

There was some interest in gaining access to all the soil datasets from one Department or another. However, most interest was related to HOST, G-BASE, RSSS, NSI and FCSC. Interest in G-BASE was particularly widespread (11 Divisions/Departments/Agencies) as it provides a geo-chemical baseline to which other site-specific data can be compared. English Nature, English Heritage and Countryside Council for Wales have all tried to access the dataset and found the cost prohibitive for their purposes.

4.1.1 3.2.1 England and Wales - overlaps in current holdings and potential for data sharing

Within Defra, all Divisions should have access to LandIS, RSSS, NSI and the ALC raw data. Outside Defra, instances of overlaps are few and far between. Both CCW and WAG have access to CS and ECN, but they are involved in similar projects. A number of Departments use the ALC land grade maps, but do not have access to the raw data. Only the EA use HOST, but another 8 Departments (outside Defra) would like to have access.

Table 2b can be used to assess the potential for sharing datasets between Departments and/or Agencies. For example, ODPM, EN, EH, CA, CCW, HA and the MOD would all like access to the LandIS datasets. The European Wildlife Division representative was not aware of the LandIS datasets, while many other Defra Divisions use them on a regular basis.

The datasets that are most used across England and Wales are MAGIC, Countryside Survey (in some cases the land classes rather than the soil data), ALC and LandIS (the derived dataset in the case of the EA). There is some shared interest in datasets. For example, the following were all using or interested in accessing LandIS, RSSS, G-BASE, CS and ECN: Defra AEQ Defra EWD English Nature Countryside Council for Wales MOD Environment Support Team

Within Defra, the greatest opportunities for data sharing are for greater awareness to be developed of Crown access rights to co-/owned datasets: LandIS (NatMAP and NSI) and RSSS. However, four Divisions also expressed a need to access G-BASE. This is due to the fact that G-BASE can be used to

14

compare baseline levels with site-specific data for mineral and waste planning applications and restorations (RDS – NLGMT); agri-environment implications (RDS – TAU); and atmospheric deposition and biodiversity impacts (Defra - AEQ and EWD).

Outside Defra, English Nature, English Heritage, CCW and MOD Environmental Support Team are all interested in a similar range of datasets. These are LandIS (NatMAP), RSSS, CS, ECN, G-BASE, NSI (not MOD) and HOST. CCW have access to CS and ECN, but would like to extend the number of Welsh ECN sites. There may be the potential to share some of this data across Departments and Agencies.

In summary, although most Departments and Agencies do not have a good grasp of what soil data sets are either available to them, or exist, the majority did express an interest in gaining access to more soil information. The only Departments that were unaware of any of the soil datasets were Defra Livestock Science, Defra Waste Strategy Division and the Department for Transport (DfT).

The main finding of the interviews is that most Departments and Agencies anticipate the need for greater use of soil information in the future. However, very few have attempted to gain access to additional soil datasets, and the majority are unaware of:

The soil datasets to which they have access The spatial resolution of datasets The spatial coverage of soil datasets Soil attributes contained within the datasets

The message that Defra intends to integrate soil metadata into SPIRE was well received and was seen as a vital tool in improving soil data use across Government. Once people are aware of what soil information exists, and there is an information repository that will enable them to identify fitness for purpose, they will be more inclined to demand access to the data.

In our opinion, therefore, the most important driver for greater use of soil information in the future will be SPIRE itself. This project has identified that other drivers exist in the form of regulation, strategy documents, and support from ministers, and the Treasury. The one factor that is missing is a cataloguing tool and information repository to enable Departments to identify useful datasets and demand access to them.

Appendix B presents the main current uses of soil data within each Department or Agency and an assessment of future requirements. The business justification for requiring soil data is supported by those recent projects that have required soil information, and the drivers (Directives, regulations, bills, strategy documents) that determine current and future requirements.

Appendix C is provided on the CD that accompanies this report. It provides an assessment of how essential each dataset is within each Department and the soil functions that are (or could be) addressed by the dataset. Each Table is referred to as the Department “Function Matrix”. Some soil attributes are more useful in assessing the soil’s ability to perform a certain function than others. By relating a Department’s use of soil data to soil functions, we can determine the principal soil attributes that a Department is interested in.

4.1.2 England and Wales – additional soil and complementary datasets

Table 3 provides information on complementary datasets. Some Departments indicated that they hold certain complementary datasets, or that they would like access. However, it was not possible to obtain comprehensive information on access to complementary datasets. CCW mentioned six datasets to which they would like access.

15

Table 3. England and Wales Government Departments and Agencies access to complementary datasets.

Dataset Defra FMD

Defra LS

Defra RDS

NLGM

Defra RDS TAU

Defra WSD

Defra WQD

Defra AEQ

Defra EWD

DfT ODPM EA EN EH CA CCW WAG

LCM 1990 1

EH designated monuments

1 1

Met Office data 1 1

Soil Drift Geology 3 3

Water course vector maps

1 3

Critical load maps 1 3

OS Mastermap 1 3

Geomorphology 3

Natural spring location 3

NVC Vegetation maps 3

National Inventory of Woodlands and Trees

3

LUCAS Survey 1

Footnotes: The numbers correspond to the following answers:1 = YES, HAS ACCESS 3 = NO, BUT WOULD LIKE ACCESS = NO, DON'T NEED

16

4.1.3 England and Wales – additional soil and complementary datasets

Table 3 provides information on complementary datasets. Some Departments indicated that they hold certain complementary datasets, or that they would like access. However, it was not possible to obtain comprehensive information on access to complementary datasets. CCW mentioned six datasets to which they would like access.

4.2 Scotland - Soil data use and future requirementsThe Scottish Soils Database held by the Macaulay Institute is one of the most comprehensive collations of national scale soil data in Europe. The database consists of a range of digitised and paper maps at a number of scales from full national coverage at 1: 50,000 scale to more local surveys at scales of 1:10,560 or larger. In addition, it contains a comprehensive database that currently contains information on over 13,000 geo-referenced soil profiles. The main emphasis of current interest is on the utilisation of the spatial and point data for a variety of purposes.

Table 4. Scotland Government Departments and Agencies responses to the question of whether they have access to soil datasets.

Dataset ForestryCommission

SEERAD SEPA Scottish National Heritage

SNIFFER

LandIS 3 2 2 2 2Soilscapes 3 4 2 2 2RSSS 2 4 2 3 2CS 1 1 1 1 1ECN 1 1 3 1 1G-BASE 4 3 3 3 1TELLUS 4 3 3 2 1MAGIC 1 4 2 2 4ALC 1 2 2 2 2SEISMIC 3 4 2 2 4NSI (part of SSD) 1 1 1 1 1HOST 1 1 1 3 1FCSC 1 3 3 1 4Additional soil datasets usedSSD 1 1 1 3HMSD 1 3 3 2 1SSFIS 4 1 3 3 1POP 2 1 2 2RSE 3 1 1 1CLD 1 1 1 1 1WMLR 4 1 2SSR 4 1 2SSKIB 1SBBI 1 1NISAD 1GLASOD 1

Footnotes: The numbers correspond to the following answers:1 = YES, HAS ACCESS2 = NO, DON'T NEED3 = NO, BUT WOULD LIKE ACCESS4 = NO, AND ARE UNAWARE OF IT

17

In order to rationalise the existing derived soil data, SEERAD has funded a project called the “Scottish Soils Knowledge and Information Base” (SSKIB). The project began in 2004 and its main aim is to develop an integrated attribute database of soil information with a common base at the chosen scale of 1: 50,000.

Another objective of the project is to consult with the principal end-users in Scotland concerning their requirements for soil data now and in the future. This current project (SP08003) therefore provides a timely update of what progress has been made in meeting the requirements of soil data users in Scotland.

Table 4 provides the responses of the five end-users in terms of their current and future requirements for UK soil datasets.

Where one Department expresses the wish to acquire a dataset (response “3”) and one or more other Departments express the same wish or already have access (response “1”), we are able to identify the potential for data sharing. For example, both SEERAD and SEPA would like access to the Heavy Metals Sensitivities Database (HMSD), while Scottish National Heritage (SNH) already has access.

As was the case in England, most of the Scottish Departments/Agencies either do not use many of the soil datasets to any great extent or are unaware of their contents (see Appendix B). However, all of the Scottish Government Departments and Agencies said that, due to the recent increased profile of soil and the need for strategies to protect Scottish soils, they envisaged an increased need to be able to access soil data in the future.

The Forestry Commission have access to many of the England and Wales datasets, including CS, ECN and HOST. However, in Scotland they stated that cost was the main reason for not using Scottish soil datasets other than their own. The Forestry Commission hold almost all of the soil data that they use regularly to plan the planting of new forests, harvesting and replanting following harvest. No soil data are regularly used other than that already held by the Forestry Commission. They use an ecological site classification system (ESC). Other soil data may possibly be used in strategic planning and research, but there have been no recent uses (to the knowledge of the interviewee).

The input from SEERAD was disappointing as they are funding the “Scottish Soils Knowledge and Information Base” (SSKIB) initiative. They were not aware of what some soil datasets contain, but did acknowledge the need to integrate SSKIB with the Defra soil data use initiative.

SEERAD use a range of soil datasets, including the Scottish Soils Database (SSD, which includes the National Soils Inventory [NSI] in relation to Scotland), the LCA classification, critical loads database, ECN, HOST and SSFIS. They would like to be able to freely access some of the other available soil datasets including Soil Erosion Risk Assessment data, solid geology, drift geology and data on soil permeability and soil organic matter. Justification for future requirements will depend on the need for research to support policy development. This will include future work to determine the chemical, physical and biological status of Scottish soils.

Although SEERAD do not need access to LandIS as it stands, if a Scottish version of LandIS was developed they would find it very useful. SEPA hold a limited amount of data themselves, which are available under the Freedom of Information Scotland Act (POP and WMLR) data. They also use data within the Countryside Survey, the NSI and SSD, HOST, SLC, RSE, SFP, CLD and SSR. SEPA envisage an increased need for soil data and a requirement for a greater range of soil data in the future.

18

This will be used to gain a better idea of the state of Scotland's soils and to implement recent and new regulations including the Water Framework Directive, the current and future NVZ regulations and possible regulations on phosphate.

SNIFFER rarely uses soil data themselves. The usual practice is to plan, supervise and coordinate projects and to use contractors to carry out the research. The contractors apply for soil data licences as required. The Function Matrix in Appendix C refers to the use of soil data by both SNIFFER and the contractors that they employ. The two SNIFFER staff interviewed felt that the need for soil data in their work (including that of contractors working on SNIFFER-funded projects) would increase in the next few years, due to the increased public and policy awareness of soils and the need for soil protection.

SNH use a considerable number of soil datasets at present (see SNH Function Matrix in Appendix C). They use soil data to help develop environmental policies or routinely during environmental impact assessments. They also use soil data to plan and carry out research projects on habitats and species and to protect vulnerable habitats and species. As with other agencies, they see their use of soil data as likely to increase in future due to increased concerns over climate change and soil protection.

SNH are actively involved in SSKIB and in encouraging the improvement of soil databases, in terms of both ease of use for non-experts and access. They provided a detailed list of datasets to which they have or would like to have access (see Appendix B).

4.2.1 Scotland - overlaps in current holdings and potential for data sharing

The datasets that are most used across the Scottish Government Departments and Agencies are the Scottish NSI and the SSD and derived datasets based on these. These two master datasets are linked (the Scottish NSI is a subset of the SSD) and are extremely large and complex. From conversations carried out with the data users, it seems that different users use different parts of these databases. It would be difficult to see how information sharing could easily take place in order to reduce the cost of access.

Of the remaining datasets used by the Scottish Government Departments and Agencies, there seems little opportunity for cost saving through data sharing, due to differences in data required by different organisations. However, in some cases, where projects are undertaken by two or more project partners, a single user licence is usually enough to release the data for use on the project. Only one organisation may actually extract the data from the dataset, but once the data has been extracted in a useful form, other partner organisations can help with interpretation.

4.2.2 Scotland – additional soils and complementary datasets

The Scottish Soils Fertility Information Service (held by SAC) is a large soil database containing information on several thousand soil samples taken from Scottish agricultural soils over the past 20 years. This amounts to over 20 000 samples in total. Only agricultural soil samples sent in by Scottish farmers and SAC advisors are included, (i.e. no soils from non-farmed or industrial sites are included). There is no fixed spatial coverage, and the better agricultural soils tend to be better represented, as it relates to soil fertility. All soils have been analysed for P, K, pH, and lime requirement. Some have also been analysed for Mg, Ca, S and trace elements. This database is partly digitised, but early data are in paper form only. It is potentially a very useful database, since it could provide an opportunity to look at trends in soil chemical (e.g. soil C) and some physical parameters over time.

Use and access to complementary datasets is detailed in Appendices B and C.

19

4.3 Northern Ireland - Soil data use and future requirementsOver 70 individuals from 30 groups in 6 Departments/Agencies (DARD, DOE, DETI, DCAL, DRD and DFP) in Northern Ireland were contacted in relation to this project (see Table 5).

The pro-forma, together with a list of available NI soil databases and a Help file, were then sent out by e-mail to 30 named individuals from the 6 Departments. The NI soil database list (with attributes listed) was included to encourage a response from those not currently using the data but who might have a use for it once they knew it existed.

Visits were made to the known ‘bigger’ users – DARD Science Service (itself comprised of 6 independent divisions), EHS WMU, DETINI GSNI and DARD Forest Service - to help complete the forms. Others received help by phone to clarify issues. All DARD divisions contacted responded in one way or another.

Thirteen pro-forma from the three major Departmental user groups (DARD, EHS and DETI) were completed and returned. The following nine groups stated that they do not currently use the available soil datasets but that there might be potential for their use in the future: DARD Vet Service DARD Countryside Management DARD Aquatics Science-Bushmills laboratory DCAL Inland Fisheries EHS-CEDaR group EHS-Built Heritage DRD Roads Service DRD Water Service The Rural Development Council (set up by DARD)All the above groups expressed an interest in finding out more about the NI soil data and how that data could be made available to them in the future for incorporation, where appropriate, into their Departmental GIS.

Table 5. NI Departments (NI executive - those consulted are shown in bold).

DOFMDFM Office of the First Minister and Deputy First Minister

DARD Department of Agriculture and Rural Development (includes Forest Service and Rivers Agency)

DCAL Department of Culture, Arts and Leisure (includes OSNI and Inland Fisheries)

DENI Department of Education

DELNI Department of Employment and Learning

DETI Department of Enterprise, Trade and Investment (includes GSNI)

DOE Department of Environment (includes EHS, CEDaR and Planning Service)

DFP Department of Finance and Personnel

DHSSPS Department of Health, Social Services and Public Safety

DRD Department of Regional Development (includes Road and Water Service)

DSD Department of Social Development

20

Table 6. Northern Ireland Departments and Agencies responses to the question of whether they have access to the main GB soil datasets.

Soil Dataset (GB and NI)

DARD Econ Stats

DARD Forest Service

DARD Sc S

AFESD

DARD ScS Ag Econ

DARD Sc S VSD

DARD ScS

APSD

DARD Rivers

Agency

DETINI-GSNI

DOE-EHS

Policy

EHS Cons Sc

EHS WMU &

AEQLandIS 2 4 2 4 4 4 4 4 4 4

Soilscapes 2 4 2 4 4 4 4 4 4 4

RSSS 2 3 3 4 4 4 4 4 4 see note 2

Countryside Survey 2 3 2 4 4 4 4 4 1 3

ECN 2 4 1 4 4 4 4 4 4 see note 2

G-BASE 2 4 3 4 4 4 3 4 4 4

TELLUS 2 3 3 4 4 4 4 4 3 3

MAGIC 2 2 2 4 4 4 4 4 4 3

ALC 2 4 3 3 4 4 4 4 4 4 4

SEISMIC 2 4 4 4 4 4 4 4 4 4

NSI 2 4 3 4 4 4 (NI only) 1 4 4 4

HOST 2 4 3 4 4 3 4 4 4 see note 2

FCSC 2 3 3 4 4 4 4 3 3

FootnotesEHS WMU & AEQ1 Note - for data sets which EHS WMU & AEQ are currently ignorant of (Score 4), EHS WMU & AEQ would potentially use them for water and or waste issues, but need to know more.

2 For data sets in GB + NI - assume use of NI element only. EHS WMU & AEQ would also be interested in equivalent ROI datasets.

21

Table 7. Northern Ireland Departments and Agencies responses to the question of whether they have access to NI specific soil datasets.

Northern Ireland Specific Datasets

DARD ES (a)

DARD FS

DARD Sc S

AFESD

DARD ScS Ag Econ

DARD Sc S VSD

DARD ScS

APSD

DARD Rivers Agency

DETINI-GSNI

DOE-EHS

Policy

EHS Cons Sc

EHS WMU &

AEQDARD Soil Classification Maps (SCM) 1:50K

3 3 1 1 1 1 1 b 4 1 1DARD SCM 1:250K 3 3 1 2 1 2 4 4 1 1DARD Geochemical Maps 3 3 1 1 1 2 3 4 3 1DARD 5K 1995 Inventory 3 3 1 1 1 2 1 4 4 1DARD 1K 1995 3 2 1 3 1 2 1 4 4 1DARD 5K 2005 3 3 1 4 1 2 4 4 4 1DARD RSSS 2005 3 4 1 4 1 2 4 4 4 1DARD Soil Carbon 3 2 1 4 1 2 4 4 4 1ALC 3 3 1 1 3 4 2 4 4 3 3HOST 3 3 1 3 4 1 1 4 1 1G-BASE-NI 3 4 1 4 4 2 1 c 4 4 3TELLUS 3 3 3 4 4 2 1 c 4 3 3ECN - NI 3 4 1 4 2 2 4 4 3 1FootnotesDARD ES(a) Re licensing: Data for NI soils known to be available free of charge for intra-Department use via C Jordan, AFESD, DARD.

DETINI-GSNI(a) In some instances 4 means DETINI-GSNI are aware of the data but don't know enough about it to decide if they would like access(b) Used for comparison with GSNI mapping of superficial deposits(c) The geophysics survey records magnetic/electrical conductivity /radioactivity of ground below from flying ht of 50m (rural) 200m (urban) - flights will cover all of NI complete Sept 2007. Main applications within GSNI: environmental baseline studies, medical geohazards, mineral resource mapping, pollution studies, groundwater vulnerability.

22

Table 8. Northern Ireland Government Departments and Agencies access to complementary datasets.

Complementary (non-soil) Datasets Used

DARD ES

DARD FS

DARD Sc S

AFESD

DARD ScS Ag Econ

DARD Sc S VSD

DARD ScS

APSD

DARD Rivers Agency

DETINI-GSNI

DOE-EHS

Policy

EHS Cons Sc

EHS WMU &

AEQLand Cover Map 2000 1 1 1 1 b 1 1GSNI 1:250k Geology 1 b

GSNI 1:50k Geology 1 c

TELLUS Geophysics Survey 1 c

CORINE 90 1 1Agricultural census 3 1Geology 1Digital Elevation Model 1Groundwater Vulnerability? 1Long-term RAIN-PT on 1km2 1DTM (50m) 1Peatland Survey 1Climate 3

Footnotes

DETINI-GSNI(b) Used for comparison with GSNI mapping of superficial deposits(c) The geophysics survey records magnetic/electrical conductivity /radioactivity of ground below from flying ht of 50m (rural) 200m (urban) - flights will cover all of NI complete Sept 2007. Main applications within GSNI: environmental baseline studies, medical geohazards, mineral resource mapping, pollution studies, groundwater vulnerability.

23

Those who replied but expressed no foreseeable interest in the use of soil data or did not return any completed forms (nil returns) amounted to 8 divisions: DARD Policy Division, DARD Grants & Subsidies, DARD Marine Fisheries, DARD-CAFRE, Greenmount, DOE Planning Service DFP Valuation and Lands Agency DFP Land Registers of NI and DRD Regional Planning and Transportation.

Information on access to the main GB soil datasets for the eleven NI Departments and Agencies that currently use soil information is provided in Table 6. For the most part, Departments either do not need these datasets or are unaware of their existence. However, the following Departments do have access to the stated datasets:

DARD Science Service Agri-environment (AFESD) – ECN Geological Survey of Northern Ireland (DETINI-GSNI) – NSI (NI only) Environment Heritage Service, Conservation Science (EHS Cons Sc) – Countryside

Survey

There was also some interest in gaining access to RSSS, Countryside Survey, G-BASE, TELLUS, ALC, NSI, HOST and FCSC by Divisions within DARD (Forest Service and Agri-environment) and the DOE (particularly EHS Water Management Unit).

4.3.1 Northern Ireland - overlaps in current holdings and potential for data sharing

Information on access to the Northern Ireland specific soil datasets for the eleven NI Departments and Agencies that currently use soil information is provided in Table 7. The responses reveal a good potential for data sharing between Divisions and Departments. Within DARD, NI soil data are available free of charge for intra-Department use via C Jordan. Some Divisions, such as the DARD Economics and Statistics Unit, are aware of this and would like to use the data in the future. For other Divisions, such as DARD Veterinary Service, DARD Applied Plant Science Division (APSD) and EHS Conservation Science, awareness of the datasets could be improved.

4.3.2 Northern Ireland complementary datasets

A number of complementary datasets were common to a number of respondents. Information on these complementary datasets and the responses from the main Divisions is provided in Table 8.

More detailed information on soil data use within the main Northern Ireland Departments is given in Appendix B sections B. 5 to B. 7.

5 LICENSING AND ACCESS ISSUES

5.1 DefraThe Defra Soils Team is responsible for licensing RSSS and has free access. Defra also owns the NSI. The main issue for Defra is the cost of accessing soil data from the National Soil Resources Institute.

As Crown Users, Defra "Divisions" have access to royalty-free data held in LandIS (Soil Survey plus NSI) by NSRI. However, Defra does have to pay a handling fee. Each year, a

sum of £5,000 is reserved for the preparation fee for bona fide and/or Defra-funded research and Crown users on a first come first served basis.

The LandIS data are effectively co-owned by the Crown and NSRI. However, Defra does not hold the data and cannot supply it to any other non-Crown users under the terms of the licence agreement with NSRI. Non-Crown users can have access to the data for a charge of a royalty and handling fee. Crown users do not have to pay the royalty, but do pay a handling fee.

The following Defra Divisions expressed an interest in accessing G-BASE:

RDS – National Land & Grazing Management Team (NLGMT) RDS – Technical Advisory Unit (TAU) Air & Environmental Quality (AEQ) European Wildlife Division (EWD)

The interest derives from the fact that G-BASE provides baseline concentrations for a number of elements to which site specific data can be compared. G-BASE would, therefore, be useful for investigating spatial and temporal variations in soil constituents. To date, Defra has not investigated gaining access to this data from the British Geological Survey. However, it is expected that costs and access would be an issue.

Another major issue for Defra in the future will be how Natural England will access soil data. When the Rural Development Service (RDS), English Nature and the Countryside Agency join to create Natural England, they will lose all Crown User rights. This would effectively mean RDS not having access to the ALC soil data. It is our understanding that this issue is yet to be resolved.

5.2 Other Departments and Agencies in England and WalesMany Departments/Agencies in England are unaware of any licensing issues, because they have not attempted to access datasets other than those they are directly involved in. This is the case for ODPM, EN, EH, the Countryside Agency, the Highways Agency and the MOD. It is likely that a number of issues will arise, once these Agencies and Departments attempt to acquire data from BGS and NSRI.

Agencies, such as English Nature and English Heritage have only just started investigating the possibilities of accessing soil data. For example, the GIS Department of English Heritage has looked into acquiring the G-BASE dataset held by BGS, but are not aware of any other soil datasets. Cost was seen as a significant and inhibiting issue to the use of G-BASE for both English Heritage and English Nature. Licensing issues for each of the Departments and Agencies interviewed in England and Wales are presented in Appendix B, sections B.1 to B.19.

5.3 Scotland licensing and accessIn Scotland, many users are interested in accessing the Scottish Soils Database held by the Macaulay Institute. However, cost is a major issue in restricting access for the majority of users. For some, use of the data is only possible on projects where the Macaulay Institute is already involved. None of the Departments supplied any specific information on how much they pay for soil data. SEPA stated that there is no regular cost, while the cost of FCSC to Scottish Natural Heritage and of Ecological site classification to the Forestry Commission was unknown.

SEPA currently has no formalised special arrangements with any dataset holders for provision of data or information. Likewise, SNIFFER has no licences for access to

commercially available datasets. However, they do negotiate licence agreements for specific datasets prior to putting contracts out to tender. Where SNIFFER work with an Agency that already has a licence, they are able to use the existing licence agreement.

Licensing issues for each Scottish Department are presented in Appendix B, sections B.20 to B.24.

5.4 Northern Ireland licensing and accessWithin DARD, NI soil data are available free of charge for intra-Department use via Crawford Jordan. DARD also has a reciprocal arrangement with NI Geological Survey for the free use of soil data. DARD soil data are also free for use by the Environment and Heritage Service (EHS) for use in specified projects. However, this current free access may be threatened when DARD becomes a Non-Departmental Public Body (an NDPB, called the Agri-Food and Biosciences Institute, AFBI) in April 2006. However, any concern that EHS may have is likely to be addressed via a Service Level Agreement (SLA) between EHS and AFBI.

It is unlikely that EHS will be charged for access to data, although this is not known for sure. DARD has already copied the soil classification maps to EHS (including HOST and ALC) for use on the EHS GIS. Moreover, it is unlikely EHS will need direct access to the DARD attribute databases, as AFBI will use them and carry out the work on behalf of EHS under the SLA.

6 COMPLEMENTARY DATASETS

Information on complementary datasets is presented for all the interviewed Departments in Appendix B and for England and Wales in Table 3 of this report. Although this provides an indication of what complementary datasets each Department is using, it does not address the potential for data sharing. To determine what complementary datasets Departments require was beyond the scope of this project. However, the information does provide an indication of what datasets are available and who is using them.

As mentioned in Table 3, Defra RDS also holds climate data that is used for Agricultural Land Classification and has the potential to complement soil data on a wide variety of environmental protection projects. Tables 9 to 11 present some detail on the data currently available to RDS. The complementary datasets used in Scotland are presented in Table 12. Northern Ireland complementary datasets can be found in the spreadsheets provided on the CD with this report.

Table 9. Defra RDS Climate data – Met Office data at 5 km interval.

Parameter Observation periodAverage annual rainfall (AAR) 1941-1970

Median accumulated temperature above 0o, Jan-Jun (AT0) 1961-1980

Median Duration of Field Capacity Days (FCD) 1941-1970

Average Summer Rainfall, April to September (ASR) 1941-1970

Median accumulated temperature above 0o, Apr-Sep (ATS) 1961-1980

Moisture Deficit (Winter wheat) (1)

Moisture Deficit (Potatoes) (1)(1) derived from ATS and ASR values

Table 10. Derived data at 1 kilometre interval (ESRI shapefiles). Interpolated from the 5 km climate variables.

Parameter File nameAverage annual rainfall genaar1k.shp

Median accumulated temperature above 0o, Jan-Jun Genat01k.shp

Overall climate limitation Nedclgrade.shp

Table 11. Derived data at 50 metre interval (ESRI grid). Interpolated from the 5 km climate variables using OS panorama as the source for altitude.

Parameter File nameAverage annual rainfall aar50

Median accumulated temperature above 0o, Jan-Jun at050

Median Duration of Field Capacity Days fcd50

Moisture Deficit (Winter wheat) mdp50

Moisture Deficit (Potatoes) mdw50

Overall climate limitation alc50

7 DESCRIPTIVE SOIL METADATA

The final objective of this project was to investigate the potential for developing descriptive soil metadata for integration into the Defra SPIRE (Spatial Information REpository) cataloguing tool. The ultimate aim is to develop useful and searchable metadata on soil attributes that would enable the user to determine the fitness for purpose of each soil data set. For example, if a user is interested in soil pH, it would be helpful to have information within SPIRE that would describe the spatial resolution, sampling methodology, depth of sampling and other metadata on how the soil pH information was derived. This would help the user decide whether a particular dataset could provide useful information for their purposes, such as monitoring change over time or providing a national picture of the recent situation.

As a first step in developing descriptive soil metadata, a survey was carried out of existing soil data set attributes. The following sections present the findings of this survey.

In general, it was possible to obtain soil metadata in a suitable form for integration into the SPIRE cataloguing tool for the majority of the England, Wales and Northern Ireland soil datasets. In Northern Ireland, soil metadata standards were largely compatible with the requirements of SPIRE. However, in Scotland and England, although a lot of information is available on almost all the datasets, some of the important information that would aid a user to determine fitness for purpose has proved difficult to obtain. Such information included sample methodology and sampling depth.

Even so, once the specific required information has been obtained, we believe it will be possible to provide useful metadata that will help all users to select useful soil data for a variety of purposes.

Table 12. Scotland Departments and Agencies access to complementary datasets.

Dataset FC SEERAD SEPA SNH SNIFFER

LCF 3 3 1LCA 1 1 3 1 2SLC 2 1 1 1SFP 2 1 2Solid geology 1 3 1Drift geology 3 1ESC 1NWM 1NVZ maps 1NVC 1LIDAR 2000 1 3LandSat 1CORINE 1Landscape Char. Assessment

1 3Geomorphology map 1 1Sand dunes 1Saltmarsh 1Footnotes:

LCF = Land Capability for Forestry

LCA = Land Capability for Agriculture, held by Macaulay Institute

SLC = Scottish Land Cover (vegetation) database, held by Macaulay Institute

SFP = Survey of Fertiliser Practice (dataset held by SEERAD for Scotland)LCF = Land Capability for ForestryNVC - National Vegetation Classification

LandSat - Satellite photography database of Scotland

ESC - Ecological site classification

7.1 SPIRE metadataTable 13 presents the SPIRE metadata collection table. It includes both the mandatory and optional elements; the mandatory elements must be completed to satisfy the requirements of the metadata standard. The mandatory elements conform to the UK Gemini metadata standard.

Table 13. SPIRE descriptive metadata mandatory and optional elements.

ID Element Optional / Mandatory1Title Mandatory2Dataset Language Mandatory3Abstract Mandatory4Topic Category Mandatory5Subject Mandatory6Date Mandatory7Dataset Reference Date Mandatory8West bounding Coordinate Mandatory9East bounding Coordinate Mandatory

10North bounding Coordinate Mandatory11South bounding Coordinate Mandatory12 Extent Mandatory13Spatial Reference System Mandatory14 Data Format Name Mandatory15Data Format Version Mandatory16Data Format Amendment Number Optional 17Data Format Specification Optional 18Data Format Decompression Techn Optional 19Data Format Distributor Optional

20Distributor Mandatory 21Distributor Contact Title Mandatory22 Name of Distributor Mandatory23Postal address of distributor Optional 24Tel No of distributor Optional 25Fax No of distributor Optional 26Email address of distributor Optional 27Web address of distributor Optional 28Frequency of Update Mandatory29Date of Update of Metadata Mandatory30Alternative Title Optional 31Originator Optional 32Lineage Optional 33Vertical Extent Information Optional 34Spatial Resolution Optional 35Spatial Representation Type Optional 36Presentation type Optional 37Supply Media Optional 38Access Constraints Optional 39Use Constraints Optional 40Additional Information Source Optional 41Online resource Optional 42Browse Graphic Optional 43Metadata Standard Name Optional 44Metadata Standard Version Optional 45Purpose Optional

ID Element Optional / Mandatory46Use Limitation Optional 47Completeness Optional 48Positional Accuracy Optional 49Temporal Accuracy Optional 50Thematic Accuracy Optional 51Unit Of Measure Optional

Each data provider was asked to fill in the SPIRE metadata spreadsheet (v1.15 provided by RDS Leeds). SPIRE descriptive metadata was requested for each dataset held by the data providers. In the main, the data providers were able to provide sufficient information to fulfil the mandatory elements of the metadata standard. In many cases, large numbers of the optional elements were also completed.

SPIRE compliant metadata were created from the spreadsheet using the scripts provided. This information is presented in xml format on the CD that accompanies this report. This information is ready to be integrated into the SPIRE cataloguing tool.

7.2 Survey of existing soil dataset attributesThrough the use of a pro-forma, we obtained information relating the attributes of each soil dataset. Table 14 presents an extract from this metadata collection exercise. The information requested related to each attribute and was designed to provide sufficient information for the user to determine the fitness for purpose of a particular attribute.

The information requested on each attribute was as follows: Sample Methodology – Whether it was Continuous, Interpolated or Lookup for example Date of Sampling – When the sample was taken or over what period it was recorded Depth of Sampling – What depth the sampling has been carried out, is it constant or does

it vary Frequency – What is the frequency of sampling, is it regular or was it a one off Consistent Collection – Was the collection methodology the same throughout the

acquisition of the data Comments – Free text to include additional information on the attribute

This approach allows the user to compare the same attributes from different datasets, for example allowing them to identify when each attribute was recorded, or to what depth it was sampled and thus assessing whether one attribute is more appropriate than another or indeed suitable at all for their purposes.

In general, the level of response to the attribute pro-forma was weak. Although attribute metadata was obtained for each dataset, the level of detail varied amongst data providers and a significant amount of manual editing was required to get them into the same format.

The electronic results of the attribute metadata collection are presented in Microsoft Excel format on the CD that accompanies this report. In order to link the attributes metadata and SPIRE metadata the Database ID field must be used. It is envisaged that each SPIRE metadata occurrence can have multiple attribute occurrences. For example, the NATMAP product has several attribute tables associated with it.

Table 14. DARDNI extract from the data collection exercise.Database

ID Attribute Sample Methodology Date of Sampling Depth of sampling Frequency Consistent

Collection CommentsDARDNI

Soil Maps 1: 50K

Polygon centroid X Lookup Surveyed 1988-97 To Parent Material by horizon One-off YES Generalised from 1:50K maps

DARDNI Soil Maps

1: 50KPolygon centroid Y Lookup Surveyed 1988-97 To Parent Material by

horizon One-off YES Generalised from 1:50K maps

DARDNI Soil Maps

1: 50KPerimeter (m) Lookup Surveyed 1988-97 To Parent Material by

horizon One-off YES Generalised from 1:50K maps

DARDNI Soil Maps

1: 50KArea (ha) Lookup Surveyed 1988-97 To Parent Material by

horizon One-off YES Generalised from 1:50K maps

Note: In this case there were two surveys:

The first in 1988-97 was the major one, to carry out the first classification of NI soils and during which a 5K grid of pits were dug on agricultural soils only and samples taken for physical and chemical analysis from each horizon (A to C).

The second survey was carried out during winter 2004-05 on a full 5K grid (including non-agricultural soils i.e. semi-natural/upland and urban soils). The sample depth was 75 mm (the agronomic depth for grass - the dominant land cover in NI) and to the A-horizon only (for comparison with the 1988-97 survey results).

DARD felt the second 5K survey was necessary to get a baseline soil quality dataset before the Nitrates Action Programme was implemented across all of NI. It will be used to identify changes in soil quality due to this programme, which is likely to cause major changes in agricultural practices in Northern Ireland (closed periods for fertilisers and slurries etc).

8 CONCLUSIONS

The main aims of this project were to provide detailed information on the soil data used by Government Departments in the UK, and to investigate the potential for providing soil dataset metadata for integration into Defra’s SPIRE cataloguing tool.

The project has confirmed that some Departments have considerable interest in using soil data, while others have peripheral or no interest depending on the degree to which they are affected by current national soil plans and EC Regulations such as the Water Framework Directive.

Almost all the Departments interviewed in England, Wales and Scotland have an interest in soil data and indicated the desire to use more soil information in the future. However, knowledge of what soil information currently exists was often limited.

Departments such as Defra RDS and ODPM have a requirement for more detailed soil information at a larger scale that is currently not available in digital format. The National Soil Resources Institute in England has started to address this issue by entering additional data into LandIS from the paper archive. Those that did not indicate the need for additional digital soil information at the national scale included:

Defra Flood Management Defra Livestock Science Department for Transport MOD Construction Support Team

In Northern Ireland, the following Departments had a particular interest in soil data:

DARD Science Service (Agri-Food and Biosciences Institute), especially DARD Agri-Environment Branch, AFESD

DETI Geological Survey Northern Ireland Department of the Environment, Environment and Heritage Service – Water Management

Unit, Air Environmental Quality Unit and Waste Management and Contaminated Land Unit

Other Departments who identified interest in the future were DARD Vet Service, DCAL Inland Fisheries, DOE Built Heritage and EHS Centre of Environmental Data and Recording (CEDaR).

This project has identified the potential for data sharing between a number of Departments interested in the same data. In England, these include Defra Air and Environmental Quality, Defra European Wildlife Division, and English Nature. Much of the information can be obtained from the Tables and Matrices provided in the main report and accompanying Appendices.

Detailed information on the datasets was obtained from all the main soil data providers in the UK, apart from the British Geological Survey (BGS) who hold G-BASE: a geo-chemical baseline survey for England and Wales. In some cases, the information was supplied directly, while for others it was necessary to enter information manually from documentation provided by the soil data providers. This was particularly the case in Scotland.

It has, therefore, been possible to provide detailed metadata for almost all the main UK soil data sets identified by this project. These data are presented in the required format for integration into the Defra SPIRE cataloguing tool. The electronic data supplied with this

report, along with the soil data use matrices and Appendix B, can therefore provide detailed information: on the soil data used within each Department and Agency that can be integrated into SPIRE to enable users to determine the fitness for purpose of

each dataset.

We have been able to acquire most of the metadata required to determine fitness for purpose. It was beyond the scope of this project to provide metadata on the sub-sets of the main datasets, or on new soil datasets. However, the project has shown that it is possible to develop descriptive metadata for integration into SPIRE for soil data at any scale. It will be possible to obtain more detailed information on specific datasets as it becomes available.

In order to facilitate ease of access to information, and as an extension to this project, a metadata database will be produced to enable the attribute metadata collected under the project to be queried. The tool will store the data in a relational fashion that will allow the user to interrogate the whole database using simple, pre-defined free text and fixed queries.

In addition to the information currently held in the metadata tables, the database will also contain information on:

licensing data owners and their contact details public bodies/business areas that have paid for or are entitled to use

each data set and contact details of a representative

Users will be able to search the database by the above attributes to determine which datasets the attribute of interest resides in. The queries will also return simple metadata relating to the attributes themselves.

A simple front end to the database will be developed that will allow non-expert database users to easily interrogate it. This form will contain all the aspects required for the user to interact with the database thus removing the need to deal with a variety of tables and queries individually.

The database will be structured in such a way that it will be possible to add new metadata relating to any given dataset. This structure will be in the form of individual tables relating to each soil dataset. The principal behind this structure is that it retains clarity and eases the substitution of updated metadata and attributes.

In England and Wales, licensing and access arrangements to soil data other than that held by NSRI and BGS are not clear. Of those that would like regular access to LandIS or G-BASE the Environment Agency, English Nature, English Heritage, and the Countryside Council for Wales all stated that cost is a major issue in restricting access to data. When English Nature combine with the Countryside Agency and Defra RDS to form Natural England there will be problems with access to soil data, unless the issue is addressed in the near future.

In Northern Ireland, access to soil data within the Northern Ireland Civil Service is currently free of charge. However, when the status of DARD becomes a Non-Departmental Public Body (NDPB) in April 2006, there may be access implications for some users.

In Scotland, the cost associated with access to the Scottish Soils Database held by the Macaulay Institute is an issue for many users. This issue may be resolved in the future. However, it is unlikely that access to soil data will ever be free, due to the need to charge a handling fee.