Stream 2 of the Regional NRM Planning for Climate Change ...€¦ · Web viewIn the year 2014-2015 impacts and adaptation projects were primarily focused on delivering climate change

Stream 2 of the Regional NRM Planning for Climate Change Fund 2013 – 2016 Final Evaluation (Final Report)Prepared for the Department of the Environment24 May 2023

Clear Horizon Consulting129 Chestnut Street, Cremorne VIC 3121 Tel: +61 3 9425 7777 www.clearhorizon.com.au

http://www.clearhorizon.com.au/

Contact details

Clear Horizon Contact Client contactStuart RaetzClear Horizon Consulting(03) 9425 [email protected]

Science Partnerships SectionScience DivisionDepartment of the [email protected]

Project Director Lee-Anne MolonyInternal Contributors Stuart Raetz, Lee-Anne Molony, Jill Campbell, Thomas

HannonExternal Contributors Patrick O’ConnorLast saved 16/11/2016 12:30 AMClear Horizon Reference Number

CH12_166

Disclaimer

This document has been produced with information supplied to Clear Horizon by projects and stakeholders involved in the Australian Government’s Stream 2 of the Regional NRM Planning for Climate Change Fund 2013 – 2016 program. While we make every effort to ensure the accuracy of the information contained in this report, any judgements as to suitability of the information for the client’s purposes are the client’s responsibility. Clear Horizon extends no warranties and assumes no responsibility as to the suitability of this information or for the consequences of its use.

Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting i

mailto:[email protected]

mailto:[email protected]

Contents1. Executive Summary.........................................................................................5

1.1. Background................................................................................................51.2. Stream 2 final evaluation...........................................................................61.3. Final evaluation findings............................................................................61.4. Key learnings from Stream 2.....................................................................8

2. Introduction....................................................................................................102.1. Background..............................................................................................102.2. Scope.......................................................................................................112.3. Purpose....................................................................................................112.4. Audience..................................................................................................112.5. Key evaluation questions.........................................................................12

3. Findings..........................................................................................................133.1. The effectiveness of Stream 2.................................................................133.2. The relevance of Stream 2.......................................................................313.3. The appropriateness of Stream 2.............................................................403.4. The efficiency of Stream 2.......................................................................433.5. The legacy and sustainability of Stream 2...............................................45

Attachment One: Stream 2 Program Logic...........................................................51

Attachment Two: The Cluster Model.....................................................................52

Attachment Three: Results chart..........................................................................53

Attachment Four: Stream 1 Report 2013-14 – 2014-15 Synthesis........................63

Attachment Five: Methodology.............................................................................70Approach...........................................................................................................70Data collection..................................................................................................70Data analysis.....................................................................................................71Data synthesis...................................................................................................71Limitations.........................................................................................................72

Attachment Six: Interview guides.........................................................................73

Attachment Seven: Output definitions.................................................................79

Attachment Eight: Glossary of terms....................................................................81

Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 2

List of figuresFigure 1. Summary of Stream 2 outputs and outcomes (2013 - 2016).................15

Figure 2. Climate Change in Australia website users per month..........................18

Figure 3. National Projections help desk client support groups............................18

Figure 4. The extent to which AdaptNRM modules met user needs.....................37

Figure 5. Stream 2 Program Logic........................................................................51

Figure 6. The Cluster Map showing the eight Stream 2 NRM clusters..................52

List of tablesTable 1. Evaluation audience and information requirements...............................11

Table 2. Summary of outputs delivered by Stream 2 (2013 – 2016)....................14

Table 3. Stream 2 products stored in Terra Nova (February 2016)......................17

Table 4. Terra Nova page views...........................................................................17

Table 5. Examples of use of Stream 2 product by NRM representatives..............21

Table 6. Modes of engagement used by Stream 2 projects..................................34

Table 7. Stream 2 Results Chart (2013 – 2015)....................................................53

Table 8. Uses of Stream 2 products by NRM regions as reported in Stream 1 reporting (2013-14 – 2014-15).............................................................................63

Table 9.Glossary of terms.....................................................................................81


AcronymsCMADCCEEDEWNRDIICCSRTE

EOPOERF

Catchment Management AuthorityAustralian Government Department of Climate Change and Energy EfficiencySouth Australian Department of Natural Resources and WaterAustralian Government Department of Industry, Innovation, Science, Research and Tertiary EducationEnd of Program OutcomeEmissions Reduction Fund

GBCMAIPCCKEQ

Goulburn Broken Catchment Management Authority…Intergovernmental Panel on Climate ChangeKey evaluation question

LLSM&E

Local Land ServicesMonitoring and Evaluation

MCAS-SMCMANACCNARCLiMNCCMANESPNGONR AMLRNRM

Multi-Criteria Analysis Shell for Spatial Decision SupportMurray Catchment Management AuthorityNorthern Agricultural Catchments CouncilNSW / ACT Regional Climate ModellingNorth Central Catchment Management AuthorityNational Environmental Science ProgramNon-Government OrganisationNatural Resources Adelaide & Mount Lofty RangesNatural Resource Management

PWG Project working groupRISCARPSCNRMSWCCUSQ

Resources InventorySouthern Slopes Climate Change Adaptation Research PartnershipSouth Coast Natural Resource ManagementSouth West Catchments CouncilUniversity of Southern Queensland


1. Executive SummaryThis report presents the final evaluation of the implementation of Stream 2 of the Regional Natural Resource Management Planning for Climate Change Fund.

1.1. BackgroundThe Australian Government’s Regional Natural Resource Management (NRM) Planning for Climate Change Fund (NRM Fund) provided $43.9 million over five years to improve regional planning for climate change and help guide the location of carbon and biodiversity activities.

The NRM Fund was administered by the Australian Government Department of the Environment1 (the Department) and delivered through two streams: Stream 1 ($28.9 million) to support regional NRM organisations to revise existing regional plans; and Stream 2 ($15 million) to produce regional level climate change information and provide guidance on the integration of that information into regional NRM and land use planning.

The objective of Stream 2 of the NRM Fund was to improve the capacity of regional NRM organisations to plan for climate change by improving the quality and accessibility of regionally relevant information on climate change impacts and potential adaptation responses.

The Stream 2 program was delivered through eight regional projects and two national projects (‘Stream 2 projects’):

a ‘National Projections’ project lead by the Commonwealth Scientific and Industrial Research Organisation (CSIRO) and Bureau of Meteorology (BoM) to deliver regionalised climate projections,

eight ‘climate impacts and adaptation’ projects (Monsoonal North, Murray Basin, Wet Tropics, Central Slopes, Southern and South Western Flatlands, Southern Slopes, East Coast and Rangelands) delivered to eight NRM ‘clusters’ based on existing regional NRM boundaries, and

a national ‘impacts and adaptation’ project (the ‘AdaptNRM’ project).

A NRM Information Management Project with Griffith University has also been supported with NRM funding to enhance the Terra Nova web portal as the repository of Stream 2 published material. Clear Horizon Consulting was engaged by the Department to provide monitoring and evaluation (M&E) support for the program and to undertake a mid-term evaluation (2014) and a final evaluation (2016).

1 There have been a number of agencies responsible for the administration of the program since its inception due to administrative arrangement order changes including the Department of Environment, the Department of Climate Change and Energy Efficiency (DCCEE) and the Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education (DIICCSRTE).Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 5

See Section 2 for more detail on the Stream 2 projects.

1.2. Stream 2 final evaluationThis final evaluation covers the entire program timeframe (from 2013 to 2016) and the investment of $15 million. The purpose of this final evaluation is to demonstrate the achievements of Stream 2, share learning, and contribute to the knowledge base for research and investment in climate change science and NRM planning.

This final evaluation was prepared by drawing on multiple data sources including: a series of 36 semi-structured phone interviews2; annual project reporting (between 2013 and 2016), Stream 1 regional reporting (39 NRM regions); NRM planning documentation (16 NRM regions); departmental records; and project documentation and resources. The evaluation findings are presented below.

1.3. Final evaluation findingsStream 2 has effectively met its overarching objective of delivering regionally-relevant climate change information and support to NRM regions via eight cluster projects, the National Projections project and the national impacts and adaption project (AdaptNRM). This was supported by fit-for-purpose, regionally-specific engagement. The findings presented below focus on what and how information has been delivered, the support given to enable planners to integrate climate science into NRM planning and the relationships and linkages that have developed as a result, and the legacy that is likely to remain from the program.

Delivery of information to assist climate adaptation by regional NRM bodies

NRM regions can now approach climate adaptation planning with more confidence – knowing they have access to scientifically-credible climate information and resources.

At the end of Stream 2 in 2016 regionally-relevant, high quality climate change impacts and adaptation information is more accessible to NRM regions than at the beginning of the program. A total of 958 NRM climate research and planning products3 have been delivered by the ten projects including 183 next generation climate change projections products. More than 268 engagement activities have been delivered across ten projects to support the use and application of climate change research and science in the NRM planning process. Through web access, up-to-date regionally relevant climate change information is more accessible to NRM organisations and stakeholders than previously.

2 Interviews were undertaken with regional NRM representatives (n=20), Stream 2 project representatives (n=13) and the Department of Environment representatives (n=3).3 Including interim, draft and final products delivered by Stream 2.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 6

Integrating climate science into NRM planning

Most NRM regions are using the climate science from Stream 2 of the Regional Natural Resource Management Planning for Climate Change Fund for their planning.

Uptake and application of Stream 2 products by NRM regions is widespread with around three quarters of the NRM regions specifying examples of using Stream 2 outputs in the planning process. Climate science outputs developed by Stream 2 have been used as technical inputs into NRM planning (e.g. use of the Agricultural Production Systems Simulator [APSIM] by Northern Tablelands in the Central Slopes cluster). Significantly, there was evidence that Stream 2 science products have influenced the planning frameworks adopted and used by NRM organisations in some regions to guide their thinking about how to plan for climate change. For instance, the Adaptation Pathways approach further developed by the Southern Slopes project was used by the Corangamite CMA and in East Gippsland CMA in Victoria to guide spatial NRM planning.

Stream 2 has had a marked influence on the NRM planning process as well as delivering products and resources to support regional NRM planning. Where Stream 2 has had a positive effect on planning capacity within NRM organisations, there tends to also be strong linkages, ties and geographical proximity to the Stream 2 project teams.

Interactions between researchers and planners

The relationships between researchers and NRM planners – developed through many types of interaction – led to enhanced capacity and understanding amongst both the science and NRM communities.

A range of engagement and delivery approaches were used by Stream 2 projects to understand and respond to the planning needs of NRM regions. For the most part, this has supported the delivery of products and support to NRM regions that has been well received, relevant and fit-for-purpose. Multiple modes of engagement were often necessary to ensure coverage across socially diverse and geographically wide-spread cluster areas. User uptake of information was supported in regions and clusters where there was a common and widely-accepted focus (e.g. agriculture) and existing networks (e.g. where there were well-established industries and linkages). The devolved funding agreements set up by the Department enabled Stream 2 projects to tailor their engagement activities and processes to the specific clusters with which they were working.

By and large the cluster approach taken in Stream 2 has led to inter-regional learning and connections across NRM groups that would not have otherwise been established. This was strongest in clusters where: (i) inter-regional forums were established and brokered by the project teams, and (ii) existing inter-regional forums existed (i.e. Victoria and South Western Australia). While the cluster model has facilitated inter-regional learning in most clusters, in others it has reinforced existing geographical and institutional divisions across state boundaries. For instance, in the Southern and South-Western Flatlands cluster, the South Australian NRM regions (Eyre Peninsula, Northern and Yorke, Adelaide and Mount Lofty Ranges and Kangaroo Island) are geographically split from the Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 7

Western Australian based project team and NRM regions (Northern Agricultural, Wheatbelt, Perth, South West and South Coast). This split was exacerbated by repeated staff turnover in the liaison point between the Perth-based project team and the South Australian NRM planners. Ultimately, inter-regional learning in this cluster was compromised as a result of these factors, which relate to both the design of the cluster model as well as project implementation (staff turnover).

Stream 2 has not fully realised considerable opportunities for greater program coordination and increased efficiencies across project and cluster boundaries. This is due largely to the fact that the program did not mandate or build any formal structures for information sharing into the program. Coordination across clusters could have been strengthened for example if the Department had established a centralised program coordination role with clear responsibilities for encouraging and supporting opportunities for information sharing.

The legacy of Stream 2

While there is uncertainty and considerable challenges ahead in regional planning for climate change the outcomes of the program are likely to be enduring well beyond the timeframe of delivery.

Reflecting the outcomes of the program, the legacy of Stream 2 includes significant advances by regional bodies in their ability to adaptively plan for climate change and the embedding of new information and knowledge into regional planning processes. As regional NRM organisations adaptively plan for climate change they will have improved access to responsive, adaptive tools and frameworks (such as the adaptation pathways approach), research outputs, online resources and platforms (such as the Climate Change in Australia website and the Terra Nova platform), as well as improved networks and relationships between researchers, planners and NRM representatives.

While scientists, researchers and NRM managers and planners will continue to derive benefits from the outcomes achieved by the program there are a range of challenges and constraints, which will affect the legacy of the program. These include:

the cluster model was designed purely for Stream 2 and has no longevity beyond the project timeframe (2013 – 2016), though it potentially has applicability for future federal NRM investment and programming

lack of ongoing funding to support the application of the Climate Change in Australia website beyond the possibility of short-term basic maintenance through a project under the Australian Government National Environmental Science Programme (NESP)

staff turnover and organisational restructuring within regional bodies resulting in a loss of corporate knowledge and capacity built through the program

low levels of awareness about Stream 2 within the Department and at a state level among relevant agencies and departments

absence of formal structures or resources to sustain linkages between researchers and planners.


Ultimately the responsibility for implementing NRM plans lies with a diverse range of state, regional and local, community stakeholders who operate within the networks facilitated by regional NRM bodies. As these stakeholders begin to implement NRM planning for climate change, the influence that the program has had on NRM organisations’ ability to adaptively plan and manage climate change impacts will become increasingly apparent.

1.4. Key learnings from Stream 2Based on the findings of this evaluation, a number of learnings for any future Australian Government investment in climate change impacts, adaptation research and NRM planning have been identified. Applying the learnings will necessitate coordination and support from other sectors such as regional NRM bodies and researchers.

The ‘impact’ of Stream 2 was largely on the NRM planning process. The update of NRM plans was a benefit of Stream 2 but the lasting value of this program will be the influence on the regional NRM planning process more generally. In this regard, the program’s impact is influenced by state policy frameworks that determine the role of NRM plans. Enhanced linkages and relationships between planners and researchers are critical in supporting the program’s contribution to the regional NRM planning process.

Learnings for consideration in future investments are outlined below.

Recognise the role of engagement in supporting science translation. Investment in climate adaptation at the science to policy and management interface must explicitly address the need for engagement between the producers and users of science. This can be done by simply requiring a percentage of funds or grants devoted to supporting science translation to be apportioned specifically to engagement activities. Early and sustained engagement between the producers and users of science is critical to support uptake.

Bridging the climate science and practitioner gap is necessarily challenging. Researchers and NRM practitioners operate with different worldviews in different systems. The process of bringing scientists and practitioners together is a dynamic process which requires adaptive, flexible approaches in procurement, programming, project management and evaluation.

Investment in science and NRM at a national scale requires sufficiently resourced coordination. To fully realise the benefits from a program of this scale - such as cross-cluster learning - it is necessary to adequately fund program coordination beyond a formal administrative capacity.

Take institutional considerations into account in further applications of the cluster model. Stream 2 has proven to be an effective trial of the cluster model for delivering NRM investment nationally. There have been significant benefits in delivering the approach on bio-climatic factors. However institutional factors, including state and institutional boundaries, should be more fully considered to maximise the benefits of this approach. This was


most acutely demonstrated in a geographical split between South Australia and Western Australia in the Southern and South-Western Flatlands cluster.

Provide sufficient flexibility in contracting arrangements in any future funding program for climate change adaptation. This program has demonstrated that program design doesn’t need to be prescriptive to effectively deliver outcomes. Indeed, it is arguable that, in order to be effective, climate adaptation investment needs to be sufficiently flexible and responsive to emergent needs, priorities and values.

Carefully consider the sequencing in designing future NRM science and planning investment. The concurrent delivery of Streams 1 and 2 presented considerable challenges as well as opportunities for projects in achieving the objectives of the NRM Fund. In some regions, NRM plans were updated with Stream 1 funding prior to receiving updated science and research information through Stream 2. This concurrent timing created difficulties but also opportunities for the program as researchers and NRM representatives were required to engage with one another in ways that would not otherwise have been possible.


2. Introduction

2.1. BackgroundThe Regional Natural Resource Management (NRM) Planning for Climate Change Fund (NRM Fund) was allocated $43.9 million over five financial years to improve regional planning for climate change and help guide the location of carbon and biodiversity activities. The NRM Fund was one of the Land Sector Measures under the Australian Government's Clean Energy Future (CEF) Plan (now defunct).

The NRM Fund was provided in two streams:

Stream 1 has $28.9 million over four financial years to support regional NRM organisations across Australia to revise existing regional NRM plans to help identify where in the landscape adaptation and mitigation activities should be undertaken. This stream was administered by the Australian Government Department of the Environment (hereafter, ‘the Department’).

Stream 2 has $15 million over four financial years to produce regional level climate change information and provide guidance on the integration of that information into regional NRM and land use planning. This stream was also administered by the Department.

The objective of Stream 2 of the NRM Fund was to improve the capacity of regional NRM organisations to plan for climate change by improving the quality and accessibility of regionally relevant information on climate change impacts and potential adaptation responses. Key components of the approach to Stream 2 delivery include a focus on delivering to end-user needs, with project teams required to work closely with the NRM clusters to identify and prioritise the impacts information they need. A further overarching program focus was on improving the way climate change information is delivered, managed and accessed by decision-makers into the future.

Of the total $15 million funding some $13.63 million was allocated to ten major projects (‘Stream 2 projects’) and an information management support project. Each Stream 2 project has designated ‘Project Leads’. The projects include the ‘National Projections’ project (supported by in-kind contribution from Commonwealth Scientific and Industrial Research Organisation [CSIRO] and the Bureau of Meteorology [BoM]) and eight ‘Impacts and Adaptation’ cluster projects4 (Monsoonal North, Murray Basin, Wet Tropics, Central Slopes, Southern and South Western Flatlands, Southern Slopes, East Coast and Rangelands) as well as one national project focused on delivering regionally-relevant impacts and adaptation information at the national level (the ‘AdaptNRM’ project). An additional project, the national information management project, was added in June 2013 (implemented between 2013 and 2015).

4 In the original project design the National Projections project was referred to as ‘Element 1’, while the ‘Impacts and Adaptation’ projects and ‘AdaptNRM’ were collectively referred to as ‘Element 2’.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 11

The National Projections project started work in September 2012, while the 8 clusters and the AdaptNRM projects started between the end of January and the end of April 2013.

2.2. ScopeThis report presents the findings of the final Stream 2 evaluation. This evaluation covers the entire program timeframe from 2013 to 2016 and investment ($15 million). The evaluation focused on program impact, effectiveness of outcomes achieved, and legacy. The evaluation has focused on the outcomes rather than the process of delivery which was considered in the interim evaluation (conducted in 2014).

The purpose of the final evaluation is consistent with the original purpose outlined in the Monitoring and Evaluation (M&E) Framework in 2013: to provide accountability, identify learnings and to contribute to the knowledge base.

2.3. PurposeThe purpose of this final evaluation report is to:

Demonstrate progress towards program goals and objectives5

Share learnings between projects and clusters Contribute to the knowledge of what works and why for future research

and investment decisions with regard to climate change science and NRM planning

Meet information requirements set by the Australian Government6.

This report specifically addresses these requirements by outlining the results of Stream 2 projects against the key evaluation questions (KEQs) of the Stream 2 M&E Framework.

2.4. AudienceThe primary audiences for this final evaluation report are the Department and the Stream 2 project teams. The primary audience for the evaluation and their information needs are set out below.

Table 1. Evaluation audience and information requirementsWho What do they need to knowDepartment of the Environment

Effectiveness and value of the program outcomes Appropriateness of the model Impact and legacy Learnings on research uptake and the model for

connecting research and planningStream 2 project teams Impact and legacy

Effectiveness and value of the program outcomesRegional NRM Impact and legacy

5 The goals and objectives for Stream 2 are outlined in Section 2.1.6 Formerly stipulated by the Land Sector Carbon and Biodiversity Board (LSCBB).Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 12

organisation representatives

Effectiveness and value of the program outcomes

Minister for the Environment

Impact of Stream 2 on NRM planning: pathways to action Legacy

Stream 1 program team Learnings from Stream 2 and its interactions with Stream 1

Legacy

Secondary users of M&E findings presented in this report include regional and cluster representatives involved in NRM planning as well as a range of other external NRM stakeholders including policy and research practitioners.

2.5. Key evaluation questionsTo address the purpose of M&E for Stream 2, a number of key evaluation questions (KEQs) have been identified. The KEQs have been used to structure and inform all M&E activities at a program level and cover the five categories of effectiveness, relevance, appropriateness, efficiency and legacy.

The KEQs for Stream 2 of the NRM Fund are:1. How effective was Stream 2 in achieving its intended outcomes?2. To what extent was Stream 2 relevant to the context and needs?3. To what extent was the implementation process appropriate? 4. How efficient was Stream 2?5. How sustainable and enduring are the outcomes of Stream 2 likely to be?

A detailed methodology for undertaking this evaluation is provided in Attachment Five.


3. Findings

3.1. The effectiveness of Stream 2This section will address key evaluation question (KEQ) one: How effective was Stream 2 in achieving its intended outcomes? The intended outcomes of Stream 2 were to:

Improve the quality and accessibility of regionally relevant information on climate change impacts and potential adaptation responses

Improve the capacity of regional NRM organisations to plan for climate change

Contribute towards government objectives of the Carbon Farming Initiative (CFI) and the Biodiversity Fund in guiding the location of carbon and biodiversity activities in the landscape.

Stream 2 has effectively met its overarching objective of delivering regionally-relevant climate change information and support to NRM regions via eight cluster projects, the National Projections project and the national impacts and adaption project (AdaptNRM).

This was supported by fit-for-purpose regionally-specific engagement. Within a broader shift towards adaptive planning approaches more generally, the influence and value of Stream 2 was highest in regions where this shift was already taking place. In these regions, e.g. Corangamite (Southern Slopes), Goulburn Broken (Murray) and South West Catchments Council (South Western Flatlands), Stream 2 has catalysed the transition towards adaptive planning approaches, which recognise the dynamic, uncertain, inter-connected, trans-disciplinary nature of NRM planning for climate change.

This section will address the overarching question of effectiveness by outlining the outputs delivered by Stream 2 (3.1.1), the quality of the science delivered (3.1.2), the accessibility of the climate change information provided (3.1.3), the use and application of Stream 2 products by NRM regions (3.1.4), the influence of the program on regional NRM capability and capacity (3.1.5) and research capability (3.1.6), as well as the contribution of the program towards Government objectives under the CFI and the Biodiversity Fund (3.1.7).

3.1.1 Outputs delivered by Stream 2

Outputs delivered by Stream 2 consisted of knowledge products and engagement processes to support the uptake of products, capacity development and ultimately improved NRM planning for climate change (Figure 1). Between 2012 and 2016 fit-for-purpose product delivery and capacity development were supported by a total of 268 engagement activities including consultations, planning meetings and workshops to understand planner information needs.


A total of 958 information products and outputs were delivered by projects (including interim and draft products), while a total of 183 next generation climate change projections products were delivered to NRM regions and project teams. The National Projections were used by all project clusters to develop communication products for NRM planners.

Table 2. Summary of outputs delivered by Stream 2 (2013 – 2016)

Indicator Activities

Outputs/products7

2013-2014 2014-2015 2015-2016PROGRAM

TOTALSub-total

Yearly

totalSub-total

Yearly

totalSub-total

Yearly

totalResearch teams have worked with regional NRM organisations, through the eight clusters, to identify priority climate change issues and the information they will need to inform planning (end year 1)

Consultations 98

268 268

Workshops 42Presentations 42

Program management meetings

75

Other 118

Quality and timely climate change information and support is delivered to regional NRM organisations via the eight clusters

Training 13

316

15

394

17

248 958

Products 1659 16510 5511

Consultations 67 60 6

Workshops 17 18 37Presentations 31 48 45Mentoring (e.g. one-to-one support)

15 66 6412

Peer reviews 8 22 5Other - - 1913

Next generation climate change projections are delivered to regional NRM organisation via the eight clusters (one-off deliverable) (year 2 through 4)

Training 1

16

4

138

2

29 183

Products 2 46 6Consultations 10 37 6Workshops 3 5 3Presentation - 45 6

8 Central Slopes: reports; Rangelands: cluster meetings, presentations; Southern Slopes: newsletters9 Summary reports 15; data sets 125; guidance documents 4; fact sheets 7; other 14.10 Summary reports 29; datasets 15; guidance documents 3; fact sheets 38; webpages 33; other 47.11 Summary reports 15; datasets 8; guidance documents 8; fact sheets 11; other 13.12 One-to-one support: Adapt NRM approx. 45; Central Slopes 7; Murray Basin 10; Southern Slopes 1. The Wet Tropics and Southern and South Western Flatlands clusters also reported providing many instances of phone and email support to NRM planners.13 Murray Basin: sub-project teleconferences 8; Southern Slopes: project management meetings 11.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 15

Indicator Activities

Outputs/products2013-2014 2014-2015 2015-2016

PROGRAM TOTALSub-

totalYearl

y total

Sub-total

Yearly

totalSub-total

Yearly

totalsTechnical support - 1 6

N/AMedia articles 6

66

158

33 54Academic publications - 9 25

2012-13In the first reporting year (2012-13) Stream 2 activity primarily focussed on planning and engagement. A number of engagement approaches were used to design, prioritise and plan projects. Consultation was undertaken via a range of means including workshops, face to face meetings, teleconferences and phone discussions. Workshops were the most common means of consultation used by projects to identify user needs and requirements.

2013-14In the second reporting year (2013-2014) impacts and adaptation projects broadened and deepened engagement with NRM regions. At this stage, they also began delivering climate change information and support to regional NRM organisations via the eight clusters and the two national projects. During the year there were some 268 engagement activities conducted by projects to identify priority issues for NRM planning; as well as the delivery of 319 outputs during this period to deliver information to NRM regions.

2014-15In the year 2014-2015 impacts and adaptation projects were primarily focused on delivering climate change information and support to NRM regions via the eight clusters and the two national projects. In 2014-2015 next generation climate change projections information were delivered to regional NRM planning organisations, cluster projects and the National Projections team. During the year there were: 141 outputs to deliver next generation climate change projections, mostly through products, presentations and consultation; and 340 outputs to deliver information to NRM regions, the majority of which were consultations, presentations and summary reports14. An important output from the National Projections project was the launch of the Climate Change in Australia website.

2015-16In the final reporting year (2015-2016) projects primarily delivered advisory activities. As expected, there were less products produced in the final year (55) most of which were summary reports, guidance documents and factsheets. In 14 See Attachment Eight for a list of definitions for outputs and products delivered.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 16

http://www.climatechangeinaustralia.gov.au/en/

the 2015-16 reporting year there were 64 one-to-one mentoring activities for NRM planners and 45 presentations were delivered by project teams. One-to-one mentoring and the presentations included support to develop content for NRM plans, frame adaptation pathways and use climate change information.

Figure 1. Summary of Stream 2 outputs and outcomes (2013 - 2016)

3.1.2 Quality of Science Delivered

Quality refers to the accuracy and degree of scientific certainty and rigour associated with Stream 2 products as opposed to the useability and relevance of products (see Section 3.2). Quality was assured, in part, through ongoing peer review, and editorial and quality assurance processes undertaken in projects. It was also supported by the application and selection process of projects undertaken by the Department, which took into account the scientific credentials of proponents. By year two of the program a total of eight peer reviews had been reported by four clusters to support the delivery of quality products. By year three, 22 peer reviews had been reported from an additional two clusters. In addition to NRM planner reviews, nine academic publications were reported for the 2014-15 reporting year. The number of peer reviewed academic publications rose to 25 in the final year (2015-16) of the program.

3.1.3 Accessibility of climate change information

Stream 2 has reached a broad range of stakeholders with climate change information including indigenous NRM groups and remote regional groups across Australia. Relevant information on climate change impacts and adaptation was made more accessible to NRM organisations between 2013 and 2016 through:

delivery of 958 outputs to support regional NRM organisations via the eight clusters and the two national projects, including: information products (reports, data-sets, guidance documents, fact sheets), training, mentoring support via workshops, presentations and other delivery methods;

delivery of 183 outputs which translate next generation climate projections to NRM planners via the eight clusters and the AdaptNRM project, including: information products (web tools, journal papers, animations, spatial layers and data sets), presentations, consultations and other delivery methods;


launch of the Climate Change in Australia website in January 2015 by the National Projections project. By 22 June 2016 the web site had received some 153,746 unique visitors with 1,011 registered users; while the AdaptNRM website has had 20,000 unique page views (February 2016) since launching in June 2014.

Throughout the program, accessibility was also reported to have been supported by engagement processes within clusters (see Section 3.2 for a discussion of engagement). At the start of the program (2013-14), Stream 2 was considered an important source of information in the majority of Stream 1 applications (39 out of 53 regions). This shows that many NRM planners were aware of and prepared to engage with Stream 2 products at the beginning of the program.

The NRM Information Management ProjectNear the end of Stream 2 over 100 content items15 developed by Stream 2 projects were stored in the Terra Nova16 repository with metadata descriptions to encourage their discovery. A summary of the products stored in the Terra Nova web-platform is provided below in Table 3 below.

15 At the time of reporting (June 2016) the number of content items stored on the Terra Nova platform exceeds 100 products, which was the amount reported in the NRM Information Management final report in February 2016.16 The Terra Nova web platform is an information hub established to store climate adaptation data. Terra Nova was used to store information developed through the Stream 2 program. Website: https://www.terranova.org.au/Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 18

https://www.terranova.org.au/

Table 3. Stream 2 products stored in Terra Nova (February 2016)Cluster Items Metadata quality17

Central Slopes 4 6/10East Coast 32 10/10Monsoonal North 8 7/10AdaptNRM 6 6/10Rangelands 7 folders 10/10Southern Slopes 14 9/10Southern and South Western Flatlands 6 8/10Wet Tropics 7 7/103C Modelling (East Coast, Central Slopes and Murray Basin)

16 9/10

Murray Basin 6 6/10

(Source: NRM Information Management Project Final Report, February 2016)

Stream 2 products have been accessed widely through the Terra Nova platform. Table 4 provides a summary of the page views for project collection (per project) and individual content (per item), which total 12,994 and 10,925 respectively.While the Terra Nova database is one of many platforms for data storage and access in the program, it was intended as the centralised repository for information. With open content and an easily accessible web-platform, Terra Nova provides “high discoverability” as stated in the NRM Information Management Project report. Because of the large number of draft and interim products developed by Stream 2 projects, not all products developed through the program are stored on the Terra Nova platform.Table 4. Terra Nova page viewsCluster Collection page views Content page viewsEast Coast 2,989 2,562Southern Slopes 2,313 1,9403C Modelling (East Coast, Central Slopes and Murray Basin)

1,498 1,321

AdaptNRM 1,112 984Central Slopes 1,085 833Murray Basin 972 779Rangelands 929 809Wet Tropics 854 735Southern and South Western Flatlands 657 556Monsoonal North 585 406

Total views 12,994 10,925

(Source: NRM Information Management Project Final Report, February 2016)

The accessibility of National Projections

Since the launch in January 2015, the National Projections Climate Change in Australia website has recorded 1,011 registered users and some 153,746 unique 17 As rated by the NRM Information Management project team using criteria including: keywords and open licensing marking.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 19

users, of which 22.6 per cent were return visitors, who undertook 202,446 sessions averaging more than 12,100 sessions per month (from 27 January 2015 until 22 June 2016). See Figure 2 for the total number of users per month from the website launch in April 2015 to February 2016.

Figure 2. Climate Change in Australia website users per month

(Source: National Projections Project)

A total of 122 client support requests were provided by the National Projections helpdesk to a range of users from universities, NRM organisations and consultants being the main user groups (see Figure 3). The helpdesk was supported by additional in-kind contribution from CSIRO to support service delivery.

Figure 3. National Projections help desk client support groups

(Source: National Projections Project)

Although the types of requests fielded by the National Projections helpdesk varied widely, assistance with NRM information was the main type of client service contact provided, with 41 per cent of requests relating to NRM. In a follow up survey of website users by the National Projections team18 the survey found that nearly 60 per cent of users were “next-users; e.g. using the information for impacts assessment”, while around 20 per cent were either “generators of climate change information; e.g. climate researcher” or “end-user of climate change information; e.g. planner and/or decision maker”.18 Sent to over 800 registered users, the survey received 99 responses.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 20

3.1.4 Use and application of Stream 2 products by NRM regions

At the end of the program in 2015-16, NRM planners are now applying and using a range of Stream 2 products, tools and approaches in the NRM planning process. The extent of use and uptake varies from region to region within each cluster. Factors which supported utilisation include the timing of release and engagement, institutional capacity, planner capability, linkages and ties to Stream 2 projects as well as peer support networks within and between regional representatives. Due to the wide regional variation and national scale of Stream 2, the actual extent of uptake has not been fully assessed19. However, based on Stream 1 reporting and interviews with NRM representatives (n=20) it is estimated that around three quarters of NRM regions have adopted, used and applied Stream 2 products in their planning processes. Ultimately the use of products may not be manifest for some time to come, with several planners stating an intention to use products, particularly the AdaptNRM Biodiversity Modules.

As would be expected, the use and uptake of Stream 2 products has largely taken place in the final year of the program (2015-16) following an extensive process of engagement and support in the preceding years (2013- 2015). At the mid-point of the program (2014), uptake of Stream 2 products was limited and primarily consisted of NRM representatives referencing and citing products in NRM plan development. Product uptake and adoption tended to be stronger in regions and clusters where there was a clear focus and strong social cohesion around a particular issue (i.e. agriculture). By contrast, biodiversity uptake was less clear due to greater uncertainty and lower levels of community support and institutional mandate.

Refer to the Northern Tablelands LLS case study below for an example where planning needs were met through engagement by Stream 2’s Central Slopes.

Case Study: Product use – Northern Tablelands Local Land Services

The Stream 2 researchers from the Central Slopes Cluster Project engaged with Northern Tablelands Local Land Services about research needs and developed products and outputs relevant to core areas of planning and program activity. These included products for planning:

Determination of the impacts of climate change on future cropping (from APSIM modelling)

Modelling of the impacts of climate change on native vegetation to identify priority areas for revegetation and protection of native vegetation (3C modelling undertaken in conjunction with AdaptNRM Project)

Development of a climate risk matrix for climate change adaptation

19 Instances of uptake of Stream 2 products presented in this section have been opportunistically sourced through interviews with a non-representative sample of NRM planners (n=20) and via a sample of Stream 1 reporting, representing around three quarters of NRM regions.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 21

planning. This information was used to determine the impacts of climate change on livestock and grazing (predominately). Information was integrated into state and transition models to determine drivers of change and management actions required in response

Development of climate temporal analogues for use as a decision support tool to investigate future climates and similar current climates

Extreme climate event information was integrated into state and transition models to assist in identification of key drivers and key management actions

Information provided on the impacts of climate change on upland wetlands on the Northern Tablelands. The information was integrated into state and transition models

The Reforestation Modelling Tool was used to estimate carbon sequestration amounts in trees and debris pools. This information provided guidance on where the best locations in the region are for sequestering carbon for mitigation.

The Central Slopes group (under Stream 2) have provided many opportunities for collaboration and have provided support wherever possible. This list [above] is not indicative of how much support they have given.

(NRM Planner20)

In the final year, use of Stream 2 products has matured and broadened with NRM representatives using products for a range of purposes including background information, to verify and confirm existing data, engagement and communication, NRM plan development and spatial analysis. In some regions, there is evidence that Stream 2 products such as ‘Adaptation Pathways’ are being used to guide thinking and strategic decision making for the management of natural resources.

In many cases, NRM plans updated under Stream 1 were informed by Stream 2 products, consistent with the intended objectives of the NRM fund.

Updating our strategy is the main work we have done with the information and products we received [through Stream 2]. We have got the majority of the Stream 2 outputs into the strategy: [Stream1 plan, including] biosecurity, temperature and rainfall, and the aquatic species report have all been incorporated. We are working on using what the community value combined with expert opinion and then we put that through MCAS-S into our strategy.

(NRM Planner)

A total of 39 Stream 1 annual project reports were received for the reporting periods 2014-15 and 2015-16, of which 35 reported using Stream 2 products21. 20 Where quotations or data references are included in the report, they are referenced as follows:

“NRM Planner” – A representative of an NRM organisation who has a role in planning, knowledge management, etc. Most likely involved in Stream 1 activities

“Cluster Project Team” – Member of one of the project teams from one of the 8 clusters “National Projections” – member of the National Projections team “AdaptNRM” – member of the AdaptNRM Team “The Department” – staff of the Department of the Environment

21 Stream 1 reporting was provided by the Department for the years 2014-15 and 2015-16. Stream 1 reporting is not representative of all regions because a limited amount of reporting was provided Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 22

Out of the 35 regions who reported using Stream 2 products a total of 85 tools and products22 were identified by regions. Three quarters of the products and tools identified (63 out of 85) were used in the NRM planning process as part of Stream 1. A variety of uses were reported by regions, ranging from inputs into the planning process, as a tool for engagement and consultation to technical uses such as vulnerability and impact assessment. Examples of the uses of Stream 2 products in Stream 1 reporting are provided in Attachment Four.

A total of 51 regional representatives were contacted from 48 regions of which 16 regions provided plans or planning documentation (i.e. technical reports). 12 of these 16 regions provided NRM plans that had used Stream 2 products. Similar to the analysis of Stream 1 reporting above, the uses of Stream 2 products are summarised for selected projects in Table 5 below.

Table 5. Examples of use of Stream 2 product by NRM representativesNRM Org. Cluster Stream 2 Products Use

Natural Resources Adelaide & Mount Lofty Ranges23

Southern and South West Flatlands

Implications of Climate Change for Biodiversity (AdaptNRM), Helping Biodiversity Adapt (AdaptNRM), Draft 3C modelling for biodiversity management under future climate (Drielsma et al, 2014), Southern and southwestern flatlands climate change project: Data layers explained (Ford et al, 2014)

Synthesis identifying implications for NR AMLR management strategies

Eyre Peninsula


Hope P. et al. (2015) Southern and South-Western Flatlands Cluster Report, Climate Change in Australia Projections for Australia’s NRM Regions: Cluster Reports, eds. Ekström, M. et al., CSIRO and BoM Australia.

Global impacts of climate change

Perth NRM


Neville, S. (2014). Planning for Climate Change - MCAS-S Analysis Process, Perth Region NRM. Consultant’s report for Perth Region NRM Inc. Ecotones & Associates, Denmark., WA.

Joint MCAS-S technical report

AW NRM Monsoonal North / Rangelands

Ninti One Ltd and CSIRO publications:Davis J (2014), Scott JK (2014), Bastin G (2014), Measham TG (2014), Pavey CR and Bastin G (2014), Pavey CR (2014), Bastin G, Stokes C, Green D and Forrest K (2014)

Fire risk impacts, aquatic refugia, vulnerability assessment, invasive species and ecology, Buffel grass management

CCMA24 Southern Slopes

AdaptNRM Biodiversity Module Adaptation pathways for NRM assets, Fire risk impacts

based on availability at the time of reporting (March 2016). Reporting is also for different time periods (2014-15 and 2015-16) for different regions.22 Some products identified included packages of information such as the AdaptNRM Modules (1 – 4).23 See AMLR NRM Case Study for further detail.24 See CCMA Case Study for further detail.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 23

In other regions planners reported that Stream 2 outputs had limited influence on their plans. For instance, in the Murray Basin cluster a planner commented that there was “too much data” and information that it was not possible to “interpret it in a way that is useful” at a regional scale. The Southern and South-Western Flatlands Case Study below provides an illustration and description of a synthesis report in the Adelaide and Mount Lofty Ranges (AMLR) region in South Australia, which found that the products did not meet their needs.

Case Study: Product applicability – Southern and South Western Flatlands Natural Resources Adelaide and Mount Lofty Ranges (NR AMLR), South Australia

NR AMLR commissioned a review of the applicability of biodiversity climate change planning products for on-ground NRM in the region1. The review was completed in October 2015 and included review of three Stream 2 products: Product A (Ford et al, 2015) 2, Product B (Williams et al, 2014)3 and Product C (Prober et al, 2015)4. The framework for review assessed the degree to which each product’s findings could directly inform NR AMLR’s on-ground management strategies and was substantially based on the perceived level of uncertainty in each of the products, relative to the quantitative uncertainty associated with NR AMLR’s management strategies. The review was very thorough and undertaken with the specific region’s planning framework as a primary consideration in assessing the utility of any planning product.

Products A and B may provide some guiding principles of relevance to the NR AMLR region’s management strategies at the community and landscape levels because product uncertainty is approximately equivalent to management decision risk. The review indicated that these products ‘may’ have some usefulness for some aspects of regional management strategies but are often limited by mismatch between the actual planning framework used in the region and the planning framework assumed in the products, by the choice of data used in the models (where the region believes more complete or superior data was available), and by the need for additional information to be brought together with the product for further planning to be useful. Product C was not considered to have direct utility for NR AMLR’s management strategies, mainly due to uncertainty in the products findings and implications.

The mismatch between the Stream 2 products and the region’s planning approach and needs is attributed to institutional barriers in knowledge brokering, some of which appear to arise from the arrangements made to cluster Western Australian NRM regions together with non-contiguous South Australian NRM regions.

…to make their (Stream 2) research useful for us it would have made sense to consult with us in the process of designing their research methodologies (for example in selecting their data sources and the parameters used for our region’s analysis), as we could have told them what would be more and less useful for us from the outset.

(NRM Planner)

1Seed Consulting Services (2015) Assessing the applicability of biodiversity climate change


planning products for on-ground NRM in the AMLR region. A report prepared for Natural Resources Adelaide and Mt Lofty Ranges. Adelaide, South Australia. 2Ford, B. & Cook, B. (2015). Southern and South-Western Flatlands climate change project: Data layers explained. Report No CENRM 139. Centre of Excellence in Natural Resource Management, University of Western Australia.3Williams KJ, Prober SM, Harwood TD, Doerr VAJ, Jeanneret T, Manion G, and Ferrier S (2014) Implications of climate change for biodiversity: a community-level modelling approach, CSIRO Land and Water Flagship, Canberra. 4Prober SM, Williams KJ, Harwood TD, Doerr VAJ, Jeanneret T, Manion G, Ferrier S (2015) Helping Biodiversity Adapt: Supporting climate-adaptation planning using a community-level modelling approach. CSIRO Land and Water Flagship, Canberra.

Adoption of planning frameworks and approaches

Adaptation pathways is an example of an approach that was been adopted by a number of regional bodies in the Stream 2 program. It is “an analytical approach to planning that explores and sequences a set of possible actions that are based on external developments over time” (Haasnoot et al. 2013:485 cited in Bosomworth et al, 2015). This evaluation has found that the Adaptation Pathways approach has had an influence on NRM planning approaches within regional bodies in the Southern Slopes, Murray and Wet Tropics clusters.

Case study: Application and uptake of the Adaptation Pathways approachThe adaptation pathways approach was utilised in the Southern Slopes cluster by a joint effort between the research teams and the NRM regions in the cluster who were actively involved in piloting the approach. In the Corangamite region, NRM representatives played a key role in developing and refining the approach with the research team.

As a process now we won’t use anything else. It allows us to put climate change at the centre of our decision making; it is through the whole planning process now.

(NRM Planner)

In the East Gippsland region in eastern Victoria, the adaptation pathways approach was used as part of the planning process to inform community discussions over contested resources.

Adaptation pathways [enabled us to] focus on places the most contested landscapes in the region and get people working together to flush out the issues where there would be a contest for resources. We used it [adaptation pathways] in a workshop with stakeholders. They key was you could draw it on a wall with issues on the vertical axis and time on the horizontal axis. By drawing lines on a grid like this it became clear to people that there were multiple pathways to the same result. For instance, you could jump off this track and get on that track as a result of changes in the issues or timing. As a result of this process we got better thinking. This was useful in difficult discussions in our public consultation.

(NRM Planner)

Meanwhile, in the Wet Tropics cluster the approach was influential in informing the structure and “core” of the Cape York NRM plan:


We are now looking at the core of the [NRM] plan being those adaptation pathways. In everything we’re planning for we’re adapting to change and identifying adaptation pathways. One we know of is fire. The entire Cape York is burnt out each year. We’re moving towards a more targeted approach. Once we reach a point it leads to change in the system. But if you burn correctly you can reduce the impact it has on heat, etc.

(NRM Planner)

In the Murray Cluster a project team member referred to the approach as being part of a broader “paradigm shift” in the way that NRM bodies approach and undertake regional NRM planning (see 3.5 Legacy). While uptake of the adaptation pathways approach was relatively concentrated to a minority of regional bodies in Southern Slopes, Murray Basin and Wet Tropics clusters, there is no doubt that it was an influential approach across the Stream 2 program.

Application of AdaptNRM products by NRM representatives

This evaluation has found specific examples of application of AdaptNRM modules by NRM planners. Interviews with NRM representatives (n=20) highlighted that there was variable uptake of AdaptNRM products by NRM regions with some regions being able to provide examples of how the AdaptNRM products were or could be used while others were not able to do so. Unlike the National Projections project that targeted products at both cluster projects and regional NRM representatives, the AdaptNRM project developed products specifically for use at a regional level.

The AdaptNRM Biodiversity Module in particular was cited by many planners as a tool that they had used or intended to use. A Greater Sydney LLS planner in particular stated that the Biodiversity Module was “transformational” in its emphasis on change and continual adaptation as a principle for biodiversity management.

The AdaptNRM module on biodiversity was transformational in my understanding of climate change. What it emphasised for me was the need to change our thinking. AdaptNRM gave us a series of modules about how we should be approaching future management. These modules are simple in concept but complex in application. For example, one of the modules was about maintaining the evolutionary character of biodiversity. These modules have an implied management principle: to implement management needs to become more interventionist. This is a move away from how things have been: we have been provoked to move from thinking in a stable to a moving world. We need to plan not just for conservation but for change through time.

(NRM Planner)

As the case study below shows, in Corangamite the AdaptNRM Biodiversity Module was also used as a tool for community discussion, which led to an elevated and more detailed understanding of the fire risk to the Otways when ‘ground-truthed’ with community knowledge. This demonstrates using the module as a tool for catalysing discussion as an input into the draft regional


Climate Change plan25. It also provides a useful example of the application of an AdaptNRM module to regional NRM planning.

Case Study: Applying AdaptNRM in the OtwaysAdaptNRM arrived at the right time in 2014-15 when there was no support or direction on Biodiversity from the State Government [Victoria]. Throughout the project we have provided feedback and input into AdaptNRM modules. All our feedback has been taken on board and has been reflected in the revised product. AdaptNRM work has directly informed the region’s draft climate change plan. AdaptNRM did a case study of the Otways, which was used directly in the draft climate change plan. AdaptNRM provided us with a case study of temperate rainforest in the Otways based on one of their modules. The study showed that temperate rainforest is projected to constrict by 2060. We showed it to our experts and they said that taking into account bushfire risk, with hotter, more intense fires a more likely scenario is that the rainforest will disappear altogether by 2060 rather than constrict, which is what the AdaptNRM module predicted. So what the module provided was a basis for discussion, from which we could make a fuller assessment of the likely impacts on the rainforest by 2060. The study [undertaken by AdaptNRM] is written up in the draft climate change plan taking into account both the module and the expert review.

(NRM Planner)

Application and uptake of the National Projections

The National Projections were delivered to NRM planners through direct engagement via a series of national workshops, consultations and presentations by the National Projections team. In addition, NRM regions also received the projections work indirectly through the use of projections in cluster project products and tools. Evidence collected for the evaluation suggests that, due to the concurrent timing of delivery of the cluster projects with the National Projections project, NRM planners were more likely to have received projections work directly from the projections team directly rather than through project products and materials.

The most important tool I have used is the CSIRO projections. Without understanding what the climate looks like we can’t do anything.

(NRM Planner)

Project reporting indicates that the National Projections have been used by all project clusters to develop communication products for NRM planners. In total 183 next generation climate change projections products have been delivered to regional NRM planners via the cluster projects (or directly to planners from the National Projections team).

This is the first time [in Australia] that we have delivered a set of projections that provide much better access to the relevant information for end users. There is evidence that it is being used by next users… the information is being used in

25 As the Draft Climate Change Plan for Corangamite was still under review at the time writing it was not possible to include an excerpt in this report.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 27

impact assessments by researchers to inform adaption planning or update existing NRM plans.

(National Projections)

When interviewed the majority of project leads (6 out of 9) provided further details on how they (AdaptNRM and the clusters) had used National Projections data. The use of the National Projections varied within and between projects depending on the needs of the planners in their regions. At one end of the spectrum summary information from the National Projections was reported to be sufficient for cluster projects to meet planner’s needs (e.g. by developing communication products and inputs to reports).

We actually produced an indigenous brochure for consultation – used in schools and public audience as well [drawing on the projections].

(NRM Planner)

We used [National Projections] summary materials as inputs to some of our outputs (e.g. Info Portal Report cited projections summary information)

(NRM Planner)

The National Projections were used by regional NRM representatives in the development of NRM plans to provide context, background information and to validate existing climate data (where it existed). Some regions (e.g. Eyre Peninsula) commented that the National Projections outputs were the most useful information that they had received through the Stream 2 program. Other regions stated that they had used the projections to facilitate community discussions such as in the Greater Sydney region and in the Alinytjara Wilurara case study below.

We used the projections [in community workshops] to communicate what bushfires might look like in the region going forward. The reason for the workshops was to look at where the community is vulnerable and manage the risk to people and property. We came up with some interesting stuff around balancing ecological values [why people live in vulnerable areas] and management actions for safety [in a changing climate].

(NRM Planner)

Case study: Applying projections in the Alinytjara Wilurara NRM Region

We had a quick look at the communications products but they are not relevant for our region. It is very remote; most people did not speak English and given people struggle with so many other areas of their lives NRM was not the most important issue. We did not want to portray too much of a dark situation [about climate change projections]. We had to be quite careful about how to engage so we used a two-way approach. We used the information and simplified it in some ways [pictorial representations] to trigger conversation. We didn’t use projections much [in the plan] but rather for community consultation. If they raised an issue, then we would bring along information on that. We were trying to see it from their angle; what they knew and then provide more information as


a result. It was being said by them and not us in the first instance.

(NRM Planner)

Several projects also cited using various modelling products (e.g. for carbon sequestration, sea level rise, agricultural productivity, species distribution etc.) which were distributed to NRM planners from the clusters using the projections data.

We find the projections extremely useful… In our project we used a lot of species distribution modelling [which integrated the projections work].

(Cluster Project Team)

We used the projections work in the science to predict things like agricultural productivity and sea level rise. We used their materials and they went straight to the NRM groups.


Where existing projections were available, such as in Tasmania, planners used National Projections outputs to validate and confirm projections made in previous and current NRM plans. While some regions and clusters used different climate data-sets such as NSW / ACT Regional Climate Modelling (NARCLiM) and the existing Tasmanian climate projections conducted by the CSIRO in 2007.

3.1.5 Planner capability and capacity

NRM planners report improved capability and capacity to use, apply and interpret climate change information and tools as a result of their involvement in Stream 2. The influence on planning capacity varies from region depending on the organisational context, level of turnover within the organisations, the background, experience and expertise of the NRM representatives. In addition, broader factors such as access to services, remoteness and proximity to high population centres, where the research teams tend to be located also influenced capacity development. Also, it is important to recognise that NRM organisations were operating from different levels of capacity prior to the program, which has necessarily influenced the capacity gains made by Stream 2 across the country.

The principle factor in the program that influenced the capability of NRM representatives was the strength of relationships and ties to researchers. Where planners reported a good relationship and direct access to researchers (i.e. being located close to researchers) they were more likely to report improvements in their climate change knowledge, understanding and ability to apply Stream 2 tools and approaches. In short, close proximity to research teams enabled greater opportunities for face-to-face contact via informal meetings as well as structured activities (i.e. workshops). By contrast, NRM representatives geographically isolated from research teams were less likely to build strong connections.


All national and cluster projects report having increased planner capabilities to plan for climate change. Project leads discussed having contributed to increasing two main capabilities for planners. Firstly, increasing planning capability to incorporate uncertainty and adaptation pathway approaches and resilience thinking into planning for climate change (AdaptNRM, Murray Basin, Monsoonal North, Southern Slopes, Southern and South West-Flatlands).

The main outcome that I would identify is that they really get that this is about decision making under uncertainty… At the start the NRM planners would have said that it was about data, they were trying to make it a predictive, problem solving exercise. Now they are saying “how do we structure our decisions in pathways and look at multiple futures?” They understand that adaption is about decision making and how you use information.

(AdaptNRM)

In addition, project leads report that the program has empowered planners to communicate climate science in their own regional and organisational contexts.

Those involved with the project feel a bit more confident to talk about climate change with people because they have climate change information for the Rangelands. It’s much easier to speak about what might and might not happen.


Planners also noted improvements in their ability to communicate climate change concepts and issues with greater clarity and ease as a result of Stream 2. In some cases, capability to communicate was enhanced by the confidence of being backed up and supported by perceived “credible” science and national institutions.

Prior to my involvement I was a bit frightened about climate change. For instance, prior to the project I didn’t know how to start a conversation with a farmer. Literally there were rooms that I would enter and could not talk about climate change but now we can discuss it. Before it was like climate change was off in the distance and all we could do was acknowledge it. There are so many good examples that I can use to communicate now; i.e. grain growers in Victoria who have a reduced harvest window etc., Tomato growers, Viticulture etc. There are some solid pieces of information that are in my anecdotes that I can draw on.

(NRM Planner)

Regional representatives stated that they had learned about how to apply and interpret climate data as a result of Stream 2 (for example see Murray CMA [MCMA] below). A Goulburn-Broken CMA representative for example commented that “I’m much more confident with interpreting climatic data and we are more aware of what uncertainty means in relation to Climate Change as planning. [As a result] adaptation planning as a theoretical concept has trickled down through the organisation: we’re getting better at it”. Likewise, the Mallee CMA representative commented in relation to the influence of the National Projections.


It’s definitely been useful for me to get an understanding of how the projections might hit the ground. But at the same time, learning about the limitations of the certainty of what we can and can’t say [has] helped me to communicate to other punters out there about what climate change might mean in the region.

(NRM Planner)

The majority of planners identified challenges in applying Stream 2 outputs. Some commented more pointedly that they had difficulties in understanding and applying Stream 2 outputs.

The projections were not quite there and so we used small scale research on particular NRM issues which were more useful than the projections themselves – like the impact of climate on agriculture and pests. Applied science was more useful for a planning and communications tool than projections which we only included in the technical reports.

(NRM Planner)

A minority of NRM representatives commented that Stream 2 has had limited influence on their ability to plan. For instance, an Eyre Peninsula representative commented that Stream 2 outputs were “too technical” and beyond the capability of the organisation. In the case of Eyre Peninsula, low uptake should be be seen in the context of turnover in the Southern and South-Western Flatlands project team (within the South Australian Department of Environment, Water and Natural Resources [DEWNR]) as well as relative geographic isolation from the project team (see Section 3.3 for a discussion of cluster model).

3.1.6 Researcher capability

All project representatives interviewed (n=13) indicated that having an explicit focus on end users throughout the life of the Stream 2 program had built their capability to deliver products to meet the needs of NRM planners. Interviewees indicated that many researchers who have been part of Stream 2 have gained and or enhanced their capabilities to: engage and communicate with NRM planners, communicate climate change science to NRM planners, be flexible and responsive to changing needs and contexts, manage expectations about what can and cannot be delivered, and ultimately deliver products that respond to NRM representative’s diverse needs and regional contexts.

Some people in our team [have gone from] fairly typical scientists to putting themselves in other people’s shoes… They have developed a better understanding of how regions work, how planning is done, the level of information that they need and the formats that they require it in. For some people it has been part of a broader shift and a really significant broadening [of] skills and capacity.


Similar to planners, researchers also stated that they had developed their ability to communicate as a result of Stream 2, as highlighted below in the East Coast and Central Slopes clusters.

We all learned a lot about presentation, where we are pitching it, they generally don’t want too much data, but we learned more about pitching and providing


information. We learned about what concerns them and their competing concerns. For instance, what we were doing is a small concern [for them]. How complex the space is that they work in.


The main change has been that some scientists have developed more of an ability to speak in plain English… They had to think about what NRM planners need to do their job. They got better at the hard sell. It has come about through dialogue with the NRM planners, their context, constraints and capacity…


Researchers, such as the Southern Slopes team, also noted that they had developed their understanding of regional NRM issues and priorities.

We have a greater understanding of the current NRM situation in this region, though recognising that the situation changes rapidly. Ability to work with NRM planners was enhanced, and some of the team are continuing to explore these relationships through other projects


A highlight cited by all project leads was the strengthening of relationships between researchers and NRM planners, particularly being able to develop a mutual understanding of each other’s needs, priorities and working contexts. For the National Projections team, being focused on planning needs with a clear purpose and mandate has increased their capability to work with end users. More broadly has contributed to a shift in thinking within the CSIRO and the BoM about the way climate projections research should be done.

At the start of the program we had a couple of researchers here who were open about not seeing the value of end user input into our work... The NRM work [via Stream 2] has consolidated our end user focus… getting to know the users is really valuable…


This focus on end users and engagement by the National Projections team has built on capacity developments achieved from previous programs delivered by the CSIRO and the BoM in the Pacific26, leveraging investments made by the Australian Government in climate change science and adaptation.

I think that there is a greater understanding of the challenges associated with delivering climate change information to user groups in an Australian context and we appreciate more the reality of what it is like to be on the land… I don't think that the context of the research we undertake has been this close to the user groups before in Australia; we have gained the capacity for a new focus on supporting next users in Australian context. This kind of work has been undertaken in the Pacific Island Nation context, applying it in the Australian context bought a new set of challenges but that has led to new ideas and tools and a new understanding of the types of tools and products that are used in NRM planning, we have learnt more about how to plug into their models.

26 Via the Pacific Climate Change Science (PACCSAP) program.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 32


AdaptNRM and cluster project leads reported that increases in researcher capability to meet the needs of planners were underpinned by relationships that have strengthened over time (East Coast, Wet Tropics, Rangelands, Southern Slopes).

The highlights have been the enhanced interaction between researchers and planners. We all got a lot out of that. Some of us, as researchers, don’t have direct input into NRM, planners particularly; we definitely got more educated in the process. We understood what their needs were. That was quite illuminating.


It’s become much more collaborative… the researchers would have developed a much better appreciation of how NRMs work, what types of information can be used, what constraints they have to work with. I also think the corollary is also true that NRMs have developed a much better appreciation for the constraints and potential the scientists are experiencing as well.


3.1.7 Supporting government objectives

Ultimately the investment in Stream 2 aims to contribute to updating NRM plans that guide the location of carbon and biodiversity activities that take into account climate change impacts and adaptation responses. Because Stream 1 of the NRM fund was designed to directly update the development of current NRM plans, Stream 2 necessarily interfaced with Stream 1 to achieve this objective. With changes in the direction of Australian Government policy on climate change during the period of delivery between 2013 and 2016, the Stream 2 program was required to continue delivery of outputs despite uncertainty over the possibility of a carbon market and economy.

Stream 2 has supported the achievement of government objectives by providing spatial guidance for biodiversity and carbon offsets under the Carbon Farming Initiative. The link between Stream 2 outputs and government objectives was more apparent in some clusters than others due to the focus and emphasis of their work. For instance, the East Coast cluster included a carbon farming sub-project.

The 2012-2013 annual review identified a clear link and alignment between Stream 2 end-of-program outcomes and broader goals articulated in the Clean Energy Future (CEF) Plan and Land Sector Package objectives. Stream 2 of the NRM Fund was designed by DCCEE (now the Department of Environment) and established by the Australian Government with the aim of contributing to the CEF Plan. More specifically, Stream 2 was designed to contribute CEF Land Sector Measures including the Carbon Farming Initiative, the Biodiversity Fund and the Indigenous Carbon Farming Fund.

In year two (2013-2014), some Stream 2 projects (3 out of 8) had made small, yet potentially significant steps towards Land Sector Package objectives, including: consultation, project development and spatial analysis being taken


towards informing the location of carbon and biodiversity activities. In years three (2014-2015) and four (2015-16) half (four out of eight) of the projects had undertaken explicit activities to contribute towards government objectives of guiding carbon and biodiversity activities. Of these projects, East Coast and South-Western Flatlands provided clear examples of activities to contribute towards the Emissions Reduction Fund (ERF), as detailed below.

East Coast: compiling biophysical, economic and other data for carbon farming assessment; attending meetings with Commonwealth Government representatives regarding new Emissions Reduction Fund framework and the implications for carbon farming (2014-15). Initiating a project to develop a spatial analysis tool to identify the best areas for soil and vegetation carbon sequestration with three NSW LLS’. Finalisation of the Queensland Government Carbon Farming report. Development of a web-based tool to visualise carbon farming opportunities from forest regrowth. Development of a joint-project between consortium researchers to realise opportunities in the ERF (2015-16).

Southern and South-West Flatlands: use of the MCAS-S spatial planning tool by South Coast NRM (SCNRM), South West Catchments Council (SWCC) and Northern Agricultural Catchments Council (NACC) to identify sites for carbon planting (2015-16).

Taking into account the project design as well as interview feedback from project leads the other clusters’ Impacts and Adaptation projects did not explicitly focus on contributing towards the ERF largely because it wasn’t prioritised by the NRM regions.

3.2. The relevance of Stream 2This Section addresses KEQ2: To what extent was Stream 2 relevant to the context and needs? The relevance of Stream 2 will be addressed by considering the engagement approaches used in the program (Section 3.2.1), the extent to which end-user needs were met by the program (3.2.2), as well as mechanisms for integrating user feedback into the research process (3.2.3) and providing support to targeted regional NRM organisations in the program (3.2.4).

The process of engaging and working with the users of products and stakeholders involved in NRM planning (i.e. other researchers and NRM stakeholders such as NRM body CEOs), supported the delivery of fit-for-purpose regionally relevant NRM planning products.

There were a wide and extensive range of engagement activities undertaken by Stream 2 projects to understand user-needs and to build the capacity of NRM organisations to plan for climate change. The quality, accessibility and usefulness of the products and services delivered through Stream 2 were strongly underpinned by the engagement activities which focused on both understanding and responding to the needs of NRM planners.

3.2.1 Engagement

As outlined in Section 3.1 and the Stream 2 output summary more than 268 engagement activities and processes were delivered by projects across eight


clusters to support product development, design and uptake between 2013 and 2016. A range of engagement processes and approaches were undertaken by projects. The Stream 2 Interim Evaluation (2014) found that there was a variety of engagement models being used by Stream 2 projects. In addition to engaging with NRM bodies, cluster projects also worked with, and alongside, other cluster projects, the National Projections project, the AdaptNRM project, and a range of other federal, state, regional and local government agencies in addition to research institutions, non-government organisations (NGOs) and industry organisations.

As projects progressed into year two (2013-14) and three (2014-15) engagement variously became more targeted, and purposeful as project teams, intermediaries (next users of research outputs whom project teams directly interface with to support uptake and use of Stream 2 products by NRM bodies) and end-users (NRM bodies) became more familiar with each other’s needs and capabilities27. Where regions experienced turnover in end-users (i.e. planners) or intermediaries (i.e. knowledge brokers or consortium partners), the process of uptake and engagement was made much more difficult and challenging. Similarly, where institutional changes took place external to the program (i.e. in the restructure of regional bodies in NSW), these changes disrupted the continuity of engagement and in many cases undid the gains made by project teams. During this period of restructuring, NRM representatives were either absent, removed or distracted by the institutional changes taking place.

Another challenge in engagement was attaining sufficient coverage across wide, geographic areas. This challenge was noted by both the National Projections team and the AdaptNRM project who engaged across all regions and clusters. At the end of the program all project leads interviewed (n=13) reiterated that the quality, accessibility and usefulness of the products and services delivered through Stream 2 were strongly underpinned by the engagement activities which focused on both understanding and responding to the needs of NRM planners. These project leads indicated that the process of engaging and working with next users allowed them to deliver products that have been integrated into NRM planning.

Tailored engagement

The diversity of needs across NRM organisations meant that engagement and consultation needed to occur in a tailored way. This resulted in different engagement approaches for different clusters, project teams and NRM groups. There were mixed results from different engagement approaches, the common theme from which was that tailored approaches at the regional body level were most effective. There were some overarching lessons to be drawn from across the projects, reflected in the comment below:

Creating effective, collaborative relationships both among the Southern Slopes Climate Change Adaptation Research Partnership (SCARP) team and with NRM agencies has required lots of time and prompt responses, face-to-face contact, listening, good intentions, and a healthy dose of humour. One size does not fit all.

(Southern Slopes, Annual Report 2013-14)27 Refer to Attachment Eight for a glossary of terminology used in this report.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 35

Largely, the implications of the diversity of NRM needs led to a recognition of the need for (a) multiple scales of engagement and, (b) multiple modes of engagement (e.g. a mix of face-to-face, telephone and web based interaction). Engagement needed to occur within project teams, across multiple regional NRM organisations at the cluster scale and then, sometimes, at varying scales within NRM bodies (i.e. at the executive as well as operational level/s). A common theme across the project reports was the need and preference for some face-to-face interaction, in combination with a range of other modes as suggested by AdaptNRM.

Multiple modes of engagement need to occur to cover the range of NRM planner needs. While, face-to-face activities are generally much more preferred and effective, particularly for achieving interactive, peer-to-peer learning and for larger groups (>10) of people who are unfamiliar with one another, other forms of engagement such as online forums, tele/video-conferences, group emails and individual consultations (phone/email) can be very effective for particular purposes.

(AdaptNRM, Annual Report 2014-15)

Models of engagement

Project self-reporting indicates that the variety of engagement approaches was effective in engaging and supporting regional NRM organisations to plan, deliver and take up planning products, resources and tools. A range of specific observations about effective engagement were made by projects. A common theme in these observations was that the means of engagement need to be targeted to the needs of intermediaries (cluster representatives, researchers and other stakeholders) and end-users (NRM representatives)28. In some projects this was facilitated by centralised, coordinating entities such as a Project Working Group (East Coast) and a Brokering Hub (Wet Tropics). The variety of modes of engagement used by Stream 2 projects is summarised in Table 6 below. While there was a wide variety of engagement modes and mechanisms used by project teams, a common approach for projects was to use one individual or lead as a focal point for engagement with NRM groups.

Overall, this evaluation found that the following engagement strategies were useful in supporting uptake of Stream 2 products and resources by NRM groups:

regular face to face workshops and meetings across the whole cluster to share information and progress projects

provision of a knowledge broker or similar position that had a defined role in the project for coordinating all interactions between NRM groups and researchers

28 Refer to Attachment Eight for the definitions of terminology used to describe ‘intermediaries’ and ‘end-users’ etc.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 36

one-on-one work with NRM groups to ensure the cluster products, tools and resources were relevant, understood and accessible

co-design of products, tools and resources to ensure that they met the needs of local users; i.e. where NRM groups were involved in the formative thinking and design of products (e.g. providing input into early decision making in product development) as well as in subsequent stages of development (e.g. through review)

direct provision of advice from researchers where NRM groups requested it

work across clusters via meetings, teleconferences and/or workshops involving planning representatives from each NRM group.

The devolved approach to engagement taken across the program resulted in a range of engagement approaches. A summary of the different engagement methods used by projects to understand and respond to user needs is provided in Table 6 below.

Regardless of the engagement approach taken, interviews with NRM representatives and project researchers showed that the real value and benefits of engagement often occurred informally as individuals and teams began to get to know one another better and develop rapport, trust and reciprocity, which in turn enabled a more productive working partnership between research producers and users. This is not to downplay the importance of formal, planned engagement, which typically provided a basis for developing, maintaining and strengthening relationships.

Table 6. Modes of engagement used by Stream 2 projectsStream 2 – Final EvaluationFinal Report Clear Horizon Consulting 37

Project Engagement methods29

National Projections

National Projections User Group Co-development of products 1:1 project engagement Presentation, feedback and consultation on outputs Engagement with NRM regions via eight Cluster Coordinators User workshops

AdaptNRM Module review process User workshops 1:1 project engagement 1:1 NRM group engagement (not all NRM groups)

Central Slopes

Joint project team comprised of researchers with NRM body representation

Engagement lead 1:1 NRM group engagement User consultation and workshops

Monsoonal North

Project steering committee with NRM body representation 1:1 NRM group engagement Presentation and consultation

Rangelands

Employment of an Engagement Officer to distribute information and share outputs

Scientific advisory committee 1:1 NRM group engagement User consultation and workshops

Southern Slopes

Joint project development Workshops, interviews, focus groups to identify needs Project Steering Committee Use of a live information portal to gain feedback 1:1 NRM group engagement

East Coast Planners Working Group (PWG) Establishment of a “Community of Practice” 1:1 NRM group engagement

Southern and South Western Flatlands

Workshops to identify needs and gain input from NRM representatives

Appointment of NRM group liaison point Product review 1:1 NRM group engagement

Wet Tropics

“Brokering Hub” established to facilitate engagement Co-development of information tools Engagement lead Identification of shared goals Workshops to identify needs and gain input from NRM

representatives Consultation on products during review Joint-publications 1:1 NRM group engagement

Murray Basin

Engagement lead Workshops to identify needs and gain input from NRM

representatives 1:1 NRM group engagement Exit interviews with NRM groups

The Wet Tropics “Brokering Hub” provides an example (see Case Study below) of a formal engagement mechanism established by the project team to connect researchers with planners in a joint partnership throughout the project. The “Brokering Hub” comprised of a focal point for engagement, which facilitated

29 Note that because the methods presented are indicative because they are drawn from self-reporting and interview data. As a result, the engagement methods listed may not be representative of the full range of approaches used by projects.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 38

connections between researchers and NRM groups via series of structured engagement activities and forums as well as ongoing, regular contact with individuals in NRM groups.

Case Study: The Wet Tropics “Brokering Hub” It was through the brokering hub process that we collectively identified projects to be developed... At first we were asked what products we wanted so the whole process was a little back the front. We definitely had the opportunity in the beginning to provide a list but in the initial year there was not a lot of interaction with Stream 2 so they didn’t meet our needs at all. It has improved enormously [since then], with a more active brokering hub and discussing what we do with the products. The engagement with the scientists and having a knowledge broker has been critical in this process. The concept of an end-user is a silly one. There is no use being part of it at the end. We are a partner and participated as a partner. I don’t think the products would have been as good if we hadn’t been a partner.

(NRM Planner)

3.2.2 End-user needs

By purposively engaging with NRM representatives and providing targeted information to address NRM planning for climate change needs, Stream 2 has met the needs of NRM groups not previously addressed. However, the extent to which end-user needs have been met varies across the regions and clusters. This is in part due to the complexity of NRM planning for climate change and the fact that ‘needs’ are context dependent (i.e. may vary according the individual and organisation involved), value-laden and often contested. In this dynamic context, Stream 2 has made a concerted effort and clear gains in addressing planning needs.

Where a clear focus and objective for engagement was present around a resource of common value, such as agriculture, end-user needs were more likely to be met. In addition, as noted in Section 3.1, end-user needs were also more likely to be met in regions where there was strong existing capacity and ability (i.e. sufficient resources, time, capability and expertise), a mandate and support to engage, as well as linkages and proximity to the project team.

The ability to meet end-user needs was in some cases hindered by a lack of flexibility in contracting arrangements, which committed project teams delivering work in groups and ‘consortia’ comprised of multiple partners before the needs of end-users were understood. For instance, in the East Coast cluster a consortium of six project research partners was established. Feedback from East Coast project representatives suggested that the consortia approach hindered the responsiveness of the team due to the large number of partners involved and because of the need to consult across all partners before acting on opportunities.

An inherent issue in Stream 2 was the tension between needs versus wants. For instance, what was ‘wanted’ by an NRM region could in fact differ from what their ‘needs’ were depending on what stakeholder values were at play (both within a


cluster, project or region). This tension was manifested in engagement activities, often early in the program, where project teams and cluster representatives negotiated what would be delivered.

‘Impacts and adaptation’ cluster projects

All clusters commented in Annual Reporting (2013-14) that they had purposefully engaged with NRM managers from the outset to ensure they would meet end-user needs. In 2014-15, three clusters had also begun to develop an evidence base (through formal surveys and interviews) to demonstrate that the products generated were meeting end-user needs, specifically citing NRM feedback about “usefulness” and “useability” (Southern Slopes, Central Slopes and East Coast). However, throughout the course of the program, all clusters reported difficulties in fully meeting end-user needs.

A key issue reported throughout the funded period by Stream 2 projects was the timing of Stream 1 and Stream 2 funding. Several project leaders commented during the period of delivery that Stream 2 should have been implemented prior to Stream 1 rather both Streams being implemented concurrently. They argued that sequential timing would have provided the necessary data and research (Stream 2) to update plans (Stream 1). However, as highlighted in Section 3.3.1 (see Case study: Co-learning between regions) and 3.2 (Support) there is a counter-argument to this perspective that Streams 1 and 2 were mis-matched; that the concurrent timing of Stream 1 and 2 in fact supported co-learning among researchers and practitioners by forcing them to work together to address the planning and research challenges.

National Projections project

The National Projections team engaged with NRM groups, researchers and the Department to determine user needs prior to starting research. This early engagement process involved establishing a User Panel and eight cluster coordinators, undertaking regular communication with cluster representatives and running nine user workshops between March and May 2013 in eight regions. This facilitated a shared understanding of user needs, research capabilities and available resources, and an opportunity to co-develop products and manage expectations from the outset.

The National Projections were received very well by NRM representatives interviewed for this evaluation who found the information highly relevant and at an appropriate scale. The National Projections user survey also found that engagement and delivery approaches taken by the project were considered effective by between 85-90% of respondents. Project representatives also uniformly praised the quality of the work and scientific output produced by the CSIRO projections team, regarded by cluster project representatives as “world-class”.

The National Projections were pitched at the right level. They provided various climate scenarios and time periods; gave us key messages and generalities for planning underpinned by rigorous science. It was a well written document; we didn’t have to apply the research.


(NRM Planner)

This positive response to the website from project teams was reiterated by the Projections team who have received feedback from project teams.

All of the feedback on the website has been positive. People appreciated the unprecedented access to data, information at a scale that was more relevant to their needs, support and guidance to use it, the elements of access and guidance were not available in our previous release in 2007.


In the survey of Climate Change in Australia website by the Projections team the majority of respondents rated the usefulness of the information highly in terms of meeting their needs; with around 44 of 99 users rating content as 4 out of 5 and a further 27 of 99 respondents rating the content as 5 out of 530.

In interviews with project leads the extent to which the National Projections project met different NRM planner needs varied. From project lead and NRM representative interviews it was clear that the Projections Project provided well received communication of the science underpinning the projections. Several project leads indicated that the National Projections team had been responsive to NRM planners’ requests for general climate change information such as clarifying the concept of scientific uncertainty in specific regional and cluster context/s. A major challenge for the projections team was in meeting diverse planner needs.

Some people wanted learning materials to share with their stakeholders. Then there were technical experts who wanted highly detailed products, through to people who knew almost nothing about climate change… [Another factor was that] some people struggled in areas where there was climate denialism [some clusters had a higher proportion of climate deniers in their communities and even on some NRM boards]. In these cases it was a major source of frustration that we could not meet their immediate needs [to address denialism] which was not in the remit of the project


In addition to meeting diverse NRM planning needs an additional challenge for the National Projections team was in meeting broader needs from government (local and state) and community stakeholders, particularly as the website gained attention through national media exposure via a featured article on The Conversation (8 April 2015). This presented a challenge for the National Projections team in balancing and prioritising their engagement time and resources for the primary audience (NRM regions) as well as unanticipated secondary audiences (the broader public).

AdaptNRM project

The AdaptNRM project final project evaluation conducted in March 2016 found that on average, AdaptNRM modules largely met the needs of NRM

30 Where 1 was ‘Strongly disagree’ and 5 was ‘Strongly agree’.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 41

https://theconversation.com/a-new-website-shows-how-global-warming-could-change-your-town-39682

https://theconversation.com/a-new-website-shows-how-global-warming-could-change-your-town-39682

representatives (n=30)31. These ratings were supported by comments from NRM representatives at the AdaptNRM workshop as reflected in Figure 4 below where a majority of respondents rated AdaptNRM modules 1 – 5 as meeting their needs ‘well’, with the exception of Module 2, which a majority of participants rated as ‘Neutral’.

Figure 4. The extent to which AdaptNRM modules met user needs

Module 1: Adaptation planning ‘The NRM Adaptation Checklist’

Module 2: Weeds and climate change Module 3: Implications of climate change on biodiversity

Module 4: Helping biodiversity adapt Module 5: Shared learning0

10

20

30

40

50

60

19

11 11

2624

54

37

52

37

44

23

48

33 33

28

4 4 4 4 4

AdaptNRM: How well AdaptNRM modules met user needs

Very well Well Neutral Not well Not at all

% re

spon

dent

s

The National Projections team noted that engagement with AdaptNRM helped the Projections project deliver information of relevance to end-users. This was because the AdaptNRM team had engaged with NRM group representatives on their climate information needs in the early stages of the project during 2013-14. The outcomes of these consultations were communicated to the National Projections team by the AdaptNRM team.

3.2.3 Feedback

End-users provided feedback to project teams on the purpose, use and format of Stream 2 products and processes throughout the program via a range of mechanisms. The diversity of engagement activities delivered by Stream 2 projects meant there was a range of opportunities for stakeholders to provide feedback to the project teams throughout the life of the program. The frequency of engagement, and hence interaction and feedback between the project teams, intermediaries and end-users32 peaked at year two, where engagement was most intensively delivered between projects. In this year (2013-14) some 268 engagements were conducted by the ten projects to identify priority climate change issues and the information they needed to inform planning.

User feedback was a critical factor in the engagement process which some projects emphasised as being a highlight of the program. Feedback between users and suppliers of research products was supported by adaptive, iterative approaches to project delivery. As the quote from AdaptNRM below illustrates, the process of engagement was necessary to support feedback in order to meet user needs.31 On a scale including ‘very well’, ‘well’, ‘neutral’, ‘not well’ and ‘not at all’.32 Refer to Attachment Eight for definitions of terms used such as ‘intermediaries’ and ‘end-users’.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 42

The value is not in the product but the process of deep engagement. We shared really early drafts, gained written feedback, organised discussion sessions and online forums. [As a result we were able to] get them discussing together and get better feedback as a result…We worked with [NRM planners] to make [the modules] specifically relevant to their region…One of our hallmarks has been discussion based delivery, where anyone who wants to participate, has a chance to explore and digest the products and then provide feedback. While the lead content scientists have been able to look on, the main aim is talk with the NRM groups in an atmosphere where they can speak openly.

(AdaptNRM)

NRM planners and project teams both described a deliberative process at the beginning of the project to determine and respond to user needs. For instance, in the projects which took a heavily consultative, participatory approach, such as SCARP, the project teams were trying to understand user needs while the users were trying to understand what the projects could offer them. Or, as it was characterised at the 2016 national workshop, projects would ask NRM groups “what do you need?”, and NRM groups would respond “what can you offer us?” This circular questioning was often described as confusing, challenging and difficult by planners and researchers. However, this situation was also described as being “dynamic” by NRM and project representatives interviewed for this evaluation as it led to a greater shared understanding of one another’s needs.

User feedback in the ‘impacts and adaptation’ cluster projects

In the first year of the program (2012-13) three of the engagement plans (AdaptNRM, Wet Tropics and Monsoonal North) clearly showed how avenues for feedback from stakeholders, including NRM planners, would be provided. At the mid-point in the program (2014), while it was apparent that many projects had implicit, informal processes for eliciting feedback from cluster representatives and users, it was less clear how feedback from users would be used to inform the development of products and modes of delivery. Despite this, the targeting of engagement activities and early evidence that products and delivery mechanisms were relevant to user needs would suggest that feedback between users and project teams was occurring informally.

The SCARP team came to the project with their own perspectives. In the first couple of meetings they brought some ideas which were quickly shot down by the group and they were put in their place. Very quickly they learned that they need to listen to us and let us guide them as to what our needs were. For instance, they were initially very focused on social and agricultural issues. We [planners] told them that our needs were to do with NRM and so this changed their focus. They responded to us.

(NRM Planner)

User feedback via the Climate Projections User Panel

In early 2013 the Climate Projections User Panel was established to provide an interface between the Projections project, AdaptNRM and cluster projects to ensure that feedback between projects was recognised and taken into account in the development of projections research products. Hosted by the Projections project, the purpose of the panel was to “ensure that the preparation and Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 43

delivery of climate change projections information for Australia’s natural resource management regions meets the needs of the user community” and to “create a mechanism for representative feedback to the CSIRO projections team” (Climate Projection User Panel TOR).

The panel provided projections users with an opportunity to provide feedback via email, video and teleconferences. It was supported by a user panel engagement portal to provide transparency in the user panel feedback and guidance process. In 2014-15 the Projections project reported that AdaptNRM feedback on different versions of the brochures, reports and website (including guidance material, tools and data) was taken into account during the review process.

User feedback in the AdaptNRM project

The national AdaptNRM project reported that, through various engagement activities, it was able to identify, understand and respond to the needs of NRM groups. Throughout the course of the design, development and delivery of AdaptNRM NRM groups were provided with several opportunities for input into AdaptNRM modules. For example, specific feedback from over 20 NRM managers was integrated into the development and draft review of delivery modules 3 and 4 (AdaptNRM, 2014-15 Annual Report).

3.2.4 Support

Peer support was a critical factor in supporting delivery between planners. Peer support between NRM group representatives was a common enabler identified across a number of regions.

One thing that was very good was they [Central Slopes] made sure we spent a lot of social time together; we had a lot of chats together to become familiar across regions, but also with the researchers. This was really setting the scene for collaboration. We weren’t shoved in a room and told to come up with answers. They were very keen to find out our requirements and needs. Our needs have evolved over this time and they have always stayed in touch and modified their approach.

(NRM Planner)

Relationships between NRM organisation representatives, researchers and agency representatives was a key factor in enabling the achievement of outcomes (see Section 3.1).

I don’t think the tools and report have been the most useful part of the project. For me it’s been the relationships the have been built [between planners and researchers] that are the most useful: having the ability to be on the phone with researchers as they are working and being able to follow up with them and thrash ideas around have been very valuable. The actual documents or tools that we get can be valuable but it’s working through together that is of higher value.

(NRM Planner)

The concurrent timing between Stream 1 and 2 also arguably further encouraged planners and researchers to work together. In some regions and clusters this led Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 44

to a deliberative process where researchers and practitioners necessarily learned from one another (See Case study: Co-learning between regions, Section 3.1).

3.3. The appropriateness of Stream 2This section addresses KEQ3: To what extent was the implementation process appropriate? Appropriateness primarily concerns the design of the Stream 2 cluster approach and model in addressing the planning for climate change needs of NRM regions across Australia (see Section 3.3.1).

The cluster model has enabled the delivery of climate change information and support to 56 regions via eight clusters. It has facilitated cross-jurisdictional learning between NRM regions and states in a way that would not have otherwise been possible.

Demarcated predominantly along bioclimatic considerations, the cluster model did not ‘work’ in all cases, particularly where it required working across jurisdictional boundaries. In one instance, the model reinforced the wide geographical distance between South Australian NRM regions (Eyre Peninsula, Northern and Yorke, Adelaide and Mount Lofty Ranges and Kangaroo Island) from the Perth based project team in the Southern and South-Western Flatlands cluster. This split between these two NRM regions and the project teams was exacerbated by high levels of staff turnover in the contact point between the project team based in Perth and the SA NRM regions.

3.3.1 The ‘cluster’ approach and model

The decision to take a cluster approach was made by DCCEE with the intent to provide an organised structure, efficiency in delivery and opportunities for shared learning across jurisdictions. The approach aimed to identify a scale of delivery that balanced the need for regionally relevant information with the requirement for national climate projections. This model was developed by the DCCEE in 2012 in consultation with the (then) 56 NRM regions, the CSIRO Adaptation Flagship, State Governments, and the Land Sector Carbon and Biodiversity Board (LSCBB).

The aim of the cluster model, articulated by the DCCEE in the Program Guidelines (August 2012), was “to provide a structure to deliver regionally relevant information, while benefitting from efficiencies in the joint delivery of relevant science information and opportunities for peer learning”. According to this document “the grouping of NRM regions into eight clusters took into account the nature of the projected change in climate, the range of adaptation options set by biogeography, and the predominant land use”. In addition, the Guidelines note that “a number of the clusters cross State/Territory boundaries, and therefore different jurisdictional arrangements may apply across some clusters. The cluster design increases opportunities for cross-learning and collaboration in similar geographical zones”.

In addition to the factors outlined above, when the clusters were demarcated DCCEE also took into consideration factors including the number of NRM regions per cluster. While clusters were based on biophysical and climatic factors, in


some cases existing institutional structures were taken into account; i.e. in the Rangelands cluster where an existing alliance was in place between organisations. Following an internal process of approval DIICCSRTE delineated eight NRM regional clusters: Monsoonal North, Murray Basin, Wet Tropics, Central Slopes, Southern and Southern and South-Western Flatlands, Southern Slopes, East Coast and Rangelands (refer to Attachment Two for the cluster map).


Inter-regional dialogue

The cluster approach facilitated inter-regional dialogue, learning and exchange between regions. As noted above (see Section 3.2) connections between planners and peer support was a critical enabler in the Stream 2 program. As a Sydney LLS planner noted: “Talking to my colleagues from the other NRM regions was a major breakthrough”. This outcome was also evident in Southern Slopes cluster where a joint working group between planners resulted in co-learning across regional boundaries (see below).

Case study: Co-learning between regionsIn 20 years of involvement with the CMA, never have I seen CMAs work so closely together as they did on this project. Similarly, never have I seen the level of collaboration with researchers and planners as on this project. The collaboration arose out of the need to work together. It was clear from the beginning that, for the CMAs to derive any benefit, we would need to cooperate. There was no competition like there is in other programs. SCARP realised this early on too. The CMAs involved in SCARP have collaborated on a number of things now. Together we conducted a vulnerability assessment for Southern Victoria. This arose out of the work that we were doing together [in Stream 1 & 2]. It is always based on trust. The relationship between the projects – SCARP and AdaptNRM – and us (CMAs). It was the opportunity to thrash things out that was important; having all CMAs in the one room; it forced us all to work together.

(NRM Planner)

A related outcome was that the cluster model enabled research to reach a wider area beyond state jurisdictions, than would have otherwise been possible, allowing greater efficiency of output. For instance, in the Central Slopes project, carbon farming research outputs produced by the Queensland Herbarium were distributed to NSW regional bodies, who would not have otherwise received these outputs without the cluster model. The cluster approach also facilitated learning at a national scale on the differences between state jurisdictions in the regional NRM delivery model. See ‘Institutional boundaries’ below.

Institutional boundaries

As noted above, the Stream 2 cluster model boundaries were developed based on biophysical rather than institutional considerations. In the Stream 2 program all clusters crossed multiple state jurisdictions with the exception of Wet Tropics in Queensland. For instance, the Rangelands cluster crosses Western Australia, South Australia, New South Wales, Northern Territory and Queensland. Meanwhile, the Southern and South-Western Flatlands covers the south west region of Western Australia and the coastal southern region of South Australia. Projects/programs had to contend with variation between each state’s legislative and institutional frameworks for NRM bodies and their varying degrees of state control. For instance, in New South Wales the Local Land Services (LLS) now have a legislated role to deliver NRM, amongst other services such as biosecurity, as an extension of the NSW State Government. By contrast, in Tasmania the NRM bodies have no such legislated function.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 47

In the Southern and South-Western Flatlands project there was a geographic and institutional split between West Australian and South Australian jurisdictions (refer to cluster map, Attachment Two). To address this split between the two states, the Southern and South-Western Flatlands partnered with the South Australian Department of Environment, Water and Natural Resources (DEWNR) to broker connections between the SA NRM regions and the Western Australian based project. As the Case Study below shows, staff turnover in the SA DEWNR contact point role during the course of the project reinforced a disconnect between the SA planners and the project teams based in Perth.

Case Study: South West and South Western Flatlands – A split cluster

It was challenge to us [Stream 2]. There has been change of personnel in our contact point [within SA DEWNR] with the SW project. Over the course of three years we have three people who were leading the project from the SA branch. There wasn’t a constant source of contact. It wasn’t until the third person who was able to show us what was available. All the support we received was via email. This didn’t necessarily reflect the project but changes in the government liaison between the project [SW Flatlands] and planners. We didn’t have much contact with WA based at UWA either. We understand why they split up the clusters based on climates and land uses. But the areas are very different. It is important to consider in the future. South Australia is very different institutionally too. This has not really been taken into consideration.

(NRM Planner)

This quote highlights the range of factors that resulted in end-users being disconnected from the project team including institutional arrangements, staff turnover and the engagement approach taken by Southern and South-Western Flatlands, which outsourced the liaison to a government department, whom made irregular contact with NRM representatives.

3.4. The efficiency of Stream 2This section addresses KEQ4: How efficient was Stream 2? In particular, this section will cover the efficiency of Stream 2 in terms of: (i) program coordination (Section 3.4.1), and; (ii) administration of funds in accordance with contract agreements (Section 3.4.2).

The efficiency of Stream 2 delivery was constrained in terms of program coordination, primarily with respect to cross-cluster coordination. The program funds were efficiently administered.

This was due largely to the fact that the program did not mandate or build any formal structures for information sharing into the program such as explicit coordination activities across clusters. It should be noted, however, that the oversight and coordination by the Department have provided a sufficient level of consistency to ensure smooth delivery of the program. Meanwhile in terms of funding disbursement, project and budget expenditure records from the


Department show that Stream 2 projects delivered outputs as per funding agreement milestones and expenditure between 2013 and 2016.

3.4.1 Coordination

Although cross-cluster coordination and collaboration was a stated aim of Stream 2 there was no formal requirement or expectation written into contract agreements for projects to work together to achieve greater coordination and consistency. As a result, there were few examples of cross-cluster coordination beyond the coordination functions administered by the Department to support program delivery. One example of cross-cluster collaboration work that stands out was the 3C Modelling jointly undertaken by Murray Basin, East Coast and Central Slopes33 to evaluate the impacts of climate change on biodiversity across an area spanning one quarter of the Australian continent over a timeframe to 2050. Meanwhile, the National Projections project coordinated a number of cross-cluster events including a side-event at the Greenhouse 2013 conference, which brought all clusters together. However, in the context of the whole program the joint cross-cluster 3C Modelling example is an isolated example.

As the lead agency responsible for administering the Stream 2 funding, the Department took responsibility for coordinating Stream 2 across eight cluster projects, the two national projects (‘National Projections’ and Impacts and Adaptation national project [‘AdaptNRM’]), in addition to the Terra Nova information management project and the Monitoring and Evaluation component of the program, which was contracted to Clear Horizon Consulting Pty Ltd.

Throughout the course of the project DIICCSRTE and then the Department undertook a number of measures to increase coordination across projects and reduce duplication of project and program outputs (as reflected in the program logic, see Appendix One). These measures included:

Hosting a program workshop in 2013, 2014, 2015 and 2016 which brought together representatives from all Stream 2 project teams, along with representatives from government departments and regional NRM planning organisations. Among other aims, these workshops sought to develop agreement among project teams on ways to coordinate and communicate across the program and identify areas for possible collaboration between project teams.

An activity calendar was developed and used in 2012-3 and 2013-4. An online program calendar included key dates for workshops and meetings for each project, providing opportunities for projects to coordinate joint activities and avoid duplicative consultation.

The appointment of NRM cluster representatives by DIICCSRTE in the early stages of the program (2012-13 and 2013-14). Representatives were appointed to act as a point of contact for DIICCSRTE and to coordinate input from their clusters during the application process.

In 2013, following the Program Workshop in February, DIICCSRTE established a program coordination group (PCG) to coordinate engagement across the Stream 2 projects.

33 Drielsma M, Manion G, Love J, Williams K, Harwood T, (2014) Draft 3C MODELLING For Biodiversity Management Under Future ClimateStream 2 – Final EvaluationFinal Report Clear Horizon Consulting 49

https://www.terranova.org.au/repository/3c-modelling-east-coast-central-slopes-and-murray-basin-nrm-collection/draft-report-3c-modelling-for-biodiversity-management-under-future-climate

There was limited evidence during the course of the program of clusters coordinating work beyond these forums facilitated by the Department, possibly because there was no formal requirement for projects to coordinate and work together across clusters. Feedback from project representatives suggests that coordination across clusters was constrained by high demands on the time of cluster representatives to engage both within as well as across clusters boundaries.

Managing the complexity of NRM Clusters and number of NRM regions within clusters has also been challenging. Establishing and maintaining cross-border collaborations and connections on a regular basis is almost impossible as each project must concentrate on within Cluster delivery within the agreed time frames.

(Murray Basin 2013-14 Annual Report)

In summary, there was a gap between the expectations of the Department and the funding and design of the program given that coordination and shared learning across clusters was part of the stated intent of Stream 2 (see 4.3 the Cluster model in ‘Appropriateness’). Like any engagement activity, cluster and project coordination requires resources, time and inputs to materialise benefits. For instance, in a recent science project funded by the Australian Government, Securing Australia’s Future , an additional science synthesis project was purposefully funded separately to integrate findings across a range of funded projects.

3.4.2 Delivery of milestones in accordance with funding agreements

Project records and budget expenditure show that Stream 2 projects mostly delivered outputs on time against funding agreement milestones between the years 2012-13 and 2015-16. While there were minor delays in projects communicating sub-project information to the Department, the issues causing the delays were subsequently resolved. In some cases, this involved the milestone payments not being fully expended by the milestone date with funds being committed (e.g. contracted). Expenditure was therefore considered to be ‘on track’ by the department.

3.5. The legacy and sustainability of Stream 2This section considers KEQ5: How sustainable and enduring are the outcomes of Stream 2 likely to be? In particular, this section will outline factors supporting the legacy of Stream 2 (Section 3.5.1) as well as challenges to the legacy of Stream 2 (3.5.2). The legacy and sustainability of Stream 2 also concerns the capacity of NRM bodies to apply Stream 2 products and resources (3.5.3), the extent to which information generated through the program is “embedded” within regional NRM bodies (3.5.4), and the ongoing use of products and resources (3.5.5). Significantly, the legacy of Stream 2 will also be influenced by the linkages and relationships developed through the program (3.5.6).

The legacy of Stream 2 includes significant advances by regional bodies in their ability to adaptively plan for climate change and the embedding of information and knowledge into regional NRM planning processes.


http://www.acola.org.au/index.php/projects/securing-australia-s-future

These organisations now have the potential to influence other regions to increase their capacity as well. As they adaptively plan for climate change they will be supported by responsive, adaptive tools and frameworks (such as AdaptNRM ‘The NRM Adaptation Checklist’), online resources and platforms (such as the Climate Change in Australia website and the Terra Nova platform), the development of networks and relationships within and between researchers, planners and NRM representatives and the publication of research outputs.

However, this legacy may be less effective over time for a number of reasons. The legacy will likely be challenged and constrained by a lack of ongoing funding to support the application of the climate projections; staff turnover and organisational restructuring; low levels of awareness about Stream 2 within the Australian Government; and a lack of formal funding and federal, state and regional support to sustain planner and researcher engagement with a climate change focus in NRM.

3.5.1 Factors supporting the legacy of Stream 2

The legacy of Stream 2 was actively considered by Stream 2 projects throughout the period of delivery. In years 2012-13 to 2015-16 the following preparations were undertaken to support the longevity of Stream 2 and durability of the outcomes post-2016:

Developing responsive tools. Evidence indicates that information, tools and resources delivered by Stream 2 were developed in response to changing climatic conditions as well as the needs and priorities of end users

Developing online resources and platforms; i.e. the AdaptNRM website and the Climate Change in Australia websites.

Implementing an information repository project (Terra Nova) led by Griffith University to provide storage for all Stream 2 outputs in a centralised place, and where outputs are searchable for the longer term, as opposed to being housed on a range of institutional websites.

Supporting the development of networks and relationships within and between researchers, planners and NRM representatives within and across cluster boundaries; i.e. through the national Stream 2 workshop and ongoing, informal linkages and connections between researchers, NRM representatives and stakeholders.

Publication of research outputs by research teams both in academic, peer-review scientific journals and in a range of commissioned reports.

3.5.2 Challenges and constraints for the legacy of Stream 2

The legacy of Stream 2 will be affected by a range of issues, constraints and challenges, some beyond the control and scope of Stream 2 including:

The cluster model was designed purely for Stream 2 and may have no longevity beyond the program timeframe (2013 – 2016), although the lessons from the cluster approach may be applicable to other forms of investment and programming federally.

The Climate Change in Australia website has no funding beyond possible basic maintenance through the NESP program. The longevity of the website and its application will depend on securing future funding, which at the end of the program (mid-2016) is uncertain.


Staff turnover and organisational restructuring within regional bodies and to a lesser extent within research institutions has resulted in a loss of corporate knowledge and capacity developed through the program. Turnover is, however, a given in programs of this kind.

There was inconsistent uptake of resources and products across different NRM regions (see 3.1) with some regions achieving clear outcomes, while others have not achieved the same results through the program. However, while capacity and uptake was variable across the program “the history of regional NRM in Australia demonstrates that concepts permeate the NRM system, moving from the early adopters and leading organisations to other regions over time” (Cluster representative).

There are low levels of awareness about Stream 2 both federally34 and at a state level among relevant agencies and departments. However, to a large degree the durability of program outcomes is not dependent on federal and state government awareness of the program.

Relationships and linkages developed through the program are not presently supported by formal structures or resources to sustain ongoing connections with the explicit goal of improving NRM planning to adapt to climate change.

Responsibility for implementing regional NRM plans typically lies outside the direct influence of NRM bodies, which have the ability to facilitate and guide NRM change but not the actual capacity to enact landscape and asset changes sought through planning, which typically occur across a range of private and public land tenure arrangements. In this context, NRM bodies are key enablers for change that must work with and through a diverse range of state, regional and local, community stakeholders to deliver NRM activities.

3.5.3 Long-term capacity for integrating NRM climate change planning

As detailed in Section 3.1, the Stream 2 program has had an inconsistent effect on capacity across regional NRM bodies involved in the program. In short, while some regional bodies developed capacity through the program and made significant advances in their ability to plan for climate change, this positive influence was not widespread and tended to be concentrated in regional bodies that were better equipped through existing capabilities, networks, resources and mandated to plan adaptively. These regional NRM bodies who have embraced planning for an uncertain climate future may be seen as leaders and ‘champions’ with the ability to influence other NRM regions. Given a reasonably high uptake of resources and outputs from Stream 2 by NRM regions, it is reasonable to assume that the majority of NRM regions are well placed to capitalise on the gains made in their capacity for NRM planning for climate change from Stream 2. This was supported by stronger relationships and ties to research institutions and the availability and accessibility of products, tools and processes to support NRM planning for climate change (see below).

3.5.4 Embedding information and knowledge into planning processes

Adaptive planning approaches and systems

Several regions are moving towards adaptive planning approaches, which emphasise the planning process rather than the plan itself. This change towards adaptive planning was described as being part of a broader paradigm shift in the

34 noting that in the funded period (2013 – 2016) the responsible agency went through three successive changes in administrative arrangements.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 52

way that NRM regions approach and undertake regional NRM planning by a Murray Basin team member:

My overwhelming reflection is how fundamentally different the conversations at these final engagement workshops have been compared to workshops and meetings even 12-18 months ago. There has been a paradigm shift in some regional planning teams in which critical climate adaptation concepts such as multiple futures, dealing with uncertainty, flexibility and learning now have primacy in the discussion. Previously these concepts were recognized but it was not fully understood how central they are to climate adaptation planning.


In this context, NRM was viewed by researchers and planners as a complex, changing system. Therefore, to be relevant, planning is necessarily a dynamic, emergent process, which must be adaptive in order to respond to climate change.

One of the overall things is we have changed our approach from looking at regional and strategic plan to taking a landscape approach taking into account resilience concepts looking at how climate change will affect different areas differently. This has helped us understand the issues around climate change and look at the issues in the future.

(NRM Planner)

Stream 2 has, in part, catalysed this shift towards adaptive planning by supporting and strengthening adaptive planning approaches. NRM plans are, for instance, increasingly being developed and updated as virtual, online “live documents” (see below).

3.5.5 Ongoing use of products, tools and information

The following online resources and platforms developed by Stream 2 will provide ongoing accessibility to Stream 2 products and resources, dependent on ongoing funding and support:

the National Projections website Climate Change in Australia provides resources, links and products developed by the National Projections project as well as links to the eight regional impacts and adaptation cluster projects.

the Terra Nova website provides an information repository for all National Projections products

the AdaptNRM website provides an online repository of AdaptNRM products and resources

the Department have committed to updating the Stream 2 web-page to provide links to Stream 2 products.

Through a linkage with the National Environmental Science Programme’s Earth Systems and Climate Change Hub, the National Projections team have potentially earmarked limited funding for basic maintenance of the Climate Change in Australia website in the short term. In addition, Stream 2 researchers, project teams and NRM organisations have produced a large number of research


http://www.environment.gov.au/science/nesp/current-projects/earth-systems

http://www.environment.gov.au/science/nesp/current-projects/earth-systems

http://adaptnrm.csiro.au/

https://terranova.org.au/

http://www.climatechangeinaustralia.gov.au/en/

outputs, which are being published in academic publications and in the wider online environment, which will provide ongoing accessibility well beyond the program timeframe.

Online plans

Online plans and portals provide an opportunity for NRM and land use planners to develop, maintain and update plans in an adaptive, responsive and flexible manner. While the exact figure is unknown it estimated that around half of the regional bodies currently have online plans (AdaptNRM). As the Rangelands NRM representative below states, online plans will support NRM plan adaption and evolution.

That’s the intent of the online plan. It involves adaptive learning as we become better informed so the plan continually evolves. It is quite a practical plan based on our delivery of projects but there is the potential to link the plan to a possible knowledge hub. The regional plan hosts the most relevant knowledge. This long term nature of [the plan online] puts it all into perspective.

(NRM Planner)

Online plans were provided by a number of regions for this evaluation including: SWCC, Territory NRM, Rangelands WA NRM, CCMA, Reef Catchments and Terrain NRM. In addition, an online forum was established during the Stream 2 period by a coalition of Victorian CMAs35.

3.5.6 Linkages and relationships supporting the legacy of Stream 2

Linkages within and between regions, researchers and agency representatives have been developed and strengthened by Stream 2. Due to the national scale of the program and size of the clusters there was necessarily a wide degree of differentiation in the quality of relationships developed across clusters. Despite this there was evidence of ongoing partnerships between researchers and end-users from all Stream 2 projects. Although most partnerships were reported as informal, some Stream 2 projects are developing formal partnerships through project work, which are expected to endure beyond the life of the project.

I think the personal interactions between researchers and NRM groups and the greater understanding of what NRM is all about. This is the most lasting. This is a shared outcome, a lot more sympathy both ways… we are more approachable. There are new projects that will come up and we are more comfortable to work together with people in the local landscape; i.e. Carbon farming.


In 2014-15 a number of projects reported that Stream 2 has contributed to a shift from a situation where there was limited interaction between climate researchers and NRM planners in Australia to a situation where strong relationships based on “trust” and a mutual understanding of research capabilities and end-user needs had been forged (Projections project, AdaptNRM, Monsoonal North, Southern Slopes, Wet Tropics, Central Slopes).

35 Funded by the Victorian State Government.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 54

Ongoing relationships will be a critical factor in supporting and enabling outcomes achieved by Stream 2.

Everyone is aware of the structural flaws [in Stream 2] but the relationships between regions and with scientists have stimulated my community. It has created networks and it was agreed it would be useful to maintain these into the future. Unfortunately, we recognise that any formal networking needs financial support, so a less approach of staying in touch and sharing resources has been maintained.

(NRM Planner)

At the time of the evaluation there was no formal mechanism to support ongoing relationships between researchers and practitioners nationally within the Australian NRM sector. There are forums where planners, researchers and agency representatives will support ongoing connections, such as the Annual NRM Knowledge Conference. Ongoing relationships between researchers and NRM practitioners are therefore likely to be largely based on informal, incidental connections and opportunities as well as through existing departmental working groups and committees.

Inter-regional networks

As noted in Sections 3.1 (Effectiveness) and 3.2 (Relevance) Stream 2 has provided an opportunity for planners to form stronger linkages and ties across regions. At the end of the program, inter-regional networks that provide a forum for planners to connect and provide peer support to one another exist in a number of state jurisdictions as highlighted below. In these jurisdictions, planners are working to apply Stream 2 products cross-regionally.

Victoria. The Victorian CMA NRM Planning for Climate Change; a forum comprised of representatives from all nine Victorian CMAs. Initially funded through Stream 1 the forum met monthly until January 2016 and now meets on a needs basis in person, via email or phone.

Western Australia. South East NRM groups are currently coordinating the establishment of a shared spatial dataset and website to support cross-regional planning.

Cross-regional linkages between planners are critical in supporting the legacy of Stream 2.

Engagement with other stakeholders

Several NRM representatives noted that the actual responsibility for implementing NRM plans often lies outside of regional NRM bodies themselves. This of course varies across states depending on the legislative framework for regional NRM in each state (refer to Section 3.3 for more detail). Notwithstanding these institutional differences across states, NRM bodies typically work with and through regional networks of industry, research and community entities to deliver NRM strategies and plans. As a representative from NRM South in Tasmania commented:Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 55

http://www.conference.nrmregionsaustralia.com.au/

http://www.conference.nrmregionsaustralia.com.au/

We are developing a strategy on behalf of the community. This isn’t NRM Souths’ Strategy; we (NRM South) are not land managers. It is their strategy.

(NRM Planner)

In Victoria the CMAs and Victorian Catchment Management Council (VCMC) committed equal amounts to fund a State-wide Climate Change Coordinator at 0.4FTE until December 2016. The position is presently hosted by the Goulburn Broken CMA. One responsibility of this position is to look for opportunities to engage with the Victorian State Government to support ongoing utilisation of Stream 1 and 2 outputs. For instance, a forum took place in May 2016 to discuss the utilisation of Stream 2 outputs in the Victorian Government Climate Change Adaptation Plan and the Victorian Biodiversity Strategy. The State-wide coordinator is also working with a national Emissions Reduction Fund (ERF) group and is in discussions with the Victorian Catchment Management Council about how the Stream 1 and 2 work can assist in the development of Catchment Condition Reporting with regard to climate change36.

36 Correspondence with the State-wide Climate Change Coordinator.Stream 2 – Final EvaluationFinal Report Clear Horizon Consulting 56

Attachment One: Stream 2 Program Logic


Figure 5. Stream 2 Program Logic


Attachment Two: The Cluster Model

Figure 6. The Cluster Map showing the eight Stream 2 NRM clusters


Attachment Three: Results chartTable 7. Stream 2 Results Chart (2013 – 2015)KEQ Summary of findings Evidence1. How effective was Stream 2 in achieving its intended outcomes?1.1. To what extent have Stream 2 projects improved the quality and accessibility of regionally relevant information on climate change impacts and potential adaptation responses?

Stream 2 has effectively increased the quality and accessibility of regionally-relevant climate change information and support by delivering information and support to NRM regions via eight cluster projects, the National Projections project and the national impacts and adaption project (AdaptNRM).

Increased accessibility evidenced by the development and delivery of a total of 828 information products by ten Stream 2 projects between 2013 and 2016; of which around 100 final products are stored on the Terra Nova web-platform, publically accessible.

Wide-spread uptake and adoption of Stream 2 tools and resources with at least three quarters of NRM regions demonstrating tangible examples of product use in the planning process.

Accessibility: Between 2012-13 and 2015-16 improved accessibility to regionally relevant information on climate change impacts and adaptation was supported by:

The delivery of 659 outputs to support regional NRM organisations to plan for climate change via the eight clusters, including: information products (reports, data-sets, guidance documents, fact sheets), training, mentoring support via workshops, presentations and other delivery methods (2013-2015)

The delivery of 157 outputs which translate next generation climate projections to NRM planners via the eight clusters (2013-2015), including: information products (web tools, journal papers, aminations, spatial layers and data sets), presentations, consultations and other delivery methods.

The delivery of the Adapt NRM outputs including: module guides, website resources, data access portal resources (maps and datasets), module delivery sessions, module review sessions, conference presentations and meetings and newsletter updates.

The launch of the Climate Change in Australia website by the Projections project From 27 January 2015 until 22 June 2016 the web site had 153,746 unique website users with 202,446 unique sessions (averaging 12,100 sessions per month).

Specific examples of increased accessibility from the final round of project reporting (2015-16) include:

Murray Basin: during initial consultation in 2013 most regions identified other regions as the main source of information; by contrast recent consultation regions cite a wider range of available sources of information, and are aware of how to access that information. Consultation by Murray Basin representatives in 2016 found a high level of familiarity in general with Stream 2 products.

Rangelands: 7 out of 11 survey respondents at the end of the project reported improved access to regionally relevant climate change information as a result of the project.

SCARP: 8 out of 9 NRM regions report that the quality and accessibility of regionally relevant climate change information improved as a result of SCARP, particularly through the information portal established by SCARP. Also in terms of enabling regions to find


KEQ Summary of findings Evidenceand access such information themselves. On average, regional bodies rated the quality and accessibility of information provided through SCARP as 7.9 out of 10.

Examples of use of Stream 2 products in the NRM planning process from project reporting (2015-16) include:

National Projections: Use of the climate futures tool for scenario planning (ACT Environment and Planning Directorate), to inform likely climate change impacts (SE NRM), to underpin second pass impact and vulnerability assessment (Wimmera CMA & West Gippsland CMA), to inform development of a Spatial Assessment Tool (GBCMA), to influence the draft NRM plan (Southern Gulf Catchments Ltd), to confirm vulnerability assessments (Burnett Mary), to inform climate refugees mapping (SEQ Catchments), to inform draft risk assessment process and spatial analysis (Hunter LLS), as a “centrepiece” in the draft NRM Strategy informing the “Possible Futures” section (NRM South), to develop sub-regional plans summarising future conditions (Eyre Peninsula), used “where appropriate” (Kangaroo Island NRM), to inform program development and implementation as well as being used on the Swan River Strategy website (Perth NRM).

Central Slopes: 3 out of 5 regional NRM plans (QMDC, NW LLS, Northern Tablelands LLS) draw on Central Slopes resources, with remaining 2 regional bodies expected to provide outputs (April 2016)

East Coast: See spatial analysis for carbon soil and vegetation carbon sequestration (KEQ1.3 below). The development of “website chapters” by GSR LLS to draw on AdaptNRM Biodiversity modules. The development of an updated plan by SEQ Catchments whereby the project team were involved in review; research included in the process of plan development rather than the end-product. Incorporation of the projections into the background of the FBA plan.

Monsoonal North: NRM plans prepared for Northern Gulf NRM, Territory NRM, WA Rangelands, NQ Dry Tropics and Cape York NRM utilised project outputs. Southern Gulf Plan (being finalised) will use project outputs. Specific examples of utilisation include the use of materials were reportedly in Indigenous schooling (further details not provided), use of the CLiMAS projections.

Murray Basin: Citation of Stream 2 sources including: North Central CMA climate adaptation plan cites the Climate Change in Australia as a key information source; Wimmera & SE NRM similarly uses projections in NRM planning documentation. Adoption of the Adaptation Pathways approach by SE NRM, GBCMA and North Central Catchment Management Authority (NCCMA).

Rangelands: seven NRM regions all report that they “have or will use [Rangelands project] information in their NRM plan”. Some sources were reported as being used while others were not by Rangelands NRM regions. NRM plans updated with Rangelands outputs include: National Projections summaries, inclusion of the “Rangelands


KEQ Summary of findings EvidenceAdaptation Table”, links to the AdaptNRM and Climate Change in Australia websites (Territory NRM), incorporation of the National Projections as the “basis for supporting climate change risk” (SW NRM), to inform the draft addendum of Strategic Plan with references to projections, likely impacts and adaptation strategies (Western LLS), to inform the location of climate refugia (Desert Channels Qld), to update the Climate Change addendum (Alinytjara Wilurara NRM), and to integrate recommendations from cluster reports (SA Arid Lands NRM). In addition, Stream 2 outputs were used by Territory NRM during the planning process where cluster projections were used during participatory planning.

SCARP: wide adoption of the Adaptation Pathways framework with the approach being used “across Victorian CMAs”. Victorian CMAs developed climate adaptation plans as adjuncts to existing plans, Tasmanian and NSW regions incorporated SCARP products & processes into scheduled strategic plan reviews.

East Coast: use of the climate impacts and adaptation information in the SEQ Catchments draft NRM plan.

Southern and South-West Flatlands: uptake of the MCAS-S spatial planning tool by some regions (SCNRM, SWCC & NACC) to guide carbon planting, and to prioritise areas for climate change adaptation strategies (Perth NRM).

Wet Tropics: updated NRM plans containing information from Wet Tropics outputs and the National Projections. NRM plans also influenced in their structure and approach by Wet Tropics; i.e. identification of sectoral themes in the Terrain and Reef Catchments’ plans, and framing of adaptation pathways in Cape York NRMs’ plan. Development of Climate Action Strategies by Traditional Owners to include in Management Plans, co-presented at the AIATISIS Native Title Conference and at a workshop on Transdisciplinary Science in 2015.

1.2. To what extent have Stream 2 projects improved the capacity of regional NRM organisations to plan for climate change?

Wide variability in the influence on climate change planning capacity within NRM regions, with some regions reporting organisational-level shifts in planning approaches (i.e. adoption of the “Adaptation Pathways” approach in CCMA), while others report limited or no influence on planning capacity.

Support for capacity development was provided by projects to NRM organisations: Adapt NRM: has provided support through the provision of module guides and web-

based resources, interactive review and delivery sessions, and by providing opportunities for shared learning between NRM groups

Monsoonal North: NRM planners have improved qualifications and training related to climate knowledge.

Murray Basin: Increased awareness of regionally relevant information on climate impacts within most NRM regions. Regions with staff continuity and stable institutional arrangements have “built personal and organisational capacity around core concepts of adaptation planning, around the conceptual difference between climate adaptation planning and other forms of planning”. Supported by self-reported capacity development; adoption of concepts (i.e. NCCMA and GBCMA have both incorporated climate adaptation pathways into their planning; examples of NRM regions actively engaging with and questioning different approaches; “The amount learnt in just three years is huge! If you look at where we were three years ago and where we are now, we


KEQ Summary of findings Evidenceare really ready to get serious about climate adaptation planning and I think you will see that happen in the next round of catchment planning” (2015-16)

Southern and South Western Flatlands: capacity among NRM bodies is variable reflected in low uptake and access of products. Capacity was increased in some regions to use the MCAS-S spatial planning tool.

Wet Tropics – Increased understanding among NRM bodies of the process of climate change research. Fostered through the Brokering Hub, which provided a forum for interactive, ongoing communication between individual researchers and NRM planners.

Rangelands – NRM regions report “increased confidence and understanding of Climate Change projections” as a result of engagement with the National Projections team. This in turn supported “improved discussions with communities” and “inclusion [out outputs]” in NRM plans. NRM regions in the Rangelands cluster also reported a “little improvement” in capacity as a result of Stream 2.

East Coast – NRM planners report “enhanced knowledge…access to tools and information for climate adaptation planning”.

In addition to developing NRM organisation capacity, Stream 2 influenced the capacity of researchers to engage with NRM organisations.

National Projections – development of knowledge brokering capability among CSIRO and BoM climate scientists; “enhanced ability to translate technical information in a way that can be understood by next/end-users” (2015-16 report).

1.3. How has the program contributed to Government objectives?

Limited direct influence and contribution to Government objectives of spatially guiding carbon sequestration and biodiversity fund activities.

Specific research outputs guiding spatial prioritisation for carbon sequestration and biodiversity fund activities include the use of the MCAS-S tool in Southern and South-Western Flatlands, carbon farming research undertaken in the East Coast cluster and the 3C Modelling jointly undertaken by Murray Basin, East Coast and Central Slopes to evaluate the impacts of climate change on

At year two (2013-2014), Stream 2 some projects (3 out of 8) were found to have made some small, yet potentially significant steps towards Land Sector Package objectives, including: consultation, project development and spatial analysis being taken towards informing the location of carbon and biodiversity activities.

At year three (2014-2015) 4 out of 8 cluster reported specific examples of progress towards these government objectives, including:

Use of MCAS-S spatial planning tool has informed the identification of sites for carbon sinks for tree planting (Flatlands).

East Coast compiling biophysical, economic and other data for carbon farming assessment and have attended meetings with Commonwealth Government representatives regarding new Emissions Reduction Fund framework and the implications for carbon farming (East Coast).

At year four (2015-16) examples included: East Coast: 3 NSW LLS have initiated a project to develop a spatial analysis tool to

identify the best areas for soil and vegetation carbon sequestration. Tool still being developed under contract by a consultant at the time of evaluation. In addition, BMRG


https://www.terranova.org.au/repository/3c-modelling-east-coast-central-slopes-and-murray-basin-nrm-collection/draft-report-3c-modelling-for-biodiversity-management-under-future-climate

KEQ Summary of findings Evidencebiodiversity up to 2050. used the Carbon Farming report to meet Stream 1 requirements. Finalisation of the Qld

Government Carbon Farming report. Development of a web-based tool to visualise carbon farming opportunities from forest regrowth. Development of a joint-project between consortium researchers to realise opportunities in the Emissions Reduction Fund (ERF).

Southern and South-West Flatlands: use of the MCAS-S spatial planning tool by SCNRM, SWCC and NACC to identify sites for carbon planting.

The 3C Modelling jointly undertaken by Murray Basin, East Coast and Central Slopes to evaluate the impacts of climate change on biodiversity up to 2050.

2. How relevant was Stream 2 to the context and needs?2.1. How well did the information and support delivered through the program meet end-user needs?

Stream 2 products and support have largely met user-needs balancing the needs for scientific rigour with a user-focused research approaches.

Over the life of the program (2015-16) the delivery of information to meet NRM planners needs was supported by a number of engagement activities:

In the first reporting year (2012-13) Stream 2 projects undertook primarily planning and engagement activities consistent with program expectations. To summarise, projects used a range of informal and formal means to engage with clusters to design and plan for national projects.

The delivery of 268 engagement activities to identify priority needs and issues for NRM planners, including: consultations, program management meetings, workshops, presentations and other delivery methods.

Feedback from NRM groups confirmed that information and support provided by Stream 2 met their needs:

Adapt NRM modules and resources were received well. For instance, the Final Adapt NRM workshop evaluation found that the workshop “increased” participant “understanding of Adapt NRM modules” moderately for 53% of participants and strongly for 30% of participants.

East Coast: 4 out of 4 planners ‘strongly agree[d]’ that the Biodiversity project was ‘relevant to NRM planning and that they ‘will be able to use the results of the project’, while participants were ‘neutral’ or ‘agreed’ that the Planning Packages ‘were useful and relevant’; at the ‘Climate Change Adaptation for NRM Planning’ Workshop (22-23 April 2015).

Rangelands: Survey respondents (2016) all (n=11) indicated that interactions between NRM regions and scientists were valuable and that these “interactions contributed to the usefulness of the end product”.

SCARP: 8 out of 9 regional bodies state that the project has taken approach that has


KEQ Summary of findings Evidencemet their needs. Face to face meetings in particular were cited as the main form of engagement that was most useful to NRM bodies. On average, regional bodies rated the information and support provided by SCARP as 7.6 out of 10 in terms of meeting their expectations, while the effectiveness of engagement approaches were rated as 7.7 out of 10 by end of project (2016) evaluation survey participants (n=9).

East Coast – the “Consortium” approach (see 2.2 below) of involving multiple (six) research partners in project design and delivery was potentially “less flexible” in responding to changing user needs than projects with fewer partners, and hence reduced consultation demands / transaction costs. Feedback from NRM regions indicate that the PWG workshops were largely appreciated but they did not always meet their needs. The PWG provided a source of focus for interactions between researchers and planners during delivery. 1:1 engagement was highly valued by researchers and planners.

Wet Tropics – emphasis on co-development of research outputs in the project. Establishment of the “Brokering Hub” mechanism to facilitate researcher-planner engagement (see 2.2 below). The research model facilitated the identification of project goals and outcomes to be met by the development of research products. Project rated as ‘Good’ in terms of providing “fit-for-purpose” outputs.

2.2. How effective were the various engagement and delivery approaches used by the cluster projects?

A range of engagement and delivery approaches were used by Stream 2 projects to understand and respond to the planning needs of NRM regions. This engagement process has supported, for the most part, the delivery of products and support to NRM regions that have been well received, relevant and fit-for-purpose.

On a project-by-project basis:

Central Slopes: the Project Team was comprised of regional NRM organisations, the university researchers and government partners. The inclusion of NRM organisation representatives on the central project team support targeted and relevant engagement by Central Slopes where “regional NRM organisation participants expressed a strong desire to be active research partners, instead of ‘end-users’” (Central Slopes, 2015-16)

Monsoonal North: NRM planners sat on a project steering committee, which provided direct feedback to projects. Planners also provided a line of communication to the project teams on NRM plan development timelines.

National Projections – climate projections User Panel, regional workshops and training courses – “The information provided is excellent and extremely valuable to awareness raising and adaptation planning at a broad level”

Rangelands – employment of an ‘Engagement Officer/Knowledge Broker’ who played a ‘key’ role in distributing information and outputs to ‘broader users’

SCARP – Founding principle of project to “engage in responsive, demand-driven research practice”. Relationships developed through “regular informal structured interactions including workshops, interviews, focus groups and surveys”. Joint development of sub-projects on adaptation planning, spatial prioritisation, impacts information synthesis and capacity building by researchers and NRM region representatives. Co-delivery of outputs via an interactive, iterative approach to project implementation. Use of a ‘container’ (a


KEQ Summary of findings Evidence“live information portal”) as a virtual space where participants could provide and receive information throughout the process.

East Coast – A consortium approach was taken whereby several research partners (six) were involved in the project. Establishment of a Planners Working Group (PWG), consisting of a series of workshops delivered to planners. Creation of a “Community of Practice” within and between the research and NRM groups.

Southern and South-Western Flatlands – consultation and information provision delivered to NRM planners via three workshops. Appointment of a representative from SA DEWNR to act as a conduit between SA NRM regions and the project team.

Wet Tropics – establishment of a “Brokering Hub” mechanism in the project to facilitate engagement between the research teams and NRM bodies. Identification of “Key Accountabilities” as well as shared goals for Stream 2 by NRM representatives and project team member, in turn setting the direction for project consultation, engagement, outputs and outcomes. Use of multiple modes of engagement including workshops and through an interactive, consultative report review process. Engagement was rated by NRM bodies as ‘Good’ to ‘Very good’ in the Final evaluation. One NRM region commented that “the researchers have been largely engaging and tailoring the process to the needs of individual researchers or NRM groups”. Some aspects of engagement were less well received; i.e. using a “list of NRM concerns” to structure the first report. In addition, differing perspectives between researchers on what constituted appropriate engagement; i.e. ‘client-provider’ model vs. ‘co-research’ models.

2.3. How well were NRM planning needs and feedback integrated into Program delivery?

Feedback was sought and used by Stream 2 projects with varying levels of success and tended to be stronger in more collaborative projects with a structured and explicit approach to engagement.

Different mechanisms for providing feedback were used by projects. Feedback was explicitly sought, gathered and used in a minority of cluster projects (Wet Tropics and Southern Slopes). In other cluster projects feedback was sought and used implicitly.

Examples of feedback provided by projects in 2015-16 reporting include: SCARP: User needs were “reportedly well-integrated into program delivery throughout

iterative process of engagement and sub-project development”. In an end of project evaluation survey, participants rated user-feedback highly, with an average score of 8.7 out of 10.

Wet Tropics: Project rated on average as ‘Good’ to ‘Very good’ by NRM regional representatives in terms of “adaptive learning and decision making” comprising incorporation of feedback. One region commented that: “Researchers have been very adaptive to respond to the requests and feedback from the NRM groups”.

3. To what extent was the implementation process appropriate?


KEQ Summary of findings Evidence3.1. How appropriate was the ‘cluster’ approach in achieving its intent?

The cluster approach taken in Stream 2 has led to inter-regional learning and connections across NRM groups that would not have otherwise been established. This was strongest in clusters where; (i) inter-regional forums were established and brokered by the project teams, and; (ii) existing inter-regional forums existed (i.e. Victoria and South Western Australia).

The cluster model has also reinforced geographical and institutional divisions across state boundaries, which has (along with other factors) effectively disconnected NRM regions from Stream 2 projects.

Refer to section 3.3

3.2. To what extent did Stream 2 promote good practice in the integration of climate change information into NRM plans?

Refer to Section 3.1 and KEQ1.1 Refer to Section 3.1 and KEQ1.1

4. How efficient was Stream 2?4.1. To what extent did projects realise opportunities to increase efficiency

Limited efficiency in terms of coordination at a program or cross-project level despite largely administrative coordination functions due to the fact that coordination was

Refer to Section 3.4


KEQ Summary of findings Evidencethrough coordination?

not budgeted for.

4.2. Are projects on track against Funding Agreement milestones and expenditure?

Stream 2 projects delivered outputs as per funding agreement between 2012-13 and 2015-16.

Refer to Section 3.4

5. How sustainable and enduring are the outcomes of Stream 2 likely to be?5.1. How well did the Program enhance the longer-term capacity for integration of climate change information into NRM plans?

Limited and variable influence on long-term capacity due to relatively high turnover in regional bodies among NRM planners who received the bulk of support provided.

Barriers to longer-term capacity include: High turnover within NRM bodies in several clusters; i.e. in East Coast and Murray Institutional restructuring; during the timeframe of delivery and following delivery, will

potentially nullify efforts and gains made.

The influence on longer-term capacity: Rangelands – regional NRM representatives (n=11) were not “as confident in [capacity]

improvement” as they were in improved access (see 5.2 below). Some regions report improvement in longer-term capacity, though on the whole regions report a “little improvement” in capacity.

SCARP – some regional bodies felt that the development of long-term capacity was outside the scope of the SCARP project. Long-term capacity received a moderate rating in the SCARP NRM region survey in 2016 with an average rating of 6.9 out of 10.

5.2. What evidence is there that products, tools, information and knowledge developed through the Program have been embedded into NRM planning processes?

Several examples of how products, tools, processes and approaches have been well integrated into the planning cycle by NRM bodies.

Integration of products, tools and approaches was supported by consultative, collaborative engagement processes to support adoption.

Examples of integration of products, tools, information and knowledge into the planning process include:

Murray Basin – Clear intent among some regions to continue to integrate climate change adaptation concepts and tools into planning at regional and sub-regional scale/s. See KEQ1 for examples of adoption of planning approaches; i.e. adoption of the Adaptation Pathways approach in GBCMA, SE NRM and NCCMA. Uptake was variable across regions in the Murray Cluster. Adoption by some regions plays a stepping stone for wider dissemination of approaches. Supported by the Victorian Climate Forum as well as the piloting of the Adaptation Pathways approach in the adjacent SCARP cluster.

SCARP – decision making frameworks and the Adaptation Pathways approach were regarded as useful in future planning processes.

Southern and South-Western Flatlands: adoption of the MCAS-S spatial planning tool by


KEQ Summary of findings EvidenceNRM regions (SCNRM, SWCC, NACC & Perth NRM)

Wet Tropics – co-development of research products in formats consistent with NRM group needs will support ongoing use and adoption; i.e. “NRM partners have expressed their intention to use the fact sheets series to initiate deeper conversations and develop scenarios with key stakeholder groups or industry sectors. Cape York NRM have used the fact sheets (as well as the foundational reports developed in this project) to formulate ‘memes’ that are published weekly on their website” (2015-16).

5.3. What evidence is there that product, tools, information and knowledge developed through the Program will be used in an on-going way, after the life of the Program?

Tools and products developed by Stream 2 will be accessible well beyond the project time-frame.

Online information storage in a range of platforms including the centralised Terra Nova repository will support ongoing accessibility.

The extent to which these products are accessed and used beyond the program timeframe will vary, with evidence (see KEQ1) suggesting that demand is relatively high and capacity to use the products among planners is sufficient.

Evidence of increased accessibility via online portals and platforms includes: Terra Nova – centralised web-portal for all Stream 2 products. National Projections – the Climate Change in Australia website provides a repository of

information developed through Stream 2. Website maintenance will be supported through the Earth System and Climate Change Hub for the next three years.

SCARP – products available on a web-portal. Southern and South-Western Flatlands: storage of products on regional NRM body web-

pages.

5.4 How well did the Stream 2 projects improve links between researchers and end-users?

Stream 2 has effectively improved links between researchers and NRM organisations across all projects.

Stream 2 has provided a forum for planners and NRM representatives to forge connections to support the program legacy.

Adapt NRM – Final workshop evaluation (March 2016) found that the workshop facilitated connections between planners; 67% ‘strongly agreed’ that workshop provided a forum for NRM planners to share adaptation ideas with peers; while 67% also ‘strongly agreed’ that the workshop helped them identify people that they could approach in the future.

Central Slopes – Communities of practice being established; to support capacity (2015-16)

Examples of improved connections between NRM bodies and research organisations include:


KEQ Summary of findings Evidence

Central Slopes – Ongoing partnership between USQ and QMDC have a formal agreement for supplementary activities; with intent to develop further agreements; interest in jointly developing a regional economic model to cost the impacts of climate change and determine relative benefits from different adaptation options (2015-16)

Monsoonal North – Enduring linkages between planners and researchers; i.e. projects have evolved into new work funded by the NESP Northern Australia Environmental Resources Hub. Linkages are supported by a deeper understanding among researchers of NRM planning needs, while NRM planners have a greater understanding of available knowledge and how it can be applied to improve planning.

Murray Cluster – NRM regional staff and researchers were brought together to develop relationships and build rapport and familiarity with one another’s perspective; i.e. one planner commented: “there was a genuine effort by people in Stream 2 to engage and better understand the needs of the regions”.

Rangelands – Project participants (n=11) including NRM regions (n=7) report that the “most useful” part of the project was enhanced connections between climate scientists and planners. Evidenced by “Inclusion of regions in on-going water refugia monitoring ARC Linkage project application lead by Dr Jenny Davis, and the interaction of CSIRO with RCP regarding its Climate Adaptation Pathways work and the use of rangeland regions as case studies (WLLS, CWLLS, North Gulf (Nth Monsoon)”.

East Coast – establishment of a “Community of Practice” to support ongoing researcher-planner engagement. Supported by ongoing relationships between researchers and planners.

Southern and South-Western Flatlands – increased interactions between researchers and end-users resulting in collaboration and improved linkages between groups. Better linkages improved by requests from groups to undertake further species modelling from project researchers.

Wet Tropics – enhanced linkages between researchers and NRM bodies developed via the Brokering Hub reflected in co-authorship of a Science Synthesis report. NRM bodies have expressed a desire to continue working together in the future.

SCARP – relationships developed through the program are now being formalised into new working projects; “links between researchers and end-users were greatly improved across the board” (SCARP, 2015-16). SCARP NRM regions reported improved links with researchers as a result of the project with an average rating of 8.5 out of 10 being given by participants in the end of project evaluation survey (n=9); “Our links with researchers is now established as far as climate change research and continues to develop and extend into other areas” (CCMA).


Attachment Four: Stream 1 Report 2013-14 – 2014-15 SynthesisTable 8. Uses of Stream 2 products by NRM regions as reported in Stream 1 reporting (2013-14 – 2014-15)

Organisation name*

Reporting period

Cluster What was used?** How was it used?**

ACT Environment and Planning Directorate

1/01/2015 to 30/06/2015

Murray Basin Decision-making framework n/aAdaptation Planning Resource Inventory

n/a

Plain Language Information Packages

Communicate the potential risks of climate change

Biodiversity Modelling under Multiple Futures

Identify potential climate corridors and climate refugia.

Exploring Adaptation Pathways Explore how NRM decisions might be structured and sequenced.

Australian Climate Futures Identify future climate scenarios for scenario planning. To help community stakeholders translate climate scenarios.

SA MDB NRM 1/07/2015 to 31/12/2015

Murray Basin NRM Adaptation Checklist To ensure that as we collectively develop actions to address NRM issues with our partners, to ensure our planning approach is flexible.

South East NRM Board

1/07/2014 to 30/06/2015

Murray Basin Projections Informed understanding of likely climate change impacts in SA regions

AdaptNRM Biodiversity analysis The Carbon and Environmental Planting Guideline for SA were informed by biodiversity analysis

North Central CMA 1/07/2015 to 31/12/2015

Murray Basin 3C Used to support the connectivity analysis.Adaptive Pathways Helped in completing 4 adaptive pathways

workshops that have supported the planning process.

Mallee CMA 1/07/2015 to 31/12/2015

Murray Basin AdaptNRM (Module 1 – 4) Sub project concepts developed to address Murray Basin Cluster specific information/data needs have been integrated into MCMA project planning to further refine regional context and data related activities.

Wimmera CMA 1/07/2015 to 31/12/2015

Murray Basin Projections Used to underpin the second pass impact and vulnerability assessment.

SCARP Adaptation Pathways Playbook

Used to help inform planning for the adaptation workshops

Goulburn Broken 1/07/2015 to Murray Basin Projections Used in the development of the Spatial Assessment


Organisation name*

Reporting period


CMA 31/12/2015 Tool that informed the development of the Climate Change Adaptation Plan.

AdaptNRM Used in the development of the Strategy. The NRM Planning Module provided useful guidance and highlighted a framework to be considered in this project (i.e. adaptive, flexible, multiple futures etc.)

Adaptation Pathways Project Principles applied to the Goulburn Broken Adaptation Pathways Trial. Several workshops took place to align the concept with the Spatial Assessment Tool and its outputs. The project manager has been sharing the information generated by the stream 2 project across the organisation.

NQ Dry Tropics 1/07/2015 to 31/12/2015

Monsoonal North Climate Knowledge Synthesis (Griffith University)

n/a

Indigenous Knowledge (CSIRO & Griffith University)

Discussed with Traditional Owners and with planners in surrounding regions to determine good ways of presenting climate data for different audiences

Decision support process map Benchmark for coverage of issuesModelling and online tools (Scaling Biodiversity Data, JCU)

Inclusion of climate issues in regional biodiversity strategies within plan

Draft Report: Guide for NRM Planners (CSIRO & JCU)

Contextual information to base NRM strategies to build resilience in rural communities

"Beef Industry Case Study" (JCU) Contextual information on socio-economic dynamics of grazing industry

Southern Gulf Catchments Ltd

1/07/2014 to 31/12/2014

Monsoonal North Regional Drivers of Change Decision Making and Planning for Natural Resource Management

AdaptNRM web platform (Biodiversity)

n/a

Social Resilience of Agricultural Landscapes (Final report)

n/a

Projections To influence the draft NRM PlanNorthern Gulf 1/07/2015 to Monsoonal North n/a n/a


Organisation name*

Reporting period


Resource Management Group Ltd

31/12/2015

North Coast LLS 1/07/2014 to 30/06/2015

East Coast 3C We have used the 3C products in our project and some of the theory out of Carbon Farming work also.

Riverina LLS 1/07/2014 to 30/06/2015

East Coast n/a Stream 2 resources have been included in all discussions with reference panel as appropriate.

Burnett Mary Regional Group for NRM Ltd

01/07/2014 to 30/06/2015

East Coast Projections To confirm assessments of asset vulnerability to climate change and to develop a region-specific infographic for plan incorporation and community engagement activities. To inform Round 2 expert panel deliberations, circulate to regional networks and incorporate into the plan.

Regrowth to remnant – veg regeneration value and mapping

Info considered in target drafting and to be incorporated into mapping for draft strategy

Burnett-Mary Natural Resource Management Region: Horticultural Sector

Material provided useful general background information to inform plan drafting

List of reference materials To check that we have covered off on our review of adaptation planning processes and also to see if any other information was available for our region that we were unaware of.

Fitzroy Basin Association Inc.

1/07/2015 to 31/12/2015

East Coast n/a n/a

SEQ Catchments Limited

1/01/2015 to 30/06/2015

East Coast Dowdy, A. et al. 2015, East Coast Cluster Report

Informed the ‘Changing Climatic Conditions into the Future’ section of the updated SEQ NRM Plan

Climate projections Climate projection data for the SEQ Region was sourced from the USQ. Data includes rainfall gradients, stable temperature zones, Cool zones & maximum temperature zones all which informed our climate refuges mapping.

Hunter Local Land Services

1/07/2014 to 31/12/2014

East Coast National Projections Informed the draft risk assessment process and given an indication of the climate variable that need


Organisation name*

Reporting period


to be factored into the risk assessment and spatial analysis aspects of the project.

AdaptNRM Accessed and have informed various aspects of the project.

Central West LLS Feb 2015 to July 2015

Central Slopes n/a Data and speakers have been utilised at a number of field days and workshops aimed primarily at landholders to ensure that they have the most up-to-date information for decision-making.

North West LLS January 2015 to June 2015

Central Slopes AdaptNRM 3C products Incorporated into addendum once they were adapted to suit regional boundaries and more easily explained products for community consultation

APSIM - Climate Change Impacts on Cropping

n/a

Grass-Gro Climate Change Impacts on Grazing

n/a

SWAT n/aCO2 Sequestration products n/a

Queensland Murray-Darling Committee Inc

July 2014 to December 2014

Central Slopes n/a n/a

Condamine Catchment NRM Corporation Ltd

1/07/2015 to 30/12/2015

Central Slopes 3C 3C Project data for consideration of how to incorporate into the wildlife section of the plan in the final stage.

Reef Catchments 1/07/2016 to 30/12/2015

Wet Tropics AdaptNRM Weeds Module A presentation was given to the Mackay Regional Pest Management Group on the outputs (and where to find spatial projections) on outputs from the AdaptNRM Weeds module.

Cape York NRM 1/07/2015 to 31/12/205

Wet Tropics n/a n/an/a n/a

FNQ NRM Limited 1/07/2015 to 31/12/2015

Wet Tropics n/a This reporting period we have referred to all of the documents and publications from the Wet Tropics Cluster in preparing the draft plan.

Projections Used and referred to the Climate Change in Australia Website published as part of the National Projections project.

Desert Channels 31/07/2015 to Rangelands n/a Stream 2 products inserted into the community


Organisation name*

Reporting period


Queensland 31/12/2015 information plan to show the current research into the rangelands and climate change. Considered to inform the NRM plan.

South East LLS 1/07/2014 to 30/06/2015

SCARP n/a SCARP assisted in the development of material for presentation at both steering committee and working group meetings and Peat Leith attended both meetings and presented on behalf of SCARP.

Adaptation Pathways Adaptation pathways continue to be examined in detail and an Adaptation Pathways Playbook was published to assist with the development of the plan.

Cradle Coast Authority

1/07/2015 to 31/12/2015

SCARP AdaptNRM website for Biodiversity n/a

NRM South 1/07/2015 to 31/12/2015

SCARP Projections Included in the final draft NRM Strategy for Southern Tasmania as a centrepiece. To inform the development of the ‘Possible Futures’ sections and to inform Target setting and Priority Actions.

SCARP Reports To inform the development of the ‘Possible Futures’ sections of the draft strategy and to inform Target setting and Priority Actions. Incorporated into the final draft NRM Strategy for Southern Tasmania.

Southern Slopes Information Portal Report

Particularly useful in articulating the ‘Possible Futures’ sections of the strategy and also guiding the threats and opportunities to the natural resources in Southern Tasmania.

Corangamite CMA 1/07/2014 to 30/06/2015

SCARP SCARP Reports A valuable source of information and resource during this reporting period to develop a planning process that is logical, comprehensive and transparent.

Adaptation Pathways for NRM Embedded capability by the Corangamite CMA to identify and use tools and frameworks in a consistent manner to inform NRM planning for climate impacts and adaptation, and its implementation.

East Gippsland CMA

1/07/2014 to 30/06/2015

SCARP SCARP Reports n/aProjections n/aAdaptNRM n/a


Organisation name*

Reporting period


Glenelg Hopkins Catchment CMA

1/07/2015 to 31/12/2015

SCARP SCARP Reports Incorporated into the development process and the draft document. During this period the Southern Slopes projections were further incorporated into the document.

PPW CMA 1/01/2015 to 30/06/2015

SCARP Four adaptation pathways planning workshops with regional stakeholders between February & June 2015.

Information from these workshops will be used to refine the NRM & Climate Change plan.

Adaptation pathways Adaptation pathways method is consistent with the intent of the Port Phillip & Western Port RCS and was used to shape our project’s direction and develop content for the plan.

Information portal Accessing information portal for literature about: Impacts of climate change to our region’s environmental assets; Adaptation pathways; Carbon sequestration and climate change mitigation.

West Gippsland CMA

1/07/2015 to 31/12/2015

SCARP Projections Used to underpin the second pass impact and vulnerability assessment.

SCARP NRM Toolkit Used to help inform writing the Strategy.SCARP Adaptation Pathways Playbook

Used to help inform planning for the adaptation workshops.

Northern Tasmanian NRM Authority

30/01/2015 to 1/01/2016

SCARP Projections Incorporation of climate projections information into the consultation draft

SCARP Spatial Prioritisation report To inform prioritisation processes. Development of the Environmental Strategic and Institutional Scans report included consideration of available tools, resources and learnings from SCARP and AdaptNRM.

AdaptNRM modules Informed planning principles for conservation management and biosecurity. Development of the Environmental Strategic and Institutional Scans report included consideration of available tools resources and learnings from SCARP and ADAPTNRM programs.

South West NRM Ltd

1/01/2015 to 30/06/2015

South West Flatlands

Scott, John K. 2014, Cenchrus ciliaris (buffel grass) and climate change, CSIRO Ecosystem Sciences, Wembley, WA.

Used for physiological information and climate change modelling distribution.


Organisation name*

Reporting period


Scott, JK, Webber, BL, Murphy, H Ota, N, Kriticos, DJ and Loechel, B 2014, AdaptNRM Weeds and Climate Change: supporting weed management adaptation

Used for flagging management actions

Eyre Peninsula NRM Board

1/07/2015 to 31/12/2015


Climate Change in Australia - Projections for NRM regions - Southern and South Western flatlands cluster

Used to develop the five sub-regional plans by including a summary of future conditions.

Kangaroo Island NRM Board

1/07/2015 to 31/12/2015


n/a Stream 2 products, including the regionalised projections and AdaptNRM modules, continue to be used where possible.

Wheatbelt NRM Inc.

1/07/2015 to 30/12/2015


CENRM Plant Refugia under Climate Change (A2- 2080).

Mapped distribution data for threatened flora, birds, fauna; revegetation species, coastal vegetation, economic vegetation, vegetation types. Includes mapping of over 700 individual species.

NCCARF Biological Refugia under climate change.

n/a

Projections n/aAdaptNRM Weeds and Biodiversity modules

n/a

South Coast Natural Resource Management Inc.

1/07/2015 to 30/12/2015


Ford, B. & Cook, B. (2015). Southern and South-Western Flatlands climate change project: Data layers explained.

n/a

Plant Refugia under Climate Change (A2- 2080)

n/a

Species distribution modelling n/a Mapped distribution data n/aNCCARF: Biological Refugia under climate change

n/a

Rogers DJ (2013) A framework for nature conservation under future climates, to inform climate adaptation planning in the Adelaide & Mt Lofty Ranges NRM region

n/a

Projections n/a


Organisation name*

Reporting period


AdaptNRM n/a

Perth NRM 1/07/2015 to 31/12/2015


CENRM Species Distribution & Bioclimatic Modelling

To inform PNRM program development through the Living Landscapes and Living Wetlands programs. Used in MCAS-S models.

Spatial data layers n/aProjections Used to inform PNRM program and project

development and implementation. CSIRO/BOM has also been used on the Swan Region Strategy website to provide an overview of the latest climate change projections for this region. To update information provided on the Swan Region Strategy website. To inform a summary report of the findings of the PCC project.

*A total of 39 Stream 1 annual project reports were received for the reporting periods 2014-15 and 2015-16 of which 35 reported using Stream 2 products.

**Responses to ‘What was used?’ and ‘How was it used?’ are extracted from: “During this reporting period, did you use any of the information and resources developed under Stream 2 of the Regional NRM Planning for Climate Change Fund?” and “If YES, what information or resources did you use and how?”.


Attachment Five: MethodologyThe overall methodology for Stream 2 M&E was designed by the M&E consultant team drawing on consultation with the Department and project representatives. This methodology is documented in an overarching program level M&E Framework. All M&E activities are independently led and coordinated by Clear Horizon but regular input is provided by cluster projects through annual project-level M&E reporting and participation in M&E activities. The final evaluation was lead independently by the Clear Horizon team, who were also active in coordinating project M&E over the time –frame of the program (2013 – 2016).

The evaluation has used mixed methods and a range of data sources to address the key evaluation questions (KEQs).

ApproachTo meet this evaluation purpose an evaluation methodology (outlined in Section 3) designed and delivered as part of a broader suite of monitoring and evaluation (M&E) activities undertaken by the evaluation team during the period of Stream 2 implementation. The approach to this evaluation was participatory in that Stream 2 projects and departmental representatives played an active role in setting the agenda and parameters for inquiry at the planning stage (2013), in providing monitoring inputs through annual reporting, (2013- 2016) and by assisting with the synthesis and interpretation of evidence at the final evaluation reporting stage (2016). By explicitly recognising and responding to the needs of the commissioners and users of this evaluation the evaluation team have taken a utilisation-focused approach (Patton, 1997).

This evaluation has also taken a theory-based approach whereby a program logic model was developed and used to articulate clarify the contribution of Stream 2 projects to the expected goals and outcomes for the program. Finally, by articulating the mechanisms for change in the program and seeking to situate outcomes achieved and case studies within the broader regional and cluster context for delivery, this evaluation has also drawn on aspects of the ‘realist’ approach to evaluation (Pawson and Tilley, 2004). Finally, by drawing on a range of qualitative and quantitative data sources this evaluation has taken a mixed methods approach to addressing the information needs for Stream 2.

Data collectionData was collected via multiple methods outlined below including desktop review, interviews and case studies.


Desktop review

Documentation including the interim evaluation (2014), project reporting (2013 – 2016) and project-specific evaluation/s (2016), as well as a sample of 16 NRM plans, a total of 39 regional Stream 1 reports submitted to the Department for the years 2014-15 and 2015-16 were reviewed.

These documents been synthesised against the KEQs to provide an evidence base from which to develop evaluation findings. Where possible, multiple data sources and lines of evidence have been used to triangulate, verify and support findings.

Interviews

A total of 20 Interviews were conducted with targeted ‘end-users’ of Stream 2 outputs NRM planners. Planners – or actors involved in ‘planning’ – were selected purposively on a cluster/project basis in accordance with their use of Stream 2 products and interaction with Stream 2. Informants were identified in consultation with the projects. Interviews were conducted face-to-face and via phone depending on the location and availability of informants.

In addition to ‘end-users‘, a total of 13 project representatives were interviewed to gather data on program effectiveness, legacy and the appropriateness of the program model. Every project was represented in this series of interviews.

To ensure privacy interviewees were assigned codes which were used for analysis and verification. See Attachment Seven for the interview guides used.

Case studies

Case studies of specific instances of practice change have been developed to provide an in-depth illustration of outcomes and the causal mechanisms that lead to the change. Case studies and ‘episodes’ for investigation were identified from the descriptive interview data provided by planners based around themes that were discussed and developed with the Department prior to investigation. Case studies have also been chosen to represent different regions and clusters in the program. They are presented in boxes throughout this report.

Data analysisQualitative and quantitative data was analysed using techniques appropriate to the data type (numerical or descriptive) and collection method. Each data source collected was analysed individually and ‘cleaned’ to ensure that the information clearly addressed the purpose of the evaluation and KEQs. Quantitative (numerical) data were analysed descriptively to identify frequency, range and distribution using appropriate statistical techniques. Qualitative data sources such as interview transcripts were analysed and cross-analysed to identify emergent themes in relation to the KEQs. Where possible multiple data sources were used to develop findings.


Data synthesisData was brought together via desktop review, interview and case study in a ‘results chart’ to collate, synthesise and develop findings. Evidence from interviews and analysis of regional NRM plans were used to develop case studies demonstrating program outcome/s; the significance of the outcome/s; and the contribution of the project/s and/or program. At the National Workshop on 15-16 March 2016 the evaluation team facilitated a half-day session (3 hrs) to interpret the results and develop interim findings. During this workshop, participants including project and departmental representatives reviewed a range of data sources and through a facilitated process began to develop likely findings and themes for the evaluation. Evaluative judgements were made at the reporting stage drawing on indicators, rubrics and through an analysis of the overall weight and value of the collected evidence against each KEQ.

Limitations This report partly relies on self-reported data contained in the annual

reports of the national projects and regional clusters. These reports have not been independently verified.

The outcomes and achievements reported in the national and cluster annual reports, and synthesised in this report, have not been independently verified with reference to original sources.

For some findings, the scale of change is not known. For example, the reports do not always provide information about the total population of ‘NRM planners’ or indicate how many planners have made changes. Similarly, with planner interviews, the extent of change reported and presented may not be representative of the region or the NRM group on which they are speaking on behalf of.


Attachment Six: Interview guidesStream 2 Final Evaluation Interview guide: NRM Planners

We have been contracted by the Department of Environment (the Department) to evaluate the program ‘Stream 2 of the Regional NRM Planning for Climate Change Fund 2013 – 2016’. This program has been established to support better NRM climate change planning by NRM bodies. Stream 2 has provided research and support to NRM decision makers through a series of projects delivered nationally. Your [NRM ORGANISATION] has participated in this program through the [PROJECT / CLUSTER]. As an NRM group planner and/or representative from [NRM REGION] we would like to interview you to understand if the climate change information, support and resources provided by the [PROJECT] have been useful and valuable for NRM planning in your region.

The interview will take around 30 – 45 minutes of your time. Just to confirm is now a good time to talk? Yes / No

We won’t identify you personally but in the report that we prepare for the Department we may associate your comments with your NRM organisation if that is ok with you? Yes / No

1. What is your NRM organisation and position?

NRM body/organisation Position

2. Which Stream 2 project/s have you been involved with?

Project/cluster (i.e. Rangelands) Length/duration of involvement with [PROJECT]

3. How have you been involved?

Project activitiesi. Consultation (i.e. workshops etc.)ii. Products

National projections project activities AdaptNRM project activities Other forums/groups to support climate change in planning?

4. What products, tools, resources or information have you received from [PROJECT]?

5. How useful have you found these products, tools, resources or information in assisting NRM planning for your region?


Format Scale Applicability Accessibility Understability/applyability (i.e. were you able to use it?) Timing

6. Have you used these products, tools, resources or information? Yes / No

If so, for what? Provide example/s (who/what/when/where etc.)

7. Do you intend to use these products, tools, resources or information?

If so, how? If not/why not?

8. How well has [PROJECT] provided support to you?

To interpret and apply tools To access advice To deal with uncertainty and risk in planning

9. How has [PROJECT] supported you to develop your own skills and ability?

To deal with uncertainty and risk in planning To interpret and apply tools To make NRM decisions taking into account climate change science

and research

10.How has [PROJECT] supported [NRM ORGANISATION] to plan for NRM under climate change?

Awareness and understanding of climate change in NRM planning Partnerships and linkages (i.e. with researchers, other NRM groups

etc.) Conceptual approaches/frameworks (i.e. resilience thinking) Planning processes and systems (i.e. for updating/renewing plans) Adaptability/flexibility and ability to update the planning

11.Do you have any other comments about [PROJECT]


Stream 2 Final Evaluation Interview guide: Project Leaders (Impacts and Adaptation)

As you know we have been contracted by the Department of Environment to provide M&E support to the ‘Stream 2 of the Regional NRM Planning for Climate Change Fund 2013 – 2016’ (Stream 2) program. We are now conducting the final evaluation for Stream 2 for the Department. This evaluation is looking at a range of things including the effectiveness and legacy of Stream 2. Your [PROJECT] has delivered climate change research and support to NRM planners as part of Stream 2 from 2013 – 2016.

The interview will take around 30 – 45 minutes of your time. Just to confirm is now a good time to talk? Yes / No

We won’t identify you personally but in the report that we prepare for the Department we may associate your comments with your Project if that is ok with you? Yes / No

1. Just to recap, can you tell me about your role in the project?

Project team Cluster Overview of activities

2. What have been the highlights of the project? Why?

3. What have been some of the challenges of the project? Why?

4. From some of the things that you have discussed, what so far has been the most significant positive or negative change or outcome for you because of this project?

Note that this is about change not process (i.e. not about timing of delivery and/or how the program has been delivered).

Capacity (Planner and/or researcher capacity; individual or organisational)

Availability / application of research Relationships, linkages etc.

5. What, if any changes or outcomes have you noticed in terms of the capability of your team to plan for NRM under climate change?

Ability to work with NRM planners Application of tools and approaches at a regional level Awareness of regional issues Projections Other


6. How did you use the National Projections work in your cluster?

As an input for products / tools etc. Provided information to planners / NRM groups etc. Didn’t / limited use Other

7. How well did the National Projections meet your projects needs in terms of delivering climate change information to NRM groups?

Quality of information Relevance at cluster/region level/s Ease of use, relevance, accessibility and application

8. How well did the National Projections project support your project to access and use the National Projections?

Access Use

9. How well did the national AdaptNRM project support NRM regions in your cluster to plan for climate change?

Access Use

10.Finally, what legacy has your project has left for ongoing benefits to be realised in the [CLUSTER]? Are there any major challenges / barriers that may prevent ongoing benefits from being realised?

Capacity; i.e. awareness of climate change in NRM planning (planner/researcher)

Partnerships and linkages (i.e. with researchers, other NRM groups etc.)

Conceptual approaches/frameworks (i.e. resilience thinking) Planning processes and systems (i.e. for updating/renewing plans) Adaptability/flexibility and ability to update planning

11.Do you have any other comments?


Stream 2 Final Evaluation Interview guide: National Projections

As you know we have been contracted by the Department of Environment (‘the Department’) to provide M&E support to the ‘Stream 2 of the Regional NRM Planning for Climate Change Fund 2013 – 2016’ (Stream 2) program. We are now conducting the final evaluation for Stream 2 for the Department. This evaluation is looking at a range of things including the effectiveness and legacy of Stream 2.Your [PROJECT] has delivered climate change projections to projects and NRM planners as part of Stream 2 from 2013 – 2016.

The interview will take around 30 minutes of your time. Just to confirm is now a good time to talk? Yes / No


1. Just to recap, can you tell me about your role in the project?

Project team Overview of activities

2. What have been the highlights of the project? Why?

3. What have been some of the challenges of the project? Why?

4. From some of the things that you have discussed, what so far has been the most significant positive or negative change or outcome for you because of this project?

Capacity (Planner and/or researcher capacity) Availability of research Application of research Relationships, linkages etc.

5. What, if any changes or outcomes have you noticed in terms of the capability of the Projections team to deliver research for NRM planning?

Ability to work with NRM planners Awareness of regional issues etc.

6. Finally, what legacy has the National Projections left for ongoing benefits to be realised? Are there any major challenges / barriers that may prevent ongoing benefits from being realised?

Data; products; website etc. Capacity; i.e. awareness of climate change in NRM planning

(planner/researcher)


Partnerships and linkages (i.e. with researchers, other NRM groups etc.)

Conceptual approaches/frameworks (i.e. resilience thinking) Planning processes and systems (i.e. for updating/renewing plans) Adaptability/flexibility and ability to update planning


Stream 2 Final Evaluation Interview guide: Department of Environment (Stream 1 and 2)

As you know we have been contracted by the Department of Environment (‘the Department’) to provide M&E support to the ‘Stream 2 of the Regional NRM Planning for Climate Change Fund 2013 – 2016’ (Stream 2) program. We are now conducting the final evaluation for Stream 2 for the Department. This evaluation is looking at a range of things including the effectiveness and legacy of Stream 2. You have been identified as a key informant who can contribute to our evaluation of Stream 2 from 2013 – 2016.

The interview will take around 30 minutes of your time. Just to confirm is now a good time to talk? Yes / No


Time & date: Interviewer:

1. Just to recap, can you tell me about your role in Stream 2?

Stream 1 Stream 2 the Department

2. Stream 2 was designed to better connect research with planning. How well do you think the cluster approach supported the program to meet this need?

What has worked well? What hasn’t worked well? Which aspects of the cluster approach have suited the intent of the

program? Which aspects could be better suited?

3. Stream 2 consisted of two national projects and eight cluster projects to deliver climate change science to planners. How well do you think this model supported the program to fulfil this purpose?

What has worked well? What hasn’t worked well?

4. Finally, what legacy has Stream 2 left for ongoing benefits to be realised?

Data; products; website etc. Capacity; i.e. awareness of climate change in NRM planning

(planner/researcher) Partnerships and linkages (i.e. with researchers, NRM groups etc.) Conceptual approaches/frameworks (i.e. resilience thinking) Planning processes and systems (i.e. for updating/renewing plans)


Adaptability/flexibility and ability to update planning


Attachment Seven: Output definitionsTable 9. Output definitions used by Stream 2 projectsPerformance Indicator Year Output type Definition

Research teams have worked with regional NRM organisations, through the eight clusters, to identify priority climate change issues and the information they will need to inform planning (end year one of the program)*

2013

Consultation

Events or activities where feedback is sought from NRM organisations. Feedback can be sought via meeting, teleconference, focus group, survey, workshops etc.

Workshop

An event hosted by the project team whereby project, cluster and regional NRM representatives come together to discuss, identify, prioritise issues and information.

Presentation

A speech or a talk delivered by a project team member where information is presented to NRM organisations for the purposes of then prioritising issues or information.

Program management meetings

Internal project meetings for the purposes of project management.

Products

Items produced and attributable to the project. Can include a range of outputs such as articles, newsletters, module outlines, reports, datasets, guidance documents and fact sheets. These products are focus on gathering, confirming, prioritising climate change issues or information.

Training

A unit, module, course or exercise delivered to provide information with the intent of identifying and prioritising NRM planning needs.

OtherAny other activities and outputs delivered by your project to identify and prioritise NRM planning needs.

Quality and timely climate change information and support is delivered to

201420152016

TrainingA unit, module, course or exercise delivered to provide information or to train people to use information.

Products Items produced and


Performance Indicator Year Output type Definition

regional NRM organisations via the eight clusters

attributable to the project. Can include a range of outputs such as articles, newsletters, module outlines, reports, datasets, articles, guidance documents and fact sheets. These products are focused on delivering information or data.

Workshops

An event hosted by the project team whereby project, cluster and regional NRM representatives come together to discuss information and products including data.

PresentationsA speech or a talk delivered by a project team member where information is presented to NRM organisations.

Mentoring (e.g. one-on-one support)

Instances of tailored advice and support provided to an individual or group within an NRM organisations around climate change information.

Peer reviewsEvaluation of scientific, academic products developed by the project by others working in the field.

Next generation climate change projections are delivered to regional NRM organisation via the eight clusters (once-off deliverable) (years two through four of the program)

201420152016 Training

A unit, module, course or exercise delivered to deliver next generation climate projections developed by CSIRO Element One National Project.

Products

Items produced and attributable to the project. Can include a range of outputs such as articles, newsletters, module outlines, reports, datasets, articles, guidance documents and fact sheets. These products are focussed on delivering next generation climate change projections.

Workshops

An event hosted by the project team whereby project, cluster and regional NRM representatives come together to discuss, next generation climate change projections.

Presentations A speech or a talk delivered by a project team member where information is presented to


Performance Indicator Year Output type Definition

NRM organisations particularly on next generation climate change projections.

Mentoring

Instances of tailored advice and support provided to an individual or group within an NRM organisations around climate change projections.

Other (Specify)

Any other activities and outputs delivered by your project to deliver next generation climate projections developed by CSIRO Element One National Project.


Attachment Eight: Glossary of termsTable 9.Glossary of termsTerm Definition Activities Activities are conducted to bring a change in a situation or behaviour

that is expected to contribute to outcomes. For example, incentives scheme advertised, workshops run, awareness raising.

Appropriateness

A measure of whether an intervention is suitable in terms of achieving its desired effect and working in its given context. Suitability may apply, for example, to whether the intervention is of an appropriate type or style to meet the needs of major stakeholder groups.

Broader goals Are long-term goals that the program outcomes are expected to contribute towards (it is acknowledged that many other factors and programs are also contributing to these broader goals).

Effectiveness The extent to which an initiative/project meets it intended outputs and/or objectives.

Efficiency The extent to which activities, outputs and/or the desired outcomes are achieved with the lowest possible use of resources.

End of Program Outcome

The desired final result of the program (may be some years after the program has finished).

End-user The ultimate persons that will use or benefit from the products and services delivered by the program.

Evaluation Essentially, evaluation is about untangling the threads of “what happened”, “why”, and “to what effect”. It can be used to assess the appropriateness, effectiveness and efficiency of the intervention, as well as to inform decision making and demonstrate accountability.

Immediate outcomes

Any immediate changes or tangible products that are a direct result of the activities – e.g. attendance workshop, awareness raised, or compliance.

Indicator A quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, and or to help assess the performance of a development action.

Intermediaries A person or group (such as a knowledge broker) who acts a link between program stakeholders (such as researchers and end-users).

Intermediate outcome

Medium term outcomes that occur as a result of the outputs and that are necessary preconditions for the achievement of end-of-program outcomes.

Legacy The extent to which the benefits of a project/initiative extend beyond its implementation.

Monitoring Monitoring is the on-going process of collecting routine data, usually internally, to track progress with previously identified activities and outputs.

Outcomes The results or changes that can be attributed to the activities (eg. changes in stakeholder’s knowledge and skills). Outcomes can be at the short, medium or long-term scale.

Outputs Outputs are usually relatively immediate and are the direct results or products of activities.

Primary The stakeholders who will receive the evaluation results directly, and


audience who will use the information for decision-making, such as program continuation or improvement.

Program logic A visual depiction of the program theory and logic behind how activities lead to outcomes. It is usually represented as a diagram that shows a series of causal relationships between inputs, activities, outputs, outcomes and goals.

Qualitative Observations or information expressed using words rather than numbers.

Quantitative Information about quantities; that is, information that can be measured and written down with numbers.

Relevance The extent to which the intended program outcomes addressed the program context and end-user needs.

Rigour Rigorous (“trustworthy”) research that applies the appropriate research tools to meet the stated objectives of the investigation. Rigorous research is both transparent and explicit; in other words, researchers need to be able to describe what they did (or plan to do) in clear, simple language.

Rubric An attempt to communicate expectations of quality around a task. In many cases, scoring rubrics are used to define consistent criteria for grading or scoring. Rubrics allow all stakeholders to see the evaluation criteria.

Scope A written description of the breadth of work for an evaluation (a description of what is included).

Utilisation Utilisation-Focused Evaluation is an approach based on the principle that an evaluation should be judged on its usefulness to its intended users. Therefore evaluations should be planned and conducted in ways that enhance the likely use of the findings to inform decisions and improve performance.


Documents

Stream 2 of the Regional NRM Planning for Climate Change ...€¦ · Web viewIn the year 2014-2015 impacts and adaptation projects were primarily focused on delivering climate change