88
Connecting the Dots between Research Priorities and Stakeholder Needs: An Evaluation of Northern Forest Stakeholder Perceptions of the Northeastern States Research Cooperative (NSRC) Grant Program FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., Principal Investigator Barbara Wauchope, Ph.D., Co-Investigator The Carsey Institute, University of New Hampshire Huddleston Hall, 73 Main Street, Durham, NH 03824 Tel: (603) 862-2821 Email: [email protected]; [email protected] and Charles French, Ph.D., Co-Investigator University of New Hampshire Cooperative Extension 204 Nesmith Hall, Durham, New Hampshire 03824 Email: [email protected]

FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

Embed Size (px)

Citation preview

Page 1: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

Connecting the Dots between Research Priorities and Stakeholder Needs:

An Evaluation of Northern Forest Stakeholder Perceptions of the

Northeastern States Research Cooperative (NSRC)

Grant Program

FINAL REPORT

Vol. I Narrative

April 16, 2012

Curt Grimm, Ph.D., Principal Investigator

Barbara Wauchope, Ph.D., Co-Investigator

The Carsey Institute, University of New Hampshire

Huddleston Hall, 73 Main Street, Durham, NH 03824

Tel: (603) 862-2821

Email: [email protected]; [email protected]

and

Charles French, Ph.D., Co-Investigator

University of New Hampshire Cooperative Extension

204 Nesmith Hall, Durham, New Hampshire 03824

Email: [email protected]

Page 2: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

ACKNOWLEDGEMENTS

The authors wish to express our appreciation to the Northeastern States Research Cooperative (NSRC) staff and leaders for their help throughout this

project. They worked with us to identify the right research questions, reviewed instruments, recommended people to interview, and provided feedback on early findings, all substantial contributions to this final product. Our special thanks to

Joe Short and his staff at the Northern Forest Center for helping us with the stakeholder survey. We would also like to acknowledge the hard work of our team of graduate research assistants: Jill Scahill Farrell, Shannon Rogers, and

Jessica Ulrich. Finally, thanks to the many NSRC principal investigators and stakeholders who graciously volunteered their time to answer our questions.

Page 3: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

TABLE OF CONTENTS

EXECUTIVE SUMMARY .................................................................................................................... 1

BACKGROUND ................................................................................................................................. 6

METHODOLOGY .............................................................................................................................. 8

Evaluation Design ........................................................................................................................ 8

Data Collection ............................................................................................................................ 8

Data Analysis ............................................................................................................................. 10

Limitations of the Research ...................................................................................................... 10

DESCRIPTION OF PARTICIPANTS ................................................................................................... 11

NSRC Principal Investigators ..................................................................................................... 11

Other Stakeholders ................................................................................................................... 12

FINDINGS ....................................................................................................................................... 15

Principal Investigator Perceptions of the NSRC Program ......................................................... 15

Other Stakeholder Awareness and Understanding of NSRC .................................................... 20

Research Needs and Priorities .................................................................................................. 23

Relevance and Usefulness of NSRC Research ........................................................................... 27

Current Outreach and Dissemination of NSRC Research ......................................................... 29

Research Products for Dissemination ................................................................................... 29

Outlets for Obtaining and Disseminating Research Information ......................................... 30

Improving Research Dissemination and Communication......................................................... 41

CONCLUSIONS ............................................................................................................................... 43

Recommendations ........................................................................................................................ 46

Page 4: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

1

EXECUTIVE SUMMARY

The Northeastern States Research Cooperative (NSRC), a grant program of the U.S. Forest Service, funds collaborative and multi-disciplinary research for the purpose of providing scientific data that can improve understanding and help solve problems facing the Northern Forest and the people who live there. The program awards grants in four broad areas:

Theme 1: Sustaining productive forest communities: balancing ecological, social, and economic considerations.

Theme 2: Sustaining ecosystem health in northern forests.

Theme 3: Forest productivity and forest products.

Theme 4: Biodiversity and protected area management.

The program, awarding grants since 2001, has had little feedback from stakeholders about how research project results are disseminated and used, and by whom, and if the research is meeting the needs of the region. In 2009 NSRC selected the Carsey Institute of the University of New Hampshire to conduct some evaluation activities to answer these and other questions.

The evaluation consisted of collecting and analyzing data from the perspectives of several diverse groups of NSRC research stakeholders. One of these groups, 80 principal investigators (PIs) of NSRC funded projects, was surveyed online. Three-quarters of the PIs were affiliated with colleges or universities and the rest from state or federal agencies, research institutes and community non-profit organizations.

To obtain the views of users or potential users of NSRC research, two groups of other stakeholders were also surveyed online with the resulting sample totaling 78. These stakeholders included people referred by the surveyed PIs and others from a list of subscribers to the Northern Forest Center News Digest. About 30% of this combined sample represented community non-profit/economic development organizations; the remainder included people from state/federal agencies, forest-serving organizations/institutions, and non-affiliated citizens. Other data came from interviews with 24 actual or potential users of NSRC research and two focus groups, one with members of the Northern Forest Higher Education Resource Network, and another with Cooperative Extension staff from several states.

All these data came from voluntary rather than randomly selected samples. Consequently, the findings presented below may not be representative of all NSRC principal investigators or of all other groups of potential and actual NSRC stakeholders. However, we believe that the methods used produced results that are valid for those groups of people who did participate in the study.

PI Perceptions of the NSRC Program

The NSRC principal investigators were overwhelmingly positive about the program, its focus and relevance, and the effectiveness of its operation.

Over half (58%) of PIs learned about the NSRC Request For Proposals from their colleagues; another 25% learned through grant alerts or announcements.

Page 5: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

2

Almost all (91%) said that the four themes that they could choose from in the RFP were broad enough for their own research.

The NSRC grantmaking process, including the RFP publicity and process, award notification and administration, and technical assistance were rated, on average, as effective to very effective.

PIs also rated the NSRC’s efforts in carrying out its mission as effective to very effective, with cross-disciplinary research receiving the lowest rating but still rated effective.

Other Stakeholder Perceptions of NSRC

The other stakeholders in the evaluation had mixed opinions about the NSRC, largely dependent upon how well they knew the organization. Across the different sources of data, about half the other stakeholders were aware of the NSRC and about half were not.

The more closely stakeholders were involved with a NSRC project or researcher, the better they understood the program and the more they were inclined to see value in it. However, when interviewed, even those who knew the program well had difficulty explaining its purpose or how it is organized.

Few stakeholders in the evaluation had collaborated on a NSRC project but even among those who had not, collaboration was perceived as desirable and they wanted to know more about it.

Research Needs and Priorities

The principal investigators and other stakeholders were compared several times during the evaluation for their different perspectives on the program.

The majority (60% or more) of the principal investigators and the other stakeholders agreed that NSRC research should result in both scientific knowledge and practical applications of that knowledge rather than either one or the other.

Of those who believed the research should serve one or the other purpose, the researchers were somewhat more supportive of need for basic research while the non-scientist stakeholders overwhelmingly said the research should address practical problems.

The principal investigators surveyed represented projects from across all four NSRC research theme areas. However, when this distribution was compared to the interests of the other stakeholders, they did not completely overlap.

More PIs were conducting research in Theme 2, Sustaining Ecosystem Health (41%), closely followed by Theme 1, Sustaining Forest Communities (36%) than in the other theme areas.

Surveyed stakeholders could identify multiple interest areas. They overwhelmingly indicated an interest in Theme 1 (85%), followed by Theme 2 (46%), Theme 3, Forest productivity and forest products (42%), and Theme 4, Biodiversity and protected area management (36%).

Page 6: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

3

This strong preference for research that focuses on forest communities, particularly economic issues, was echoed in other stakeholder interview and survey questions.

Relevance and Usefulness of NSRC Research

Those who know NSRC research best, i.e., the NSRC principal investigators and their collaborators, believe in its relevance and importance to the region. The PIs believe the research is of most relevance to other researchers and state and regional policymakers and non-profit organizations.

Almost two-thirds of the PIs use NSRC research and their collaborators say it is important work.

Stakeholder responses regarding NSRC relevance and usefulness were more varied. Some interviewed stakeholders not directly involved in the research said they did not find the research particularly useful or having much impact.

Outreach and Dissemination of NSRC Research

Principal investigators and other stakeholders were compared for how NSRC research is disseminated and where they go for information and research about Northern Forest issues. Ideally, the two would be the same if the goal is for NSRC research to reach an audience of research consumers who can apply the research to their own work. However, there were some notable differences found.

The primary products of NSRC research and the means for disseminating them appear to be

targeting the researchers’ peers rather than a broader stakeholder audience.

Over 80% of NSRC PIs reported peer-review journal articles as the product of their research, followed by professional meeting presentations (50%), and seminars or workshops (37%).

Three-quarters or more of PIs said that their primary means for dissemination were professional meeting presentations, the NSRC website, academic journals, and seminars and workshops.

The greatest potential for overlap here is in the use of the NSRC website and professional seminars and workshops.

About 80% of other stakeholders surveyed said they go to informational websites, which could be the NSRC website or any Internet site, to find information on their area of interest. A majority also said they go to seminars and workshops for such information.

However, findings from the interviews and focus groups indicate that only those stakeholders who had experience with the NSRC knew its website. Seminars or workshops serve specific professional audiences and the researchers and other stakeholders may not cross paths if they work in different fields.

The most striking disconnect between PIs and the other stakeholders is in the other media, such as newspapers, trade publications, policy documents, and best practice guides, which so few PIs use for dissemination.

Page 7: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

4

Depending on the source, from 33% to 65% of the stakeholders said they rely on these other, non-academic resources for information about Northern Forest research. At the same time, fewer than one-quarter of the PIs are using these resources for dissemination.

Various explanations for this difference were explored. The clearest concerned barriers to dissemination faced by the researchers:

Lack of funding to disseminate beyond peers was the most cited problem followed by lack of networks for dissemination, difficulty in translating the research into lay terms and lack of time.

At the same time, the stakeholders acknowledged their own problems in seeking out research.

Over half of the surveyed stakeholders said they do not have time to seek out information.

Interviews clarified that much of the problem is information overload, making it challenging and time-consuming to find the information relevant to them.

Finding information specific to the NSRC projects is even more daunting for most stakeholders who either are unaware of the program or even if they are, have difficulty locating the information they want.

Conclusions & Recommendations

Results of the evaluation are mixed. The principal investigators who have benefitted most from NSRC hold strong and positive perceptions of the value and operations of the program. They rated highly aspects of the program’s grantmaking and efforts to achieve its mission. With the exception of the few who have collaborated with the PIs, other stakeholders who know the program like the concept, but say it has little impact on their work. They have difficulty accessing the research, or once they learn about it, find the research does not meet their needs. In particular, they are looking for more focus on topics relevant to economic and community development. A third group, potential users of the research, is interested but knows little to nothing about the program.

Both NSRC principal investigators and the other stakeholders had a number of ideas for ways to

improve NSRC’s outreach and dissemination. Most were variations on two themes:

increase the publicity about the program and the results of projects, particularly through the NSRC website, e-mail and other online means of communication; and

increase the opportunities for the NSRC researchers to engage with practitioners and other stakeholders through, for example, presentations and collaborations.

The evaluation concludes with these additional recommendations:

1. Increase the responsiveness of academic researchers to the needs of stakeholders for more research targeting issues related to the Northern Forest economy and its communities, for example, by making these issues a funding priority or encouraging researchers to use practitioners as co-principal investigators or collaborators on all research teams.

Page 8: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

5

2. Increase the production of applied research by requiring that project proposals include plans for the utilization of research products.

3. Increase the dissemination of applied research by requiring that project proposals include a dissemination plan with named media and venues for sharing the research with non-scientist stakeholders.

4. Identify and facilitate opportunities for grantees to disseminate and communicate their research to stakeholders, particularly conferences, seminars and workshops where scientist/non-scientist professional fields converge or overlap; and ensure that the researchers know about them and are encouraged or incentivized to attend.

5. Improve the NSRC website’s visibility and usability, particularly by ensuring that it is linked to other websites and that all documents and e-mails include the Web address.

Page 9: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

6

BACKGROUND

The Northeastern States Research Cooperative (NSRC), a grant program of the U.S. Forest Service, funds collaborative and multi-disciplinary research for the purpose of providing scientific data that can improve understanding and help solve problems facing the Northern Forest and the people who live there. Since 2001 over 240 research grants were awarded to researchers studying the Northern Forest in Maine, New Hampshire, Vermont and New York.1

According to the authorizing legislation, the purpose of the program is to “coordinate and improve ecological and economic research relating to agriculture research, extension, and education” by conducting research, disseminating information, and analyzing land conservation strategies.2 The program has accomplished this by awarding grants in four broad areas over a period of eight years:

Theme 1: Sustaining productive forest communities: balancing ecological, social, and economic considerations (funding begun in 2002);

Theme 2: Sustaining ecosystem health in northern forests (the oldest area of funding, begun in 2001);

Theme 3: Forest productivity and forest products (funding begun in 2006); and

Theme 4: Biodiversity and protected area management (the latest theme, begun in 2008, just prior to this evaluation).

Grantees produce reports on their research which are available through the NSRC website. However, the program has had little feedback about how the results are otherwise disseminated and used, and by whom, and if the research is meeting the needs of the region.

The NSRC program staff and theme directors asked the Carsey Institute of the University of New Hampshire to conduct several data collection activities with NSRC stakeholders to investigate these questions. Stakeholders were defined as 1) NSRC grantees, i.e., principal investigators for the research grants; 2) practitioners and others who partner with grantees; and 3) other practitioners, policy-makers, and citizens with an interest in or a use for the research. Working with guidance from staff and the theme directors, the Carsey evaluation team specified the following questions to be answered by the evaluation:

1) What are the current research needs and priorities of the Northern Forest region’s stakeholders and what research topics or directions might be beneficial or relevant to them?

2) How relevant and useful is the current NSRC research focus to the region’s stakeholders, i.e., does it match the current needs and priorities described by the stakeholders?

3) What mechanisms are grantees using to communicate the results of their research to these stakeholders and how effective are they?

4) What other communication/outreach strategies might be more effective?

1 From the NSRC website: http://www.uvm.edu/rsenr/nsrc/about.php

2 Ibid.

Page 10: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

7

A fifth area, knowledge and perceptions of the NSRC grantmaking program, was added by Carsey after the evaluation was underway.

The following report describes the methodology used for the evaluation followed by findings in five areas: Principal investigator perceptions of grantmaking; stakeholder awareness and understanding of the NSRC; research needs and priorities; relevance and usefulness of NSRC research; and current outreach and dissemination of NSRC research. The report closes with a summary of findings, conclusions, and set of recommendations.

Page 11: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

8

METHODOLOGY

Evaluation Design

The objective of the evaluation was to obtain data on the perceptions, opinions, and attitudes about the NSRC grant program from people with a wide variation in knowledge and experience of the program. These people include 1) scientists and practitioners who participate in the program as grantees or have applied for but not yet received grants, 2) stakeholders who know about and use the research produced by the program, and 3) stakeholders who do not know about the program but because of their work in a relevant field, are potential beneficiaries of the research. Each of these groups was in a position to provide different types of data that could inform the evaluation questions.

The first and most knowledgeable group was included to provide data on the program’s implementation, i.e., how well it is achieving its mission of funding multi-disciplinary, collaborative research projects focused on the Northern Forest, producing results that can be applied to problems and questions confronting its people, forests, and economy. The second group, i.e., current users of the research, was expected to provide feedback on how well the program is doing, in terms of the quality, quantity, and relevance of the research conducted and the products of that research. The third group, lacking knowledge of the NSRC, would provide unbiased views of their need for the type of research produced by NSRC researchers and how such research might best reach them.

In addition to the data that could be provided by each group, the evaluation team also sought out areas where these groups might overlap in their knowledge, opinions, and perspectives. For example, all participants potentially had opinions on the needs of the Northern Forest region, and all potentially could respond to questions about how they access and use research findings. To the extent possible, the evaluation team took advantage of these likely commonalities and designed data collection tools that would allow for collecting the same data from two or more groups to elicit different perspectives on the same topic.

Data Collection

Surveys and telephone interviews were the primary methods of data collection from stakeholders, supplemented by several focus groups and informal interviews. NSRC grantee publications in the public domain were also reviewed.

Online surveys. Questionnaires were developed for online surveys of three stakeholder groups: NSRC grantee principal investigators, a group of users or beneficiaries of NSRC-funded research identified by the principal investigators in their survey responses, and actual or potential users or beneficiaries of NSRC-funded research who were subscribers to the Northern Forest News Digest. Each questionnaire was tailored to some extent to the recipients, particularly their expected knowledge and experience with the NSRC program. For the Principal Investigator Survey (or PI Survey), topics included: their status and involvement with the program, methods for dissemination of their research, opinions about the value and relevance of their project(s),

Page 12: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

9

effectiveness of and barriers to dissemination, and opinions about the NSRC grantmaking process and mission.

The two stakeholder surveys asked somewhat different questions because of different assumptions made about the likely respondents. Across the two, topics included their interest in the Northern Forest and other topics, where they go for information on research, awareness of NSRC and collaboration on projects, barriers to obtaining information, usefulness of NSRC projects, the type of research that makes the greatest contribution to the Northern Forest, other topics needing research, and benefits of NSRC research. For copies of the three questionnaires, see Appendix A.

All participants were contacted about the survey through e-mail solicitations. The NSRC provided e-mail contact information for all their principal investigators. Those grantees gave contact information for the people they identified as users or beneficiaries of their research. The second Stakeholder Survey was conducted with the cooperation of the Northern Forest Center which sent out an e-mail to their list of Northern Forest News Digest subscribers inviting them to participate. All people solicited to participate were sent an e-mail that provided them with a link to the survey and placed their responses in a database separate from their e-mail addresses to ensure anonymity. Respondents to all three surveys were volunteers and received no compensation for participation.

The Principal investigator Survey was administered online from March-April 2010; the first Stakeholder Survey of research users/beneficiaries followed in September-October 2010; and the second Stakeholder Survey of actual or potential users/beneficiaries was conducted from December 2010-January 2011.

Interviews. Carsey evaluation staff, NSRC program staff and theme directors developed a second list of actual or potential users or beneficiaries of NSRC-funded research for interviews by telephone. The purpose of the interviews was to probe with more depth into some of the issues raised by the findings of the surveys. The interview protocol began with a screening question that assigned the interviewee into two groups: those who are aware of the NSRC program and those who are not. Each group had its own protocol asking similar questions but relevant to their level of knowledge of the program. See Appendix B for a copy of the Interview Protocol.

These interviews took place in March-May 2011. Carsey staff supplemented these interviews with two focus groups and informal interviews conducted with members of the Northern Forest Higher Education Resource Network (NFHERN) and with Cooperative Extension staff from several states.

Publications review. A review of NSRC project publication citations was undertaken in early 2011. A search was conducted for all the citations for the publications listed on the NSRC website at that time to use as one objective measure of the extent of NSRC research dissemination. The following databases were searched: Web of Knowledge, Citation Linker, Google Scholar, citebase, CiteSeer, RefWorks, and EndNote. The type of publication in which the citation was found was recorded. Citations for 22 projects from the 2001-2007 award years were found.

Page 13: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

10

Data Analysis

The surveys produced a combination of quantitative and qualitative data; interviews and focus groups resulted in qualitative data. Summary statistics were calculated for quantitative survey data. Open-ended survey and interview question data were coded and categorized and analyzed to identify the range of responses and dominant themes.

Limitations of the Research

The data collected for this evaluation came from voluntary samples of principal investigators and other stakeholders with the consequence that one must be cautious about drawing conclusions about their responses. The decision to use volunteers rather than randomly selected samples was made for two reasons. First, the questions guiding the evaluation required obtaining as broad a range of stakeholder feedback as possible rather than collecting data to develop estimates of some opinion or behavior that could be statistically generalized to a population. Second, although the NSRC principal investigators were an easily defined and limited population from which a random sample could have been drawn, the potential and actual users of NSRC research were not and had to be identified and solicited through a variety of methods described above.

As a consequence of this decision, the findings presented in the sections that follow are based on samples that may not be representative of all principal investigators or of all other groups of potential and actual NSRC stakeholders. In particular, although we believe from examining the samples of these groups that they captured a wide range of opinion and perceptions, there is no way to know what is not represented in their responses or what biases exist in the data. If the same questions were asked of different samples of participants, the results of the evaluation might be different. Nonetheless, we have confidence in the methods we chose and believe that the results are valid for those groups of people who did participate.

Page 14: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

11

DESCRIPTION OF PARTICIPANTS

The charts in this section describe each group of participants in the evaluation. Both principal investigators and other stakeholders participating in surveys were asked questions about their background and chose their answers from a set of predetermined categories. Stakeholders who were interviewed were either asked about or volunteered their backgrounds but without predetermined categories to choose from, the evaluators had to derive categories from their responses, resulting in categories which were not always the same as the survey categories. As a result interviewees are described separately, following descriptions of each of the surveyed participants below.

NSRC Principal Investigators

Principal investigators for 209 grants awarded through 2009 were invited to participate in an online survey. Of those, 80 or 38% responded, more than the typical 25% for online surveys. Figure 1 describes the affiliation of the respondents.

Figure 1. Description of principal investigator survey respondents (n=80).

76.3%

5.0%

6.3%

2.5%

5.0%

1.3%1.3%

2.5% University/college

Research institute

State/federal agency

Forest-serving organization

Non-profit communityorganization

Business/industry

Non-affiliated

Other

Three-quarters (76.3%) of principal investigators responding were affiliated with a college or university.

Other affiliation categories with more than 2 respondents were state or federal agencies, research institutes, and community non-profit organizations (totaling 16%).

Page 15: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

12

Other Stakeholders

Two online surveys were conducted with actual or potential users of NSRC-funded research. The first was sent to 25 people identified by the principal investigators in their survey as stakeholder who had used or knew about the results of the grantees’ research. Of those 25, nine or 36% responded.

The second survey was sent to subscribers of the Northern Forest News Digest, a list of approximately 329. Of those, 69 responded for a response rate of 21%. Because many of the questions were the same on both surveys, the two samples were combined, resulting in a sample of 78 stakeholders, and were analyzed together for the findings presented in this report. For the few questions that were asked in the first survey for which we have nine responses, we report those responses separately in the findings.

Figure 2 below describes the affiliations of the respondents to the two stakeholder surveys combined.

Figure 2. Description of the two groups of other stakeholder survey respondents combined (n=78).

5%

16%

13%

4%

29%

3%

16%

16%

Scientist/researcher

State/federal agency

Forest-servingorganization/institution

Forest-basedbusiness/industry

Community/economicdevelopment organization

Policy-maker

Citizen

Other

Stakeholder affiliation was diverse, with the largest representation from community non-profit/economic development organizations (29%).

Page 16: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

13

Three groups each were represented by 16% of respondents: state/federal agencies, respondents identifying themselves as “citizens,” and a miscellaneous group identified as “other.”

Forest-serving organizations/institutions and forest-based businesses/industries also represented 16% of the respondents.

Carsey staff and NSRC program staff and theme directors generated a list of 34 actual or potential users or beneficiaries of NSRC-funded research to be interviewed by telephone. Of these, 24 or 71% participated in interviews which ranged in length from 10-45 minutes.

As noted above, stakeholders were asked about their backgrounds with the resulting categories derived from their responses. Figure 3 presents the categories of affiliation and/or occupation that resulted. Respondents could be placed in multiple categories so the percentages sum to more than 100.

Figure 3. Description of other stakeholder interviewees (n=24).

0%

20%

40%

60%

80%

100%

The majority (19) of the interviewees described themselves as practitioners.

Four people said they were Northern Forest residents but had they been asked specifically, most of the people interviewed were probably residents of the region, based on other information they provided.

The practitioners identified themselves as working for non-profit organizations, commissions, councils, and/or as consultants to organizations. The policy-makers said they worked for state or federal agencies, commissions, or councils.

Five of the interviewees worked in the forest industry; 14 did work related to planning and/or economic development.

Page 17: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

14

All were asked to characterize their interest in the Northern Forest.

Interests were about equally divided between forest -related issues (15) and community and/or economic development (13), with two mentioning conservation and land protection. Several people’s interests fell in more than one area.

- Several additional groups of people were interviewed more informally, to obtain insight into the perspectives of other groups not interviewed. One focus group included twelve members of the Northern Forest Higher Education Resource Network (NFHERN), members of which include university and college presidents, deans, and faculty. The focus group was held during an annual meeting of NFHERN in May 2011. A second focus group included a small (4) group of forest and wildlife staff from New Hampshire, Maine, and Vermont Cooperative Extension. Finally, two municipal employees were interviewed from the New Hampshire Local Government Center. These last two groups were interviewed during the summer 2011.

Page 18: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

15

FINDINGS

Principal Investigator Perceptions of the NSRC Program

The evaluation was an opportunity to obtain information about the program from the perspective of those stakeholders who benefit most directly from it: the principal investigators of the grants awarded. This section presents the results of the questions in the PI Survey that asked about their experience and perceptions of the program’s grantmaking process.

To understand where this particular sample of PIs was in the grantmaking process they were asked about the status of their grants (see figure 4).

Figure 4. NSRC grant status of principal investigator respondents (n=80).

0%

20%

40%

60%

80%

100%

Completed = 42

In-progress = 38

Of the 80 participating principal investigators, 52.5% had completed their NSRC-

funded projects and 46.5% had grants that were still in-progress3.

Across all the respondents, the number of projects funded per grantee over the years ranged from 0 (grant awarded but not yet funded) to 6 projects, with a median of 2 and mean of 1.89 projects.

PIs were then asked how they hear about NSRC Request For Proposal announcements (see figure 5).

3 Some PIs have been funded for multiple projects. In the survey each PI was asked to choose one project to

report on, either a NSRC project that they had completed or the NSRC project that they believe to have had the most impact if they had more than one. Consequently, 80 projects were represented in the survey which is fewer than the actual number of projects of the participating PIs.

Page 19: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

16

Figure 5. Percentage of principal investigators reporting source for information about the NSRC Request For Proposal (n=80).

0%

20%

40%

60%

80%

100%

Colleagues Grant alert Professionalmeeting

NSRCwebsite

Academicjournal,

newsletter

Other

Over half (58%) of the PIs said they hear about the RFP from colleagues.

One quarter (25%) obtain the information from grant alerts or bulletins.

Of particular interest to the NSRC is the relevance of the four theme areas it has chosen to fund. Grant applicants are asked to submit applications for research in one of the theme areas. The survey asked a general question applicable to all the theme areas: Was the theme under which you applied for NSRC funding broad enough to support the research that you wanted to conduct? Figure 6 summarizes their responses.

Figure 6. Principal investigators’ opinion on whether NSRC themes are broad enough to support their research (n=76).

0%

20%

40%

60%

80%

100%

Yes No Not sure

Almost all (91%) of the respondents agreed that the themes were broad enough for their research.

Page 20: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

17

Principal investigators were also asked their opinions on the effectiveness of the NSRC in two areas: aspects of NSRC grantmaking that the PIs would be the most familiar with, and implementation of the NSRC mission. Figure 7 shows their views on the grantmaking process.

Figure 7. Average effectiveness of aspects of the NSRC grantmaking process from the perspective of principal investigators: 4=Very effective, 3=Effective, 2=Somewhat effective, 1=Not effective at all. (n=76).

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Request For Proposalspublicity

Grant administration Technical assistance duringproposal

All aspects of the grantmaking process that all applicants would experience, from the Request For Proposals to grant administration were given high ratings, as effective to very effective.

Technical assistance, which is not offered formally as part of the program, was still rated highly, suggesting that when applicants or grantees did need help, they were satisfied with the people and/or process they experienced.

Principal investigators were similarly positive about the effectiveness of the NSRC in carrying out its mission (figure 8).

Page 21: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

18

Figure 8. Average effectiveness of the NSRC in carrying out its mission from the perspective of principal investigators: 4=Very effective, 3=Effective, 2=Somewhat effective, 1=Not effective at all. (n=734).

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Research onNorthern Forest

Collaborativeresearch

Researchrelevant to

Northern Forestresidents

Cross -disciplinary

research

Overalleffectiveness

All aspects of the NSRC mission, including principal investigators’ rating of the

effectiveness overall, were perceived as effective to very effective, with a median response of 4, very effective.

Efforts to do research that is cross-disciplinary was the area that received the lowest average rating, with a median of 3, effective.

The survey invited principal investigators to comment on the program and provide suggestions to improve the program’s grantmaking. From the 80 PIs in the survey, 48 wrote comments which were coded for the type of content.

Overall, almost half (48%) of the comments were positive, with 10 (21%) negative comments which were largely constructive criticisms. Four people had comments that were neither positive nor negative but general suggestions.

Comments included positive statements about:

the program overall, particularly its efficiency and effectiveness;

the program’s promotion of useful research relevant to the Northern Forest region;

the program’s promotion of collaboration – between scientists, across disciplines, and with non-scientists;

the increased research on the Northern Forest that has resulted;

the broad scope of projects and disciplines that are supported; and

4 This chart summarizes responses to five questions. The n reported above is the average number of PIs

responding across all items; the range of those responding varied from 70-76.

Page 22: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

19

the program’s promotion of interdisciplinary research.

Some of the specific comments were:

“Without this program there would be very little research that directly deals with issues important to this region.”

“[It is] done well to fund a breadth of projects and expertise across disciplines and regions”

“ [NSRC is] exceptionally well-positioned to allow researchers to leverage resources from other sources”

“[NSRC promotes ] great collaboration and synthesis within themes. Less opportunity to cross themes.”

The negative comments of the principal investigators fell into two areas:

the focus of the grants: one said that NSRC is funding research that is not relevant enough to Northern Forest residents; two said that the focus on the Northern Forest or the requirement to address one of the themes means that research on other important scientific questions are not being funded; and one person said that not enough social science research is being funded.

the scope and structure of the grant program: three said the small size of the grants limits the scope of what can be studied, the ability to foster graduate students, and the resulting impact of the research; and three people said there were problems with the process: the scientific review process not followed in some cases, there is bias in the awards made, and rejected applicants receive inadequate communication about their proposals.

Some of the specific comments were:

“[NSRC] projects in isolation fail to convey full impact – needs a vehicle for nesting and connecting to NSRC’s work in the big picture”

“The funding pool is so small that it’s hard to develop a good collaborative, cross-disciplinary proposal”

“Not all themes are equally "open" to competition”

“The themes divide up the pie and a proposal that crosscuts themes is disadvantaged”

“A short proposal precludes effective review by panels who are quite mixed in discipline, depth, and breadth of knowledge”

Overall, the principal investigators were very positive about the NSRC program and expressed appreciation for the grants.

Page 23: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

20

Other Stakeholder Awareness and Understanding of NSRC

The NSRC grant program has many potential stakeholders, people who live and/or work in or for the Northern Forest region, in many different professional fields and disciplines, occupations and organizations. These are the people who could most benefit from the research produced by the NSRC grant program. However, they must know about the program to learn about the research.

Awareness of the NSRC program. The evaluation’s surveys of these other stakeholders included both people who were aware of the program, with a few even participating as collaborators in the research, and others who knew nothing about the program except possibly the name or acronym. Most of these people were located through the Northern Forest News Digest subscription list. Figure 9 shows the extent of this samples’ awareness of the NSRC program.

Figure 9. Percentage of other stakeholders who said they were aware of the Northeast States Research Cooperative (NSRC) (n=76).

0%

20%

40%

60%

80%

100%

Yes No Not sure

Almost half (45%) of the other stakeholders said they were aware of the NSRC; 41% were not aware of the program.

In the stakeholder interviews, 63% or 15 of the 24 people interviewed knew about the NSRC, although in varying degrees. They were asked to describe the program’s focus and their answers were coded and categorized.

All 14 interviewees knew that the program had something to do with scientific research.

Of those 14, 79% knew that the program provided funding for research.

Several people knew something about the type of research supported: a balance of applied and basic science research (3); applied research (2); and basic science research (1).

Most of the interviewees said who the program serves: a broad range of stakeholders (8), or a targeted audience (5), primarily academic scientists.

Several people said the program is not transparent enough and wanted to know more about it:

Page 24: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

21

“Who is it? I have this vision of some group of academics sitting in a room somewhere but I don’t know who it is and how it’s organized.” (partner grantee)

“I want to know who is on the RFP decision committee, rather, what is the mix of representation? Are there non-technical, scientific people on the committee? It shouldn’t be all scientists.”

Collaboration. The stakeholder surveys also asked respondents whether they had collaborated on a NSRC grant. Few people responded in the affirmative (figure 10).

Figure 10. Percentage of other stakeholders who had collaborated on NSRC grants (n=75).

0%

20%

40%

60%

80%

100%

Yes No Not sure

Out of the 75 stakeholders responding, 9% knew they had collaborated on a grant and another 7% were not sure, most likely because they did not know if the grant was funded by the NSRC.

To find out more about the collaboration relationship, the 15 stakeholders who knew of the NSRC were asked if they had collaborated on a NSRC-funded research project. Five of the interviewees had. Collaboration had included

joint submission of proposals (3, with 1 of the proposals successful);

partnering with a university research unit on a project (5); and

informal sharing of information or planning a proposal (2). In a small focus group with four Cooperative Extension staff, the participants asked several questions about the NSRC grants. Specifically,

they asked questions about how the research generated actually hits the ground, whether there is a process for integrating the research, and why application of the research is not more emphasized during the Request For Proposal process (this latter point was emphasized by an individual who had collaborated on an NSRC project).

Page 25: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

22

One participant observed that the NSRC does not seem to reach out to Extension to serve as a principal investigator on projects but instead encourages them to be project collaborators.

The group agreed that the focus on research in the RFP is appropriate but wanted to see a mechanism in the process to ensure that the research gets applied by integrating Extension or other practitioners into the projects and should be a requirement for proposals.

A member of the NFHERN focus group commented:

“Really integrative, holistic collaborative engaged work is a far better way to go. I would really like to see that value really written into the RFP and the grant proposal evaluation process.”

In sum, about half of the other stakeholders who were surveyed and/or interviewed for the evaluation had some familiarity with the NSRC grant program and a few had experience with it through collaboration with researchers. However, among those interviewed who knew the program, few had a clear understanding and some misunderstandings about what the program does or who it serves. For those who knew something about the NSRC, collaboration was often mentioned as an important strategy for the program.

Page 26: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

23

Research Needs and Priorities

To assess whether NSRC research was addressing the priority topics and meeting the information needs of Northern Forest stakeholders and other researchers, the evaluation team asked a series of questions in online surveys of both Principal Investigators and Northern Forest stakeholders and through telephone interviews with selected stakeholders.

Perceived benefits of NSRC research. Principal investigators were asked about how they perceived the benefit of their NSRC-funded project, for scientific knowledge, practical applications or both, while surveyed stakeholders were asked which of these three benefits did they find the most useful (figure 11).

Figure 11. Benefits of NSRC research as perceived by principal investigators and other stakeholders.

0%

20%

40%

60%

80%

100%

Scienceknowledge

Practicalapplication

Both

OtherStakeholders(n=75)

PrincipalInvestigators(n=77)

The majority, 60% or more, of both types of respondents indicated that both

scientific knowledge and practical application were important.

The most interesting finding from these questions, however, comes from those who felt one or another of the benefits of the research was more important.

Stakeholders said that practical application was more useful than scientific knowledge by a more than ten to one margin (37% to 3%), while more than twice as many principal investigators chose scientific knowledge over practical application as the benefit of their research (20% to 9%).

The 21 responses received from interviewed stakeholders to a question about their thoughts on the contribution of NSRC research tracked very closely with the surveyed stakeholders.

More than half (57%) of the other stakeholders interviewed felt that contributions to both scientific knowledge and practical applications were the most important, while

Page 27: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

24

38% thought practical applications were the most critical and one person (5% of the sample) favored research that contributed to scientific knowledge.

In sum, a significant majority of principal investigators and stakeholders contacted believe that the utility of NSRC-supported research is to contribute to scientific knowledge and to practical applications, but among those who leaned towards one benefit or the other, PIs felt scientific knowledge to be more important while the stakeholders favored practical applications.

Comparison of NSRC research themes. To assess whether stakeholder interest in each of the four different NSRC research themes lines up with the actual distribution of research across the themes, we compared the other surveyed stakeholders’ expressions of interest in the particular themes with the research themes of the principal investigators’ research projects (figure 12) Other stakeholders could choose more than one theme so responses total more than 100%; PIs reported on one project so their responses total 100%.

Figure 12. Comparison of NSRC themes of principal investigator projects with NSRC themes of interest to other stakeholders.5

0%

20%

40%

60%

80%

100%

1: Sustainingforest

communities

2: Sustainingecosystem

health

3: Forestproductivity

4: Biodiversity

OtherStakeholders(n=78)

PrincipalInvestigators(n=80)

Other stakeholders expressed interest in all four of the themes, but there was overwhelming support for Theme 1, Sustaining Forest Communities, with 85% responding positively.

Interest in each of the other themes was approximately half as strong with each theme at around 40%.

By contrast the theme with the largest number of PI projects was Theme 2, Sustaining Ecosystem Health at 41%, followed closely by Theme 1 (36%) and more

5Figure 12 combines findings from two surveys with slightly different questions. PIs chose 1 theme from a list of

four that best represented their most current project. Stakeholders could choose more than 1 theme of interest from the list of four.

Page 28: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

25

distantly by Themes 3 and 4, representing 13% and 10% of the total respectively, differences that correspond to the length of time each theme has been funded.

In the interviews with other stakeholders, the findings were similar for the top choice but slightly different for the others.

Over half (55%) of interviewed stakeholders indicated Theme 1 as their primary interest, while one-third (32%) chose Theme 3, and both Theme 2 and 4 had only one or two respondents each.

Finally, from another source of stakeholder opinion, the focus group conducted with four Northern Forest-based Cooperative Extension educators produced similar responses. There was general consensus among the group that themes 1, 3, and 4 had the most relevance to Extension work and their typical audience (landowners, managers, forestry professionals, forest industry, etc.).

These findings concerning the themes are not all that surprising given the particular mix of backgrounds and occupations in the sample of stakeholders who participated in the evaluation. Conceivably a different sample could have indicated preferences for other themes. However for this group there was a clear indication of the strong support that exists for research on Northern Forest economic and community issues.

Another survey question added more support to this finding. Stakeholders were given a list of specific topics and asked to check their interest (see Figure 13).

Figure 13. Other stakeholders’ primary Northern Forest research interests (n=69).

0%

20%

40%

60%

80%

100%

Page 29: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

26

90% of stakeholders expressed interest in research on the Northern Forest economy.

Here again, other stakeholder’s preference for issues related to Northern Forest economic, community and human land use topics dominated bio-physical science topics.

Topics needing more research. The surveys, interviews and focus groups provided additional opportunities for both principal investigators and stakeholders to suggest additional areas for research. Among the PIs there was broad agreement that the four existing NSRC themes were the correct ones, though there were a few suggestions for targeted research on topics such as:

understanding local community need and demand for biological data (and how to get them what they wanted);

effects of forest habitat supply on featured wildlife species;

forest hydrology; and

forest – stream ecosystem connections.

The PIs’ preference for more bio-physical science topics, which may reflect the larger percentage of Theme 2 projects in the evaluation, was in contrast to the additional research areas suggested by stakeholders that emphasized economy, community and practical applications.

Energy issues, specifically biomass energy, were the most frequently mentioned topics.

Other topics were mentioned as well and these are included here more or less in order based on the number of responses:

effects/impacts of renewable energy development;

climate change impacts on economy, community, ecosystem and vice-versa;

tourism’s potential to turn around the current economic downturn;

local food and agricultural systems impact on economy and ecology; and

economic and ecological impact of community forests.

The integral and central connection between a healthy economy and community and a healthy forest came out most strongly in the stakeholder interviews and the focus group with NFHERN. A few key quotes sum up:

“What’s true in the rainforest is true in the Northern Forest – we have to deal with the poverty.”

“My main interest is the economic success of this Northern Forest. So that’s what you’re hearing from us. I can tell you a lot about problems with invasive species and sustainable harvesting, but if we don’t solve the economics then we won’t have a sustainable community and I won’t have a college. That’s where the research needs to be done. If we don’t have a healthy community, we won’t have a healthy forest. A healthy forest is a working forest.”

Page 30: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

27

Relevance and Usefulness of NSRC Research

The principal investigators and stakeholders were asked several questions about the relevance of individual research and NSRC research in general to different audiences.

Perceived relevance of NSRC research. In the PI Survey, principal investigators were asked to rate how relevant they thought the findings of their specific research were to a variety of different groups. Figure 14 lists these different groups and the researchers’ mean rankings in terms of relevancy for each.

Figure 14. Average relevance of NSRC research to various other stakeholder groups as perceived by principal investigators: 4=Very relevant, 3=Relevant, 2=Somewhat relevant, 1=Not at all relevant (n=716).

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Principal investigators perceived other researchers as the primary audience for their research, followed by state-level policymakers.

Principal investigators were also asked if they used the results of other NSRC research in their own work (figure 15).

6 This chart summarizes responses to ten questions. The n reported above is the average number of PIs

responding across all items; the range of those responding varied from 65-76.

Page 31: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

28

Figure 15. Principal investigators’ use of results from other NSRC research (n=77).

0%

20%

40%

60%

80%

100%

Yes No Not sure

More than 62% of the principal investigators said they use other NSRC research.

Stakeholders who were familiar with the NSRC were asked about the relevance and usefulness of NSRC research. Their responses were quite varied, from expressing enthusiast support to describing NSRC as a “closed club” for academic scientists. Experience with or collaboration on some aspect of a NSRC-funded research activity generally prompted positive responses, while those not directly involved with NSRC research or without a reason to use particular research findings tended to be much more critical of the program’s relevance. Specific quotes from each angle make this point fairly clearly. On the more positive side people made remarks such as:

[NSRC research is] absolutely critical…it’s given us the basis of our economic development program.”

[NSRC research is] “helpful in that it confirms other research that has been conducted.“

“The [project] that I collaborated with was really valuable to my department, and my staff.”

[NSRC research is] “great information for scientists and researchers.”

“I was able to expand my own work through the channels of NSRC project PIs.”

And from those more critical we received comments such as:

“I haven’t found them to be very relevant; too theoretical and abstract … most of the work is ecologically focused and not relevant for communities and specifically for their economic growth.”

“I can’t think of anything that has had a direct impact.”

“I don’t feel really connected to the research to be honest … but I can see the potential.”

“There doesn't seem to be an emphasis on application of the research in the real world.”

“Some of it, truthfully, just goes over my head.”

Page 32: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

29

Current Outreach and Dissemination of NSRC Research

In order to assess the effectiveness of outreach and dissemination strategies incorporated by NSRC and ensure that the research is utilized by stakeholders – including forest industry, policy-makers, forest-supporting organizations and institutions, community decision-makers, and the general public – the evaluation team looked at this process from the perspective of both principal investigators and other stakeholders, that is, actual and potential users of the research. This process included identifying how and where grantees disseminate the findings of their research, where they and other stakeholders access and how they utilize findings, and barriers to both dissemination and utilization, including any incongruities in those patterns.

Research Products for Dissemination

Each principal investigator was asked in the PI Survey to identify how they disseminate their research, that is, the most important product of their research (see figure 16).

Figure 16. Percentage of principal investigators identifying the most important products of their research (n=78)78.

0%

20%

40%

60%

80%

100%

7 In this section of the report and in the one that follows, with a few exceptions, the response categories in the

charts have been standardized to make it easier for the reader to compare across charts. The standardization includes all possible response categories asked in similar questions across several different surveys. Categories with responses of 0% or a 0 mean, unless otherwise noted, indicate that the category was not included in that particular survey question. The categories were slightly different in the surveys because of differences in the target audience for the surveys.

8 Percentages do not sum to 100% because respondents could identify more than one product.

Page 33: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

30

The most frequently identified product was an article in a peer-reviewed journal (83%) followed by professional meeting presentation (50%), and seminar or workshop (37%).

Conversely, products that target broader stakeholders – technical assistance providers, policy-makers, economic development entities, and forestry practitioners – were considered less important (30% or fewer responses).

In sum, the priority of most principal investigators is to write products from their research that serves their own professional needs while writing for public or practitioner audiences is of secondary interest.

Outlets for Obtaining and Disseminating Research Information

In the surveys for the evaluation, both principal investigators and the other stakeholders were asked where they go to find research. PIs and stakeholders familiar with NSRC were asked specifically about NSRC research; other stakeholders were asked more generically about Northern Forest research.

Principal investigators’ sources. As stakeholders for each other’s research, principal investigators were asked about where they go to obtain information about their NSRC-funded colleagues’ research (figure 17).

Figure 17. Principal investigators’ sources for others’ NSRC research (n=63).

0%

20%

40%

60%

80%

100%

The majority (60%) of principal investigators use colleagues as a major source of information about research.

Page 34: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

31

Fewer than half of the PIs go to academic journals (44%), an informational website - the NSRC website (43%), and professional meetings or conferences (41%).

Other stakeholders’ research sources. Since communicating the project results to broader public stakeholders is a goal of NSRC, stakeholders other than PIs were asked two related questions about where they obtain research on the Northern Forest. The first asked about their likelihood of using a particular source to find information about Northern Forest research and the second asked for their use of different sources (figure 18).

Figure 18. Average likelihood of other stakeholders’ use of source for research on Northern Forest: 4=Very likely, 3=Likely, 2=Not very likely, 1=Not at all likely (n=759).

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Stakeholders, on average, are likely to use a wide variety of sources to obtain

information about Northern Forest research, with informational websites and colleagues the most likely to be used.

Only trade publications fell into the “not very likely” range.

Other stakeholders’ responses were similar when asked about what sources they actually use (figure 19).

9 This chart summarizes responses to ten questions. The n reported above is the average number of PIs

responding across all items; the range of those responding varied from 73-77.

Page 35: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

32

Figure 19. Other stakeholders’ use of sources to find information on their areas of interest (n=69).

0%

20%

40%

60%

80%

100%

Of the 69 respondents, 80% indicated they use informational websites, which could include the NSRC website although this was not asked specifically in this question, and colleagues, respectively. Regional network and seminars, workshops and conferences are the sources of information used by 68% of the stakeholders responding.

Peer review journals were used by less than half (42%) of stakeholders.

A focus group interview with one group of stakeholders corroborated these findings. When asked where they get Northern Forest information that is of interest and relevance to them, Extension foresters indicated that they generally hear about it through colleagues and their regional networks. As one County Forester with UNH Cooperative Extension pointed out,

“I rely on my colleagues to stay up to date with the latest research findings. That is where I learned about NSRC….I have not been to their website, however.”

Comparison of PIs to other stakeholders as consumers of information about research. Both PIs and other stakeholders are users of research so presumably they would both face the same challenge in locating Northern Forest and/or NSRC research. To explore this question the two figures above, figure 17, where PIs go to find NSRC research, and figure 19, where other stakeholders go to find information on their areas of interest, were combined in one chart (figure 20 below).

Page 36: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

33

Figure 20. Comparison of principal investigator and other stakeholder sources of information.1011

0%

20%

40%

60%

80%

100%

OtherStakeholders(n=69)

PrincipalInvestigators(n=63)

Obtaining information from colleagues was the source listed by more respondents in

both groups.

Almost twice as many other stakeholders than PIs obtained information from websites. They were also more likely to use less academically oriented outlets such as newspapers, trade publications, and best practice guides.

About the same percentage in both groups preferred using academic/peer-review journals as a source of information.

Comparison of dissemination and utilization sources. We also compared the outlets where principal investigators disseminate their research to the sources used by other stakeholders to find research. Figure 21 first shows the results of principal investigators’ outlets for communicating the findings and accomplishments of their NSRC-funded research followed by a chart showing the comparison (figure 22).

10

The questions were not exactly comparable. Stakeholders were asked about general sources of information for their areas of interest and PIs were asked about sources of information about NSRC research. We made an assumption that at least some of the information that other stakeholders search for is research, which could include NSRC-funded research. 11

Categories with 0% response were not included in the question asked in the survey of that respondent group.

Page 37: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

34

Figure 21. Principal investigators’ outlets for dissemination of their own NSRC research (n=6612).

Conference presentations (88%), the NSRC website (78%), peer reviewed journal (76%) and seminars and workshops (73%) were the primary means used by principal investigators to disseminate their research findings.

Except for informational websites, fewer than one-quarter of the PIs reported using the other means of dissemination.

Next these data were combined with data from figure 21 above, stakeholders’ sources of research information, for figure 22 below.

12

This chart summarizes responses to nine questions. The n reported above is the average number of PIs responding across all items; the range of those responding varied from 60-73.

Page 38: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

35

Figure 22. Comparison of principal investigators’ outlets for research dissemination to other stakeholders’ sources for research.1314

0%

20%

40%

60%

80%

100%

OtherStakeholders(n=69)

PrincipalInvestigators(n=66)

Of the sources of information that both groups used, websites were the source

preferred by most PIs for disseminating their research, specifically the NSRC website, and were also the other stakeholders’ preferred means of obtaining research.

Overlap is most evident in seminars and workshops, although without knowing the audience for these, it is difficult to know if grantee research is being disseminated at the same seminars and workshops that other stakeholders attend.

PIs were most likely to present their research at professional meetings and conferences.15

PIs used academic journals as a primary outlet for dissemination, but it was one of the other stakeholders’ less preferred sources of research information.

These findings indicate that there is some disconnect between where principal investigators share and promote their research and where end-users obtain their information. In particular, two of the outlets most preferred by researchers, academic journals and professional meetings, most likely serve national audiences of fellow researchers. Searching through these journals

13

The questions were not exactly comparable. Stakeholders were asked about Northern Forest research and PIs were asked about their NSRC research. 14

Categories with 0% response were not included in the question asked in the survey of that respondent group. 15

Other stakeholders were also asked about conferences but it was included along with the category of seminars and workshops. Nonetheless, even if the other stakeholders’ use of seminars and workshops is completely accounted for by attendance at conferences, it seems likely that they are more likely to attend conferences for their own professions and fields which may not overlap with the scientific meetings and conferences of the researchers.

Page 39: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

36

and attending national meetings is an inefficient and costly means for other stakeholders, representing Northern Forest-serving institutions and entities, to seek out the information they need on the regional issues they are most interested in.

On the other hand, this comparison suggests the possibility inherent in several of the outlets for dissemination. Because the other stakeholders rely on websites for much of their information on research, the fact that the PIs are using the NSRC website for dissemination means that it has the potential for becoming a major source of information about Northern Forest research for the public. Also, although PIs and other stakeholders probably attend different seminars and workshops, the fact that they share a preference for that way of communicating and receiving research suggests that it could become a major vehicle for disseminating and translating research to other stakeholders.

Evidence from publications. The review of NSRC publications conducted for this evaluation found additional support for the finding that the audience for NSRC research is probably primarily academic.

Of the 141 total citations found for the 22 NSRC projects reviewed, 84% were in articles found in journals. Many of the citations were in articles by the NSRC researchers themselves, citing their own reports or published journal articles on the research.

The other citations were conference proceedings (9%), and a variety of other publications (7%), including theses, book chapters, websites, state reports, and non-technical reports or pamphlets.

We did not determine if the journals are peer-review or who their intended audiences are but it is likely that some of the journals do target scientists and non-academic practitioners. It also seems likely that some of the products of the research are used but not cited. Nonetheless, the predominance of journal article citations supports the earlier finding of the important role that journals play in dissemination.

Possible explanations. Several explanations are possible for the difference found in the way the research is being disseminated and where stakeholders look for research. First , grantees might believe some outlets are more effective than others in communicating their research to the desired audiences. The PI Survey included a question about grantees’ opinion of the effectiveness of the outlets they used for dissemination of research (figure 23).

Page 40: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

37

Figure 23. Averaged principal investigator ratings of the effectiveness of research dissemination outlets: 4=Very effective, 3=Effective, 2=Somewhat effective, 1=Not effective at all. (n=79)

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Principal investigators were more likely to rate academic journals, presentations at professional meetings, and seminars or workshops as the most effective, the same outlets that they reported preferring to use.

All the other outlets were rated as at least effective with the exception of the NSRC website, which on average, PIs rated as a “somewhat effective” means of dissemination.

Given that the audience for academic journals and professional meeting presentations are typically other academic researchers, these findings help explain why the PIs would choose more academic venues. However, they do not explain why the PIs do not choose some of the other outlets for research that, with the exception of seminars and workshops, are preferred by the other stakeholders, since all are rated as effective except the NSRC website.

Some possible reasons were presented to the PIs in the survey to learn which are perceived as barriers in the dissemination of their research (figure 24).

Page 41: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

38

Figure 24. Principal investigators’ perceived barriers to dissemination of their research (n=57).

0%

20%

40%

60%

80%

100%

The perceived barrier to dissemination reported by the most PIs (40%) was lack of funding.

The lack of a network for disseminating the research (23%), difficulty in translating the research into lay terms (21%), and lack of time (19%) were the other three significant challenges.

In the ‘other’ category, 5 people identified institutional barriers such promotion and tenure priorities that do not reward outreach and dissemination (4) and getting a peer-reviewed article published slows down dissemination of the public version (1). Four people said there were no barriers.

Very few principal investigators indicated that they did not like to disseminate their work or that they did not know how. This suggests that they would strive to share the results with a broader public audience if there was some level of support available through additional funding for dissemination, access to networks, and translation to convey the research findings in lay terms.

Several researchers did not believe that dissemination outside the academic sphere was expected of them. When asked why they focused their energies on writing and presenting to the research community, as opposed to practitioners and policy-makers, one Principal Investigator noted,

“Our project is focused on the research because that is what the funding is for. We need additional support to translate the research into a product for potential users”.

Challenges of Disseminating and Communicating Research to Other Stakeholders

The challenges of getting the word out about NSRC research are not limited to those who are producing it. The other stakeholders who are the actual and potential users of the research have their own problems obtaining the information. Some of these are related to the way the

Page 42: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

39

grantees are communicating the results of their research, but others are not and are problems for all Northern Forest research, not just the research produced by NSRC grantees.

Northern Forest research dissemination challenges. We asked stakeholders in our survey to tell us about the barriers to obtaining research about the Northern Forest in general, not NSRC research specifically16 (see figure 25).

Figure 25. Other stakeholders’ barriers to obtaining Northern Forest–relevant research (n=72).

0%

20%

40%

60%

80%

100%

Informationdoesn't get to

me

Not enoughtime

Informationdifficult to

access

Topics not ofinterest

Other

Over half (53%) of the respondents said that they do not have the time to seek out

information.

This finding alone may explain why colleagues and websites, probably the easiest and fastest sources to solicit for information, rank so high among stakeholders (see figure 19).

Only 19% indicated that the information does not get to them and few identified problems with accessing (11%) or irrelevant (6%) information.

This suggests that many other stakeholders would be interested in the research findings if they were communicated to them in ways that make it fast and easy for them to find and read it, an idea expressed in various ways in our interviews with stakeholders not familiar with NSRC. These stakeholders included representatives from forest serving institutions, organizations, practitioners, communities, and policy-makers. They described a number of barriers to accessing the research:

“There is too much information to weed through.”

“Information is not well summarized into one or few sources – it's all over the place.”

“Information about Northern Forest research is scattered and self-serving [for researchers].”

16

Many of the stakeholders responding to the survey were not expected to know the NSRC so the question was asked in more general terms.

Page 43: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

40

“Often times we’re singing to the choir and [the research results] lack application to practitioners.”

“There is a cost to obtain peer-reviewed articles.”

“We have restricted travel to meetings [so miss opportunities to learn about relevant research].”

What emerged from these interviews was the need for research that is organized for ease of access, communicated in a manner that is clear and concise, and speaks to the needs and interests of the practitioner community, not just the research community.

In interviews with stakeholders who knew the NSRC program, they were asked specifically about the effectiveness of its dissemination.

All 14 stakeholders who had heard of the NSRC said that the program’s dissemination was ineffective.

The primary reason given by most interviewees was that they either never or rarely receive or hear anything directly from NSRC about the program, grants that are funded, or completed research.

A variety of barriers were listed for why this might be, including:

“There is a lack of easy access to information about the grants and the project findings, as well as a lack of time to sort through lots of information”

“I tried to click on more than one of [NSRC’s research links] and it didn't work!”

“Some potential audiences are not aware of NSRC.”

“Forestry practitioners and forest-serving institutions typically hear about NSRC research findings through word-of-mouth.”

“There is no mechanism to drive people to NSRC’s website.”

Members of the NFHERN focus group (n=14) noted that there is no organized structure or central place for disseminating the research aside from the NSRC website. One participant said:

“In [rural Vermont], for us it is about getting our faculty to a hub where there’s an inventory at UVM website where they can find what’s out there”.

However, for the potential stakeholders we interviewed who did not know about the NSRC, the major barrier to using the NSRC research is that they are simply not aware of or have only barely heard of the program.

Page 44: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

41

Improving Research Dissemination and Communication

Both those who were surveyed and those interviewed had opinions about how dissemination and communication about Northern Forest research and NSRC funded research, in particular, could be improved. Suggestions from the PIs fell into the following categories:

improving online access to NSRC publications and perhaps make project data available;

as projects are completed or at the end of each project year, issue a NSRC press release and/or hold a press conference to highlight project results;

increase the opportunities for face-to-face contact between the researchers and the research users/stakeholders; and

develop an outreach document at the conclusion of each project, summarizing and translating project results for relevant audiences and including strategies for their application;

encourage more collaboration between researchers local stakeholder groups such as environmental managers;

increase the informal outlets for research that reach the other stakeholders , for example, workshops for professionals and articles in general interest magazines or newspapers;

increase the opportunities for informal public presentations, for example, setting up a

voluntary speaker’s bureau using the researchers, and seek co-sponsorship with aligned local organizations; and

improve the website so it is more interactive and content-rich.

Generally speaking, other stakeholders who were familiar with NSRC indicated that they keep up-to-date with the research through their personal connections to the researchers they know. However, they did not feel that there was an effective communication mechanism to enable them to learn about research findings outside of those relationships. One interviewee representing an economic development/regional planning organization noted that he doesn’t have time to dig through complex research findings in a journal. He said,

“For me, a simple and short summary of the research would be helpful, particularly if it were distributed through regional networks such as the Northern Forest Center.”

Other suggestions included:

actively reach out to practitioners and other stakeholders;

use the Northern Forest News Digest to disseminate key findings and links to the research reports;

develop e-mail lists and send out announcements about the research with links to the website and the research reports;

better use of forest-serving organizations’ websites to share research and grant information;

ensure that whatever information is shared is up-to-date, otherwise, the program will lose credibility;

Page 45: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

42

keep summaries of findings placed elsewhere very short and simple and link them to the NSRC website for more information; and

“just getting an email saying, ‘hey, we updated the project list and here is what is going

on across the region’ would be helpful… don’t even need to send the project abstracts…

just knowing that they have been updated is helpful.”

Page 46: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

43

CONCLUSIONS

The Northeastern States Research Cooperative (NSRC) grant program has been operating since 2001 with little feedback about its implementation or impact. The purpose of this evaluation was to obtain this feedback from the program’s primary stakeholders, the principal investigators of the funded research projects and potential or actual consumers of the research from a variety of local, state, regional, and federal agencies and organizations with an interest in the Northern Forest lands and communities.

The results of the evaluation are mixed. The principal investigators who have benefitted most from NSRC hold strong and positive perceptions of the value and operations of the program. They rated aspects of the program’s grantmaking and efforts to achieve its mission as effective to very effective. With the exception of the few who have collaborated with the PIs, other stakeholders who know the program like the concept but say it has little impact on their work. They have difficulty accessing the research, or once they learn about it, find the research does not meet their needs. A third group, potential users of the research, is interested but knows little to nothing about the program.

Much of the data collected for the evaluation focused on understanding the perceptions and experiences of these groups in more detail and were investigated in the context of a set of specific evaluation questions. The findings from these data are summarized briefly below.

1) What are the current research needs and priorities of the Northern Forest region’s stakeholders and what research topics or directions might be beneficial or relevant to them?

The majority of NSRC principal investigators and the other stakeholders agreed that the need is for Northern Forest research that results in both scientific knowledge and practical applications of that knowledge rather than either one or the other. Of those who believed the research should serve one or the other purpose, the PIs were somewhat more supportive of need for basic research, particularly forest bio-physical research, while the non-scientist stakeholders overwhelmingly said the research should address practical problems such as energy and development impacts.

Of the four NSRC research theme areas, the stakeholders expressed overwhelming support for Theme 1, Sustaining Forest Communities. Interest in the other themes was about half as strong among surveyed stakeholders and among those interviewed, Theme 3, was more popular than Themes 2 or 4. The majority of stakeholders expressed interest in a variety of Northern Forest topics but almost all listed the economy and most said research on the communities were of interest.

2) How relevant and useful is the current NSRC research focus to the region’s stakeholders, i.e., does it match the current needs and priorities described by the stakeholders?

The principal investigators surveyed represented projects from across all four NSRC research theme areas. Representation of each theme in their research corresponded to the length of time each theme has been funded. The majority of projects fell into the two themes funded the longest: Theme 2, Sustaining Ecosystem Health (41%), the oldest, closely followed by

Page 47: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

44

Theme 1, Sustaining Forest Communities (36%). As noted above, Theme 2 was one of the two themes of least interest to the other stakeholders.

Almost all of the PIs said that the NSRC’s focus on the four themes was adequate and broad enough to accommodate their own research. Almost two-thirds said they value the research produced by other NSRC researchers to use it themselves, and their collaborating non-scientist stakeholders also said the work is important, even critical to the Northern Forest. However, when the PIs were asked who their work is most relevant to, they said their researcher peers were the primary beneficiaries of the research and only secondarily, state and regional policymakers and practitioners.

Stakeholders who were the most familiar with the NSRC noted the bias of the research toward academic researchers as well. Those who had collaborated with or knew the program well generally accepted this bias; others less involved were more critical. They and other stakeholders expressed instead a preference for translated research that they could more easily understand and that addressed the pressing economic questions facing Northern Forest communities.

3) What mechanisms are grantees using to communicate the results of their research to these stakeholders and how effective are they?

Given that most principal investigators see their scientific colleagues as their primary audience, it was not surprising to find that the products of their research were largely peer-review journal articles with the means for dissemination traditional academic venues: professional meetings, the NSRC website, academic journals, and seminars and workshops. With the exception of the website, these were also rated as the most effective means for disseminating their research.

The NSRC website was rated as “somewhat effective” but received the lowest rating of the nine possible responses. Criticisms of the website concerned its limited content and usability

problems, but primarily people’s lack of awareness of its existence.

Most of the stakeholders, including the PIs, said that they are most likely to use informational websites as their source for information about research. Given this, it appears that the greatest potential for reaching other stakeholders appears to be through the NSRC website and other

websites with links to it, a potential inhibited by the website’s current limitations.

A similar potential exists in the use of seminars and workshops for sharing information about the research. Many principal investigators use this method for dissemination and many of the other stakeholders access research this way but both groups have to attend the same workshops and seminars and the research has to be understandable by a broad audience.

The most challenging area for dissemination appears to be in the use of other media such as newspapers, trade publications, policy documents, and best practice guides which few researchers use for dissemination but are widely read by non-scientist practitioners and policymakers. The PIs identified various barriers they face in using these nontraditional media. The primary ones are lack of financial and institutional support for doing so. Without funding available specifically for the purpose of translating and communicating their research to non-academic audiences, they cannot afford the time it takes to do so. For non-tenured and junior faculty, this problem is complicated by the promotion and tenure process. They also are less

Page 48: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

45

likely to have established the networks and collaborations needed for disseminating to non-academic colleagues.

The communication problem is not entirely one way. The other stakeholders also acknowledged their own problems in seeking out research. Over half of the surveyed stakeholders said they do not have time to seek out information on research. Lack of time is complicated by the information overload of the Internet and all the other sources of available information on research, making it a challenging and time-consuming process to find information relevant to them, a problem for all the stakeholders, including the PIs.

Given this problem, it was not surprising to learn that more PIs and more of the other stakeholders rely on their colleagues for information on research than on almost any other source. Such colleagues can act as filters, making recommendations and offering references to other relevant research. For the stakeholder who has never heard of the NSRC and is without a colleague who knows about it, discovering research identified as such seems a low probability.

4) What other communication/outreach strategies might be more effective?

Both NSRC principal investigators and the other stakeholders surveyed and interviewed had a

number of ideas for ways of improving NSRC’s outreach and dissemination. Most were variations on a theme:

Increase the publicity about the program and the results of projects, particularly through e-mail and online means of communication.

Increase the opportunities for the NSRC researchers to engage with practitioners and other stakeholders, through presentations and collaborations.

Their specific suggestions are found in the last section of the findings. Our own recommendations build on these suggestions and other ideas suggested by the findings.

Page 49: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

46

Recommendations

1. Increase the responsiveness of academic researchers to the needs of stakeholders for more research targeting issues related to the Northern Forest economy and its communities, for example, by making these issues a funding priority or encouraging researchers to use practitioners as co-principal investigators or collaborators on all research teams.

2. Increase the production of applied research by requiring that project proposals include plans for the utilization of research products.

3. Increase the dissemination of applied research by requiring that project proposals include a dissemination plan with named media and venues for sharing the research with non-scientist stakeholders.

4. Identify and facilitate opportunities for grantees to disseminate and communicate their research to stakeholders, particularly conferences, seminars and workshops where scientist/non-scientist professional fields converge or overlap; and ensure that the researchers know about them and are encouraged or incentivized to attend.

5. Improve the NSRC website’s visibility and usability, particularly by ensuring that it is linked to other websites and that all documents and e-mails include the Web address.

At the heart of these recommendations is improved communication – between NSRC and its grantees, between grantees and stakeholders, and between the program, grantees and the public. Some of the strategies require a commitment of resources that may or may not be available. However others require instead a commitment or will to modify traditional attitudes and behaviors about the purpose and value of research prevalent in universities, not just among the NSRC grantees. Given both the emphasis of the program’s authorizing legislation on dissemination and its own mission’s focus on research that is of relevance and benefit to the people engaged with the Northern Forest in some way – the ultimate stakeholders of the research – we hope these evaluation findings and recommendations contribute to this process.

Page 50: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

Connecting the Dots between Research Priorities and Stakeholder Needs:

An Evaluation of Northern Forest Stakeholder Perceptions of the

Northeastern States Research Cooperative (NSRC)

Grant Program

FINAL REPORT

Vol. II. Appendices

April 16, 2012

Curt Grimm, Ph.D., Principal Investigator

Barbara Wauchope, Ph.D., Co-Investigator

The Carsey Institute, University of New Hampshire

Huddleston Hall, 73 Main Street, Durham, NH 03824

Tel: (603) 862-2821, Co-Investigator

Email: [email protected]; [email protected]

and

Charles French, Ph.D.

University of New Hampshire Cooperative Extension

204 Nesmith Hall, Durham, New Hampshire 03824

Email: [email protected]

Page 51: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

2

Appendix A

Survey of Principal Investigators

Survey of Stakeholders (1) - Northern Forest Center New Digest Subscriber List

Survey of Stakeholders (2) - PI referred

Page 52: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

3

Survey of Principal Investigators

Northeastern State Research Cooperative

On behalf of the Northeastern States Research Cooperative (NSRC), we ask your participation in the following survey designed to help us assess the NSRC grant program's responsiveness to regional needs. To adhere to the highest professional research standards, we request your consent to complete the following questions. Your participation is voluntary and we will not be asking about regulated activities, personal information, or other sensitive matters. The survey is divided into three sections and we estimate that it will take you 10-15 minutes to complete the survey. The information you provide will be held confidential. Data are coded to mask each individual's identity. Your name will not appear in any written, published materials from this study. However, please be aware that there are rare instances when the researcher is required to share personally-identifiable information (e.g., according to law, policy, or regulations). The findings of the study will be available to all participants who request them. If you have questions about your rights as a participant in research you may contact Julie Simpson at the UNH Office of Sponsored Research at 603-862-2003 or [email protected] to discuss them. Please direct all other questions about this study to [email protected] or call him at 603-862-3778. If you are not able to complete the survey in one sitting, you can complete the survey at a later time.

SECTION 1: PROJECT BACKGROUND

1. Principal Investigator’s name

2. How many NSRC-funded projects have you received funding for as Principal Investigator (PI) or Co-PI?

Page 53: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

4

We would like you to answer the following questions regarding 1) a NSRC project that you have completed, or 2) the NSRC project that you believe to have had the most impact, if you have had more than one.

3. What is the status of the project?

Completed

In-progress

4. Please select one of following categories that best describes the entity under which you as PI were awarded the grant:

University/College

Research Institute

State/Federal Agency

Forest-serving Organization

Non-profit Community organization

Business/Industry

Non-affiliated

Other (please specify)

Page 54: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

5

5. Which theme did your project fall under?

Theme 1: Sustaining productive forest communities: Balancing ecological, social, and economic considerations

Theme 2: Sustaining ecosystem health in northern forests

Theme 3: Forest productivity and forest products

Theme 4: Biodiversity and Protected Area Management

6. Where did you first hear about the NSRC request for proposals?

From a colleague(s)

Professional meeting

Academic journal or newsletter

NSRC website

Grant alert/bulletin/email

Other (please specify)

Section II: Dissemination of Findings

Page 55: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

6

7. Which of the following best describes how you perceive the benefit of your NSRC-funded project?

Contributing to the scientific body of knowledge

Contributing to practical application(s)

Contributing to both the scientific body of knowledge and practical applications

Other (please specify)

8. What have been, or do you expect to be, the most important products of your NSRC-funded project?

Seminar/workshop

Peer reviewed journal article

Trade publication

Best practices/management manual

Policy document/plan

Newspaper Article

Meeting and conference presentation

Informational Website

Creation of a network or alliance

Collaborative partnership with stakeholders

Page 56: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

7

Technical assistance and capacity-building

Testimony (e.g. congressional)

Other (please specify)

9. Please rate how effective that you think each of the following resources have been for communicating the results and accomplishments of your NSRC-funded project .

Very effective

Somewhat effective Not very effective Not at all effective Does not apply

Professional seminar/workshop

Peer reviewed journal article

Trade publication

Best practices/management manual

Policy document/plan

Newspaper Article

Meeting/conference or other presentation

Informational Website

NSRC Website

Other

Page 57: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

8

10. If you had the opportunity to revise the way you communicated the results and accomplishments of your NSRC research, how would you do it?

11. Which of the following, if any, do you perceive to be barriers to getting the results/findings of your NSRC-funded project into the hands of potential users or others interested in your research?

Not having the networks/connections

Translating the science to lay terms

I don't like disseminating the results/findings

I don't know how to disseminate the results/findings

Outreach and dissemination of results/findings is not part of my work, i.e., not mandated as part of grant

Potential users are not interested in my research

Lack of funding

Other (please specify)

12. How relevant do you think that the findings of this research are to the following groups?

Page 58: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

9

Very relevant Relevant Not very relevant Not at all relevant Does not apply

Science/social science researchers

Local public policy-makers (municipal officials, etc)

Regional policy-makers (planning commissions, etc.)

State policy-makers (agency staff, legislators, etc.)

National policy makers (agency staff, legislators, etc.)

Businesses/industries

Community/economic development practitioners

Other nonprofit agencies

Consultants

The general public

Other

13. Can you give us examples of how others (for example, researchers, practitioners, policymakers) have used the findings/results of your project?

Page 59: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

10

14. Can you provide us with the name, and e-mail or phone, of one or more of the people who have used your findings/results or others who are aware of your project? (We may invite them to complete a survey about NSRC research.)

Section III: Your Experience with NSRC

For the following questions, please answer based on your entire experience with the NSRC.

15. Was the theme under which you applied for NSRC funding broad enough to support the research that you wanted to conduct?

Yes

No

Not sure

Page 60: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

11

16. If you answered “No” to the above question, please list or describe any themes (or sub-topics or aspects of existing themes) that you think might be missing?

17. Please rate the effectiveness of each of following aspects of the NSRC granting process.

Very effective

Somewhat effective Not very effective Not at all effective Does not apply

Publicizing the Request For Proposals

The Request For Proposals process

Technical/informational support provided to you by NSRC during your proposal writing process

Your award(s) notification

NSRC's administration of your grant(s)

Technical/informational support provided to you by NSRC during your grant

Other

18. Do you use the results of other NSRC-funded research? (If no, go to question 20.)

Page 61: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

12

Yes

No

Not sure

19. Where do you go to find out the results/findings of other NSRC-funded research?

From a colleague(s)

Professional seminar/workshop

Professional meeting/conference or other presentation

Academic journal or newsletter

Trade publication

Best practices/management manual

Newspaper article

NSRC website

Other (please specify)

20. What suggestions do you have for NSRC to help ensure that the results/findings of your NSRC-funded research are accessed and used by others?

Page 62: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

13

The NSRC's mission is to support "cross-disciplinary, collaborative research in the Northern Forest — a 26-million acre working landscape that is home to over a million residents and stretches from eastern Maine through New Hampshire and Vermont and into northern New York. Broad research goals are stipulated in the NSRC congressional authorization (Public Law 105-185). A central component of the program is the importance of the Northern Forest to society and the need for research activities to have relevance and benefit to "the people who live within its boundaries, work with its resources, use its products, visit it, and care about it."

21. From your experience as a NSRC researcher, please rate how effective the NSRC is in carrying out the following aspects of its mission:

Very effective

Somewhat effective Not very effective Not at all effective Does not apply

Cross-disciplinary research

Collaborative research

Research on the Northern Forest

Research relevant to the people who live in the Northern Forest

Overall effectiveness of the NSRC

Page 63: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

14

22. If you answered any of the items in the question above, please say why you feel this way.

Thank you for Completing this Survey!

Page 64: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

15

Survey of Stakeholders (1)

Northeastern States Research Cooperative

The Northeastern States Research Cooperative (NSRC) is a competitive grant program that supports cross-disciplinary, collaborative research in the Northern Forest — a 26-million acre working landscape extending from eastern Maine through New Hampshire, Vermont and northern New York that is home to over a million residents. On behalf of NSRC, we ask your participation in the following survey. Your input will help us assess the NSRC grant program's responsiveness to the region's needs. To adhere to the highest professional research standards, we request your consent to complete the following set of questions. Your participation is voluntary and we will not be asking about regulated activities, personal information, or other sensitive matters. We estimate that it will take 10 minutes for you to complete this survey. The information you provide will be held confidential. Data are coded to mask individual's identity. Your name will not appear in any written, published materials from this study. Please be aware that there are rare instances when the researcher is required to share personally-identifiable information (e.g., according to law, policy, or regulations). The findings of the study will be available to all participants who request them. If you have questions about your rights as a participant in this research, you may contact Julie Simpson at the UNH Office of Sponsored Research at 603-862-2003 or [email protected] to discuss them. Please direct all other questions to [email protected] or call him at 603-862-3778.

1. Which of the following best characterizes your professional or personal interest in the Northern Forest?

Page 65: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

16

I am a scientist/researcher

I work for a state/federal agency

I work for forest-serving organization/institution

I work for/own a forest-based business/industry

I work for a non-profit community/economic development-focused organization

I am a policy-maker (state or local)

I am a citizen with a stake in the Northern Forest

Other (please specify)

2. Which of the following topics are of interest to you? (check all that apply)

Northern Forest economy

Northern Forest communities and culture

Forest products industry (e.g. paper, lumber, furniture, etc.)

Forest management

Land conservation/Wildlife habitat

Land use change/conflicts

Outdoor recreation

Page 66: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

17

Forest science (nutrient cycling, disease pathology, etc.)

Ecosystem services (water filtration, carbon sequestration, etc).

Other (please specify)

3. Where do you go for information pertaining to your area(s) of interest? (check all that apply)

Seminar/workshop/conference presentation

Professional colleagues or acquaintances

Peer-reviewed journal article

Trade publication

Best practices/management guide

Policy document

Newspaper article

Informational Website

Regional network

Extension/educational outreach

Other (please specify)

4. Which of the following do you perceive to be the greatest barrier to staying

Page 67: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

18

informed about Northern Forest topics that are of interest to you?

Information that is relevant to my interests does not get to me

I do not have the time to seek out information

Information that is relevant to my interests is difficult to access

Information of interest to me is not shared in a form that I use

There is little information available about topics of interest to me

Other (please specify)

5. Which of the following do you feel would have the greatest contribution to the body of knowledge about the Northern Forest?

Research that contributes to the scientific body of knowledge

Research that leads to practical application(s)

Research that both contributes to the scientific body of knowledge and practical applications

6. Please rate how likely it is that you would go to each of the following sources for information about Northern Forest research that is of interest to you?

Very likely Likely

Not very likely

Not at all likely

No opinion/don't

know

Page 68: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

19

Seminar/Workshop/Professional meeting

Peer reviewed journal article

Trade publication

Best practices/management manual

Newspaper article

Informational Website

Northeastern States Research Cooperative Website

A colleague

A Northern Forest researcher

7. Are there specific topics or issues pertaining to the Northern Forest that you feel should be addressed through research?

8. What suggestions do you have to ensure that the results/findings from Northern Forest research are made accessible to diverse stakeholders?

Page 69: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

20

9. Are you aware of the Northeastern States Research Cooperative (NSRC)?

Yes

No

Not sure

10. Have you ever collaborated on - or received an award for - a Northeast States Research Cooperative-funded research project?

Yes

No

Not sure

11. If you answered yes to the above question, please briefly describe the project and how you collaborated.

Page 70: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

21

12. Each year, the NSRC supports research that fits into four research themes. Which of the following themes are, or would be, of interest to you? (please check all that apply)

Theme one: Sustaining productive forest communities: Balancing ecological, social, and economic considerations

Theme two: Sustaining ecosystem health in northern forests

Theme three: Forest productivity and forest products

Theme four: Biodiversity and Protected Area Management

Thank you for Completing this Survey!

Page 71: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

22

Survey of Stakeholders (2)

Northeastern States Research Cooperative

The Northeastern States Research Cooperative (NSRC) is a competitive grant program that supports cross-disciplinary, collaborative research in the Northern Forest — a 26-million acre working landscape extending from eastern Maine through New Hampshire, Vermont and northern New York that is home to over a million residents. On behalf of NSRC, we ask your participation in the following survey. You were identified by a NSRC-funded project investigator as a stakeholder who could benefit from its Northern Forest research findings. Your input on this survey will help us assess the NSRC grant program's responsiveness to regional needs. To adhere to the highest professional research standards, we request your consent to complete the following set of questions. Your participation is voluntary and we will not be asking about regulated activities, personal information, or other sensitive matters. We estimate that it will take 10 minutes for you to complete this two-part survey. The information you provide will be held confidential. Data are coded to mask individual's identity. Your name will not appear in any written, published materials from this study. Please be aware that there are rare instances when the researcher is required to share personally-identifiable information (e.g., according to law, policy, or regulations). The findings of the study will be available to all participants who request them. If you have questions about your rights as a participant in this research, you may contact Julie Simpson at the UNH Office of Sponsored Research at 603-862-2003 or [email protected] to discuss them. Please direct all other questions to [email protected] or call him at 603-862-3778.

Page 72: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

23

PART 1: FAMILIARITY WITH NSRC RESEARCH

Please answer the following questions regarding NSRC-funded research.

1. Which of the following best characterizes your interest in the Northern Forest?

I am a scientist/researcher

I work for a state/federal agency

I work for forest-serving organization/institution

I work for/own a forest-based business/industry

I work for a non-profit community organization

I am a policy-maker (state or local)

I am a citizen with a stake in the Northern Forest

Other (please specify)

2. Are you aware of the Northeastern States Research Cooperative (NSRC)?

Yes

No

Page 73: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

24

Not sure

If you answered "no" or "not sure" to the question above, please skip the rest of the questions on this page and click the 'next' button at the bottom of the page to go to Part 2.

3. Have you ever collaborated on a NSRC-funded research project?

Yes

No

Not sure

4. Where have you heard about NSRC-funded research? (check all that apply)

NSRC research investigator

A colleague

Seminar/workshop/conference presentation

Peer-reviewed journal article

Trade publication

Best practices/management guide

Page 74: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

25

Policy document/plan

Newspaper article

Informational Website

Regional network

Extension/educational outreach

Testimony (e.g. congressional)

Other (please specify)

5. Have you ever used the findings from NSRC-funded projects in your own work?

Select one of the following

Yes

No

Not sure

6. How useful do you feel the findings from NSRC-funded projects are in your work?

Very useful

Somewhat useful

Page 75: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

26

Neutral

Not very useful

Not at all useful

Don’t know

7. How effective do you feel that NSRC project partners have been at conveying relevant research findings to you?

Very effective

Somewhat effective

Neutral

Not very effective

Not at all effective

Don't know

8. Which of the following, if any, do you perceive to be the greatest barrier to staying informed about NSRC-funded projects that are relevant to you?

Information about relevant projects does not get to me

I do not have the time to look at the research findings

The information NSRC-funded projects is difficult to access

Page 76: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

27

Information about NSRC-funded projects is not shared in a form that is useful to me

The NSRC-funded projects are generally not relevant to my work

Other (please specify)

9. Which of the following do you perceive to be the greatest contribution of NSRC-funded projects?

Contribution to the scientific body of knowledge

Contribution to practical application(s)

Both the contribution of the scientific body of knowledge and practical applications

Other (please specify)

10. Please briefly describe any tangible benefits that you feel have resulted from NSRC-funded projects?

Page 77: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

28

PART 2: NORTHERN FOREST RESEARCH

11. Please rate how likely it is that you would go to each of the following sources for information about Northern Forest research that is of interest to you?

Very likely Likely

Not very likely

Not at all likely

No opinion/don't

know

Seminar/Workshop/Professional meeting

Peer reviewed journal article

Trade publication

Best practices/management manual

Newspaper article

Informational Website

Northeastern States Research Cooperative Website

A colleague

A Northern Forest researcher

12. Each year, the NSRC supports Northern Forest research that fits into four research themes. Which of the following NSRC research themes are, or would be, of interest to you? (please check all that apply)

Page 78: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

29

Theme one: Sustaining productive forest communities: Balancing ecological, social, and economic considerations

Theme two: Sustaining ecosystem health in northern forests

Theme three: Forest productivity and forest products

Theme four: Biodiversity and Protected Area Management

13. Are there other topics or issues pertaining to the Northern Forest that you would like to see addressed through the Northeastern States Research Cooperative request for proposal (rfp) process?

14. What suggestions do you have to ensure that stakeholders like you access and use the results/findings of NSRC-funded research?

Page 79: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

30

Thank you for Completing this Survey!

Page 80: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

31

Appendix B

Protocol for Northern Forest Stakeholder Interview

Protocol for Northern Forest Stakeholder Focus Group

Page 81: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

32

Protocol for Northern Forest Stakeholder Interview

Northeast States Research Cooperative

Project Title: Connecting the Dots between Research Priorities and Stakeholder Needs:

An Evaluation of Northern Forest Stakeholder Perceptions of NSRC Research

Project Goal: Assess stakeholder perceptions of the Northern States Research Cooperative (NSRC) grant program’s responsiveness to regional needs, specifically the relevance of research focus areas selected by NSRC and the strategies it and its grantees currently use to disseminate project results.

Field Method: Interview

Target Audience: Northern Forest researchers, organizational representatives, policy makers, foresters, managers, and educational outreach practitioners

Time Interview:

Date:

Place:

Interviewer:

Interviewee:

Anonymity Status of Interview Subject:

Specific Interviewee Instructions:

Introduction Statement:

The Northeastern States Research Cooperative (NSRC) is a competitive grant program that

supports cross-disciplinary, collaborative research in the Northern Forest — a 26-million acre

working landscape extending from eastern Maine through New Hampshire, Vermont and

northern New York that is home to over a million residents. You were identified by a NSRC-

funded project investigator as a stakeholder who could benefit from Northern Forest research

findings. Therefore, on behalf of NSRC, we ask your permission to conduct a personal interview

Page 82: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

33

to help us assess NSRC’s responsiveness to regional needs. The interview will take 15 – 25

minutes.

The researchers seek to maintain the confidentiality of all data and records associated with your

participation in this research. You should understand, however, there are rare instances when the

researchers are required to share personally-identifiable information (e.g., according to policy,

contract, and regulation). For example, in response to a complaint about the research, officials at

the University of New Hampshire, designees of the sponsor(s), and/or regulatory and oversight

government agencies may access research data.

For our part, we will secure all documents, transcripts, and tape recordings. Subsequent to the

transcription of the interview tapes, they will be destroyed and the files will remain protected.

The purpose for recording the interview is to ensure that we capture the information that you

share accurately. The information that you provide will be analyzed along with the other

interviews and the results of the study will be aggregated so that no individuals or communities

can be identified in published documents.

SCREENER: All Stakeholders

Interview Questions:

1). How would you characterize your interest in the Northern Forest? (e.g. are you a researcher,

policy-maker, interested citizen, employed by the forest industry, etc)

[Awareness of NSRC]

2). Are you aware of the Northeast States Research Cooperative or NSRC research grants

program?

[If YES – go to Protocol 1]

[If NO – go to Protocol 2]

Page 83: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

34

PROTOCOL 1: Stakeholders Familiar with NSRC

Interview Questions:

[Knowledge of NSRC]

3)What is your understanding of what the NSRC does?

4) What is your understanding of who NSRC funded research is designed to serve?

[Level of engagement with NSRC research]

5). Have you collaborated with a researcher on a NSRC-funded research project? If so, please

describe the nature of the project and your involvement.

[Experience with NSRC dissemination]

6) How do you find out about Northeast States Research Cooperative-funded research?

7). How effective is the NSRC in the way it conveys or disseminates its research findings to you?

8). How would you like to receive information about Northern Forest-related topics of interest to

you?

[Research interests]

9). How useful are the findings from NSRC-funded research projects for your work?

Probes: Can you give some examples?

What have you found that was particularly useful?

10). Northeast States Research Cooperative research grants are typically awarded in four

thematic areas. Which of the following NSRC research areas is, or might be, of most interest to

you?

Balancing ecological, social, and economic considerations

Sustaining ecosystem health in northern forests

Forest productivity and forest products

Biodiversity and Protected Area Management

Probe: What is it about this area that is [or these areas that are] of particular interest to

you?

11). Are there other specific topics or issues pertaining to the Northern Forest that you would

like to see addressed through the Northeast States Research Cooperative program’s grant-

making?

12). Which is more useful to you: research that contributes to the scientific body of knowledge

or research that contributes to practical applications or both?

Probe: What is it about this type [or these types] of research that is most interesting or

useful to you?

[Dissemination suggestions]

Page 84: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

35

13). In general, what do you perceive to be the greatest barrier to staying informed about

research related to the Northern Forest?

14). What suggestions do you have for the NSRC to ensure that the results of their research

projects are accessed and used by more people like yourself?

15). Do you have any other comments you would like to add about Northern Forest-related

research or the Northeast States Research Cooperative?

Page 85: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

36

PROTOCOL 2: Stakeholders NOT Familiar with NSRC

Interview Questions:

[Research interests]

[Insert description of NSRC]

3). Northeast States Research Cooperative research grants are typically awarded in four thematic

areas. Which of the following NSRC research areas is, or might be, of most interest to you?

Balancing ecological, social, and economic considerations

Sustaining ecosystem health in northern forests

Forest productivity and forest products

Biodiversity and Protected Area Management

Probe: What is it about this area that is [or these areas that are] of particular interest

to you?

4). Are there other specific topics or issues pertaining to the Northern Forest that you would like

to see addressed through the Northeast States Research Cooperative program’s grant-making?

5). Which is more useful to you: research that contributes to the scientific body of knowledge or

research that contributes to practical applications or both?

Probe: What is it about this type [or these types] of research that is most interesting or

useful to you?

[Dissemination suggestions]

6). In general, what do you perceive to be the greatest barrier to staying informed about research

related to the Northern Forest?

7). What suggestions do you have for the NSRC to ensure that the findings of their research

projects are accessed and used by more people like yourself?

8). How would you like to receive information about Northern Forest-related topics of interest to

you?

9) Do you have any other comments you would like to add about Northern Forest-related

research or the NSRC?

Page 86: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

37

Protocol for Northern Forest Stakeholder Focus Group

Northeast States Research Cooperative

Project Title: Connecting the Dots between Research Priorities and Stakeholder Needs:

An Evaluation of Northern Forest Stakeholder Perceptions of NSRC Research

Project Goal: Assess Northern Forest stakeholders’ perceptions of regional needs around .

Field Method: Focus Group

Target Audience: Northern Forest researchers, organizational representatives, policy makers, foresters, managers, and educational outreach practitioners.

Time Focus Group:

Date:

Location:

Focus Group Facilitator:

Focus Group Audience: ___________________________________________________

Anonymity Status of Focus Group Participants:

Introduction Statement:

The Northeastern States Research Cooperative (NSRC) is a competitive grant program that supports cross-disciplinary, collaborative research in the Northern Forest — a 26-million acre working landscape extending from eastern Maine through New Hampshire, Vermont and northern New York that is home to over a million residents. You were identified by a NSRC-funded project investigator as a stakeholder with an interest in Northern Forest research. Therefore, on behalf of NSRC, we ask your permission to participate in a focus group to help us assess regional needs related to Northern Forest research and the application of that research. The focus group will take approximately 45 minutes to an hour of your time.

The researchers seek to maintain the confidentiality of all data and records associated with your participation in this research. You should understand, however, there are rare instances when the researchers are required to share personally-identifiable information (e.g., according

Page 87: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

38

to policy, contract, and regulation). For example, in response to a complaint about the research, officials at the University of New Hampshire, designees of the sponsor(s), and/or regulatory and oversight government agencies may access research data.

You should also understand that what you say in the focus group may be repeated by other participants outside the sessions.

For our part, we will secure all documents, transcripts, and tape recordings. Subsequent to the transcription of the interview tapes, they will be destroyed and the files will remain protected. The purpose for recording the focus group is to ensure that we capture the information that you share accurately. The information shared during the focus group will be aggregated so that no individuals can be identified in published documents.

Focus Group Questions:

1). How would you characterize your interest in the Northern Forest? (e.g. are you a researcher, policy-maker, interested citizen, employed by the forest industry, etc). (Do a round-robin).

2). How many of you are aware of the Northeast States Research Cooperative (NSRC)? What is your perception of NSRC and what its mission is? How many have actually collaborated on NSRC-funded projects?

3). What specific topics or issues pertaining to the Northern Forest are of particular interest to you?

4) Are there specific topics or issues pertaining to the Northern Forest that are not being addressed or that you would like to see better addressed through research? Please explain.

5) What type of research do you feel would be most beneficial – basic research focusing on the hard sciences, applied research aimed at addressing Northern Forest stakeholder needs (policy-makers, businesses, conservation groups, etc.), or a combination of the above?

6). Where do you go for Northern Forest information that is of interest to you? Do you find that the information you need is accessible?

Page 88: FINAL REPORT Vol. I Narrative - Home Page | … · 2012-09-13 · FINAL REPORT Vol. I Narrative April 16, 2012 Curt Grimm, Ph.D., ... stakeholders were also surveyed online with the

39

7). What are the greatest barriers to staying informed about Northern Forest research that is relevant to you?

8). What suggestions do you have to ensure that the results/findings of Northern Forest-relevant research are accessed and utilized by stakeholders such as yourself?