RESEARCH AND EVALUATION

ON A DIME

Religion Communicators Council

2015 National Convention

Alexandria, VirginiaDouglas F. Cannon, Ph.D., APR+M, Fellow PRSA

Virginia Polytechnic Institute and State University

April 10, 2015

WORKSHOP DESCRIPTION

Research is fundamental to public relations

planning and evaluation. Religion communicators

often say they don’t have the time, money or

background for research. This session looks at

simple, cheap methods—qualitative and

quantitative, primary and secondary—for

gathering information about publics and

communication effects. Communicators can use

these techniques to determine realistic

objectives, plan programs, assess results and

demonstrate effectiveness.

Topics

Introduction• What’s research? What’s evaluation?• Why do we research? Why evaluate?• Research and religion communicators• 4-step public relations process

Research• Types of research• Secondary research• Research techniques• Primary qualitative, quantitative research tools

Evaluation• Things to evaluate• Basic evaluation questions• Measurement levels

Introduction

What’s Research?

• Gathering information (listening,

observing, monitoring)

• Testing hypotheses, strategies,

messages

• Judging results

What’s Evaluation?

• Judging results against agreed-upon

objectives established during planning

• Determining value of effort

Why do research?

• To learn about, understand or assess

situations

• To gather information about publics

• To gauge public opinion, track issues, monitor

industry trends

• To test ideas, plans, messages, products

• To monitor, evaluate progress or feedback

• To make decisions, prepare for planning

• To generate publicity

Why Evaluate?

• To judge effects, outcomes

• To determine value of what you did

• To maintain accountability

• To document contributions to your organization

• To learn from successes, failures

• To provide information for future efforts

When to research?

• Continuously: Environmental scanning

• Before a campaign• Assess problem

• Assess publics

• Assess media, social climate

• During a campaign• Implementation checking

• In-progress monitoring

• Benchmarking

• After a campaign• Evaluate outputs

• Evaluate outcomes

Research & religion communicators

I conduct communication audits to identify

communication problems between the

organization and various publics.

• 2006: 1.96/5 (n=115)

• 2011: 1.98/5 (n=150)

• 2013: 1.72/5 (n=46)

Research & religion communicators

I report public opinion survey results to keep

management informed of the opinions of

various publics.

• 2006: 2.96/5 (n=115)

• 2011: 2.34/5 (n=150)

• 2013: 1.67/5 (n=46)

4-step

Public Relations process

• RACE

• ROPE

• R-PIE

Copyright © 2013 Pearson Education, Inc. All

rights reserved.

Public Relations as a Process:

RACE

ResearchEvaluation

ActionCommunication

Research

Types of PR research

• Secondary: Information gathered by

others

• Primary: Information gathered firsthand

• Qualitative: impressions

• Quantitative: numbers

Secondary research

• Search engines: Google Scholar, Google, Yahoo

• Research organizations

• Gallup (http://www.gallup.com)

• Pew Research Center (Religion and Public Life) (http://www.pewforum.org/)

• Institute for Public Relations (http://www.instituteforpr.org/)

• The Association of Religion Data Archives (http://www.thearda.com/)

• Academic journals

• Public Relations Journal (http://www.prsa.org/Intelligence/PRJournal/)

• Research Journal of the IPR (http://www.instituteforpr.org/research-journal/)

• Journal of Public Relations Research

• Public Relations Review

• Trade publications/blogs

• Mr. Magazine (https://mrmagazine.wordpress.com/)

• Bulldog Reporter’s Daily Dog (https://www.bulldogreporter.com/dailydog)

• CommPRO (http://www.commpro.biz)

• Ragan’s PR Daily (http://www.prdaily.com)

• RTDNA Daily (http://rtdna.org/)

Other sources

• University departments, schools, colleges• Advertising

• Communication

• Journalism

• Mass Communication

• Public Relations

• University agencies• Bradley Public Relations, Brigham Young University (http://www.bradleypr.com/)

• Cardinal Communication, Ball State University (http://cardinalcomm.org/blog/)

• Center for Communication Research, Texas Tech University

(http://www.depts.ttu.edu/comc/ccr/index.php)

• Lindsey + Asp Advertising + Public Relations, University of Oklahoma

(http://www.lindseyandasp.com/)

• The Plank Center for Leadership in Public Relations, University of Alabama

(http://plankcenter.ua.edu/)

research techniques

• Asking questions• Direct—to gauge knowledge

• Indirect—to probe motivations

• Observing

• Consulting sources

• Reviewing lessons learned

Primary research tools

• Surveys

• Direct observation

• Interviews• One-on-one (guided, depth

or informal)

• Group (focus groups)

• Experts/panels

• Content analysis

• Copy/message testing

Qualitative tools

• Interviews (formal, informal)

• Focus groups (Sunday school

classes)

• Direct observation (meetings, events)

• Expert panels (Lectionary study

groups)

• Message tests (Sunday school

classes)

• Textual/discourse analysis (reading)

Quantitative tools

• Surveys• SurveyMonkey.com

• Qualtrics.com

• Facebook

• Content analysis

• Readability tests

Survey samples

• Random

• Self-selected

• Purposive

Evaluation

Evaluation

• Measurement: Using metrics tied to

objectives (quantitative)

• Assessment: Judging success, failure

(qualitative)

• Evaluation: Determining value (Were

results worth the effort?) (qualitative)

Things to evaluate

• Preparation (research, planning)

• Implementation (execution)

• Output (messages, efforts)

• Outcomes (immediate results compared to

objectives)

• Outgrowths (long-term or societal changes)

Basic Evaluation Questions

• Adequately planned?

• Message(s) understood?

• How could strategy have been more effective?

• Audiences reached?

• Objectives achieved?

• What was unforeseen?

• Budget met?

• Future improvements?

Measurement Levels

Message-Exposure Measures

• Clippings/mentions

• Media impressions

(Placement x circulation/viewership/listenership)

• Internet hits, likes, follows, retweets

• Requests for information

• Systematic tracking/online analytics

(Analyze volume and content of media placements or online engagement)

• Advertising equivalency (not acceptable)

(Space/time x advertising rate)

Summary

Introduction• What’s research? What’s evaluation?• Why do we research? Why evaluate?• Research and religion communicators• 4-step public relations process

Research• Types of research• Secondary research• Research techniques• Primary qualitative, quantitative research tools

Evaluation• Things to evaluate• Basic evaluation questions• Measurement levels

Contact

Information

Douglas Cannon

Department of

Communication

Virginia Polytechnic

Institute and State

University

dfcannon@vt.edu

540-231-2331

106 Shanks Hall (0311)

Blacksburg, VA 24061