Upload
dinhthu
View
217
Download
2
Embed Size (px)
Citation preview
Final Case Study: Methodology for Improving Alumni Relations
ROBERT ROBINSON
December 9th, 2016
Com 564 Research Methods for Professionals
Washington State University
COLLEGE OF
ENGINEERING AND
ARCHITECTURE,
WESTERN STATE
UNIVERSITY
Final Case Study: Methodology for Improving Alumni Relations
1
Table of Contents
Overview……………………………………………………………………………………..1-3
Focus Group Proposal………..………………………………………………………3-4
Survey Study Proposal……………………………………………………………….4-5
Experimental Study Proposal………………………………………………………...5-6
Focus Group Study…………………………………………………………………………..7
Moderator’s Guide……………………………………………………………………8-11
Discussion…………………………………………………………………………….12-13
Survey Study…………………………………………………………………………………14
Cover Letter………………………………………………………………………….14-15
Questionnaire…………………………………………………………………………15-22
Pretest Results…………………………………………………………………...........22-24
Discussion…………………………………………………………………………….24-25
Experimental Study…………………………………………………………………………..25
Design………………………………………………………………………………...25-27
Discussion.....................................................................................................................27-28
References…………………………………………………………………………………….28
Final Case Study: Methodology for Improving Alumni Relations
2
Overview
The purpose of this case study is to develop and review methodology for improving
alumni relations at the College of Engineering and Architecture at Western State University.
This study was commissioned by the CEA staff members. They are interested in understanding
and improving relations with their 20,000 alumni. The specific goals include (1) evaluate the
effectiveness of current communication strategies and recommend preferred strategies of alumni
(2) strategies for increasing alumni donations. The department is nationally recognized for the
quality of its education and research. Specifically, the CEA representatives are interested with (a)
assessing how young adults value their education as a means of developing fundraising, (b) how
alumni appreciation of student experiences can be translated as a fundraising appeal, (c) how
targeted donation options for specific programs or problems might generate funds, (d) what are
the preferred media contacts between the department and alumni, (e) how to improve alumni
website contact, (f) how website content should be presented for maximum interest, (g) how to
message effectively to under 35 Alumni as a means of building a lifetime relationship.
A major focus in the development of this research methodology is the alumni. It is clear
they are a major part of the overall CEA-WSU community. Therefore the findings of the
proposed research will impact all stakeholders which include WSU administrators, CEA faculty
and staff, current graduate and undergraduate students, alumni, employers, public and private
donors. Finding more effective channels to message as a means to reach out to alumni may
influence how the CEA department communicates accomplishments and improve fundraising
appeals. The College has established a positive reputation for its quality degree programs and
innovative research. Many of the graduates work in similar industries and tend to remain in the
Final Case Study: Methodology for Improving Alumni Relations
3
Pacific Northwest near major urban centers. Consequently, research may center on CEA alumni
who work in similar industries within specific geographic locations.
CEA fundraising officers have lingering questions about the best way to contact alumni.
Specifically, they question the benefits of e-mail versus regular mail fundraising solicitations and
the potential benefits of e-zines to replace expensive, hard-copy publications. Anecdotally, many
alumni seem to prefer infrequent contact with development staff members.
It is the purpose of this case study to investigate these issues through a mixed mode of
research approaches which includes qualitative research in the form of focus groups, quantitative
research through survey sampling and finally experimental research. The key components to
investigate include feedback on how to strengthen the CEA website. Specifically, identify
content areas that will resonate with alumni. Also, the CEA administrators request feedback on
the use of e-zines to replace print information. In relation to fundraising, they want to know
whether email or regular mail appeals for donations are preferred and how this compares to new
donation software on the website. This reflects a specific need to test and evaluate different
fundraising appeals. One of the key outcomes is to provide insights into the best way to develop
near term and long term donations with recent graduates. In order to achieve the desired
outcomes for the College of Engineering and Architecture three studies are proposed to generate
data upon which the department can build their growth strategies.
Focus Group Proposal
A key consideration in the generating feedback from alumni is to seek opinions. Focus group
methodology involves listening to the opinions of alumni in a small safe group through
discussion. It is proposed that a group of six to ten alumni participate in an open discussion with
a moderator. Using the resources of the alumni office a random sample of proposed participants
Final Case Study: Methodology for Improving Alumni Relations
4
will be selected. The study will take advantage of the fact that most alumni work in similar
industries and live in the Pacific Northwest. Although focus groups represent qualitative
research with a limited sample, efforts will be made to include alumni representative of the
graduate population. One of the main characteristics important to this study is determining
opinions of under 35 alumni and alumni over that age. A purposeful sample will be recruited
through the WSU alumni relations office with consideration to age, industry, geographical
location, type of degree, income, race and current activity with alumni activities and donations. It
is proposed that six focus groups be selected; three under 35 and three over 35. Six participants
in each group will comprise a total of 18 participants under 35 and 18 participants over 35, with
a total of 36 participants. Letters of invitation will be sent to 100 alumni members above and
below age 35 who are located within similar geographical areas like Spokane, Tacoma, Olympia
or Seattle. A small stipend of $50 will be offered as an incentive for their time. In the letters of
invitation, there will be an opportunity to indicate preferred times for meeting, so that preferred
times can be considered.
Survey Study Proposal
The public relations staff at the CEA Department indicated that there are 20,000 CEA alumni
registered with the school. Currently about 20 percent donate to the department. The goal is to
reach at least a 35 percent level. In order to get a more complete picture of the issues, a survey is
developed to generate feedback. The key content areas covered in the targeted questions included
website evaluation, fundraising appeals and preferred methods of communication with CEA-
WSU interactions. The method used to generate the sample is based on a random sampling from
the alumni members. Since one of the research goals is to reach out to the younger alumni as
well as the older alumni, demographic responses in the survey will include gender, age (Under-
Final Case Study: Methodology for Improving Alumni Relations
5
35 and Over-35), and geographical location. Using the total population of alumni at 20,000 it
would be estimated that with a 95% confidence level and a .02 confidence interval the sample
size should be 5,000. A randomly selected sample of 5,000 will be emailed the survey. With
survey return rates within organizations generally at 35% that would generate approximately
1,750 survey responses for analysis (Baruch & Holtom, 2008). A cover letter is also prepared to
highlight the importance of the survey to all alumni and to the College of Engineering and
Architecture. The questionnaire will include the key content areas of improving communications
with alumni through website evaluation, fundraising appeals and preferred methods of
communication with the department.
Experimental Study Proposal
An experimental study was designed to test the impact of a fundraising appeal on alumni over
age 35 and under age 35. The College of Engineering and Architecture is trying to build a
motivating connection with the under 35 alumni as a means of building a future fundraising
relationship. Survey research by the department indicates there is enthusiasm to give among
donors under 35 years of age. It is suggested that the younger donors have the highest motivation
but are likely to have the least capacity to give. In order to examine attitudes and response
behaviors two participant ages will be studied: alumni over 35 and under 35. The design of the
video fundraising appeal will be developed based on the feedback from the focus group and
survey research studies. The goal of the department is to increase donations from 20 percent to
35 percent. Therefore, evaluating the impact of the fundraising appeals is essential to achieving
that goal. Departmental research cites the fact that under-35 alumni are willing to give back, but
within their budget. Appeals to diversity and global consciousness are motivating and should be
considered within the fundraising appeal.
Final Case Study: Methodology for Improving Alumni Relations
6
The independent variable will be a video fundraising appeal which includes department
chairperson explaining the importance of donations to the success and leadership of the
department. They will highlight current research, student projects and how alumni have made a
difference. It will end with a link to online donations. Experimental Group 1 (over 35) will
receive exposure to the fundraising video. The Control Group 1 will not receive the treatment.
Both groups will receive a post-test attitude measure to determine differences between alumni
who watched fundraising appeal compared to those who did not. The post-test attitude
instrument will be the dependent variable. Group 3 Experimental participants under 35 will be
presented with the same fundraising appeal. Group 4 Control participants will not view the
fundraising appeal. As with Groups 1 and 2, 3 and 4 will complete the same dependent variable
using the post-test attitude measure. A between group design will be used in which n = 100
randomly sampled alumni within the above and below age framework will be assigned to one of
the four groups. With n = 100 in each group, that would comprise a total of N = 400.
Final Case Study: Methodology for Improving Alumni Relations
7
Focus Group Study
Focus Group Letter of Invitation
College of Engineering and Architectures
Western State University
Alumni Association
College of Engineering & Architecture
Western State University
Pullman, Washington, 99164
[Date]
[Name]
[Address]
[City, State, Zip]
Dear WSU College of Engineering and Architecture Alumnus,
As a graduate of the College of Engineering and Architecture at Western State University
we would like to invite you to attend a focus group discussion. The purpose is to help us
improve our communications with alumni, share our department innovations and
accomplishments and encourage fundraising appeals. One of our primary concerns is how you
evaluate the current website in terms of effective communication and outreach. The session will
be December 3rd
, 2016 from 6:30 to 8:30. The focus group will provide you with an opportunity
to find out what other graduates think about how to improve our outreach to alumni.
Your opinion will provide valuable insight to the department. The department has
contracted with the consultant firm of Robinson Research Associates. They will moderate the
discussion and analyze the results. Your responses will remain anonymous, only themes and
ideas will be reported on the department website when the results are completed. This study is
part of a longer evaluation process. Your feedback will provide a valuable starting point for our
study. If you would like to take part in the focus group on December 3rd
, 2016, please sign the
reply box at the bottom of this letter and email it back to us. You will receive a $50 gift card for
your participation. If you received a mail copy, please return it in the pre-paid return envelope.
If you have any questions, please contact the department at 206-xxx-xxxx.
Sincerely
Final Case Study: Methodology for Improving Alumni Relations
8
John Smith
Director of Alumni Relations
(xxx) xxx-xxxx
---------------------------------------------------------------------------------------------------------------------
Yes I will be glad to attend the Focus Group Session
No, I will not be able to attend
Contact information”
Name_________________________________________________
Email address__________________________________________
Phone:________________________________________________
Focus Group Moderators Guide
A moderator’s guide is included to guide the participant’s discussion and to generate information
that will help the CEA representatives make decisions on how to guide alumni participation and
fundraising appeals. The goal of the focus group interactions is to provide a structure that will
facilitate open discussion.
Introduction and Welcome:
Welcome and thank you for participating in this CEA alumnus focus group.
Introduction:
I would like to introduce myself. My name is _________ and I am the moderator for today’s
discussion. I would like to present ________who is our assistant moderator. They will be
recording our responses so that we may learn from them.
Purpose of the Focus Group:
We have been asked by the public relations department of College of Engineering and
Architecture at Western State University to seek your opinions about alumni relations. We
appreciate your willingness to participate and value your input. As moderator I will work to
guide our questions with follow-up to focus on your responses. The opinions you share with us
today will help guide future planning for CEA alumni interaction and communication.
Final Case Study: Methodology for Improving Alumni Relations
9
Specifically we are looking for ways to increase alumni interaction with the website, increasing
online communications with alumni and what fundraising appeals might be attractive. You were
randomly selected for this focus group. We centered in on (geographical area), as a large
representative sample of graduates are located here. Although not required by the University
Institutional Review Board, we ask that you review and sign the consent form for our records.
We need your input and want you to share your honest and open thoughts with us so that we may
better meet your needs as alumni.
Ground Rules:
This focus group will be about two hours long.
We want you to do the talking.
o We would like everyone to participate
o We may call on you if we have not heard anything from you
Remember, there is no right or wrong answer.
o Each person’s opinion is important
o Please share with us even if you agree or disagree
o We want to learn what you are thinking
Your opinions will remain anonymous
o We want you to share freely knowing that what is said in this room stays here
We will be tape recording our discussion
o We want to capture the responses, but we don’t identify anyone by name.
Please turn off all cell phones
Speak clearly one at a time
Are there any questions?
Icebreaker:
Let’s go around the room and briefly introduce ourselves.
Briefly share with us what you value most about your education at the College of
Engineering and Architecture at Western State. What do you appreciate about your
education and experience at CEA-WSU?
Key Questions:
The questions prepared today are designed to tap into your thoughts about the alumni website,
fundraising appeals and preferred methods of communication. I will present each one
Final Case Study: Methodology for Improving Alumni Relations
10
individually. We will spend about 25 minutes discussing the topic. When we are done we will
move on to the next question.
Topic: Website (Use computer to link to website on large screen. Present the landing page.)
Moderator: Most of you are familiar with the CEA-WSU website. Have a look at the landing
page.
1. How many of you have visited the alumni website in the last year? Show of hands.
2. For those who visited the website, how long did you spend on the website?
Was there anything specific you were looking for?
3. Was the content relevant to your interests?
4. If it was not, why not?
5. Those of you who did not visit the website, what were the main reasons?
6. What would encourage you to visit the website?
Fundraising Appeals: Present current website fundraising appeal
Moderator: Thank you for your website discussion and feedback. We now want to move on to
your ideas on improving fundraising appeals.
Moderator will share with the Under 35 Alumni Focus Group: Survey research results
indicate enthusiasm to give among donors under 35 years of age. These younger donors
have the highest motivation but are likely to have the least capacity to give. Try to
consider how this issue applies to you as you answer and discuss the questions.
Moderator will share with the Over 45 Alumni Focus Group: Giving to education
increased to $41.33 billion in 2012, an estimated 7 percent increase (not adjusted for
inflation). Four-year institutions receive roughly 75 percent of the donations in this
category. We would like you to consider ways to improve our fundraising appeals as you
answer and discuss the lessons.
1. How many of you have ever donated to the CEA College at Western University? Show of
hands.
How many of you have donated in the past year?
Can you identify how you were motivated to donate?
Do you feel your donation will make a difference?
Final Case Study: Methodology for Improving Alumni Relations
11
2. Are there any specific programs or ideas you would be motivated to donate to within the
school?
3. Would you like to have a choice of donation options so you can designate where you
want your money to go?
4. Would you like feedback on the impact of your donations within the CEA-WSU?
5. The department website is now implementing an online donation form to streamline the
process. Currently, it is through the mail. Do you think this will improve opportunities
to donate?
Preferred Methods of communication
1. What is your main channel of communication with the CEA college?
2. How many of you have visited the website in the last month, last year? Show of hands.
3. Does it provide you with the information you are seeking or would like to know more
about?
4. As you all know the CEA-WSU Department is well known for its excellent education and
outstanding research. They want to share with alumni their accomplishments.
What is your preferred method for receiving updates?
Would you like updates through email, e-magazines, Facebook, or other social
media?
5. How often would you expect updates and messaging? When would you consider
messaging to be too often? What do you consider a balance?
6. What type of messaging to encourage fundraising would you consider appealing or
motivating to you?
Ending Questions/General Discussion:
I want to take this time to thank everyone for participating in today’s focus group. As we end the
session, is there anything else that you feel is most important to this discussion? Have we missed
anything?
Conclusion:
I want to thank everyone for coming today. We appreciate your time and participation.
Final Case Study: Methodology for Improving Alumni Relations
12
Focus Group Discussion:
The data derived from the focus group will provide an in-depth look into the current ideas of
alumni Above 35 and Under 35. The key content areas covered in the targeted questions for
discussion included website evaluation, fundraising appeals and preferred methods of
communication with CEA-WSU interactions. It is the goal of the focus group research to
uncover information about these issues that might not be discovered in survey or individual
interviews (Austin & Pinkleton). The social interaction of the alumni group should work to
stimulate shared observations. It also provides an opportunity to probe positive and negative
responses. In this way, the results provide good face validity in the sense that they can assess
what is said with the context of what was meant by the participant. The moderator can probe to
confirm and clarify responses. However, since the groups are small, the insights gained may not
be translated to generalized conclusions. The sample was not selected randomly. It is non-
parametric with low predictability. The purpose of the focus groups in this study is to explore
alumni insights, examine themes across statements, dig deeper into these issues and generate
some ideas and themes that may offer some possible strategies to consider.
The characteristics of focus groups represent qualitative responses. They provide an opportunity
to get feedback from the respondents about the characteristics that influenced how the ad
campaign. Focus groups are generally much smaller than quantitative survey research. Sample
size is generally not an issue in focus groups as it is not designed to draw definitive conclusions.
In other words, it is not possible with a limited sample to project the findings as representative of
the larger population. Therefore, the small sample size limits external validity. Also each of the
four focus groups, even when presented with similar questions may follow different lines of
Final Case Study: Methodology for Improving Alumni Relations
13
thematic discussion. Therefore there may be less consistency or reliability with this type of
sampling.
Final Case Study: Methodology for Improving Alumni Relations
14
Survey Study
Survey Cover Letter
College of Engineering and Architectures
Western State University
Alumni Association
College of Engineering & Architecture
Western State University
Pullman, Washington, 99164 [Date]
[Name]
[Address]
[City, State, Zip]
Dear WSU College of Engineering and Architecture Alumnus,
Our college of Engineering and Architecture at Western State University is respected for
its achievements in research and teaching. Many alumni have told us of the value of their
education and how it has helped their careers. One of our goals is to improve our
communications with alumni. We are conducting a survey of alumni to give us feedback on how
we can serve you more effectively. The purpose is to help us improve our communications with
alumni, share our department innovations and accomplishments and encourage fundraising
appeals. The key content areas covered in the targeted questions included website evaluation,
fundraising appeals and preferred methods of communication with CEA-WSU interactions. In
the last year we have observed many alumni visit our website but don’t stay long to read the
information. With your feedback we may find better ways to meet your needs. We have
contracted with Robinson Research Associates to develop and analyze the survey. This is part of
a continuing mixed-method approach to study the problem. When the study is completed we
will share the findings on our website.
Your participation in the survey will be completely confidential and data will be reported
in general summaries in the written reports. This request has been sent to you by email. We can
mail you a copy if you would prefer. It should take no more than fifteen minutes to complete.
Once you return the survey, your contact information will be removed. Your participation is
voluntary.
We want to thank you for participating in our survey. Your insights and opinions will
help our alumni organization provide more meaningful communications. Not only will you
provide guidance in improving our website design abut also guide us in fundraising appeals.
Please call us at 206-xxx-xxxx if you have any questions. There is a three week period for all
alumni selected to return the survey. The final return date will be xx/xx/xxxx. Upon receipt of
Final Case Study: Methodology for Improving Alumni Relations
15
the survey we will send you an email VISA prepaid gift card for $10 in appreciation for your
time. When completed, return by sending back through email.
Sincerely, [Name]
Director of Alumni Relations
College of Engineering and Architecture
Survey Questionnaire
College of Engineering and Architectures
Western State University Alumni Survey
Introduction
We would like to thank you for participating in our alumni survey. The purpose of our survey is
to generate feedback from our alumni so that we may better serve you. Specifically we are
looking for ways to improve our communications with alumni through our website. We value
your opinion about the best ways to share our department innovations and accomplishments and
encourage fundraising appeals. Give us your feedback on topics, events and programs that
interest you. Please provide us with your honest opinions so that we may improve our alumni
outreach. The average time to complete this questionnaire is approximately 15 minutes. Mark
the questions with an x to indicate your opinion or choice.
College of Engineering & Architecture Experiences
1. What was your major focus of study
Bioengineering
Chemical engineering
Civil engineering
Computer engineering
Electrical engineering
Materials science & engineering
Mechanical engineering
Software Engineering
Construction Management
Infrastructure Engineering
Interior Design
Landscape Architecture
Structural Engineering
Other__________________
Final Case Study: Methodology for Improving Alumni Relations
16
2. How satisfied are you with your overall college experience at WSU?
Strongly Satisfied Strongly
Dissatisfied
1 2 3 4 5
3. How satisfied are you with the College of Engineering and Architecture experience
at WSU?
Strongly Satisfied Strongly
Dissatisfied
1 2 3 4 5
4. Please indicate how you rate your continuing activity with the College of
Engineering and Architecture at WSU.
Hardly Ever Occasionally Sometimes Frequently Often
How often do
you participate
in Alumni
Activities?
5. How would you rate alumni outreach experience?
Effective Ineffective
1 2 3 4 5
The College of Engineering & Architecture Website
6. How many times have you visited the CEA-WSU website in the last year?
None
1-3
4-6
More than 6
7. How would you rate the overall impression of the website?
Ineffective Effective
1 2 3 4 5
Final Case Study: Methodology for Improving Alumni Relations
17
8. Mark the type of information you were seeking. Mark as many that apply.
New research and innovations by CEA-WSU
Department Initiatives
Student accomplishments
Alumni accomplishments
Alumni Activities - Contacts
Donations
Other_______________
9. How would you evaluate the quality of information you were seeking on the
website?
Excellent
Very Good
Satisfactory
Poor
Unacceptable
10. Please mark all content areas on the website that you would like to see improved.
Mark as many that apply.
News about current research led by professors in the department
Current news about goals and department objectives
Information about student accomplishments
Information about alumni accomplishments
Donations that have made specific research and support possible
Alumni activities locally and around the country
Other_______________________
11. What additional features would you like to see integrated into the website to
improve interest? Mark all that apply.
Expanded links to social media like Facebook and Twitter
Highlighted news with video
Opportunities to respond and interact to department news stories
New programs
More multi-media presentations
Other_____________________________
Final Case Study: Methodology for Improving Alumni Relations
18
The College of Engineering & Architecture Preferred Channels of
Communications
12. What is your preferred channel of communication with the alumni office?
E-Newsletter
E-magazine
Print Newsletter
Print magazine
Website
Other_______________________________
13. What is your preferred channel of communication with the CEA-WSU updates?
E-Newsletter
E-magazine
Print Newsletter
Print magazine
Website
Other_______________________________
The College of Engineering & Architecture Fundraising Appeals
14. How often have you donated to CEA-WSU since graduation?
0
1-3
4-6
More than 6 times
15. How would you rate your overall donations to all organizations in the last year?
Final Case Study: Methodology for Improving Alumni Relations
19
$0
$10 - $20
$30 - $40
$50 - $60
$70 - $80
$90 - $100
$101 - $500
Above $500
16. How have you received fundraising requests?
Fundraising
Newsletter
Annual giving campaign
Phone call
No contact
17. How would you rate current fundraising appeals at the CEA-WSU website and
communications with you?
Outstanding
Better than expected
Meets expectations
Unremarkable
Poor
18. What improvements would you suggest in our fundraising appeals? Mark all that
apply.
Produce multi-media to illustrate new
research and innovations
Produce multi-media presentations of
faculty at leading conferences
Produce multi-media presentations of
outstanding students
Offer alumni memorabilia with
donation requests
Provide more fundraising events
Produce multi-media presentations and
interviews with outstanding alumni
Produce multi-media stories on alumni
activities
Provide alumni with choices in what
program they would like to make
donations.
Provide donation links on the website.
Donation opportunities to support
scholarship
Final Case Study: Methodology for Improving Alumni Relations
20
The College of Engineering & Architecture Demographic Information
In order to help us understand our alumni response patterns, please indicate what characteristics best
represent you. We wish to reassure you that all information will remain confidential and will not be used
for any other purpose.
19. Please confirm that you are a graduate of the College of Engineering and Architecture at Western
State University
Yes
No
20. What year did you graduate?
______________________
21. What is your gender?
Male
Female
Other
22. Please indicate your race/ethnicity.
American Indian or Alaskan Native
Asian
Native Hawaiian or Other Pacific Islander
Black or African American
Hispanic or Latino or Spanish Origin of any race
White
Two or more races
Race and Ethnicity unknown
23. Please indicate your marital status.
Single
Married
Separated
Divorced
Widowed
Other
24. What age range are you in?
Under 35
Final Case Study: Methodology for Improving Alumni Relations
21
36-39
40-50
60+
25. What is the highest level of education you have completed?
Bachelor’s Degree
Master’s Degree
Graduate or Professional Degree
Doctoral Degree
26. Please indicate the range of your personal yearly income before taxes during the past 12
months?
Less than $25,000
$25,000 to $34,999
$35,000 to $49,999
$50,000 to $74,999
$75,000 to $99,999
$100,000 to $149,999
$150,000 or more
Prefer not to answer
27. Describe your current employment status
Full-time employment
Part-time employment
Self-employed
Unemployed
Looking for employment
Retired
Final Case Study: Methodology for Improving Alumni Relations
22
Military service
Other_________________________
28. Please add any other comments or thoughts to help us improve our alumni outreach
Pretest Results
The preliminary cover letter and survey questionnaire were presented to six college graduates.
Time taken to complete the survey was recorded. On the average it took approximately 10-15
minutes. There were a total of 26 questions. The key content areas included feedback on the
evaluation of their college experience and continuing activity, interactions with the College of
Engineering and Architecture website, preferred channels of communication, fundraising appeals
and demographic information. Feedback from respondents was used to modify cover letter and
questionnaire.
Cover Letter
Feedback was generally positive for the cover letter. There was consensus that the key
points were clear.
o That the college of Engineering and Architecture at Western State University is respected
for its achievements in research and teaching.
o Many alumni have told us of the value of their education and how it has helped their
careers. One of our goals is to improve our communications with alumni.
Final Case Study: Methodology for Improving Alumni Relations
23
o The purpose is to improve communications with alumni, share department innovations
and accomplishments and encourage fundraising appeals.
The key content areas covered in the targeted questions included website
evaluation, fundraising appeals and preferred methods of communication with
CEA-WSU interactions.
In the last year many alumni have visited our website but don’t stay long to read
the information.
Feedback will help the department find better ways to meet your needs.
o Participation in the survey will be completely confidential and data will be reported in
general summaries in the written reports that will be shared with alumni.
o The request has been sent by email. There is an option to receive the survey by mail.
o It should take no more than fifteen minutes to complete. Email information will be
deleted from the survey. Your participation is voluntary.
Questionnaire
Feedback on the introduction to the questionnaire was considered to be succinct and to
the point. It states the purpose about the importance of obtaining feedback to better serve
alumni.
It states that the feedback is targeted to communications with alumni and effectiveness of
the website. It seeks honest opinions about their experience at the College of
Engineering and Architecture. It also requests information about fundraising appeals.
It states clearly that the questionnaire should take no longer than 15 minutes.
The primary mode of questionnaire response is through email. It is requested that
respondents mark an X on the appropriate response. Pretest takers requested a statement
about completing questionnaire and returning to alumni office. This change was made.
Final Case Study: Methodology for Improving Alumni Relations
24
It was suggested that question 18 should be changed from “What would you suggest as
improvements in fundraising appeals?” to “What improvements would you suggest in our
fundraising appeals?”
Demographics
Three participants noted that there was no marriage status or race/ethnicity. Review of
current recommendations by the Washington State Department of Health guided the
categories (VanEenwyk, 2010). These categories were added to the questionnaire.
Four participants requested in Question 26 that when answering income range, there
should be the option of “preferring not to answer.”
Survey Discussion
This survey designed for the public relations staff at the CEA Department will be administered to
randomly sample of 1,750 alumni of which there are 20,000 CEA alumni registered with the
school. The survey will be emailed and self-administered along with a cover letter detailing the
survey. The purpose of the research is to generate feedback on current departmental
communications and fundraising appeals. The current estimation is that only about 20 percent
donate to the department. In order to reach at least a 35 percent level the department has to
develop a more complete picture of the issues. The key content areas covered in the targeted
questions included website evaluation, fundraising appeals and preferred methods of
communication with CEA-WSU interactions. Findings will be analyzed separately by under 35
and over 35 age demographics.
The sample generated for this study provides a relatively accurate assessment of the
alumni population from which a degree of external validity can be generated. The pre-testing of
Final Case Study: Methodology for Improving Alumni Relations
25
the survey provided feedback on the internal validity of the questions. At face value, the general
question content appears to be representative of the departmental objectives. The revision was
valuable in developing the final survey questionnaire. The questions were relatively consistent
and should offer a format and a reliable survey format. The results will be summarized by
question type and analyzed for alumni preferences. One of the challenges is obtaining an
adequate return rate (Fink, 2017). Surveys generally have a low response rate. With the cover
letter, there is incentive to return the survey in which participants will receive a $10 email gift
card. This may work to improve return rate which will result in a more representative sample of
alumni.
Experimental Study
An experimental study was designed to test the impact of a fundraising appeal on alumni over
age 35 and under age 35. The College of Engineering and Architecture is trying to build a
motivating connection with the under 35 alumni as a means of building a future fundraising
relationship. The department observed that generally survey research indicates enthusiasm to
give among donors under 35 years of age. These younger donors have the highest motivation but
are likely to have the least capacity to give. In order to examine attitudes and response behaviors
two participant ages will be studied: alumni over 35 and under 35. The design of the video
fundraising appeal will be developed based on the feedback from the focus group and survey
research studies. The goal of the department is to increase donations from 20 percent to 35
percent. Therefore, evaluating the impact of the fundraising appeals is essential to achieving that
goal. Departmental research cites the fact that under-35 alumni are willing to give back, but
within their budget. Appeals to diversity and global consciousness are motivating and should be
considered within the fundraising appeal.
Final Case Study: Methodology for Improving Alumni Relations
26
The independent variable will be a video fundraising appeal which includes department
chairperson explaining the importance of donations to the success and leadership of the
department. They will highlight current research, student projects and how alumni have made a
difference. It will end with a link to online donations or request for mailed forms. Experimental
Group 1 (over 35) will receive exposure to the fundraising video. The Control Group 1 will not
receive the treatment. Both groups will receive a post-test attitude measure to determine
differences between alumni who watched fundraising appeal compared to those who did not.
The post-test attitude instrument will be the dependent variable. Group 3 Experimental
participants under 35 will be presented with the same fundraising appeal. Group 4 Control
participants will not view the fundraising appeal. As with Groups 1 and 2, 3 and 4 will complete
the same dependent variable using the post-test attitude measure.
A between group design will be used in which n = 100 randomly sampled alumni within the
above and below age framework will be assigned to one of the four groups. With n = 100 in
each group, that would comprise a total of N = 400. If pre-tests were used, then the experimental
design would be increased to 8 groups with a total of 800 participants. Pre-tests can work as
intervening variables by alerting participants to the content messaging. The department has the
option of expanding the design if it chooses. In the proposed design, it is recommended that two
experimental groups above and below 35 receive the fundraising appeal. The control groups will
receive the same post-test to see if there a significantly different response between groups. In
other words, does the fundraising appeal make a significant difference in alumni responses. A
total random sample of 400 will allow for sufficient data to determine the impact of the
fundraising appeal. This design can be replicated as needed for additional evaluation.
Final Case Study: Methodology for Improving Alumni Relations
27
Between Groups Design
Groups Random Sample Fundraising
Appeal
Post-test
Group 1 (Alumni
Experimental +
35)
R n = 100
X
01
Group 2 (Alumni
Control +35)
R n= 100
02 Group 3 (Alumni
Experimental- 35)
R n = 100
X
03 Group 4 (Alumni
Control - 35)
R n = 100
04
Discussion
In this design, one treatment condition across two experimental groups with two levels by age
will compared responses on an attitude measure towards donations with two control conditions
by age. This will allow researchers to assess the effects of the independent variable, which is the
fundraising appeal. It is recommended that the attitude towards fundraising measure be assessed
for content validity. Multiple questions should be designed to comprise multiple item indexes
related to the fundraising appeal (Austin &Pinkleton, 2015). They can be pre-tested for
reliability prior to use. The indexes can be evaluated through an item analysis to determine
relatedness to the key questions: 1. Importance of fundraising, 2. Effectiveness of fundraising
appeal presentation, 3. Ad impact on personal donations, 4. Suggestions for improvement.
This between groups design combines a treatment and control group with random assignments of
participants. The sample size and assignment will work to increased reliability and external
validity. It is designed to demonstrate that differences between treatment and control group are
due to chance. With 100 participants randomly selected within each of the four groups it
decreases selection bias of pre-existing group effects. Random assignment also works to increase
external validity which represents generalizability of results to the larger alumni population.
Final Case Study: Methodology for Improving Alumni Relations
28
Taken together, the focus group study, survey study and experimental study will provide
valuable feedback to the College of Engineering and Architecture on how to improve
communications and relations with alumni in achieving their goal of increased donations.
References
Austin, E. & Pinkleton, B. (2015). Strategic Public Relations Management. New York,
Routledge.
Baruch, Y. & Holtom, B. (2008) Survey response rate levels and trends in organizational
Research. Human Relations, 61; 1139-1160. Retrieved from
http://hum.sagepub.com/content/61/8/1139.abstract
Fink, A. (2017). How to conduct surveys. Los Angeles, Sage.
VanEenwyk, J. (2010) Revised guidelines for using racial and ethnic groupings in data
analyses. Washington State Department of Health. Retrieved from
http://www.doh.wa.gov/portals/1/documents/5500/raceethnguidelines.pdf