Embed Size (px)
DESCRIPTION
ECUR 809 Final Project
Citation preview
Running Head: YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Evaluation of Drop-in Fitness Classes, YMCA Saskatoon.
Bill Kapphahn
University of Saskatchewan
ECUR 809
Jay Wilson
March 30, 2013
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 2
Contents
Page
Letter of Agreement ................................................................................................................ 3
Abstract of Proposal................................................................................................................. 4
Program Goals.......................................................................................................................... 5
Overall Purpose and Method of the Evaluation........................................................................ 6
Objectives and Approaches....................................................................................................... 7
Evaluation Plan ........................................................................................................................ 8
Limitations.................................................................................................................................9
Timeline................................................................................................................................... 10
Budget...................................................................................................................................... 11
Linkages to External Documentation ……..............................................................................12
Appendix 1 - YMCA Overview ..............................................................................................13
Appendix 2 – Drop-in Fitness Logic Model............................................................................ 14
Appendix 3 – Member Survey................................................................................................. 15
Appendix 4 – Survey Response Example Spreadsheet............................................................ 20
Appendix 5 – Program Planning Worksheet............................................................................ 21
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 3
Proposal Title: Evaluation of Drop-in Fitness Classes, YMCA Saskatoon.
Project Evaluator: William Kapphahn
Project Director: Fitness & Funds Development Director, Gail McDougall
Email: [email protected]
Approved by: Chief Executive Officer, Janet Giles
Email: [email protected]
Address: 25-22nd
Street East, Saskatoon, SK S7K 0C7
Phone: 306.652.7515 ext 224
Fax: 306.652.2828
________________________________
Signature
Gail McDougall
Fitness & Funds Development Director
________________________________
Signature
Janet Giles
Chief Executive Officer
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 4
Abstract of Proposal:
Drop-in fitness programs fulfill a major role of the YMCA in building healthier communities.
Methods used by this study will include research, surveys, observations and interviews. The
study will include all major stakeholders within the Saskatoon YMCA (participants, volunteers,
and staff). The goal of this evaluation is to measure the effectiveness of the drop-in fitness
programs. This evaluation, the first of this program, will be both diagnostic and prescriptive. The
evaluation will attempt to answer;
Does the Saskatoon YMCA land fitness program fulfill the existing needs? For example;
are there enough different fitness choices, time slots, equipment, and facilities?
What additional new programming may be added?
Will these new programs increase participation?
Are there any impediments to participation?
Funding Period: Summer 2013, winter 2013-14
Projected Number of Program Participants: 2500
Funds Requested: $2400.00
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 5
PROPOSAL NARRATIVE
Program Goals
The Canadian YMCA is dedicated to the growth of all persons in spirit, mind and body, and to a
sense of responsibility to each other and the global community (see appendix 1).The YMCA is a
charitable organization. Through child care, recreation, fitness, leadership, camps and youth
clubs, the YMCA helps families lead well rounded lives. The Saskatoon YMCA drop-in fitness
programs specifically address the goal of health, fitness, and recreation (see appendix 2).
One of the major tenants of the YMCA is that all programming should be periodically evaluated
to ensure the organization remains vibrant and pertinent. The proposed study will be part of this
diagnostic process. The evaluation will ensure the membership has input into programming at the
local YMCA.
The short-term outcomes of this evaluation will include increased awareness, membership,
knowledge and fitness. As a local YMCA, the long-term outcomes will be significant. Potential
exists for the drop-in fitness programming to have a greater impact and extended reach to the
general public. More specifically, the goals of drop-in fitness programs are greater health,
personal growth, and community service. These are currently fulfilled through volunteer
leadership, volunteer training, and participation.
Drop-in fitness programs at the YMCA are quite diverse. They include: Y-Cycle; Athletic
Circuit; Cardio Combo; Choreographed Step; Low Impact Aerobics; Muscle Sculpt; Tai Chi;
Zumba; Aquafit; Aquastep; Mommy and Baby Aquafit; Boot Camp; and, Yoga. In addition to
these drop-in programs, support programs are offered for instructor training. These additional
support programs include the instructor specific courses, the CPR first aid training, and fitness
retreats. Members are involved exclusively in drop-in programs. The importance of membership
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 6
satisfaction cannot be understated. General membership fees subsidize other programming
within the YMCA. A growth in membership results in substantial increases to funding, and
hence, overall programming. While fundraising drives have been quite successful in the past,
ongoing membership fees are a more stable and reliable funding source. There is a large,
unfulfilled capacity for membership at the Saskatoon YMCA. Increased participation in drop-in
fitness programming will be reflected in increased membership.
Overall Purpose and Method of the Evaluation
The goal of the evaluation is to measure the effectiveness of drop-in adult physical fitness
programs. Input from each of the stakeholder groups will be solicited in the form of
questionnaires and interviews. Primary stakeholders include the CEO, directors, staff, volunteer
leaders, and participants. Feedback regarding the type, quality, scheduling and participant
success in the drop-in fitness programs will be solicited. The study will utilize a hybrid CIPP
(content, input, process, product) with a cost-benefit analysis, and participant interviews.
The YMCA is a non-profit organization, and always concerned with funding models. So, while
the CIPP process is valuable in outlining the effectiveness of the program, the addition of a cost-
benefit analysis and participant interviews will be valuable to current and future boards of
directors of the Saskatoon YMCA. This data is will inform future planning. There must be a
concern not only for the monetary cost, but also the effect on the general membership of the
YMCA. A consequence of declining adult membership could be systemic underfunding of other
outreach programs.
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 7
Objectives and Approaches
Goal of Drop-in Fitness Program
Increased health, fitness and recreation
Objectives Measures Approaches
Provide a variety of activities Historical records and
membership perception
Observation, and survey data
Activities offered throughout
the day
Historical records and
membership perception
Observation, and survey data
Multiple opportunities for
activities
Historical records and
membership perception
Interview, and survey data
Create a positive fitness
community
Volunteer interviews and
membership perception
Observation, interview and
survey data
Fitness education Volunteer interviews and
membership perception
Interview, and survey data
Safe fitness environment Employee and volunteer
interviews and membership
perception
Observation, and survey data
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 8
Evaluation Plan
Background research and survey data from this evaluation will create a baseline for future
evaluations. Research of existing YMCA documentation at both the national and local level will
be done by the evaluator. This will allow the ability to match local data to national standards.
The survey will measure several quantifiable demographics as well as qualitative comments and
suggestions. It will be promoted through e-mail, monthly newsletter, and current events monitors
to the general membership. At the completion of each class and participants will be asked to
complete an online or written survey. Surveys can be completed at the front desk through a
membership computer. The computer will be made available at the front for the generation of the
survey. Written surveys will be available at the front desk upon request. These can be mailed or
dropped off at the front counter.
YMCA front desk staff will be responsible for the input of written surveys results into the
computer. An example of a draft participant survey is attached (see appendix 3). The staff
member will simply have to transcribe the data through the online form. Upon completion of the
survey, the results can be e-mailed to participants. This option can be disabled upon request.
The survey identifies demographic factors and quantitative data to measure programming
success. The demographic factors identify the current participating population. This data is
similar to that described in the national YMCA Serving All Members (SAM) survey. These
measurable outcomes will include: maintenance of ongoing programming; increased
participation; and alignment of local standards to YMCA national standards. This will allow the
Saskatoon YMCA to assess its performance against existing national standards.
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 9
Qualitative outcomes can be measured through the survey and an interview process. A third-
party impartial interviewer will be asked to talk to key stakeholders. Interviews of the program
director, volunteer leaders, and participants will be conducted. Interview notes will be collated to
establish criteria for success, and suggestions for improvement. Some of this information will be
found in the comment sections of the survey data (see appendix 3). The interviewer will
summarize all qualitative data into the report. This will avoid bias in the final report that could be
present because of close personal involvement with the YMCA.
All information will be kept confidential in the evaluation. The guiding ethical principles of the
American Evaluation Association will be strictly followed. Conclusions will be generalized, and
anonymous.
The final report will be presented to the Fitness and Funds Development Director and CEO of
the Saskatoon YMCA. Additional presentations to board members can be organized at your
convenience.
Limitations
While this evaluation was carefully planned, there still exist potential limitations. An outside
evaluator will be used to interpret data collected. This will eliminate most evaluator bias in the
study. However, the initial design was created by a member and volunteer of the YMCA so may
include some bias. This evaluation marks the beginning of an ongoing assessment process. The
process of implementing ongoing assessment practices is fundamental to positive change within
the organization. This evaluation is one of many tools which will improve the policies and
organizational practices of the YMCA. This initial assessment will set benchmarks of customer
service and practice. Follow-up evaluations are necessary for further ongoing developmental
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 10
changes. The evaluation is not the ultimate solution to all YMCA problem areas. This evaluation
will serve to identify the level of satisfaction of participants within the organization, and the
strengths and weaknesses of the existing drop-in programming. While exposing strengths and
weaknesses, the evaluation will not correct problems. Future actions taken by stakeholders will
be guided by this evaluation, and those actions will bring about change.
Timeline
Research of national and local publications June to August 2013
Survey window September and October 2013
Staff and volunteer leaders interviews September 2013
Program observation September 2013
Midterm progress report October 2013
The compilation of final document December 2013
Report to executive and CEO January 2014
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 11
Budget
Transportation: $50
Observer mileage to the Saskatoon YMCA - $50
Wages/Honoraria: $2000 [$8000 in-kind, $50 internal]
Graduate student observer - $25 per hour [approximately 80 hours] ~ $2000
Evaluator supervisor - $50 per hour [approximately 160 hours] ~ $8000 (in-kind)
Front desk staff - $10 per hour [approximately 5 hours] ~ $50 (internal)
Office Expenses: $350
General office materials - $100
Mailing expenses for survey forms - $1 per form [approximately 100 forms] ~ $100
Photocopying - additional forms for the front desk ~ $50
Final document publication - $10 per copy [approximately 10 forms] ~ $100
Software and other resource items: $0 [utilize existing computers]
Computer at the front desk - $1000 (utilize existing computers)
Survey forms and software - no cost ( using Google tools)
Advertising: $0
Utilize existing systems - Mass E-mail, newsletter, bulletin boards, display screens ~ no cost
Other: Survey incentives offered by YMCA Saskatoon [$250 internal]
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 12
Linkages to External Documentation
Canadian YMCA Health Fitness and Recreation
YMCA Saskatoon drop-in fitness programming
American evaluation Association guiding principles for evaluators
University of Wisconsin - program evaluation
Logic Model - YMCA (Appendix 1)
Logic Model - Drop-in Fitness Program (Appendix 2)
Draft Participant Survey (Appendix 3)
Survey Response Spreadsheet (Appendix 4)
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 13
Appendix 1
YMCA Overview
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 14
Appendix 2 – Drop-in Fitness Model
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 15
Appendix 3 – Member Survey
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 16
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 17
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 18
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 19
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 20
Appendix 4 – Survey Response Example Spreadsheet
Timestamp
Which type of membership do
you currently hold?
How long have you been a
member of the YMCA?
What is your current age?
Which time of the day do you
tend to work out?
3/15/2013
12:13:36 3/15/2013
12:27:06 Adult + 5-10 years 50-60 At lunch [12 to 1]
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 21
Appendix 5 – Program Planning Worksheet
Direction of evaluation: Evaluating the effectiveness of drop-in adult physical fitness programs. I am
currently a volunteer fitness instructor and a member of the YMCA. This gives me access to most of the
staff, other volunteer leaders and participants.
Engage Stakeholders
Who should be involved?
There a broad range of stakeholders to consider in this evaluation. These include: CEO, directors,
general membership, staff, volunteer leaders, and participants. Indirectly this will impact other
stakeholders from the general community, more specifically subsidize family programs.
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 22
How might they be engaged?
Input from each the primary stakeholders will be requested in the form of questionnaires and interviews.
These primary stakeholders will include the CEO, directors, staff, volunteer leaders, and participants.
Input regarding the type, quality, scheduling and participant success in the program
Focus the Evaluation
What are you going to evaluate? Describe program (logic model).
Organization: YMCA in the City of Saskatoon, Saskatchewan, Canada
Program: Land Fitness Classes, YMCA Saskatoon.
Model of evaluation assessment: I will use a hybrid CIPP (content, input, process, product) with a cost-
benefit analysis and participant interviews. The YMCA being a non-profit organization is always
concerned about funding models. So while the CIPP process is valuable in outlining the effectiveness of
the program, the addition of a cost-benefit analysis and participant interviews are valuable to current
board of directors of the local YMCA in making decisions. There has to be concerned not only for the
monetary cost but also the effect on the general membership of the YMCA. The consequence of declining
adult membership could be systemic underfunding of other outreach programs.
What is the purpose of the evaluation?
To determine if existing programming is meeting its expected outcomes, also to determine the future
programming options to increase adult participation.
Who will use the evaluation? How will they use it?
Who/users How will they use the information?
Directors Modify adapt the current program to increase
participation rates.
Programming coordinator Determine the support required to facilitate
greater participation in land fitness programs.
The plan future training for the creation of
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 23
additional volunteer leaders.
Volunteer leaders Participate in additional training required to
offer effective programming.
What questions will the evaluation seek to answer?
Does the existing land fitness program fulfill the existing needs? These will include are there enough different fitness choices, time slots, equipment, facilities, appropriate time in the day.
What additional new programming may be added? Will these new programs increase participation? Are there currently any impediments to participation?
What information do you need to answer the questions?
What I wish to know Indicators – How will I know it?
Should additional classes be added? Data collected from membership.
Are classes offered at appropriate times? Data collected from membership.
What type of classes will increase
participation? Data collected from membership.
Participant success? Increased participation, fitness leader
feedback.
When is the evaluation needed?
December 2013, in time for creation of the winter programming schedule.
What evaluation design will you use?
For this evaluation overusing a hybrid CIPP (content, input, process, product) with a cost-benefit analysis
and participant interviews.
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 24
Collect the information
What sources of information will you use?
Existing information: Past local participant surveys, national YMCA research, models
of surveys from other Saskatchewan YMCAs
People: Fitness leaders, participants
Pictorial records and
observations: Surveys, interviews sheets, participant numbers
What data collection method(s) will you use?
x Survey
x Document review
x Interview Testimonials
x Observation Expert panel
Group techniques Simulated problems or situations
Case study Journal, log, diary
Tests Unobtrusive measures
Photos, videos Other (list)
Instrumentation: What is needed to record the information?
Tracking of current participant numbers by fitness leaders. Survey recording sheet Interview recording sheet
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 25
When will you collect data for each method you’ve chosen?
Method
Before
program
During
program
Immediately
after Later
Literature and evaluation
reviews June 2013
Participant surveys Sept 2013
Fitness leader/participant
interviews Sept 2013
Results reported Nov 3013 Jan 2014
Will a sample be used?
No
Yes x
The general membership will voluntarily participate.
Procedure: The survey will be promoted in an e-mail as well as in the monthly newsletter to the
general membership. At the completion of each class, each participant will be asked to complete
a written survey. Surveys will be online and available upon request at the front desk. A drop box
will be located at the front counter as well.
Pilot testing: when, where, how?
Changes will be made on an ongoing basis, so no pilot program will take place. With the initial
implementation in September 2013.
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 26
Analyze and Interpret
How will the data be analyzed?
Data analysis
methods:
Responses to surveys/questionnaire will be compiled by hand and
summarized in a spreadsheet. Interviews will be conducted by an outside
assistant not directly involved in the YMCA.
Who
responsible:
YMCA front desk staff is responsible for the compilation of the surveys.
The outside assistant will compile all interview questions.
How will the information be interpreted—by whom?
Information will be interpreted by the adult fitness program coordinator in conjunction with the
evaluator.
What did you learn? What are the limitations?
I will use an outside assistant to conduct and compile all interviews. This is to avoid bias in the
final report because of my close involvement with the YMCA.
Use the Information
How will the evaluation be communicated and shared?
To whom When/where/how to present
Directors Written summary results and recommendations
to be shared December 2013.
CEO Informal meeting November 2013.
Adult fitness coordinator Informal meeting November 2013
Fitness leaders Written summary results and recommendations
to be shared January 2014.
YMCA DROP-IN FITNESS PROGRAM EVALUATION PROPOSAL
Page 27
Next steps?
Put forward proposal to CEO and Director.
Manage the evaluation Standards
Human subject’s protection Utility
Management chart Feasibility
x Timeline
Propriety
x Responsibilities
Accuracy
Budget
Note: Surveys will be voluntarily filled out by participants. Exact sample size will need to be determined
after surveys are completed.
