Fix it or Flee it?Proven approaches for dealing with

failing, flagging and floundering association programs

Greg Melia, CAE @gmeliacae

Photo credit: hebedesign on flickr

Program Assessment is a lot of GRIEF

Program Assessment is a lot of GRIEF

• Goals

• Research

• Impact

• Efficiency • Finances Credit: Angie Torres on Flickr

Defining the Goal of the Review

– Work with key stakeholders and decision-makers

– Solicit their key questions:• What do they want to know?• What information will help their decision-making?• What do they think needs to be evaluated?

– Refine questions to be measurable and defined

Exploring Program Goals

Investigate the Key “Whys”:• Why was the program originally established?• Why has it continued to be offered?• Why has it change over time?

Review the Original Intended Approach:• Who, what, when, where, why and how?• What was implemented? • What was not?

Lessons Learned

• Verbalizing goals of the review are an important part of preparing for change

• Seek superordinate goals

• Some agreements are easier reached upfront

Research3 kinds of data:

Corpus Inscriptionum

Opinions and Descriptions

Imponderabilia of behavior

Bronislaw Malinowski

Credit: Photographie et Ethnologie: Les photographes français au XXème siècle

devant d’autres formes de cultures.

Lessons Learned

• Historical minutes and documents can be VERY informative

• Seek to understand program mutations

• Good decisions based on bad data usually give bad results

• Remember they may have been involved

Impact Measure

• Outputs = Immediate I attended

• Outcomes = Short-term I gained knowledge

• Impacts = Long-termI served more members as a result

Efficiency

Staff and Volunteer Time

Complaints and Comments

Marketing/Communication

Registration/Orders

Technology Photo credit: mansikka on Flickr

Lessons Learned

• How do people feel about it? Do they have suggestions to increase efficiency?

• What are the trends in terms of level of effort?

• What is cost/benefit of doing it the same way versus trying a new approach?

• Can improved efficiency save it?

Finances

• Direct fixed expenses– Incurred regardless of how many people participate, buy,

or are served (e.g., office rent)

• Variable expenses– Incurred per each additional unit or person that is

serviced (e.g., per person meal charge)

• Mixed expenses– Incurred per each additional set of units or persons

serviced (e.g., room rental, temporary help)

Key Calculations

• Revenue – Expense = Margin

– Margin per attendee/unit

– Margin per staff hour

– Margin per impact

Lessons Learned

• One full flight is more profitable than two half full ones.

• Most do not realize the true full cost.

• Understanding costs helps determine price.

• Price increases (and closing programs) take time.

Implementation

• Who to involve– Consultant?– Stakeholders– Staff– Non-users

• Who to interview–Former volunteers–Former staff–Vendors

Photo credit: kierenmccarthy.co.uk

Communication of Results

• Review purpose of review

• Review what was done

• Arrange key findings in logical order, highlighting interpretations where appropriate

• Close with recommendations or issues to be addressed

Credit: EE.UTD.Events on Flickr

Lessons Learned

• Equivocal recommendations get equivocal results

• Decision-makers need synthesized data, not the opportunity for data analysis

• Survey data is important, but the proof is in historical performance

• Plan and communicate transitions

Thank You!Greg Melia, CAE

gmelia@asaecenter.org@gmeliacae

http://www.slideshare.net/gregmelia