Surviving and Thriving in Programmatic Self-Assessment...

Surviving and Thriving in Programmatic Self-Assessment:

Faculty and Administrative Perspectives

Presented at the 2015 HLC Annual Meeting By:

David G. Brailow, Ph.D., Vice President for Academic Affairs and Dean, Franklin College

Allison Fetter-Harrott, Ph.D., Assistant Professor of Political Science, Franklin College

Randall D. Smith, Ph.D., Assistant Professor of Political Science, Franklin College

INTRODUCTION AND INSTITUTIONAL BACKGROUND

By the Numbers

• ENROLLMENT: Just over 1000 full-time students.

• ACADEMICS: 55 majors, 41 minors, and 11 pre-professional programs

• FACULTY: 74 full-time professors; Average class size = 15, Student-to-Faculty ratio = 12:1.

• STUDENT CHARACTERISTICS: 96% receive financial aid; 35% participate in NCAA Division III sports; 30% first generation.

• OUTCOMES: 95% graduates employed/attending graduate school within six months of graduation.

Franklin College and HLC

• Higher Learning Commission reaffirmed Franklin College accreditation in 2012.

• 2012 Systems Appraisal Feedback Report reinforced opportunity for more systematic program review vis-à-vis SLOs.

• Transitioned to Open Pathway. • Program assessment planning AY 2012-13,

executed AY 2013-14. • Administration invites Dr. Tami Eggleston,

McKendree University, for faculty-wide training on mission statement formation and SLO setting.

And the HLC

Program Review Protocol

• Departments assess at least one student learning outcome per year

• Departments submit assessment reports every three years

• Departments undergo full program review every seven years

• VPAA and Associate VPAA review and discuss results with each department

Program Review Protocol

Franklin College Political Science

• Political Science pilots program assessment at Franklin College.

• Review Process permitted: – Clarification of department mission and vision,

– Linking SLOs and Dept. Goals to mission and vision,

– Seeking empirical evidence to measure achievement of SLOs and Goals.

• Results were mixed – Some clear success,

– Empirical evidence supports curricular revision, etc. to better fulfill mission and vision.

Political Science

SEVEN STRATEGIES FOR SUCCESSFUL ASSESSMENT

1. Don’t Panic!

• Faculty should pursue their vision for their program.

• “Skillful teachers” naturally critically reflect on their work (Brookfield, 2006). Programmatic assessment formalizes practice.

• Embrace assessment that fulfills your goals. • Resistance to assessment may be

rational/expected (Bloxman & Boyd, 2008; Knight & Yorke, 2003), but should not obstruct.

2. Let Vision Guide You

• Begin with mission and vision (Eubanks, 2010).

• Then, identify key student learning outcomes (SLOs) and programmatic goals (Bloxham & Boyd, 2008; Eubanks, 2010).

• Brainstorm about how SLOs and goals are, and might be, met.

• The assessment world is in your hands

2. Let Vision Guide You

3. Don’t Reinvent the Wheel (KISS)

• REDUCE: Do not assess everything, only progress toward SLOs/goals (Bloxham & Boyd, 2008). Example: See Supplement.

• We put a spin on Eubanks (2010) and others:

• REUSE: Existing classes and assignments may fit SLOs/goals. Example: POL 110 Pre-Post.

• RECYCLE: Rubrics currently in use (slightly altered and/or AAC&U’s rubrics) may assist in assessment Example: POL 360 and Critical Thinking.

3. Don’t Reinvent the Wheel

4. Assessment Doesn’t Have to Be Quantitative

• Employ a diverse range of measures (Bloxham & Boyd, 2008; Shipman, et. al., 2003).

4. Assessment Doesn’t have to be Quantitative

• Example: Success via qualitative exit interviews and focus group with our graduating seniors (using mostly open-ended questions).

5. If you are going to use numbers, don’t over-complicate

• Colleagues can help.

• Do not be put-off by a “small n.” Take the first step. The “n” will come with time. Even a “small n” result may be useful.

5. If you are Going to use Numbers, Don’t Overcomplicate

• Avoid elaborate/cumbersome systems (Eubanks, 2010).

• Much may be learned from t-tests and z-scores.

6. Value to Students

• Build around your best learning experiences.

• Identify best part of program, and see if students agree (Eggleston, 2013; Bain, 2004).

6. Value to Students

7. Intrinsic Value to the Craft

• Use results to inform teaching and innovate, becoming more efficient and effective (Asand, Amidon & Huffman, 2010; Eubanks, 2010; Knight & Yorke, 2003; Shipman, et. al., 2003 ).

• If you don’t assess yourself; someone else might!

• Example: Results indicated possible changes and methods to improve student performance on comps.

7. Intrinsic Value to the Craft

QUESTIONS?

American Association of Colleges and Universities. "VALUE (Valid Assessment of Learning in Undergraduate Education)." Association of American Colleges & Universities. N.p., 07 Oct. 2014. Web. 11 Feb. 2015. <http://www.aacu.org/value>.

Asand, Hardin L., Stevens Amidon, and Debrah Huffman. "Slouching Toward Assessment: One Department's Journey Toward Accountability Bliss." Coming to Terms With Student Outcomes Assessment: Faculty and Administrators' Journeys to Integrating Assessment in Their Work and Institutional Culture. Ed. Peggy L. Maki. Steerling, VA: Stylus, 2010. 107-22. Print.

Bloxham, Sue, and Pete Boyd. Developing Effective Assessment in Higher Education: A Practical Guide. Maidenhead: Open UP, 2007. Print.

Bain, Ken. What the Best College Teachers Do. Cambridge, MA: Harvard UP, 2004. Print.

Brookfield, Stephen D. The Skillful Teacher: On Technique, Trust, and Responsiveness in the Classroom. 2nd ed. San Francisco, CA: Jossey-Bass, 2006. Print.

Eggleston, Tami. "Outcomes, Rubrics, Assessment Workshop I." Assessment Workshop. Franklin College, Franklin, IN. 2-3 Jan. 2013. Lecture.

Eggleston, Tami. "Outcomes, Rubrics, Assessment Workshop II." Assessment Workshop. Franklin College, Franklin, IN. 20 Jun. 2013. Lecture.

Eubanks, David A. “Some Uncertainties Exist." Coming to Terms With Student Outcomes Assessment: Faculty and Administrators' Journeys to Integrating Assessment in Their Work and Institutional Culture. Ed. Peggy L. Maki. Steerling, VA: Stylus, 2010. 47-58. Print.

Knight, Peter, and Mantz Yorke. Assessment, Learning and Employability. Maidenhead: Society for Research into Higher Education & Open UP, 2003. Print.

Rhodes, Terrel L., ed. Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics. Washington DC: Association of American Colleges and Universities, 2010. Print.

Shipman, Donald, Susan L. Aloi, and Elizabeth A. Jones. "Addressing Key Challenges in Higher Education Assessment." Journal of General Education 52.4 (2003): 335-46. JSTOR. Web. 13 Feb. 2015. <http://www.jstor.org/stable/27797968>.

Bibliography: