APSU Program Review Handbookapsu.edu/dsir/prresources/apsu_program_review_handbook.pdf · January 9‐11, 2019 Finalize On‐site Visit Team schedule and hotel ... Prepare and submit

PROGRAM REVIEW HANDBOOK

2015‐20 Quality Assurance Funding Reporting Cycle

Austin Peay State University

August 13, 2018

2 | P a g e APSU Program Review Handbook


Table of Contents

What is Program Review? 4

Why is Program Review important? 4

What is the schedule for the Program Review Process? 5

How do we identify potential reviewers? 7

What is included in the self‐study? 8

What additional information might the reviewers need? 8

How do the reviewers assess the program? 8

What are the responsibilities of the program reviewer(s)? 9

How is the site visit scheduled? 9

Appendices

Appendix A: Sample Itinerary of Program Review Visit 10 Appendix B: Associate Program Review Self‐Study Report Template 12

Appendix C: Baccalaureate Program Review Self‐Study Report Template 18 Appendix D: Graduate Program Review Self‐Study Report Template 24 Appendix E: THEC Program Review: Certificate and Associate Programs Rubric 30 Appendix F: THEC Program Review: Baccalaureate Programs Rubric 33 Appendix G: THEC Program Review: Graduate Programs Rubric 36


What is Program Review? Academic program review is a peer review process designed to improve the quality of the university’s academic programs. Program reviews provide a systematic method to evaluate quality, productivity, and need, both in the university and across the state and region. Historically, Austin Peay State University participated in Academic Audit, which was a similar process where reviewers were provided by the Tennessee Board of Regents. The 2018‐19 academic year marks the first year since 2006 that Austin Peay has elected to use Program Review for its evaluation process.

Each academic program, not accredited by a recognized agency which accredits programs for that field and degree level, must participate in the Program Review process. Programs must undergo an evaluation once in every five year reporting cycle as part of the Tennessee Higher Education Commission’s (THEC) Quality Assurance Funding process. During the designated year of review, an academic program will collaborate with the Office of Institutional Effectiveness and Assessment and their respective college dean to complete a self‐study report and site visit. After reading the report and participating in the site visit, reviewers of the program will compile a narrative report that includes recommendations for improvement. Programs create action plans based on these recommendations and present the outcomes of the Program Review to the Provost and senior administration involved in implementing changes. Ongoing tracking of recommendations and outcomes continues until the next scheduled review. The program review cycle provides the vital link that enacts improvements brought to light in self‐study and peer review processes.

The Office of the Institutional Effectiveness and Assessment coordinates all reviews. This office serves as a resource for you as work begins on your self‐study. Decision Support and Institutional Research can also offer historical data needed for this report.

Why is Program Review important? Program review is an important tool designed to help you identify the strengths and weaknesses of your program so improvements can be recommended and implemented. The follow‐up process after program review makes sure these improvements are applied.

Program Review and Accreditation serves as one of five Student Learning and Engagement standards of THEC Quality Assurance Funding. Each year, the university receives recommended points for each of these standards as well as a standard for Student Access and Success. The “Academic Programs: Accreditation and Evaluation” standard offers Austin Peay the opportunity to earn 25 of the 100 points awarded annually by THEC.


What is the schedule for the Program Review Process?

APSU Program Review Timetable 2018‐19 Timeframe Activity Responsibility

Programs Health & Human Performance (BS & MS), Psychological Science (BA/BS), Political Science (BA/BS), Liberal Arts (AS), English (BA/BS & MA) and Industrial‐Organizational Psychology (MSIO)

March 2018 Attend APSU Program Review Orientation VP/AVPAA , Dean, Chair, PR Team Lead*

April 2018 Lead an organizational meeting with departments to begin self‐study process

Dean

April 2018 Begin self‐study process PR Team Lead

Pre‐Semester Week August, 2018

Lead a program meeting to discuss program data PR Team Lead and Program Faculty

August 17, 2018 Submit data request to DSIR for needed data not provided on the DSIR web site**

PR Team Lead

September 17, 2018 Send reminder to PR team lead, Chairs & Deans regarding APSU self‐ study template

VP/AVPAA

October 2018 Attend APSU Academic Audit presentations from 2017‐2018 cohort VP/AVPAA Dean, Chair, PR Team Lead

November 9, 2018 Submit recommendations by each self‐study team for reviewers (2) withCVs & preferred Program Review team site visit date to Quality Assurance Coordinator

PR Team Lead

November 16, 2018 Submit draft of self‐study to Chair for review and feedback PR Team Lead

November 29, 2018 Return draft to PR Team Lead with suggestions for revision Chair

December 3‐13, 2018

Incorporate Chair feedback and revisions, reaching consensus with Chair

PR Team Lead & Chair

December 14, 2018 Submit revised draft to Chair for signature on or prior to Dec. 14 PR Team Lead

December 17, 2018 Submit revised draft with Chairs’ signature to Dean for review and feedback; for graduate programs being audited, a revised draft also submitted to the Dean of College of Graduate Studies

PR Team Lead

January 4, 2019 Return draft with suggestions for revision to the PR Team Lead; Dean of College of Graduate Studies returns draft with feedback to graduate program

Dean & Graduate Dean

January 7‐9, 2019 Incorporate Dean’s feedback and revisions, reaching consensus with Deans

PR Team Lead & Dean, Graduate Dean

January 10, 2019 Submit revised draft to Dean for signature on or prior to Jan. 10 PR Team Lead

January 11, 2019 Submit revised draft of self‐study with Chair and Dean signatures to VP/AVPAA for review and feedback

PR Team Lead

January 9‐11, 2019 Finalize On‐site Visit Team schedule and hotel reservations PR Team Lead


January 15, 2019 Return report with suggestions for revision to the PR Team Lead VP/AVPAA

January 15‐17, 2019 Incorporate VP/AVPAA feedback and revisions, reaching consensus with VP/AVPAA

PR Team Lead & VP/AVPAA

January 18, 2019 Submit final report to VP/AVPAA for signature on or prior to Jan. 18 PR Team Lead

January 18, 2019 Submit final self‐study to Provost for review with Chair, Dean, and VP/AVPAA signatures****

VP/AVPAA

January 25, 2019 Return final self‐study with Provost signature to PR Team Lead Provost

January 25, 2019 Submit final self‐study, on‐site visit schedule, and hotel reservations to Quality Assurance Coordinator for distribution to External Reviewers

PR Team Lead

January 25, 2019 Send Outlook meeting invitations to Dean, VP/AVPAA , and others who should attend the introductory session and the exit meeting

PR Team Lead

January 28, 2019 Submit to External Reviewer(s) the final self‐study, on‐site visit schedule, and hotel reservations

Quality Assurance Coordinator

February 2019 Host a Pre‐Site Visit prep meeting with deans, department chairs, and PR Team Lead

Chairs, PR Team Leads, Deans, Graduate Dean, QA Coordinator VP/AVPAA

March 11‐April 19, 2019

Site visits for all programs Chair and PR Team Lead

Within 30 calendar days after site visit

Obtain written report from reviewers Quality Assurance Coordinator

Within 1 week of receiving all written reports from reviewers

Submit signed rubrics, reviewers’ narrative reports and CVs of reviewers to THEC staff


Within 2 Weeks after receiving written report from reviewers

Discuss preliminary observations concerning the program, criteria ratings, and recommendations for improvement during exit meeting

Chair, PR Team Lead, VP/AVPAA , Dean, Graduate Dean

September 13, 2019 Prepare written responses to the reviewer’s report and ratings; the response should address the observations and recommendations in the reviewers’ report and identify appropriate actions to be taken; submit to Dean and Quality Assurance Coordinator

Chair, PR Team Lead and Program Faculty

October, 2019*** Present response to PR report, rating and recommendations (in a meeting with senior administration)

Chair, PR Team Lead

Ongoing Program improvement activities Implemented by program

*PR Team Lead: Program Review Team Lead, faculty member of department leading the program audit

**www.apsu.edu/dsir/data

*** Italicized dates are estimated dates

****PR Team Lead be available for discussion of report with Provost


How do we identify potential reviewers? Two to three external reviewers (from outside the state of Tennessee) and two or three internal consultants should be identified for this process. Please contact potential reviewers to ask if they are willing to be considered and able to serve. Once you have established those that have agreed to serve, provide a list of these reviewers to the Quality Assurance Coordinator as early in the academic year as possible, but no later than Friday, November 9. All reviewers must meet the qualifications listed below. The VP/AVPAA will select from your list the external reviewers who will conduct the review, based on their credentials and availability. The external reviewers must be professionals in the field of study under review. The reviewers chosen cannot have personal or professional affiliation with members of APSU’s faculty within the program under review and avoid any conflict of interest. (See below for further specifications.) The Vice Provost and Associate Vice President for Academic Affairs (VP/AVPAA) and THEC will have final approval of the review team.

When you provide the list of potential reviewers, please include contact information (including email address), and a copy of their vita (or links to web versions).

External Reviewers must: 1. Be professionals in the field under review; 2. Hold a terminal degree; 3. Hold an academic position, preferably at a regional public university comparable to APSU; in some cases

a practicing professional in the field or a retiree is an appropriate substitute; 4. NOT be APSU graduates; 5. NOT have active or previous professional or personal affiliations with faculty or staff in the

department to be reviewed, or with other reviewers (co‐author, classmate, professor/student, former colleague, etc.);

It is recommended that each of the following qualifications is held by at least one reviewer: 6. Department Chair or coordinator experience; 7. Training/experience as a program reviewer.

Internal Consultants Two to three internal consultants are included on every review team. One should be from within the same college as the program under review; one should be from outside that college. We give preference to consultants from programs that will be reviewed in coming years, in order to mentor those future reviews and provide a fresh perspective. The internal consultants can provide important campus‐related information to external reviewers, but they are still key members of the reviewing team, providing insight from within the university but outside of the program.

Internal consultants must: 1. Be faculty members outside program being reviewed; 2. Not be co‐author or co‐creator with faculty within the program being reviewed. 3. Be APSU Graduate Faculty (if graduate program is reviewed);

Tips for identifying potential reviewers:

Ask appropriate professional associations for help in identifying potential reviewers. Many disciplinary organizations provide training for program reviewers and can provide names of experienced/trained individuals.

Ask department faculty for suggestions.


Contact comparable programs at other regional universities to learn who successfully reviewed their programs.

The Office of Institutional Effectiveness and Assessment pays reviewers and will reimburse program reviewers for travel costs and provide the per diem rate for meals and incidentals. The Office of Institutional Effectiveness and Assessment does not provide a stipend or other compensation for the internal consultants.

What is included in the self‐study? The narrative of the self‐study should be constructed after an open and frank discussion by program faculty and staff members as they prepare for the review. Individual faculty and/or committees can be appointed to write the self‐study, but the program chair is responsible for the final product.

Use the self‐study report templates for associate, baccalaureate and graduate programs (Appendices B, C, and D) to organize the self‐study. While completing the self‐study, use the following points as a guide:

Use a five year timeframe when referencing program history and accomplishments.

Avoid naming particular faculty members when citing examples.

Contact Information Technology, Distance Education, Library, Finance and Administration, Decision Support and Institutional Research and Enrollment Management offices for resource assistance.

When preparing the listing of program faculty, include faculty who may only teach in your program once every year or two. Specify what courses faculty teach and how often.

What additional information might the reviewers need?

If any of the data requested in the outline above is too cumbersome or lengthy to include in full in an appendix,

then summarize and include either a) a website where the exhibits are posted; or, b) a note that the exhibit will

be available at the site visit. Physical exhibits should be gathered in a convenient location in case they are

requested by the review team at the time of the site visit.

Among those materials that could be available:

Written exams, reports, projects, etc. used for Institutional Effectiveness over the past five years;

Previous Academic Audit or program review narrative report and summary document;

Syllabi for all courses in the program;

Journal articles from students or student/faculty collaborations;

Research presentations from students.

How do the reviewers assess the program? The review team reads the self‐study and related materials before the visit and notes questions and concerns to

be addressed during the review. During the site visit, they observe, question, and assess the program in light of

the self‐study. They may also examine additional information that you will prepare for their perusal.


What are the responsibilities of the program reviewer(s)?

Read the program’s self‐study and the applicable THEC program review rubric before arriving on campus.

Conduct interviews of faculty, administrators, students, and employers/internship supervisors.

Before adjourning, the program reviewers complete the THEC checklist for the designated program level

(see Appendices E, F, and G). (When writing the self‐study, address every checklist point included in the

rubric. Using this outline will make it easier for reviewers and ensure compliance with performance

funding guidelines.)

Provide a verbal report to the faculty, staff and administrators before leaving campus.

Prepare and submit a narrative report to the Director of Institutional Effectiveness and Assessment

within 30 calendar days after the visit.

How is the site visit scheduled?

After reviewers are selected and approved, the Quality Assurance Coordinator will correspond with them to

confirm their participation and send them all of the necessary materials regarding the self‐study, including

reviewer guidelines, university bulletins, and self‐study documents. Once the reviewers arrive in Clarksville,

the academic department will act as their host and therefore be responsible for transportation and

information.

The faculty and staff of the program under review are responsible for:

Scheduling rooms for departmental sessions (see Appendix A for details) and collaborating with

Quality Assurance Coordinator to create complete schedule;

Schedule participation of departmental faculty, students, and stakeholders;

Distributing schedule to departmental participants (Quality Assurance Coordinator will distribute to

administrators and reviewers);

Arrange meal events and refreshments;

Provide local transportation for review team.


APPENDIX A Itinerary of Program Review

Visit Schedule

Academic Department Name

Degree and Title of Program

Date of Visit

External Program Reviewer(s)

Name, Title, Institution

Name, Title, Institution

Internal Program Reviewers

Name, Title, Department

Name, Title, Department

Day 1 Date (Academic Department schedules travel to and from hotel.)

7:30 a.m. Pick up from hotel

Program Affiliate

8:00 a.m. THEC Quality Assurance Funding Program Review Orientation and Breakfast

meeting (Institutional Effectiveness and Assessment schedules location and attendees.) Review Team (External and Internal)

Dean of the College of Graduate Studies

Vice Provost and Associate Vice President for Academic Affairs Director of Institutional Effectiveness and Assessment Quality Assurance Coordinator

9:00 a.m. Tour and Overview of Department (Academic Department schedules location.) Review Team (External and Internal)

Chair of Department

9:45 ‐ 10:00 a.m. Break

10:00 a.m. Faculty Interviews (Academic Department schedules location and attendees.) Review Team (External and Internal) Faculty

11:00 – 11:50 a.m. Interview with Dean (Academic Department schedules location.)

Review Team (External and Internal) Dean of College (and Dean of the College of Graduate Studies, if applicable)


12:00 p.m. Luncheon Meeting (Academic Department schedules location, attendees, and

transportation.)

Review Team (External and Internal)

Chair of Department

Available Faculty

Employers and Internship Supervisors

1:15 p.m. Interview with Students (Academic Department schedules location.) Review Team (External and Internal)

Students (majors only)

2:00 – 3:00 p.m. Administration (Institutional Effectiveness and Assessment schedules location and attendees.) Review Team (External and Internal)

Provost and Vice President for Academic Affairs Vice Provost and Associate Vice President for Academic Affairs Executive Director for Decision Support and Institutional Research

Director of Institutional Effectiveness and Assessment

Director of Library

3:00 p.m. Review Team Work Session (Academic Department schedules location.) Reviewers begin outline of summary report.

External Reviewers complete forms: THEC Appendix E: Program Review: Certificate and Associate Programs THEC Appendix F: Program Review: Baccalaureate Programs THEC Appendix G: Program Review: Graduate Programs

4:00 p.m. Concluding Session (Institutional Effectiveness and Assessment schedules location and attendees.)

(Evaluation forms are collected from program reviewers.)

Review Team (External and Internal)

Chair of Department

Dean of College

Available Department Faculty

Provost and Vice President of Academic Affairs

Vice Provost and Associate Vice President for Academic Affairs

Dean of College of Graduate Studies (Graduate programs only)

Director of Institutional Effectiveness and Assessment


(Addresses and Phone Numbers for Reference)

Chair of Department

Office of Institutional Effectiveness and Assessment


Appendix B

Associate Program Review Self-Study Report

Department of Department Name


Self-Study Report for Degree in Program Name

Program Review Audit Year

Program Review Team Lead: Name

Focal Area 1: Name(s) Focal Area 2: Name(s) Focal Area 3: Name(s) Focal Area 4: Name(s) Focal Area 5: Name(s) Focal Area 6: Name(s) Focal Area 7: Name(s)


I. INTRODUCTION

Department or Program Mission:

Program History and Structure:

Introduce the program. Describe program structure such as college and department program is housed, if program offered partly or entirely online, other special characteristics of program. Include a brief history if applicable to understanding of program’s current status.

Faculty:

Full Time Part Time Full Professor

Associate Professor

Assistant Professor

Instructor

Adjunct

Figure 1. Head count of program name current faculty.

Other faculty and staff assigned administrative duties, etc.

Student Demographics:

Figure 2. Head count in program name fall 2018.

Describe appropriate unique characteristics of program students. Program Review Process

Describe how the program conducted its self-study process – who was engaged (faculty, adjunct faculty, stakeholders, students); how they were engaged (meetings, online methods, focus groups, survey, etc.); and how the self-study report was drafted, reviewed, and finalized.

Minority Non-Minority Total Freshmen

Sophomore

Junior

Senior

Total


II. OVERALL PERFORMANCE

This section is essentially the Executive Summary or Abstract of the self-study report. Begin this section with a brief assessment of the unit’s education quality assurance processes and how you work together as a faculty and with stakeholders to improve quality. The Program Review Team will ask about the logic and evidence behind the assessment, but it will not collect additional evidence nor substitute its judgment about education quality. The objective is to provide an accurate state of the program in terms of curriculum, student experience and faculty. It is not expected that the program flawlessly delivers exemplary quality education. For example, candid descriptions of areas that will benefit from attention and improvement, supported by evidence, will be received better than unsupported claims of excellence. A summary statement of how the Program Review self-study processes benefited the program should be included in this section.

III. FOCAL AREAS

Focal Area 1: Learning Outcomes

Program learning outcomes: 1. 2. 3. 4. 5. 6. 7. 8.

Student learning outcomes of core courses:

Course #1 1. 2. 3. 4.

Course #2

1. 2. 3.

Etc.

Describe the process for evaluating program and student learning outcomes. This process should be taking place on a regular basis taking into account best practices, stakeholder feedback, and appropriate benchmarks in the field.

Reviewers will be looking to identify the following information:

Program and student learning outcomes are clearly identified and measurable.

Program uses appropriate indicators to evaluate achievement of program and student learning outcomes.

Program makes use of information from its evaluation of program and student learning outcomes and uses the results for continuous improvement.

Program directly aligns with the institution’s mission.


Focal Area 2: Curriculum

Describe the process of how the faculty regularly and effectively reviews the design of, and identifies and makes improvements to the curriculum content and organization. How often does this occur, and who is involved in this process? Give examples.

Describe the process of ensuring courses are offered regularly and that students can make timely progress towards their degree. Give examples.

Describe the process of how the faculty incorporates appropriate pedagogical and/or technological innovations into the curriculum that enhance student learning. Give examples.

Describe how the curriculum is aligned with and contributes to the mastery of program and student learning outcomes identified in Focal Area 1. Give examples.

Describe how the curricular content of the program reflects current standards, practices, and issues in the disciplines. Give examples.

Describes how the curriculum fosters analytical and critical thinking and problem-solving. Give examples.

Describe how the design of the program’s specific courses provides students with a solid educational foundation. Give examples.

Describe and explain how the curriculum is appropriate to the level and purpose of the program. Give examples.

Focal Area 3: Student Experience

Describe how the program provides students with opportunities to apply what they have learned to situations outside the classroom. Give examples.

Describe how the program provides students with the opportunity to regularly evaluate faculty relative to the quality of their teaching effectiveness.

Describe how the program ensures students are exposed to professional and career opportunities appropriate to the field of study.

Describe how students in the program have access to appropriate academic support services.

Focal Area 4: Faculty (Full-time and Part-time)

Describe how all faculty (full-time and part-time) meet high standards set by the program and expected SACSCOC guidelines for credentials.

Describe how the number of faculty in the program are able to meet the needs of the program with appropriate teaching loads.

Describe how faculty strives to cultivate diversity with respect to gender, ethnicity, and academic background, as appropriate to the demographics of the discipline.

Describe how the program uses an appropriate process to incorporate the faculty evaluation system to improve teaching, scholarly and creative activities, and service. Give examples.

Describe how faculty engages in regular professional development that enhances their teaching, scholarship and practice.


Give examples.

Describe how faculty actively engage in planning, evaluation and improvement processes that measure and advance student success. Give examples.

Focal Area 5: Learning Resources

Discuss how library, equipment, and facilities are regularly evaluated, encouraging necessary improvements within the context of overall institutional resources.

Discuss how the program has access to learning and information resources that are appropriate to support teaching and learning. Give examples.

Focal Area 6: Economic Development

For transfer programs:

Describe how the program provides and promotes clear transfer pathways supported by curricular maps, advising, and other means to support student articulation. Gives examples.

Describe the success of graduates who pursue baccalaureate degrees in related programs. Give statistics.

For career programs:

Describe how the program demonstrates responsiveness to local and regional workforce needs through an advisory committee, partnerships with industry and/or other means. Give examples.

Describe how the program identifies applicable workforce trends and uses the information to improve the program. Give examples.

Focal Area 7: Support

Demonstrate how the program’s operating budget is consistent with the needs of the program.

Provide historical program enrollment and degree awards to demonstrate they are sufficient to sustain high quality and cost- effectiveness.

Figure 3. Head count of program name fall enrollments between 2011 and 2016.

Enrollment

2016 2015 2014 2013 2011

250

200

150

100

50

0

English BA/BS Fall Census Enrollment


English BA/BS Degree Awards

60

50

40

30

20

10

0

2011‐12 2012‐13 2013‐14 2014‐15 2015‐16

Degree Awards

Figure 4. Number of degrees in program name awarded per year between 2011 and 2016.

IV.

APPENDICES

Appendix 1:

Appendix 2:

Appendix 3:

Appendix 4:

Appendix 5:

A Table of Contents for the Appendices is needed.


Appendix C

Baccalaureate Program Review Self-Study Report






Focal Area 1: Name(s) Focal Area 2: Name(s) Focal Area 3: Name(s) Focal Area 4: Name(s) Focal Area 5: Name(s) Focal Area 6: Name(s)


I. INTRODUCTION




Faculty:


Associate Professor

Assistant Professor

Instructor

Adjunct








This section is essentially the Executive Summary or Abstract of the self-study report. Begin this section with a brief assessment of the unit’s education quality assurance processes and how you work together as a faculty and with stakeholders to improve quality. The Program Review Team will ask about the logic and evidence behind the assessment, but it will not collect additional evidence nor substitute its judgment about education quality. The objective is to provide an accurate state of the program in terms of curriculum, student experience and faculty. It is not expected that the program flawlessly delivers exemplary quality education. For example, candid descriptions of areas that will benefit from attention and improvement, supported by evidence, will be received better than unsupported claims of excellence. A summary statement of how the Program Review self-study processes benefited the program should be included in this section.

Minority Non-Minority Total Freshmen

Sophomore

Junior

Senior

Total


III. FOCAL AREAS




Course #1 1. 2. 3. 4.

Course #2

1. 2. 3.

Etc.

Describe the process for evaluating program and student learning outcomes. This process should be taking place on a regular basis taking into account best practices, stakeholder feedback, and appropriate benchmarks in the field.

Reviewers will be looking to identify the following information:


Program uses appropriate indicators to evaluate achievement of program and student learning outcomes.




Describe the process of how the faculty regularly and effectively reviews the design of, and identifies and makes improvements to the curriculum. How often does this occur, and who is involved in this process? Give examples.

Describe how the program has developed a process to ensure courses are offered regularly and that students can make timely progress towards their degree. Give examples.

Describe how the program incorporates appropriate pedagogical and/or technological innovations that enhance student learning in the curriculum. Give examples.

Describe how the curriculum is aligned with and contributes to the mastery of program and student learning outcomes identified in Focal Area 1. Give examples.

Describe how the curricular content of the program reflects current standards, practices, and issues in the discipline. Give examples.


Describes how the curriculum fosters analytical and critical thinking and problem-solving. Give examples.

Describe how the design of the program’s specific courses provides students with a solid educational foundation. Give examples.

Describe how the curriculum reflects a progressive challenge to students and that depth and rigor effectively prepares students for careers or advanced study.

Describe how the curriculum encourages the development of and the presentation of results and ideas effectively and clearly in both written and oral discourse. Give examples. Describe how the curriculum exposes students to discipline-specific research strategies from the program area.


Describe how the program provides students with opportunities to regularly evaluate the curriculum and faculty relative to the quality of their teaching effectiveness.

Describe how the program ensures students are exposed to professional and career opportunities appropriate to the field of study.

Describe how the program provides students with the opportunity to apply what they have learned to situations outside the classroom.

Describe how the program seeks to include diverse perspectives and experiences through curricular and extracurricular activities.

Describe how students in the program have access to appropriate academic support services.

Focal Area 4: Faculty (Full-time and Part-time)

Describe how all faculty (full-time and part-time) meet high standards set by the program and expected SACSCOC guidelines for credentials.

Describe how the number of faculty in the program are able to meet the needs of the program with appropriate teaching loads.


Describe how the program uses an appropriate process to incorporate the faculty evaluation system to improve teaching, scholarly and creative activities, and service. Give examples.

Describe how faculty engages in regular professional development that enhances their teaching, scholarship and practice.

Describe how faculty actively engage in planning, evaluation and improvement processes that measure and advance student success. Give examples.


Discuss how library, equipment, and facilities are regularly evaluated, encouraging necessary improvements within the context of overall institutional resources.

Discuss how the program has access to learning and information resources that are appropriate to support teaching and learning. Give examples.







Describe how the program is responsive to local, state, regional and national needs. Give examples.


250

200

150

100

50

0

2011 2013 2014 2015 2016

Enrollment


60

50

40

30

20

10

0

2011‐12 2012‐13 2013‐14 2014‐15 2015‐16

Degree Awards


IV.

APPENDICES

Appendix 1:

Appendix 2:

Appendix 3:

Appendix 4:

Appendix 5:



Appendix D

Graduate Program Review Self-Study Report






Focal Area 1: Name(s) Focal Area 2: Name(s) Focal Area 3: Name(s) Focal Area 4: Name(s) Focal Area 5: Name(s) Focal Area 6: Name(s)


I. INTRODUCTION




Faculty:


Associate Professor

Assistant Professor

Instructor

Adjunct








This section is essentially the Executive Summary or Abstract of the self-study report. Begin this section with a brief assessment of the unit’s education quality assurance processes and how you work together as a faculty and with stakeholders to improve quality. The Program Review Team will ask about the logic and evidence behind the assessment, but it will not collect additional evidence nor substitute its judgment about education quality. The objective is to provide an accurate state of the program in terms of teaching and student learning. It is not expected that the program flawlessly delivers exemplary quality education. For example, candid descriptions of areas that will benefit from attention and improvement, supported by evidence, will be received better than unsupported claims of excellence. A summary statement of how the program review audit self-study processes benefited the program should be included in this section.

Minority Non-Minority Total First Year

Second Year

More than 2 Years

Total


III. FOCAL AREAS




Course #1

1. 2. 3. 4.

Course #2

1. 2. 3.

Etc.

Describe the process for evaluating program and course-level learning outcomes. This process should be taking place on a regular basis taking into account best practices, stakeholder feedback, and appropriate benchmarks in the field. Reviewers will be looking for the following information:


Program uses appropriate evidence to evaluate achievement of program and student learning outcomes.




Describe the process of how the faculty regularly and effectively reviews the design of, and identifies and makes improvements to the curriculum. How often does this occur, and who is involved in this process? Give examples.

Describe the process developed to ensure courses are offered regularly and that students can make timely progress toward their degree. Give examples.

Describe how the program reflects progressively more advanced in academic content than its related undergraduate program(s). Give examples.

Describe how the curriculum is aligned with and contributes to mastery of program and student learning outcomes identified in Focal Area 1. Give examples.

Describe how the curriculum is structured to include knowledge of the literature of the discipline. Give examples.


Describe how the curriculum strives to offer ongoing student engagement in research and/or appropriate professional practice and training experiences. Give examples.

If the program is offered entirely online, describe how the program is evaluated regularly to assure achievement of program outcomes are equivalent to on-campus programs. Give examples if applicable.

Describe how the program incorporates appropriate pedagogical and/or technological innovations into the curriculum that advance student learning. Give examples.


Describe how the program ensures a critical mass of students to ensure an appropriate group of their peers participating in course work. Give examples.

Describe and explain how the program provides students with the opportunities to regularly evaluate the curriculum and faculty relative to the quality of their teaching effectiveness.

Describe how the program provides adequate professional development opportunities, such as encouraging membership in professional associations, participation in conferences and workshops, and opportunities for publication. Give examples.

Describe how the program provides adequate enrichment opportunities, such as lecture series, to promote a scholarly environment. Give examples.

Describe how the program seeks to include diverse perspectives and experiences through curricular and extracurricular activities. Give examples.

Describe and explain how students have access to appropriate academic support services.

Focal Area 4: Faculty

Describe how all faculty, full-time and part-time, meet the high standards set by the program and expected SACSCOC guidelines for credentials.

Describe and explain how faculty teaching loads are aligned with the highly individualized nature of graduate instruction, especially the direction of theses or dissertations.


Explain how faculty engages in regular professional development that enhances their teaching, scholarship and practice.

Describe how faculty is actively engaged in planning, evaluation and improvement processes that measure and advance student success.

Demonstrate how the faculty uses an appropriate process to incorporate the faculty evaluation system to improve teaching, scholarly and creative activities, and service. This could include analysis of course evaluations, peer observations, SLO assessments, etc. How often does the faculty do this?


Describe how the program regularly evaluates its equipment and facilities, encouraging necessary improvements within the context of overall institutional resources.

Explain how the program has access to learning and information resources that are appropriate to support teaching and learning. Give examples.

Explain how the program provides adequate materials and support staff to encourage research and publication. Give examples.







Enrollment

2016 2015 2014 2013 2011

250

200

150

100

50

0


Degree Awards

2015‐16 2014‐15 2013‐14 2012‐13 2011‐12

60

50

40

30

20

10

0



Describe and explain how the program is responsive to local, state, regional, and national needs.

Describe how the program regularly and systemically collects data on graduating students and evaluates placement of graduates. Give examples.

Describe how the program’s procedures are regularly reviewed to ensure alignment to institutional policies and mission. Give examples.

IV.

APPENDICES

Appendix 1:

Appendix 2:

Appendix 3:

Appendix 4:

Appendix 5:



Appendix E 2015-20 Quality Assurance Funding Program Review: Certificate and Associate Programs

Institution: Program Title: CIP Code: Embedded Certificates:

Embedded Certificates:

Embedded Certificates:

Instruction for External Reviewer(s)

In accordance with the 2015-20 Quality Assurance Program Funding guidelines of the Tennessee Higher Education Commission (THEC), each non-accreditable certificate and associate program undergoes either an academic audit or external peer review according to a pre-approved review cycle. If the program under review contains embedded Technical Certificates, the names of each certificate should be included above. The review of embedded certificates must be included as part of the review of the program in which they are embedded. Embedded certificates do not require a separate Program Review Rubric.

The criteria used to evaluate a program appear in the following Program Review Rubric. The Program Review Rubric lists 30 criteria grouped into seven categories. THEC will use these criteria to assess standards and distribute points to certificate and associate programs. The five criteria noted with an asterisk are excluded from the point calculation but will be used by the institution in their overall assessment.

For each criterion within a standard, the responsible program has provided evidence in the form of a Self-Study. Supporting documents will be available for review as specified in the Self-Study. As the external reviewer, you should evaluate this evidence and any other evidence observed during the site visit to determine whether each criterion within a standard has been met. A checkmark should be placed in the appropriate box to indicate whether the program currently exhibits poor, fair, good or excellent in meeting the criterion. If a particular criterion is inappropriate or not applicable to the program under review, the item should be marked NA.

This evaluation becomes a part of the record of the academic program review. The rubric will be shared with the department, college and central administration, as well as the Tennessee Higher Education Commission. When combined with the written report, prepared by the entire program review committee, the Program Review Rubric will facilitate development of a program action plan to ensure continuous quality improvement.

Your judgment of the criteria will be used in allocating state funds for the community college's budget.

Name Name

Title Title

Institution Institution

Signature Signature

Date Date

Name, Title and Institutional Affiliation of Reviewer(s)


Program Review Rubric Certificate and Associate Programs

Directions: Please rate the quality of the academic program by placing a checkmark in the appropriate box to indicate whether the program currently exhibits poor, fair, good or excellent evidence of meeting the criterion.

1. Learning Outcomes N/A Poor Fair Good Excellent

1.1 Program and student learning outcomes are clearly identified and measurable.

1.2 The program uses appropriate indicators to evaluate achievement of program and student learning outcomes.

1.3 The program makes uses of information from its evaluation of program and student learning outcomes and uses the results for continuous improvement.

1.4 The program directly aligns with the institution's mission.

2. Curriculum N/A Poor Fair Good Excellent

2.1 The curriculum content and organization are reviewed regularly and the results are used for curriculum improvement.

2.2 The program has developed a process to ensure courses are offered regularly and that students can make timely progress towards their degree.

2.3 The program incorporates appropriate pedagogical and/or technological innovations that enhance student learning into the curriculum.

2.4 The curriculum is aligned with and contributes to mastery of program and student learning outcomes identified in 1.1.

2.5 The curricular content of the program reflects current standards, practices, and issues in the discipline.

2.6 The curriculum fosters analytical and critical thinking and problem-solving.

2.7 The design of degree program specific courses provides students with a solid foundation.

2.8 The curriculum is appropriate to the level and purpose of the program.

3. Student Experience N/A Poor Fair Good Excellent

3.1 The program provides students with the opportunity to apply what they have learned to situations outside the classroom.

3.2 The program provides students with the opportunity to regularly evaluate faculty relative to the quality of their teaching effectiveness.

3.3 The program ensures students are exposed to professional and career opportunities appropriate to the field.

3.4 Students have access to appropriate academic support services.

4. Faculty (Full-time and Part-time) N/A Poor Fair Good Excellent


4.1 All faculty, full time and part-time, meet the high standards set by the program and expected SACSCOC guidelines for credentials.

4.2 The faculty are adequate in number to meet the needs of the program with appropriate teaching loads.

4.3* The faculty strives to cultivate diversity with respect to

gender, ethnicity, and academic background, as appropriate to the demographics of the discipline.

4.3 The program uses an appropriate process to incorporate the faculty evaluation system to improve teaching, scholarly and creative activities, and service.

4.4 The faculty engage in regular professional development that enhances their teaching, scholarship, and practice.

4.5 The faculty are actively engaged in planning, evaluation and improvement processes that measure and advance student success.

5. Learning Resources N/A Poor Fair Good Excellent

5.1* The program regularly evaluates its equipment and facilities, encouraging necessary improvements within the context of overall institutional resources.

5.2 The program has access to learning and information resources that are appropriate to support teaching and learning.

6. Economic Development N/A Poor Fair Good Excellent

6.1 For transfer programs: The program provides and promotes clear transfer pathways supported by curricular maps, advising and other means to support student articulation.

6.2* For transfer programs: Graduates who transfer to baccalaureate programs in a related area are successful.

6.3 For career programs: The program demonstrates responsiveness to local and regional workforce needs through an advisory committee, partnerships with industry and/or other means.

6.4 For career programs: The program identifies applicable workforce trends and uses the information to improve the program.

7. Support N/A Poor Fair Good Excellent

7.1* The program's operating budget is consistent with the needs of the program.

7.2* The program has a history of enrollment and/or graduation rates sufficient to sustain high quality and cost-effectiveness.

*Criteria not scored as part of Quality Assurance Funding.


Appendix F 2015-20 Quality Assurance Funding Program Review: Baccalaureate Programs

Institution: Program Title: CIP Code:


In accordance with the 2015-20 Quality Assurance Program Funding guidelines of the Tennessee Higher Education Commission (THEC), each non-accreditable baccalaureate program undergoes either an academic audit or external peer review according to a pre-approved review cycle.

The criteria used to evaluate a program appear in the following Program Review Rubric. The Program Review Rubric lists 30 criteria grouped into six categories. THEC will use these criteria to assess standards and distribute points in to baccalaureate programs. The four criteria noted with an asterisk are excluded from the point calculation but will be used by the institution in their overall assessment.



Your judgment of the criteria will be used in allocating state funds for the university's budget.

Name Name

Title Title


Signature Signature

Date Date



Program Review Rubric Baccalaureate Programs




1.2 The program uses appropriate evidence to evaluate achievement of program and student learning outcomes.

1.3 The program makes use of information from its evaluation of program and student learning outcomes and uses the results for continuous improvement.



2.1 The curriculum content and organization are reviewed regularly and results are used for curricular improvement.


2.3 The program incorporates appropriate pedagogical and/or technological innovations that enhance student learning into the curriculum.


2.5 The curricular content of the program reflects current standards, practices, and issues in the discipline.

2.6 The curriculum fosters analytical and critical thinking and problem-solving.

2.7 The design of degree program specific courses provides students with a solid foundation.

2.8 The curriculum reflects a progressive challenge to students and that depth and rigor effectively prepares students for careers or advanced study.

2.9 The curriculum encourages the development of and the presentation of results and ideas effectively and clearly in both written and oral discourse.

2.10 The curriculum exposes students to discipline-specific research strategies from the program area.


3.1 The program provides students with opportunities to regularly evaluate the curriculum and faculty relative to the quality of their teaching effectiveness.

3.2 The program ensures students are exposed to professional and career opportunities appropriate to the field.

3.3 The program provides students with the opportunity to apply what they have learned to situations outside the classroom.

3.4 The program seeks to include diverse perspectives and experiences through curricular and extracurricular activities.



4. Faculty (Full-time and Part-time) N/A Poor Fair Good Excellent


4.2 The faculty are adequate in number to meet the needs of the program with appropriate teaching loads.

4.3* The faculty strives to cultivate diversity with respect to gender, ethnicity, and academic background, as appropriate to the demographics of the discipline.


4.5 The faculty engages in regular professional development that enhances their teaching, scholarship and practice.

4.6 The faculty is actively engaged in planning, evaluation and improvement processes that measure and advance student success.







6.3 The program is responsive to local, state, regional, and national needs.



Appendix G 2015-20 Quality Assurance Funding Program Review: Graduate Programs

Institution: Program Title: CIP Code: Degree Designation:


In accordance with the 2015-20 Quality Assurance Program Funding guidelines of the Tennessee Higher Education Commission (THEC), each non-accreditable graduate program undergoes either an academic audit or external peer review according to a pre-approved review cycle.

The criteria used to evaluate a program appear in the following Program Review Rubric. The Program Review Rubric lists 32 criteria grouped into six categories. THEC will use these criteria to assess standards and distribute points in to graduate programs. The four criteria noted with an asterisk are excluded from the point calculation but will be used by the institution in their overall assessment.



Your judgment of the criteria will be used in allocating state funds for the university's budget.

Name Name

Title __________________________________ Title


Signature Signature

Date Date



Program Review Rubric Graduate Programs




1.2 The program uses appropriate evidence to evaluate achievement of program and student learning outcomes.

1.3 The program makes use of information from its evaluation of program and student learning outcomes and uses the results for continuous improvement.



2.1 The curriculum content and organization is reviewed regularly and the results are used for curricular improvement.


2.3 The program reflects progressively more advanced in academic content than its related undergraduate programs.


2.5 The curriculum is structured to include knowledge of the literature of the discipline.

2.6 The curriculum strives to offer ongoing student engagement in research and/or appropriate professional practice and training experiences.

2.7 Programs offered entirely through distance education technologies are evaluated regularly to assure achievement of program outcomes at least equivalent to on-campus programs.

2.8 The program incorporates appropriate pedagogical and/or technological innovations that advance student learning into the curriculum.


3.1 The program ensures a critical mass of students to ensure an appropriate group of peers.

3.2 The program provides students with the opportunities to regularly evaluate the curriculum and faculty relative to the quality of their teaching effectiveness.

3.3 The program provides adequate professional development opportunities, such as encouraging membership in professional associations, participation in conferences and workshops, and opportunities for publication.

3.4 The program provides adequate enrichment opportunities, such as lecture series, to promote a scholarly environment.


3.5 The program seeks to include diverse perspectives and experiences through curricular and extracurricular activities.


4. Faculty N/A Poor Fair Good Excellent


4.2 The faculty teaching loads are aligned with the highly individualized nature of graduate instruction, especially the direction of theses or dissertations.

4.3* The faculty strives to cultivate diversity with respect to gender, ethnicity, and academic background, as appropriate to the demographics of the discipline.

4.4 The faculty engages in regular professional development that enhances their teaching, scholarship and practice.

4.5 The faculty is actively engaged in planning, evaluation and improvement processes that measure and advance student success.





5.3 The program provides adequate materials and support staff to encourage research and publication.




6.3 The program is responsive to local, state, regional, and national needs.

6.4 The program regularly and systematically collects data on graduating students and evaluates placement of graduates.

6.5 The program's procedures are regularly reviewed to ensure alignment to institutional policies and mission.


Documents

APSU Program Review Handbookapsu.edu/dsir/prresources/apsu_program_review_handbook.pdf · January 9‐11, 2019 Finalize On‐site Visit Team schedule and hotel ... Prepare and submit