Benchmark Service Area Review Institutional Research · • Conducting and supporting institutional research analysis; • Complying with state, federal, and agencies’ requests

Benchmark Service Area Review

Institutional Research FY 2005-06

Completed by: Tom Hughes Director of Institutional Research

Diane Mazmanian Senior Research Analyst

Sandra Thurman-Jackson Research Analyst

N:\IR Service Area Review\2005-06\IR Service Area Review - 2005-06 Final.doc i

TABLE OF CONTENTS

PAGE

A. Introduction

1

B. Mission Mission

1

Historical Sketch

1

C. Service Area Personal

2

D. Service Area Outcomes

3

Product Services and Core Activities

3

Ad Hoc Services

3

E. Strengths, Weaknesses, Opportunities, Threats

4

F. Budget and Staffing

6

G. Benchmarks

6

H. Recommendations

8

I. Action Plan

9

N:\IR Service Area Review\2005-06\IR Service Area Review - 2005-06 Final.doc 1

A. INTRODUCTION Benchmarking is a process of identifying and sharing best practices. Through this process Yavapai College will use the information gathered to better serve its students and community. Using enrollment, full-time faculty counts, and multiple campus status as selection criteria the following benchmark colleges were selected by the Service Area Review Committee: Arizona Western College, Central Arizona College, Cochise College and Mohave Community College. Selection was carefully determined to best match Yavapai College characteristics. B. MISSION Mission Statement To develop an institutional assessment system in support of academic student learning, administrative decision making and college planning. The office accomplishes this by:

• Conducting and supporting institutional research analysis;

• Complying with state, federal, and agencies’ requests for information and required reports.

Historical Sketch The Office of Institutional Research (IR) was created in 1995 as a result of the 1994 self-study process and North Central Association (NCA) visiting team recommendation. From 1995 to 1999, IR support of college analysis needs was at a minimum due to small staffing and a lack of automated analysis and reporting. In 2000, the college determined there was a need for increased analysis driven solutions and the department was granted one senior level analyst position and one administrative position to join the current director and senior research analyst positions. In 2003, due to budgetary reasons, one senior level analyst position was phased out and the administrative position was reclassified as a research analyst. Beginning in 2000, IR added a part-time programmer position to increase efficiency and accuracy of data reporting. Today the IR office processes close to 300 ad hoc data requests annually, conducts 12 survey/study projects annually, and has created automated programs to respond to 90% of data and reporting requests. As of 2002, IR was charged with leading the college’s strategic planning and Higher Learning Commission accreditation processes and related activities.


C. SERVICE AREA PERSONNEL Director of Institutional Research Director of Institutional Research develops and implements a comprehensive system to assess effectiveness of district programs and services. Includes enrollment research, survey development, core institutional studies, data reports, quantitative and qualitative methods, complex research designs and making presentations and recommendations based on findings for institutional improvement; work with district staff, faculty, and outside entities such as state and federal agencies. Additional duties include serving as chair of the Strategic Planning Committee and self-study coordinator for Higher Learning Commission accreditation, and service on other district committees. Senior Research Analyst Senior Research Analyst uses qualitative and quantitative methods to provide analysis and expertise in the areas of enrollment research, survey development, core institutional studies, and data and demographic reports. Presents data and interpretation to district staff and faculty to support analysis driven solutions for institutional improvement. Additional duties include serving on the Strategic Planning Committee to assist academic planning and budgeting, and service on other district committees. Research Analyst Research Analyst analyzes data and provides critical support to core and ad hoc research activities (i.e., surveys, studies, enrollment reports, program reviews and audits, etc.), resulting in institutional assessment information for the purpose of supporting academic and administrative decision making; supervises the Administration Assistant position. Additional duties include serving on the Strategic Planning Committee to assist academic planning and budgeting. Information Systems Analyst (Part-Time) Information Systems Analyst develops computer programs to assist IR in automating the analysis of data and survey projects. This position also creates and maintains all on-line functions such as, on-line surveys, the IR website, and the college’s Strategic Planning website. Administrative Assistant (Part-Time) Administrative Assistant provides administrative assistance by conducting survey mailings (course/instructor surveys for each course YC offers, graduate follow-up, employers of graduates, ad hoc surveys, etc.), processing/scanning returned surveys, entering narrative data from surveys, and assisting with special projects.


D. SERVICE AREA OUTCOMES Products and Services Core Activities The IR office serves the campus as the primary source of student, course and faculty data. In this capacity, IR supports such primary campus activities as budgeting, enrollment management, assessment, program review, retention efforts, quality improvement, student satisfaction survey administration, and strategic planning. In its capacity as the enrollment reporting office for the college, IR provides mandated data to the State of Arizona, the U.S. Department of Education, and the Higher Learning Commission. Ad hoc Services In addition to the products and services above, the IR office handles a large number of ad hoc data requests from various departments and divisions district-wide. In 2004, the first year IR formally tracked ad hoc requests,166 projects were completed. In 2005, this number increased 66% to 276 projects and is expected to top 300 in 2006. Below is a sample of recent ad hoc data requests. A full list of ad hoc projects is available upon request:

• Develop a facilities usage report for the Office of Academic Affairs

• Outdoor recreation course survey – analysis and report for Health, Physical Education and Recreation division

• Outcomes assessment portfolio scores data entry and report for English

• Database matching project for the Learning Center

• Demographic reports for Verde Campus and surrounding communities

• Develop on-line technology questionnaire for Dick Meyer consulting

• Develop program for nursing division to comply with mandated tracking of

nursing students

• Analysis of students with veteran status for Financial Aid

• Annualized FTSE reports by course and location including trend information Section G. of this report presents a benchmark analysis, and the appendix contains specific counts of activities and services conducted by IR.


E. STRENGTHS, WEAKNESSES, OPPORTUNITES, THREATS ANALYSIS IR’s Strength, Weaknesses, Opportunities, Threats (SWOT) analysis revealed that the office’s strengths were its highly qualified staff and the staff’s ability to provide timely and accurate scheduled and ad hoc data reports. Additionally, IR has automated much of its data reporting using internal resources with reduced impact to other college resources. These strengths tie directly to the college’s mission in providing high quality, convenient and cost-effective support to student learning. The college’s strategic initiatives, especially Excellence in Education and Long-Range Facilities Planning, are supported by IR’s strengths. The office’s primary weaknesses were determined to be an increasing inability to respond to many requests as quickly as customers would like due to increased volume of requests. IR will need to improve its ability to assess the scope and resources needed to complete a project while also prioritizing the project within the framework of the college’s strategic plan. The primary opportunities identified were technology based. Increased use of technology such as on-line surveys and optical scan based surveys would save time and resources while providing quicker turnaround of analysis findings. Additionally, Yavapai College has many talented individuals and using their expertise for specific projects could be beneficial and cost-effective. Seemingly endless increased requests for analysis driven assessment and decision making from internal and external sources threatens to overload the IR offices current capacity. IR’s strength of developing programs in-house has a potential liability in that all programming is done by a part-time employee with no ITS support or backup. If the need arises to replace the current programmer, it would most likely require converting the position to full-time or sacrificing IR’s current response time to data requests.


F. BUDGET AND STAFFING YC full-time staffing was similar to the other benchmark colleges. Full-time salaries budget was 3% less than the average; however, YC’s total budget was 10% higher than benchmark participants. The discrepancy can be explained by IR’s utilization of more part-time staff. Part-time staff has allowed YC to automate and cover many functions, thus allowing full-time staff time to focus on new responsibilities in the areas of strategic planning and accreditation. Please refer to the appendix to see budget and staffing figures. G. BENCHMARKS Findings Below is a summary of the findings from the analysis of the IR benchmark data. Please refer to the appendix to see the benchmark survey data. Strategic Planning YC was the only institution to report high involvement in strategic planning. IR coordinates strategic planning for the college including: scheduling meetings, creating and maintaining strategic planning documents (strategic plan, planning guide, year-end report, assessment survey, meeting minutes, and environmental scanning report), maintains the strategic planning website, and ensures that the strategic planning process is aligned with accreditation expectations. Program Review YC was above the average in annual completion of non-academic program reviews. This new benchmarking model replaces the old model of assessing non-academic program reviews. Additionally, IR is instrumental in bi-annual process of reviewing auxiliary service areas. At the time of the benchmark study YC was completing an average of six academic program reviews annually, while other colleges were averaging ten reviews. However, beginning fall 2006, IR is now reviewing all degree/certificate program areas (32) annually. In academic year 2007-08, IR will add 27 general education areas to bring the total number of annual academic program reviews completed to 59.


Data Reporting YC completes an average of 250 ad hoc data requests annually, which is higher than the benchmark average. Additional data services provided by IR include comprehensive enrollment reporting, enrollment forecasting, providing data to state and national projects such as ASSIST and the National Community College Benchmark Project. YC also surveys all credit courses for instructor/course evaluations. IR is also responsible for all state and federal mandated reporting. Studies YC, like the majority of benchmark participants, completes graduate follow-up and exiting student surveys annually. Only one benchmark institution does not conduct exiting student surveys. The majority of benchmark participants conduct current student surveys on an annual basis, while YC conducts this study every other year. The majority of benchmark participants conduct former student surveys; however, there is a lot of variability in the frequency of how often this type of study is conducted. YC conducts a former student survey every other year. YC also conducts employer of graduates surveys in selected career areas either annually or bi-annually.


H. RECOMMENDATIONS Recommendation #1 No changes in staffing or budgeting are warranted. However, the addition of strategic planning duties to the office of IR have stretched resources and it is recommended that the college explore participating with ACT, Noel Levitz, or a like partner to conduct the college’s current student and former student surveys. An additional benefit of this relationship will be the gaining of national normative comparisons. Recommendation #2 Based on the Strengths, Weakness, Opportunities, and Threats (SWOT) analysis IR should work closely with the college’s survey panel to assist with the setting of survey project priorities to streamline work and avoid redundancy. Recommendation #3 As a result of the SWOT analysis IR identified automation of functions, which is performed by a part-time employee, as one of the offices greatest strengths and threats. It is recommended that IR explore alternative avenues for automation in the event that the part-time programmer leaves.


I. ACTION PLAN Recommendation #1 Objective Explore partnering with Noel Levitz, ACT, or similar partner, to conduct current and former student studies. Activities Contact potential partners for scope of service and cost. Prepare a report that documents benefits and limitations to such a partnership and include cost benefit to the college. Responsibility Director of Institutional Research Timeline June 30, 2007 Resource Allocation None for exploration. Cost for potential implementation will be included in the report document prepared for this recommendation. Assessment Report Recommendation #2 Objective Coordinate IR survey planning with the college’s survey panel. Activities IR full-time staff to serve on survey panel and incorporate panel decisions into IR project list.


Responsibility IR staff Timeline June 30, 2007 Resource Allocation None Assessment Year-end IR office report will address the assessment of this process. Recommendation #3 Objective Develop a succession plan for the part-time programmer and office automation functions. Activities Talk with current part-time programmer for suggestions and ideas. Explore cost of replacing the programmer position. Assess current IR office skill sets and determine if in-house programming is feasible. Determine if Information Technology could be used to support automation functions. Responsibility Director of Institutional Research Timeline June 30, 2007 Resource Allocation None Assessment Report


APPENDIX



Documents

Benchmark Service Area Review Institutional Research · • Conducting and supporting institutional research analysis; • Complying with state, federal, and agencies’ requests