Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 93
AQIP Category Seven: MEASURING EFFECTIVENESS examines how your organization collects,
analyzes, distributes, and uses data, information, and knowledge to manage itself and to drive
performance improvement.
Processes related to Measuring Effectiveness vary in their level of maturity. Processes related to
selecting, managing, and distributing data related to programs, planning, and improvement (7P1 and 7P2)
are aligned. These processes are stable, well-managed by the University’s vice presidents, and
continuously improved in collaboration with affected units. Stakeholders understand the processes, have
input in the selection and analysis of data, and share results. Results related to these processes track
progress on the University’s key strategic and operational goals. Processes for selecting, managing, and
distributing data at the department level (7P3) are systematic, with data collected through repeatable
processes that are being evaluated for improvement. Preparation of the University’s first systems
portfolio, however, revealed an obvious area for improvement—a number of process teams discussed
ways to improve access to institutional information and recommended improvements related to collection
of data for the annual faculty performance reviews and related to centralizing online access to more
institutional data.
Processes related to analyzing and sharing data at the institutional level (7P4) are aligned. These
processes are stable, well-managed by the University’s vice presidents, and continuously improved in
collaboration with affected units. Institutional analysis is expanding to incorporate measures required by
the state’s new performance funding system. The Vice Presidents Council (VPC) provides key leadership
in analyzing and sharing information with stakeholders.
Processes related to determining the needs for comparative data (7P5) are systematic. These processes
provide valid, comparable, longitudinal data for institutional and unit planning. Processes for ensuring
that department and unit goals align with organizational goals (7P6) are also systematic. The VPC, deans,
and directors work together to ensure that goals, objectives, and action steps address University goals.
Processes for ensuring timeliness, accuracy and security of information system (7P7) are systematic.
Procedures for internal control of information are well-managed, under the oversight of the vice president
for finance and the director of information technology services.
PROCESSES
7P1. How do you select, manage, and distribute data and performance information to support your
instructional and non-instructional programs and services?
Southern Arkansas University uses multiple methods to select, manage, and distribute data and
performance information to support instructional and non-instructional programs and services. The
University selects data to be collected based on the informational needs of the data users. These data
needs may be defined by strategic goals, state and federal requirements, specialized accreditation
requirements, and general education and program assessment requirements. In addition, departments may
identify data needed to effectively manage work processes and measure departmental operating goals.
As previously discussed in Category 1, the University has clearly defined learning goals and a systematic
process for assessing student learning. The Office of Institutional Effectives (OIE) compiles assessment
data and prepares an annual report, which is maintained and shared as a LiveText project. As previously
discussed in Category 1 and Category 6, the University has a systematic process for assessing support
programs and services that is under the oversight of program directors and the Vice Presidents Council.
Table 7-1 provides a list of data used to support instructional and non-instructional programs and services.
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 94
Table 7-1: Data Supporting Instructional and Non-instructional Programs and Services
Data Selected Managed By Distribution Method
Admissions data Dean of Enrollment Services
EMAS platform
Daily enrollment and weekly application
reports shared within unit and with VPC
Student information: ACT scores,
transcripts, current schedule
Institutional Research
Registrar
POISE database
Online access for faculty and staff for
individual students, specific reports
available upon request
Student schedules Information Technology Services
Campus Connect platform
Online access for students, faculty and
staff
Instructional resources, student
grades, online instruction, course
embedded assessments
Information Technology Services
Blackboard LMS
Online access for students, faculty, and
staff
Course evaluations by students Office of Human Resources
Blackboard LMS
Results of course evaluations distributed
to faculty, chairs, deans
Student learning outcomes –
program level
Assessment Review Council and
OIE
LiveText web-application
Shared online access
Student learning outcomes –
general education
General Education Committee
and OIE
LiveText web-application
Shared online access
Student success rates in transitional
classes
Director of Developmental
Studies
Shared within division, with VPC;
summary results posted on website
Student success rates for
supplemental instruction program
Director of Student Support
Services
Shared within division, with VPC; report
posted on website
Tracking data for early alerts Director of Early Intervention
Services
Shared within division, with VPC
ACT scores for incoming students Dean of Enrollment Services
Campus Connect platform
Advising folders and online access for
advisors
Library usage data Director of Library Shared within division, with VPC
Program reviews Academic Department Chairs Shared with department and VPAA;
submitted to ADHE
Specialized accreditation reports Academic Deans Shared within program and with VPAA
and president; submitted to accrediting
agency
Continuous Quality Improvement
Initiatives
AQIP Council and AQIP
Executive Council
Updates shared through council meetings,
faculty and staff meetings; annual reports
on HLC website
NCAA student athlete reporting:
enrollment, retention, grades,
graduation rates
Athletic Director Shared with VPC, submitted to NCAA
Institutional Fact Book: student
demographics, enrollment, SSCH,
ACT scores, retention, degrees
awarded, faculty composition
Institutional Research Shared with VPC, deans, department
chairs, and directors, Institutional Fact
Book posted on website
7P2. How do you select, manage, and distribute data and performance information to support your
planning and improvement efforts?
Addressing Core Component 5D
SAU uses multiple methods to select, manage, and distribute data and performance information to support
planning and improvement efforts. The University selects data based on the information needs of the
institution that are defined by strategic goals, state and federal requirements, and specialized accreditation
requirements, with a commitment to collecting data that helps the University improve performance.
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 95
Planning for continuous quality improvement takes place at multiple levels. At the college and
department level, deans and department chairs base planning on enrollment trends, student assessment
data, course and faculty evaluation data, requirements for specialized accreditation, and feedback from
student satisfaction surveys. At the institution level, strategic planning incorporates key performance
indicators, such as enrollment, retention, and completion rates; local student satisfaction surveys; and
recommendations emanating from continuous quality improvement initiatives projects. The University
also relies on comparative student satisfaction data provided by the NSSE and SSI to identify areas for
improvement. Table 7-2 identifies data that is used to support planning and improvement. (5D)
Table 7-2: Data that Supports Planning and Improvement
Data Selected Managed By Distribution Method
Strategic plan Strategic Planning Council, VPC,
president, and Board of Trustees,
Submitted to ADHE, shared with deans and
directors, goals posted on University
website
Blue and Gold Vision President and SAU Foundation Distributes regular reports on progress and
giving. Updates posted on Foundation
website.
SAU Budget Book VP Finance Budget book is distributed yearly to deans,
vice presidents, president and board of
trustees
Budget reports VP Finance
POISE software
Online access to current budget activity and
balances available for department chairs,
deans, directors, faculty, administrators
Admissions and enrollment
data
Dean of Enrollment Services and
Registrar
Office of Admissions sends daily
enrollment and weekly application reports
to VPAA. Registrar sends reports to ADHE.
Retention and progression data VPSA, Institutional Research
(IR)
VPSA distributes yearly and semester
reports to deans, VPC, and directors. IR
sends reports to IPEDS, ADHE.
Graduation rates Registrar, VPAA Registrar distributes semester reports to
deans, VPAA.
Student diversity Office of Multicultural Affairs Distributes yearly minority report to VPC
and submits to ADHE.
Safety and security data University Police Department Cleary reports distributed to VPC and
available on the University website.
Dining surveys Aramark Results provided to VPSA and VPC
Occupancy rates in residential
living
VPSA Results shared with dean of students, VPC
Housing exit surveys VPSA Results shared with dean of students, VPC
Cost containment initiatives VP Facilities Results shared with VPC and available on
the University website
Student learning outcomes OIE
LiveText web-application
Shared online access and summary reports
provided for VPAA
Course evaluations by students Office of Human Resources
Blackboard LMS
Results distributed to faculty, chairs, deans,
VPAA
Faculty composition Office of Human Resources Results distributed to VPC
Faculty and staff retention Office of Human Resources Results distributed to VPC
Average faculty salaries Office of Human Resources Results distributed to VPC
Faculty peer evaluations Office of Human Resources
Blackboard LMS
Results distributed to faculty, chairs, deans,
VPAA
Staff evaluations Office of Human Resources Results distributed to staff member,
supervisor
Faculty annual summary of
professional activity
Office of Human Resources Results distributed within departments to
facilitate peer review, filed in HR
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 96
Promotion and tenure
documents
Office of Human Resources and
VPAA
Recommendations forwarded from VPAA
to president and Board of Trustees
Advising surveys Dean of Enrollment Services Results are distributed to VPSA and
advising director
BAM Orientation surveys Dean of Enrollment Services Results are distributed to VPSA,
admissions, and advising director
Continuous Quality
Improvement Initiatives
AQIP Council and AQIP
Executive Council
Updates shared through council meetings,
faculty and staff meetings
NCAA student athlete
reporting: enrollment,
retention, grades, graduation
Athletic Director Shared with VPC, submitted to NCAA
SAU Foundation Report Director of Foundation Distributed to president and financial
reporting posted on website
Institutional Fact Book: student
demographics, enrollment,
SSCH, ACT scores, retention,
degrees awarded, faculty data
Institutional Research Shared with VPC, deans, department chairs,
and directors, Institutional Fact Book
posted on website
IPEDs: enrollment, student
financial aid, degrees awarded,
retention, graduation rates,
institutional resources
Institutional Research Submitted to U.S. Department of
Education; online access for VPC, extracted
data provided as requested
Annual state reporting,
including performance funding
data
Institutional Research Submitted to ADHE, shared with VPC;
reports distributed by ADHE to all
institutions
National Survey of Student
Engagement
VPSA Reports shared with VPC, distributed to all
departments
Noel Levitz Student
Satisfaction Inventory
VPSA Reports shared with VPC, distributed to all
departments
In 2009, the State of Arkansas adopted a performance funding system (PFS) for institutions of higher
education. This model is based on outcomes, such as retention, degree completion, and job placement,
rather than on inputs, such as enrollment. By adopting this model, the state is rewarding effective and
efficient management, which is essential as institutions face increasing demands on higher education and
reductions in state funding. The state will phase in the PFS beginning in the next fiscal year, with 95% of
funding based on established needs and 5% based on performance. Original legislation called for
allocations to shift by 5% annual increments until funding reached 75% for need and 25% for
performance. Recent legislation, however, froze future performance funding at 10% until all institutions
reach needs funding at the 75% level. As presented in 7R3, SAU achieved the performance targets for the
first measurement year and received the allocated performance funding. Table 7-3 briefly summarizes the
mandatory and optional measures included in the Arkansas PFM. (5D)
7P3. How do you determine the needs of your departments and units related to the collection,
storage, and accessibility of data and performance information?
Table 7-3: Arkansas Performance Funding Measures
Mandatory
Measures
4 point max
Optional Measures
6 point max
Compensatory
Measure
1 point max
Bachelor Degrees Minority Credentials Non-Traditional Credentials Students Receiving Pell
All Credentials Remedial Credentials Transfer Student Credentials
STEM Credentials High Demand Course Completions
Progression of Cohort External Grants and
Awards Expenditures
Regional Critical
Need/Economic Development
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 97
Each department or unit plays a role in determining their own needs for data and performance
information. All faculty and staff have access to the University’s primary data collection and sharing
systems—POISE and Campus Connect—which provide access to current student data and budget
information. All staff members in the Office of Enrollment Services has access to EMAS, the
University’s data collection and sharing system for enrollment management. All faculty advisors have
access to the degree audit program. All stakeholders have access to the University’s Fact Book, which is
accessible on the University website. The Fact Book contains historical data related to enrollment,
retention, degrees awarded, student demographics, and faculty composition, as well as comparative data
for other state institutions. Any department may request data from the Institutional Research, which
maintains IPEDS data and state reporting data, or from the registrar, who maintains the POISE database
related to current students.
Recommendations for improvements in data collection, storage, and accessibility usually emerge from
dialog between department chairs, deans, committees, quality improvement teams, and administrators.
Table 7-4 summarizes recommendations and solutions to recent data needs.
Table 7-4: Methods for Determining Department-level Information Needs
Recommended by Need Solution
Faculty and department
chairs
Need for online resources to support advising Degree audit program
Assessment Review
Council
Need for integrated online sharing of assessment
reports
LiveText portfolios for
assessment
Action Project team Need for integrated online sharing of general
education assessments
LiveText portfolios for general
education assessment
AQIP coordinating team Need for online data sharing for systems portfolio SugarSync cloud sharing
College of Education
NCATE team
Need for online data sharing for preparing NCATE
report
Drop Box cloud sharing
Office of Institutional
Effectiveness
Need for online data sharing for preparing state
program review
LiveText portfolio for program
review
Department chairs Need for online tools to collect and compile annual
faculty performance reviews
Assigned to Action Project
team for solution
Department chairs Need for better system for student advising Assigned to Action Project
team for solution
Collection and accessibility of advising information has been the focus of recent dialog between
department chairs and the administration. The University’s degree audit program assists advisors in
reviewing degree requirements and courses taken to track a student’s progress toward graduation. Using
degree audit, advisors can better identify students satisfying associate degree requirements—one of the
performance indicators included in the state’s performance funding model. Last year degree audit
identified 75 additional students who had completed associate degree requirements. However, advisors
have encountered problems using the degree audit program, particularly in evaluating transfer students.
Department chairs communicated these concerns, as well as other issues with the advising process, to the
administration. To provide immediate relief for a clearly-identified problem, the University adjusted
planned staffing to add a half-time transcript analysis evaluator. In addition, the University launched a
new action project in fall 2012 to identify and implement strategies for improving degree audit and the
advising process.
As discussed in 1I1, the University has significantly improved the processes for collecting, storing, and
accessing data related to student learning outcomes. To implement these processes campus-wide, the
University adopted the LiveText web-based application as SAU’s platform for collecting general
education and program assessment data.
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 98
Preparation of SAU’s first systems portfolio revealed areas for improvement in the collection and
accessibility of data. Several teams discussed ways to improve the process for collecting and compiling
data for faculty performance reviews, which resulted in an action project to identify online solutions for
collecting and compiling this data. Many teams discussed the need for centralizing online access to more
institutional data. For example, the team collecting data about faculty research (Table 2-4) reported that
scholarship data was difficult to aggregate. SAU recognizes this as an opportunity for improvement.
7P4. How, at the institutional level, do you analyze data and information regarding overall
performance? How are these analyses shared throughout the institution?
Addressing Core Component 5D
At the organizational level, SAU’s senior administrators participate in state meetings and conferences
with legislators and the ADHE to discuss significant educational issues related to the new performance
funding system and related to the University’s needs-based funding. SAU was a member of the four-year
planning committee that made recommendations related to developing the performance funding process.
The University collects standardized data and submits reports for the Integrated Postsecondary
Educational Data System (IPEDS). The University provides annual reports to the ADHE, including
mandatory state reporting, performance funding data, and the University’s annual budget. Senior
administrators review IPEDS and ADHE summary reports to identify areas of strength and opportunities
for improvement. For example, the most recent reports indicated that SAU’s faculty composition is more
than 100 percent of that for comparable schools, but classified and professional staff composition ranges
from 40-60 percent. Faced with budget reductions for 2012-2013, the University chose to implement a
reduction in force related to non-tenured faculty rather than staff. (5D)
Since enactment of performance funding in 2009, SAU has worked aggressively to monitor and improve
measures identified in Table 7-3. For example, SAU recognized an immediate need to improve retention
and remediation. In response, a continuous quality improvement team was appointed to design and
implement improvements; this resulted in redesign of the transitional development curriculum and
creation of the Office of Early Intervention Services. These improvements are further discussed in 1R2.
As another example, in May 2013 the VPC appointed a team to review all 810 students who did not make
satisfactory progress. The team analyzed academic records, developed a plan for each student to get back
on track through summer school, and assisted students in finding summer jobs to pay for the classes. (5D)
At the institutional level, senior administrators analyze data and share information on overall performance
through formal and informal processes. The University distributes the Fact Book, which contains current,
longitudinal, and comparative key performance indicators, to senior administrators, deans, directors,
department chairs, and the library. The Fact Book is also posted on the SAU website and can be accessed
by any stakeholder. The president presents regular reports about University performance to the Board of
Trustees, Faculty Senate, Staff Senate, SAU Foundation Board of Governors, and at faculty and staff
meetings. Informally, senior administrators meet regularly to analyze data and information about
performance. Table 7-5 lists significant institutional data that is analyzed and distributed. (5D)
Table 7-5: Institutional Data Analysis and Distribution
Analysis by Performance Data Reporting and Distribution
President Foundation fundraising SAU Foundation Board of Governors
Updates on capital projects Board of Trustees, legislature, ADHE
Operational data for all divisions Board of Trustees with recommendations
Performance funding report Board of Trustees, VPC, deans, directors,
faculty and staff
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 99
VP Academic
Affairs
Annual college assessments Deans, chairs, VPC
Summary of pending legislation, new acts Legislative liaisons, president, VPC
VP Administration Summaries of course evaluations by students Faculty members, department chairs and
deans
Summaries of peer evaluations Faculty members, department chairs and
deans
Summaries of staff evaluations Staff member, appropriate VP
VP Student Affairs Enrollment, retention, and graduation data VPC, deans and directors
Housing applications and occupancy VPC, dean of students
Annual departmental assessments Department heads, VPC
Support program assessments (BAM, SI) Groups administering program, VPC
NSSE, SSI VPC, deans, directors, department chairs
VP Finance Comparison of budget to actual expenditures President, VPC, Deans, Directors
ADHE financial reporting ADHE, legislature, VPC, public
information
Grant reporting ADHE, USDED, ADE, appropriate grant
project coordinator, and senior
administrator
University comprehensive financial
statements
VPC, Board of Trustees, general public,
ADHE, DFA, division of Legislative Audit
Title IV Student Financial Aid data USDED
VP Facilities Quarterly and annual job performance ratings Supervisors
In-house construction project updates Senior staff and supervisors in physical
plant
Energy usage Senior staff in physical plant, VPC
Safety compliance reports Senior staff at physical plant; VPC
Since transitioning to a focus on quality improvement, the University has increasingly shared information
about continuous quality improvement initiatives (CII). The AQIP Council, which includes over 25
faculty and staff members, meets each month to share progress reports on action projects and other CIIs.
The three-member AQIP coordinating team meets each month with the AQIP Executive Council to share
these updates. The coordinating team also presents regular reports at faculty and staff meetings.
7P5. How do you determine the needs and priorities for comparative data and information? What
are your criteria and methods for selecting sources of comparative data and information within and
outside the higher education community?
The University determines comparative data needs and priorities by reviewing key processes that are
essential to attaining SAU’s mission, vision, and strategic plan. Comparative data is primarily derived
from reporting provided by the state (ADHE) and from Federal reporting (IPEDS). These comparative
and validated sources are readily available when needed, but these sources are often one year behind the
current calendar year. Within the full-composition of state institutions, SAU primarily compares our data
to peer institutions within the state—institutions with similar levels of enrollment and similar programs.
SAU also participates in ACT and CSRDE studies that gather retention and graduation data on cohorts.
That data is then compared to participating institutions of our selection.
Comparative data is required for the ten-year reviews of academic programs coordinated by ADHE. State
reporting and IPEDS provide comparative data related to enrollment, retention, graduation, etc.
Additional sources of comparative data are needed to evaluate student and employee satisfaction. To meet
this need, the University uses collects data on a three-year cycle through the National Survey of Student
Engagement (NSSE), the Noel Levitz Student Satisfaction Inventory (SSI), and the Chronicle of Higher
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 100
Education survey of ―Greatest Colleges to Work For.‖ Results of these surveys are linked to their
respective normalized data and provide comparison with peer groups and national averages.
The Arkansas performance funding score card also provides comparative data. The first score card was
recently released by ADHE and was presented at the April 26, 2013, meeting of the Arkansas Higher
Education Coordinating Board.
7P6. How do you ensure department and unit analysis of data and information aligns with your
institutional goals for instructional and non-instructional programs and services? How is this
analysis shared?
As described in 8P3, the SAU Strategic Planning Council communicates the University’s strategic goals
to the colleges, departments, and divisions. In response, each unit develops action plans to support the
achievement of the goals and objectives. Each unit also selects measurable outcomes associated with the
action plans. The University is completing the first-year of a new three-year strategic planning process,
and units are currently preparing progress reports. Table 7-6 provides methods in place to ensure that data
analysis and information aligns with organization goals and describes how unit results are shared.
Table 7-6: Alignment of Unit and Organizational Goals
Administrator Action Assurance Dissemination
Instructional Goals
VPAA Department chair submits strategic
plans to deans, VPC
VPC review Analysis shared with dean
Budgets linked to strategic plan,
submitted by chairs through deans
VPAA reviews and
forward to VPC
Budget and analysis shared
with deans, chairs
Fact Book used as basis for enrollment
and graduation data by academic unit
Fact Book maintained by
IR and reviewed by VPC
Analysis shared with dean
Projected enrollment data included in
proposals for new courses and
programs
Reviewed by Academic
Affairs Committee and
VPC for budget
implications and need
Minutes of AAC
published, new programs
reviewed by Board of
Trustees
Personnel requisitions to substantiate
need and budgeting for new positions
Reviewed by VPAA and
HR
Approved positions
advertised through HR
Department chair reviews faculty’s
development plan and prepares
performance evaluation
Reviewed by dean and
VPAA
Individual meetings for
evaluation confidentiality
Requests for library resources Revised by dean, librarian,
library committee
Acquisitions available on
library website
President Reviews and prioritizes proposed
quality improvement initiatives
Reviews by AQIP
Executive Council
New CII presented at
faculty and staff meetings
Non-Instructional Goals
VP Finance Financial goals and objectives must
provide fiscal support to University
and data necessary for compliance
VPC reviews Analysis distributed to
Board of Trustees, ADHE,
auditors
VP Student
Affairs
Directors submit strategic plans to
VPSA
VPC reviews Analysis shared with
directors
Budgets linked to strategic plan,
submitted by directors
VPSA reviews and
forward to VPC
Budget and analysis shared
with directors
VP
Administration
Reviews state and federal updates
related to HR and updates handbooks.
VPC reviews Handbooks on website,
provided to new
employees
Reviews strategic planning process
with SPC and Administrative Council
VPC reviews Analysis provided at
faculty/staff meetings, and
to units and departments
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 101
7P7. How do you ensure the timeliness, accuracy, reliability, and security of your information
system (s) and related processes?
Each department at SAU has a responsibility to collect and enter data accurately and in a timely manner.
Student data initiates in the Admissions Office, where staff members enter student admission data into
EMAS, the enrollment management system. After admission, student data is transferred by the Office of
the Registrar into the POISE system. The Registrar also enters new courses and programs, which have
been approved by Academic Affairs Committee and by ADHE, into the POISE system. Course
information is maintained and schedule information updated by the Office of Academic Affairs with data
integrated from the POISE system into Campus Connect, the platform for registration management.
Faculty verify rosters and enter and verify grades through Campus Connect, from which grades are posted
onto academic transcripts in the POISE system.
Reporting of mandatory state and federal data is centralized in the Institutional Research. Access to the
data is restricted, primarily for security purposes. Specialized training is necessary to query, extract, and
format valid, meaningful data from the POISE system. The ITS staff, including the director of
institutional research (IR), are well trained to effectively and accurately use the system. Data compiled for
state and federal reporting are summarized in the Institutional Fact Book, which is posted on the website.
When requested, IR extracts and provides data need by the offices, departments, and colleges across
campus. Integration of some data, particularly data which related to graduate studies, currently lies
outside the current parameters for student data collection for tracking retention and enrollments. The new
upgrade for the EMAS software will allow greater integration of POISE and EMAS and incorporate the
graduate department into the system.
Information Technology Services supports and manages the software and hardware necessary for POISE,
Campus Connect, and Blackboard, the University’s learning management system. Many controls are in
place for security, prevention of erroneous data entry (data validation), and error report checks after data
entry. Campus data, including budgets and student records, are centralized in the POISE database system.
The POISE system is based on Open VMS, one of the most secure operating systems in the world.
OpenVMS and POISE are very stable less expensive than similar systems, requiring less staff to manage.
SAU’s financial information system, under the oversight of the vice president for finance, incorporates
internal control procedures appropriate for higher education. These internal controls are reviewed
annually by the Arkansas Division of Legislative Audit to determine the audit procedures that will be
applied in order to express an opinion on the financial statements. Internal controls extend to control over
the administrative systems and student information system, which are centrally managed by director of
information technology services. Both systems have generator backup systems that automatically start in
the event of loss of power. This configuration provides high availability for our business critical systems.
ITS also monitors system performance and provides downtime notification. SAU has 24/7 access,
uninterruptable power, backups (regular and off-site disaster recovery backup) of the administrative
system. The security for SAU’s system requires user authentication, application access, off-site disaster
recovery backup, intrusion detection/monitoring, SSH encryption, and web site encryption.
VP Facilities Crosswalks organizational goals with
Blue and Gold Vision campus
development plan
VPC reviews Construction progress
updates for faculty, staff,
and Board of Trustees
President Reviews and prioritizes proposed
quality improvement initiatives
Reviews by AQIP
Executive Council
New CII presented at
faculty and staff meetings
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 102
RESULTS
7R1. What measures of the performance and effectiveness of your system for information and
knowledge management do you collect and analyze regularly?
Table 7-7 lists the key performance indicators for measuring effectiveness.
Table 7-7: Key Performance Indicators for Measuring Effectiveness
Indicators Data Presented in
Student entry indicators Admissions updates
ACT scores for beginning students Figures 7-1 and 7-2
Demographics of beginning students Fact Book (p. 2+)
Process indicators Daily enrollment updates
Enrollment data for university Figures 7-3–7-6
Enrollment data by major and by college Fact Book (p. 41+)
International enrollment Fact Book (p. 2+)
Diversity enrollment Fact Book (p. 2+)
Enrollment data by feeder schools Fact Book (p. 55+)
Retention rates Figure 7-7
Graduation data Figures 7-8—7-11
Housing occupancy Figure 7-12
Housing exit surveys See Figures 6-2
Dining surveys See Table 6-8
Student learning outcomes See 1R2 and 1R3
NSSE Table 7-12
SSI Figure 7-14
University resource indicators SSCH by department and college Fact Book (p. 81+)
Faculty composition See 4R2
Faculty/staff diversity See 4R2
Faculty scholarship and creativity See 2R2
Faculty salary comparisons Table 7-11
Employee satisfaction surveys Figure 7-13
Financial management Utility usage and cost Table 7-8
Cost containment initiatives
External accountability Mandatory performance funding measures for graduation
rates and cohort progression Table 7-9
Optional performance funding measures, including
graduation rates for minority students, remediation students,
transfer students, students in critical needs programs, and
completion data based on SSCH
Table 7-10
IPEDS
7R2. What is the evidence that your system for Measuring Effectiveness meets your organization’s
needs in accomplishing its mission and goals?
SAU’s system for measuring effectiveness responds to environmental changes, changes in stakeholder
needs, and
continuous quality improvement initiatives. The University is committed to communicating with all
constituencies to ensure that the University community is aware of progress in meeting various
objectives. The data presented below represent key measures of effectiveness.
Student Entry Indicators. The Office of Admissions provides weekly updates to the VPSA related to
the number of students applying for admission and completing admission to SAU. Student admission data
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 103
0
200
400
600
800
20
00
20
02
20
04
20
06
20
08
20
10
2500
2600
2700
2800
2900
3000
20
00
20
02
20
04
20
06
20
08
20
10
19.5
20
20.5
21
21.5
22
2000 2002 2004 2006 2008 2010
Figure 7-1: Average ACT Scores
0
20
40
60
80
100
ACT>27
ACT<16
Figure 7-2: Enrollment by ACT
Figure 7-3: Total Enrollment Figure 7-4: Beginning Freshmen
Figure 7-5: Undergraduates Figure 7-6: Graduate Students
50.00%
55.00%
60.00%
65.00%
70.00%
Figure 7-7: Retention Rates
provides a profile of entering students, including demographics, high
school transcripts, and ACT scores. As shown in Figure 7-1, the
average ACT scores for beginning freshmen continues to trend upward.
As shown in Figure 7-2, data also indicates an upward trend in entering
students with ACT scores about 27, coupled with a downward trend in
entering students with ACT scores below 16. The University compiles
and analyzes demographic data of entering students to monitor student
diversity initiatives. Data on diversity is provided in the SAU Fact Book
(p. 2+)
Process Indicators. The University collects and analyzes data to measure
the effectiveness of its instructional and no-instructional
programs, including enrollment and retention data, graduation
data, and measure of satisfaction with key services.
Enrollment data indicates the effectiveness of SAU’s
recruiting efforts, instructional processes, and support services.
At the beginning of each semester, the Office of Enrollment
Services provides the deans and VPC with daily updates on
enrollment until the official census date. Institutional research
compiles current and longitudinal data on enrollment. As shown in
Figure 7-3, total enrollment has
increased since 2003—with total
enrollment of nearly 3,400 in 2010
and 2011. Enrollment of beginning
freshmen, shown in Figure 7-4, has
increased after a modest decline in
2007. Graduate enrollment, shown in
Figure 7-5, has increased
significantly. The University’s
strategic plan includes an objective
to increase the number of graduate
programs and graduate enrollment.
Data indicates that the University is
accomplishing this objective.
Additional analysis of enrollment
data, including analysis by college
and major, based on diversity, and by
feeder schools, is provided in the
SAU Fact Book.
Retention data indicates effectiveness of the University’s retention efforts,
instructional programs, and support services. The University compiles
current and longitudinal data on first-year retention and cohort progression.
As shown in Figures 7-7, the University’s retention rates have not
consistently improved. The University identified retention as an area for
improvement in 2010 and appointed an Action Project team to develop and
implement new retention strategies. Data for the next two years will
0
100
200
300
400
500
20
00
20
02
20
04
20
06
20
08
20
10
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 104
0
200
400
600
800
20
00
-01
20
02
-03
20
04
-05
20
06
-07
20
08
-09
20
10
-11
0
50
100
150
200
20
00
-01
20
02
-03
20
04
-05
20
06
-07
20
08
-09
20
10
-11
Figure 7-8: Total Degrees
Figure 7-9: Associate Degrees Figure 7-10: Bachelor’s Degrees Figure 7-11: Master’s Degrees
Figure 7-12: Residence Hall Occupancy
indicate the degree of effectiveness of these new initiatives.
Graduation data indicates success in accomplishing the University’s
mission and effectiveness of the University’s retention efforts,
instructional processes, and support services. The University compiles
current and longitudinal data on graduates from associate, bachelors,
and masters programs. As shown in Figures 7-8 through 7-11, total
degrees awarded by SAU continue to increase in all categories. As
previously noted, advisors are now using the degree audit system to
better identify students satisfying associate degree requirements,
which resulted in identifying 75 additional students who had
completed associate degree requirements in 2011-12.
Residential occupancy and satisfaction
are indicators of effectiveness related to
campus life. As shown in Figure 7-12,
trend analysis indicates that SAU is
meeting the housing needs of students.
With occupancy nearing full capacity, the
University continues to add new
residential living facilities. As previously
reported in 6R2 (Figure 6-2), residence
hall students complete exit surveys at the
end of each semester. Results consistently
show that over 85 percent of residents
were satisfied with their room.
Dining services are an essential service provided for students, faculty, and staff. The University collects
and analyzes two measures of satisfaction with campus dining services—surveys by the dining service
provider and results of the SSI questions related to dining. Results of the dining survey can be referenced
in 6R2 (Table 6-8) and consistently indicate lower scores related to variety of food. Results of the most
recent SSI corroborates this weakness with results that are statistically lower than the national averages
(4.04 versus 4.57; significant at the .001 level).
The University identified dining services as an area for improvement in 2011 and requested input from
the Student Government Association about options for improving the variety of food. An SGA poll
indicated student’s preference for replacing Quizno’s with Subway and for adding Chick-fil-A and Grill
Works. By late 2012, the three new restaurants opened in the food court. Results of future surveys will
indicate if these changes improved satisfaction.
0
50
100
150
200
20
00
-01
20
02
-03
20
04
-05
20
06
-07
20
08
-09
20
10
-11
0100200300400500
20
00
-01
20
02
-03
20
04
-05
20
06
-07
20
08
-09
20
10
-11
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 105
Student learning outcomes are a key measure of the effectiveness of the University’s academic
programs and support services. Student learning outcomes are discussed at length in 1R2 and 1R3.
University Resource Indicators. SAU collects and analyzes data to measure the effective use of faculty
and staff resources, including analysis of student semester credit hour production by department and by
college. This analysis is available in the Fact Book (p. 81+). The University monitors faculty productivity
in terms of scholarly contributions and creativity; these measures are discussed in 2R2. The University
monitors faculty composition by rank and diversity; these measures are discussed at length in 4R2.
Financial Management. Cost containment measures are key indicators of effective management of
financial resources. Energy consumption and costs are tracked by the physical plant as a targeted area for
critical cost containment. As shown in Table 7-8, the University’s energy usage and per unit energy costs
have also increased significantly over the last five years. Increased usage is attributed to three factors:
new construction that
increased campus square
footage by 14%, enrollment
increased 7.9%, and record-
high summer temperatures.
Faced with increasing utility
costs, the University
invested $1.4 million in the new district heating and cooling system, which is providing estimated annual
savings of $340,000 and resulted in SAU’s energy costs dropping to $1.20 per square foot, which is
among the most efficient in the state. Selecting energy efficient equipment has resulted in over $100,000
in rebates from power and gas companies. Other cost containment initiatives are shared on the University
website.
External Accountability. Arkansas’ performance funding measures include four mandatory measures
related to graduation and cohort progression. In terms of graduation data, the PF formula established
2009-10 as the base year and set goals of increasing graduation rates by 4.73% each year for bachelor’s
degrees, total degrees, and
degrees in science, technology,
engineering, and mathematics
(STEM). Cohort progression is
a measure of two-year retention, in which the goal is to increase the current two-year progression rate
over the average rate for the three preceding years. Table 7-9 summarizes SAU’s performance results
released in April 2013. As shown, SAU exceeded targets for three of the four measures.
Each institution may choose to be evaluated based on up to six optional Arkansas performance funding
measures. At this time, SAU has elected to be evaluated on five optional categories plus the compensatory
category that factors in a low-income student population. Each optional measure is based on the goal of
increasing the institution’s current two-year results over the average rate for the three preceding years and
one point is awarded for meeting each goal. Table 7-10 summarizes the points awarded on SAU’s
performance score card released in April 2013. As shown, SAU earned full points in the five optional
measures with additional points based on the number of students who receive Pell grants.
Based on SAU’s results, the University received the full allocation of performance funding provided.
Table 7-8: Analysis of Energy Usage and Costs 2008-2012
Electricity Natural Gas Water Wastewater
Increase in use 4.6% 15% 22% 13.5%
Change in unit cost .4%
Increase
15%
Decrease
102%
Increase
96%
Increase
Percentage of total utility 60% 21% 11% 8%
Table 7-9: Performance Funding Scorecard for Degrees and Progression
Bachelor’s Degrees Total Degrees STEM Degrees Progression
97% 101% 127% 106%
Table 7-10: Performance Funding Scorecard for Optional and Compensatory Measures
Minority
Degrees
Remedial
Degrees
Transfer
Student Degrees
Regional Critical
Needs Degrees
Course
Completion
Compensatory Points
for Pell
1 point 1 point 1 point 1 point 1 point .56
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 106
0% 20% 40% 60% 80%
Survey Average
Fairness
Communication
Senior Leadership
Pride
Policies, Resources & Efficiency
Compensation, Benefits &…
Teaching Environment
Figure 7-13: Employee Satisfaction Survey
7R3. How do your results for the performance of your processes for Measuring Effectiveness
compare with the results of other higher education organizations and, if appropriate, of
organizations outside of higher education?
SAU processes for measuring effectiveness are evaluated using comparative data to identify market
trends, areas of success, and opportunities for improvement. Comparative data related to enrollment,
graduation rates and employees are provided by IPEDS and ADHE reporting and are summarized in the
SAU Fact Book. Comparative data related to student satisfaction is provided through regular
administration of the National Survey of Student Engagement (NSSE) and the Noel Levitz Student
Satisfaction Inventory (SSI). Comparative data related to employee satisfaction is provided through
participation in the Chronicle of Higher Education’s survey of the Greatest Colleges to Work For and
analysis of average faculty salaries provided by AAUP.
At the institutional level, the administration
carefully monitors faculty salaries in comparison to
state and national averages. As shown in Table 7-
11, AAUP data indicates that faculty salaries at
SAU compare favorably with state averages but are
below the national averages.
As shown in Figure 7-13, results of
the Chronicle survey identified an
opportunity for improvement relation
to communication between faculty,
staff, and administrators. Reflecting
on these results, the University has
taken steps to improve
communication between constituents.
For example, the AQIP coordinating
team strategically selected faculty and
staff members to jointly co-chair each
team working on the systems
portfolio and continuous quality
improvement initiatives.
Results provided by the NSSE generally
indicate that student satisfaction at
SAU is similar to that at peer
institutions. The NSSE results in
Table 7-12, indicate statistically
significant differences related to
the level of academic challenge,
which is below peer averages, and
student-faculty interaction, which
is above peer averages.
Summary results of the 2012 SSI
are provided in Figure 7-14 and
indicate a relatively high level of
student satisfaction with little
difference when compared to peer
Table 7-11: Average Faculty Salaries
Based on 2012 AAUP Data
SAU Arkansas National
Professor $74,700 $67,125 $88,021
Associate Professor $57,300 $56,745 $70,312
Assistant Professor $49,600 $49,721 $59,318
Color
Coding:
Red Flag
0%-
44%
Yellow Flag
45%- 54%
Fair to Mediocre
55%- 64%
Good
65%- 74%
Very Good to
Excellent
75%- 100%
Table 7-12: 2012 NSSE Results
SAU Southeast
Public
Sig Carnegie
Class
Sig NSSE
2012
Sig
Level of Academic Challenge
First-year 51.3 53.1 54.8 * 54.5 *
Senior 54.6 57.1 * 58.6 *** 58.4 ***
Active and Collaborative Learning
First-year 43.8 43.5 45.8 44.2
Senior 54.1 51.8 53.7 52.1
Student Faculty Interaction
First-year 37.0 36.4 37.5 35.9
Senior 50.3 44.1 *** 44.5 ** 42.9 ***
Enriching Educational Experiences
First-year 27.0 28.1 28.0 28.4
Senior 39.4 39.7 39.8 40.4
Supportive Campus Environment
First-year 62.5 64.0 64.8 63.5
Senior 64.2 60.8 * 62.2 60.5 * *p<.05 **p<.01 ***p<.001
Southern Arkansas University – Systems Portfolio – June 1, 2013
Category Seven: Measuring Effectiveness Page 107
1 2 3 4 5 6
Academic Advising
Campus Life
Concern for the Individual
Recruitment & Financial Aid
Responsiveness to Diverse…
Service Excellence
How Satisfied Are Our Students Compared to
other Institutions?
National Four-Year Publics
Figure 7-14: Student Satisfaction Inventory
results. Results of detailed questions on
the SSI, however, identify opportunities
for improvement, such as the results
previously discussed related to dining
services.
7I1. What recent improvements have you made in this category? How systematic and
comprehensive are your processes and performance results for Measuring Effectiveness?
The University’s processes and performance results for measuring effectiveness are comprehensive,
providing information to those engaged in planning and managing the institution. Performance results are
available and shared to support data-based decision making.
The University has made many improvements related to this category. For example, effective collection
and sharing of information is linked directly to the University’s reliable web-based information systems.
Over the past few years, our Campus Connect web-based information system has become more and more
utilized by the faculty and staff of SAU. The Campus Connect portal is a web extension of POISE which
allows a few of the commonly used academic applications from POISE to be accessed anywhere. Course
enrollments, student advising, degree audits, grade submissions, and student demographic data are all
accessible through the Campus Connect web portal. This has greatly streamlined data access for students
and faculty. Campus Connect also allows faculty to complete routine tasks online, such as certifying
rosters, submitting early alert referrals, and printing student transcripts.
7I2. How do your culture and infrastructure help you to select specific processes to improve and to
set targets for improved performance results in Measuring Effectiveness?
SAU uses many processes to improve its performance. Since transitioning to a focus on continuous
quality improvement in 2010, the University community has worked together to improve processes
related to retention, remediation, assessment of student learning outcomes, and employee development.
Teams continue to work on issues related to improving academic advising and the faculty evaluation
process.