Assessment of Student Learning at Frederick Community College Evaluating Institutional Learning...

Assessment of Student Learning atFrederick Community College

Evaluating Institutional Learning Centeredness Conference

San Diego, California · July 2007

Presenters

• Dr. Richard HaneyVice President of Learning Support

• Dr. Debralee McClellanAssociate Vice President for Student Development

About Frederick Community College• Located in central Maryland

– 45 miles west of Washington D.C. and Baltimore

– Northern end of the I-270 high-tech corridor

– County population of 220,000

– Largest county by land mass in the State

• Enrollments

– 4,800 credit students per semester

– 11,000 continuing education students annually

About FCC• Average student age of 27

• 62% attend part-time

• 19% are students of color

• 61% are enrolled in transfer programs

• Annual operating budget of $36 million

Genesis for AssessmentNational focus• For 10 years, professional organizations have underscored the need to

assess student services

• NASPA conferences – emphasis on learning outcomes assessment in student services

• Middle States Commission on Education’s Standards of Excellence

Genesis for AssessmentState focus• Maryland Higher Education Commission (MHEC)

– State accountability reports

State focus• Maryland Association of Community Colleges (MACC) Affinity Groups

• Chief Student Services Officers– Registrars and Admissions Officers– Financial Aid– Athletics– Student Life– Academic Advising/Counseling– Testing Center– Career Counseling– Tutoring

Genesis for Assessment

State focus• Maryland’s Community College Deans/VP of Student’s Learning

Outcomes Project– Developed process for statewide assessment– Contracted with Dr. Marilee Bresciani to guide process– Conducted series of workshops for Deans/VPs and affinity group

members– Developed core goal – “Self-directed learner” to be applied by the

affinity groups

Genesis for Assessment

Context for AssessmentCollege focus• FCC’s re-organization as a “Learning College”

– Vision – “Student Learning First”– Mission – “FCC, as a learning college prepares individuals to meet

the challenges of a diverse, global society through quality, accessible, innovative, lifelong learning. We are a student-centered, community-focused college. FCC offers courses, degrees, certificates, and programs for workforce preparation, transfer, and personal enrichment. Through these offerings, FCC enhances the quality of life and economic vitality of our region.”

College focus• Division of Student Development became Division of Learning

Support

• Learning Support mission statement – “To provide services in a changing, professional environment that encourages and supports lifelong learning.”

Context for Assessment

Learning Support Organizational Structure

• Athletics

• Enrollment Management

• Financial Aid

• Information Technology

• Student Development

• Student Life

• Welcome & Registration Center

Overview of Assessment Process in Learning Support

• College’s transition to a learning college shifted the focus of assessment to student learning – Shift from student satisfaction to student learning– Movement away from evaluating programs by the numbers to

measuring the learning that has occurred as a result of the student’s involvement in the program and/or services

– Forced departments to re-think their true purpose – Emphasis shifts from what we do to what we want students to be

able to do

What we did• Division-wide retreats in 2003 – “The Student Learning Imperative”

and “Outcomes Assessment in the Learning College”

Key questions we asked of ourselves as we developed our assessment plan

• What are we trying to do and why?• How does my program contribute to student learning?• How well are we doing?• How do we know?• How do we use the information to improve or celebrate successes?• Do the improvements we make work?

• Provided professional development and encouraged staff to become active participants in the statewide affinity group

• Created assessment template as a tool to provide structure and to facilitate the development of a comprehensive assessment process for each unit in Learning Support

What we did

Learning Support Assessment Plan FYAREA:

GOAL/OBJECTIVE:

EXPECTED OUTCOMES

1.1

1.2

MEASUREMENT BENCHMARK ASSESSMENT TOOLS

1.1

1.2

STRATEGIES ASSIGNED TO

1.1

1.2

STATUS/FEEDBACK LOOP (How has data been used to enhance learning, services, processes?)

Learning Support Assessment Plan FY - continued

• Used the statewide goal of a self-directed learner as a model for developing learning outcomes across functional areas

– Managers charged to develop assessment objectives for their program areas to address the goal - “Student will become a self-directed learners”

What we did

Definition of Self-directed LearnerThe student will:

• Define a need or problem and employ effective decision-making to resolve it

• Plan ahead/set goals

• Acquire knowledge

• Use available resources

• Seek assistance from appropriate people/experts

• Apply critical analysis to consider options

• Evaluate decisions

Examples of Learning Outcomes in Learning Support Areas

• Athletics – Goal - Assist students in becoming self-directed learners by promoting development of sportsmanship and citizenship skills – Learning Outcome - Student-athletes exhibit good sportsmanship and

citizenship skill• Welcome & Registration Center – Goal -Assist students in becoming self-directed learners by enhancing

one-stop operations – Learning Outcome - Students demonstrate the ability to navigate registration processes in subsequent semesters

• Financial Aid – Goal – By providing information, assistance, and directions to students regarding sources of financial aid and the application process students will more effectively negotiate the financial aid process – Learning Outcome – Students will demonstrate increased awareness of financial aid opportunities – Learning Outcome – Students will receive an increased number of Maryland State Scholarship Awards

Examples of Learning Outcomes in Learning Support Areas

Incorporating Assessment into Student Affairs Programming

Academic Advising

Steps in Assessment1. Develop Functional Area Goal(s)

– Goal should be broadly defined

– What is the overall purpose of the area?

2. Develop Outcomes that Operationalize the Goal

– What do we want students to know or to be able to do?

– Must be measurable – how will you know if student accomplished the outcome?

Steps in Assessment3. Develop Strategies to Ensure Outcomes will be Met

– Services, resources, programs offered

– Opportunities for students to accomplish what you want them to do or to learn what you want them to know

4. Establish Benchmarks

– The level of student accomplishment will you accept as evidence that students have met the outcome

Steps in Assessment5. Develop Assessment Instrument to Measure each

Outcome

– Survey, focus group, portfolio of student work, rubrics, pre and post measurement

6. Use Findings to Make Improvements

– Can include changing a program or a service, or developing a new service to address assessment findings

Assessment of Academic Advising• Advising Goal – Students will become self-directed learners• Advising Outcomes – Students will:

– Demonstrate knowledge of academic requirements– Articulate an educational goal– Develop an educational plan outlining steps necessary to

reach their goals– Demonstrate knowledge of educational policies and

procedures– Demonstrate awareness of how to access college resources

and services

Assessment of Academic Advising• Advising Strategies to Address Outcomes

– First Year Advising Program

– ICAP – Individualized College Academic Plan• tool which helps an academic advisor to frame an advising

session within the context of goal development and establishment of an educational plan

Assessment of First Year Advising Program

• Applied Advising Outcomes to First Year Advising Program– Assessment instrument created to measure attainment of

outcomes– Questions developed to address each advising outcome

• What would student have to be able to do or to know in order for us to feel they met the outcome?

• Grouping survey items by outcome gave us the ability to evaluate each outcome independently

Sample Outcome/Survey ItemsAdvising Outcome

• Students will demonstrate knowledge of academic requirements

– e.g., curriculum for intended major, how to select general education courses

Survey Items

• “I know how to select general education courses”

• “I understand how to select courses for my major”

• “I understand how placement test results determine which classes I can take”

Assessment of First Year Advising Program

• Highest mean scores were obtained for outcomes relating to student’s:

– Knowledge of academic requirements

– Ability to articulate an educational goal

– Ability to develop an educational plan outlining steps needed to reach their goal

Assessment of First Year Advising Program

• Lowest mean scores were obtained for outcomes relating to student’s:

– Knowledge of educational policies and procedures (related to the registration process)

– Awareness of how to access college resources and services (related to setting career goals)

Assessment of First Year Advising Program

• Evaluated effectiveness of ICAP

– Reviewed scores obtained on survey items related to effectiveness of goal setting, degree of assistance obtained with goal setting and educational planning

– Compared to baseline scores from prior years

Spring 2005 – Pre-ICAP

Spring 2006 – ICAP Piloted

Summer 2006 – ICAP fully implemented

Effectiveness of the ICAPSurvey Items

• “I feel that I was helped with establishing realistic academic goals”

• “My advisor was able to help me to start to develop an educational plan to meet my goals”

Mean Scores

– Spring 05 3.68– Spring 06 3.73– Summer 06 3.81

– Spring 05 3.52– Spring 06 3.72– Summer 06 3.81

Using Results to Improve Practice• Reviewed outcomes and individual survey items that

received lowest scores from students in 2006 to revise First Year Advising Program for 2007

– Focused on two lowest rated outcomes and the lowest rated survey item within a highly rated outcome

Using Results to Improve PracticeOutcomes to be Addressed• Students will demonstrate

knowledge of academic requirements (general education courses)

• Students will demonstrate knowledge of educational policies and procedures (related to registration)

• Students will be aware of how to access college resources and services (career goal setting)

Strategy Implemented• Developed consistent delivery

regarding how to select general education courses

• Implemented Registration Services module

• Implemented Career Development module

Assessment of Student Leadership• Goal – Students will become self-directed learners• Outcome – Student leaders will develop competency in six areas:

• personal development• organizational awareness• community awareness• communication skills• interpersonal skills• critical-thinking/problem solving skills

• Assessment tool – Co-curricular portfolio• Measurement – Evaluation of portfolio using leadership rubric

• Set expectation that assessment is a requirement of each manager’s and department’s annual performance

• Annual Assessment Report required for each department

• Learning outcomes published in College catalog

• Results reported to Student Learning Outcomes Assessment Council

• Tie assessment results to the budget process

• Assessment is a formative process – the process and the findings should not be feared

Building a Culture of Assessment

Did it work?• Middle States Commission on Education Accreditation Team

Feedback – March 2006

“The Division of Learning Support deserves enthusiastic commendation for their work with outcomes assessment. There are articulated assessment plans in place that include clear functional area mission statements, maintain realistic benchmarks, and demonstrate evidence of continuous quality improvement. The model created by this division should serve as a standard.”

What we learned• Provide professional development on assessment of student

learning for staff

• Involve staff at the beginning of process to facilitate buy-in

• Template was an asset in helping staff understand assessment

• Encourage staff to read the literature on assessment

• Provide time to do the work

• Keep process as streamlined as possible

What are the challenges?• Change in staff and acclimating new staff to assessment

• Ensuring results lead to improvements

• Interfacing departmental assessment with the College’s strategic plan

• Expertise in research and data analysis

– Office of Institutional Research resources

– Variability of skills of individual staff

Where do we need to go?• Deepen the staff development program• Have managers present their assessment results on an ongoing

division-wide basis • Review and update assessment plans• Expand assessment plans to address additional “student

learning” outcomes in Learning Support• Develop student learning outcomes in Learning Support based

upon the College’s general education program outcomes• Infuse Learning Support efforts into the institutional

effectiveness process

Contact Information

Rhaney@frederick.edu

DMcClellan@frederick.edu