Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)

8/9/2019 Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)

http://slidepdf.com/reader/full/evaluating-and-assessing-ipas-projects-stories-from-the-field-261569331 1/46

Evaluating & Assessing IPAS Projects:

Stories from the Field

2015 NERCOMP Annual Conference, 31 March 2015

D. Christopher Brooks, Ph.D.

Senior Research Fellow

ECAR

Laura Damm

Student Success Coordinator

East Mississippi Community College

Lois Smith

Vice President, Organizational Growth, Development & Effectiveness

Stanly Community College



Overview

What is IPAS?

ECAR IPAS Projects

Methodological Issues

Case Studies





ECAR

IPAS Research Hub

http://tinyurl.com/k95hr7t



IPAS Evaluation & Assessment Guide

•

Why the evaluation and assessment to IPASprojects is important

• Issues and approaches to designing an evaluation

and assessment protocol

•

Issues and approaches related to measuring andanalyzing outcomes data

• Identifying good practices in assessing and

evaluating IPAS projects

• How IPAS evaluation and assessments lead to the

identification of actionable outcomes



Evaluation versus Assessment

#

Evaluation

Assessment

Question

How do we know

the efforts are orhave been worth it?

How do we know if

it works or worked?

Objects

Worth or merit of a

project Impact of a project

Aspects

Procedurally-focused

Judgmental

Contextual

Outcomes-focused

Diagnostic

Empirical



Evaluation versus Assessment: Timing



Evaluation & Assessment:

Levels of Analysis



Assessment Designs



Assessment Designs



Case Studies


29-30 January 2015

Montgomery County Community College

2 February 2015


17-18 December 2015

Colorado State University

5 February 2015




Expand the Pride

March 31, 2015

Laura Damm

Student Success Coordinator

[email protected]



Campuses

Scooba- The main campus (est.1927) is

residential, host to athletics, band, and chorus inaddition to academic and limited technical

programs. 1,126 students

Mayhew- The most populous, commuter campus

(est. 1968) houses GED and Workforce training

services in addition to academic and technical

programs. 3,673 students

Four extension centers offer unique technicalprograms. Online course offerings began in 2000.



Demographics

EMCC enrolls about 6,900 students annually

Ethnicity is 49% White, 47% Black, 4% Other

Female (58%)

Attends full time (57%)

Average age is 23.5

Most are commuters



Technology solutions

1. DropGuard by Smartevals

Early Alert system driven by instructor input

Replaced homegrown process initiated 2009and stalled in 2010-11. Went live fall 2012.

2. Student Planning by Ellucian

Education planning tool, asynchronousinteraction with advisor, capability to register

with one click. Went live fall 2013.

1.



Project Goals

1. Increase retention of new cohort 10%

2. Early Alert used by 100% FT instructors;75% adjuncts in year 1 and 85% in year 2

3. Student Planning used with 25% new cohort inyear 1 and 50% in year 2

4. New cohort demonstrates 5% improvement in

retention, persistence, and graduation/transfer

1.





Usage of Student Planning by Students

in FTFT Fall 2013 cohort



This is a viable tool that allows a second

party to reach out to my students viaemail contact and phone calls. Through

this tool, students are encouraged to

meet with tutors, use technology more

effectively, and have a one-on-one

question and answer session thataddresses EMCC policy and procedures.

-S. Frey, faculty



CompletionCohort: First time, full

time students enrolled at

10th day audit.Q: Of those who started,

how many completed?



Retention/Course Completion



Re-enrolledCohort: First time, full

time students enrolled

at 10th day audit.Q: Of those who started,

how many re-enrolled?



Retention/Re-enrolled



WarningsStudents who received a warning for

poor test, missing assignment,

online absence, absent from class,and missing English essay.



Warned with Early Alert System



IntervenedIntervened: Defined as

on ground tutoring

due to inability to trackindividual students in

NetTutor



Intervened with on ground tutoring



Warned and Finished

How many students in the

cohort were warned and

finished the term?



Warned and Finished



Tutored and Finished

How many students in the

cohort were tutored and

finished the semester?



Tutored and Finished



Lions’ Brew opened Sept 2014. Usage

of tutoring services increased 155%.



Evaluating and Assessing IPAS

Projects: Stories from the Field

March 31, 2015

Lois Smith

VP Organizational Growth, Development, and

Effectiveness

Stanly Community CollegeNorth Carolina




!

A little about us! Small, primarily rural, community college in south

central North Carolina

! One of 58 community colleges in NC

!

SCC serves Stanly County




! A member college of Achieving the Dream

and of the Completion by Design NC cohort

! Curriculum student headcount of

approximately 2800

! Smallest college in the Gates FoundationIPAS Grant Program




! IPAS Project at Stanly• The technology selected was Student Success Plan (SSP)

• SSP is open source software that was integrated with our

Moodle LMS and our Ellucian SIS

• Used two partners to assist with implementation - Unicon and

Gateway to College

• Users throughout the institution are utilizing SSP for: Early Alert, Journaling, and My Academic Plan (MAP) for academic

planning




! Goals and Outcomes

Upon completion of the IPAS adoption, SCC:

! will report improvement in cross department

collaboration for the benefit of student success andcompletion

! will be able to offer students a holistic approach toadvising and intervention strategies with the goal of

student retention

!

will be able to compare course availability toprojected student needs via MAPs






! Assessment Strategy

! Long Range and Short Range Measures

! Few Examples:

!

Indirect measures – student surveys (those whoreceived early alerts or received a MAP)

! Usage reports within SSP such as # ofcompleted MAPs or # of early alerts generated

! Institutional data such as Student Persistence,

Success Rates, and Course Completion Rates




! Assessment Strategy - Example

! Holistic approach to advising and interventionsmeasured by tracking early alert activity and

student persistence




! Assessment Strategy - Example

! Student Persistence rates, are so far,

remaining steady when compared back to

baseline

Spring 2013 to Fall 2013: 52.8% (Baseline)

Spring 2014 to Fall 2014: 52.9%




! Assessment Strategy – Example

! Overall goal of improving student success andcourse completion measured by:

! Student Success Rates

Fall 2013 (Baseline) = 75%

Fall 2014 (Post-SSP) = 78%

! Student Course Completion Rates

Fall 2013 (Baseline) = 83%

Fall 2014 (Post-SSP) = 86%





Questions?



D. Christopher BrooksComing: May 2015 to

http://tinyurl.com/k95hr7t

!"#$ &'()*(+,- (-.

#//0//10-2 3*4.0



Thank you!

Lois Smith

[email protected]

D. Christopher Brooks

[email protected]

@dcbphd

Laura Damm

[email protected]

Documents

Evaluating and Assessing IPAS Projects: Stories from the Field (261569331)