Table of Contents Introduction 1 Retention Analysis 3...retention through predictive modeling, as will be discussed in the next section of this report. High school GPA is also strongly

____________________________________Scannell & Kurz, Inc.____________________________________

Draft: June 15, 2011

Table of Contents

Introduction ................................................................................................................................... 1

Retention Analysis ........................................................................................................................ 3

Retention to Graduation by Cohort ............................................................................................. 3

First to Second Year Retention for Freshman Cohorts ............................................................... 4

Table Analysis ........................................................................................................................ 4

Predictive Modeling ................................................................................................................ 5

Second to Third Year Retention for Freshman Cohorts ............................................................. 8

Table Analysis ........................................................................................................................ 8

Predictive Modeling .............................................................................................................. 10

First to Second Year Retention for Transfer Cohorts ............................................................... 12

Table Analysis ...................................................................................................................... 12

Predictive Modeling .............................................................................................................. 13

Recommendations ....................................................................................................................... 15

Conclusion ................................................................................................................................... 18

1



Introduction

Scannell & Kurz, Inc. (S&K) was invited to the City College of New York (CCNY) to

provide advice and counsel regarding the use of institutional financial aid in support of

enrollment goals for new and continuing students. Because of the time required to compile the

requested data set, S&K provided initial observations and recommendations related to

recruitment, financial aid, and retention programs on April 7th, based on site visit interviews and

a review of various off-the-shelf materials. This report provides more detailed observations and

strategic recommendations related specifically to retention, based on an analysis of the retention

patterns of the freshman and transfer cohorts that enrolled from fall 2005 through 2009.

It is important to note that pulling this retention data file together represented a

significant effort for the campus, in large part because the data are stored in so many different

systems. The file needed to be re-pulled several times in order to ensure the data were being

drawn from the most accurate source, and even the final file still had the following limitations:

● Attempted hours were not available for most records.

● Earned credits were cumulative and included credits not earned at CCNY.

● Transfer GPA was not available for most students because of problems with the way

entering GPA data are stored.

● Institutional aid data for returning students were eventually pulled from student

account records, but it was not possible to separate out different types of institutional

aid with any accuracy.

2



● It was not possible to distinguish between Macaulay and CCNY Honors students.

Although the data file included flags for both, almost all students with the Macaulay

flag also had the CCNY Honors flag.

● Many fields ideally provided for retention analysis are simply not captured by CCNY

including legacy status, first generation, extracurricular participation, and the college

they transferred to (from the National Student Clearinghouse). Consequently, it was

not possible to test some of the hypotheses about retention expressed by campus

members during the site visit.

Clearly, Ed Silverman is to be commended for his diligent efforts in providing the

requested data. However, if CCNY is to continue to conduct detailed retention analysis moving

forward, consideration must be given to how to improve data capture protocols and streamline

the reporting process.

3



Retention Analysis

Retention to Graduation by Cohort

S&K first analyzed overall retention rates from year to year for each cohort in order to

understand at which transition points CCNY experienced the most significant attrition. As can

be seen in Attachment #1a, approximately 20% of each freshman cohort was lost by the fall of

their second year (term 3). Note that the most recent cohort is an exception, when only 17% were

lost. Another 14-19% was lost by the fall of the third year (term 5). Between term 5 and term 7

another 7-9% was lost. Clearly, the biggest losses occur in the first two years of enrollment.

Consequently, S&K focused on those two transition points for more detailed analysis.

Similar patterns are found for transfers, although the losses between term 1 and term 3

have been larger than for freshmen, averaging 30%. (See Attachment #1b.) Then, another 10-

15% of each transfer cohort was lost between term 3 and term 5. Losses were minimal after that

point. For transfers, therefore, the focus was placed on the term 1 to term 3 transition. (Note:

The cohort sizes and retention rates differ somewhat from those reported in off-the-shelf

materials produced by CCNY. See Attachments #2a and #2b. However, the differences are

not material, and S&K believes that they are most likely a function of differences in when the

data were pulled.

4



First to Second Year Retention for Freshman Cohorts

Table Analysis

As a first step in understanding the factors impacting the retention of first time freshmen

to their second year, S&K examined retention rates for all five cohorts combined, segmented by

various subpopulations. As can be seen in Attachment #3, retention during the period under

study has been lower for Caucasian students than for students of color, which differs from

national trends. As is often the case, retention of out-of-state students is lower than for in-state

students. It is also lower than for international students. Students who participated in athletics in

their first year of enrollment have much higher retention to term 3 than non-athletes (91% versus

79.3%).

Retention of students achieving a term 1 GPA of less than 1.5 is much lower than for

those with higher GPAs. Consequently, a 1.5 GPA was used as a break point for exploring

retention through predictive modeling, as will be discussed in the next section of this report.

High school GPA is also strongly correlated to retention, although SAT is not. (Note also that

SAT is missing for many students in the cohorts under study.) Students intending to major in

engineering, the Sciences, Social Science, and Medicine have higher retention rates than students

in other academic divisions or undecided as to major. Honors participants were also more likely

to retain to term 3. However, these patterns were explored in more detail with predictive

modeling to better understand the influence of major and honors holding all other factors (e.g.,

student quality profile) constant.

5



There were no consistent retention trends by need, EFC, or grant level although, overall,

aid filers are more likely to retain than non-aid filers. It is also important to note that retention

rates do not decline as unmet need (defined as need minus all grants) increases, even when

unmet need rises above $12,000. Retention rates do generally decline as admit phase increases,

although the trend reverses for students admitted in Phases 13 and 14. Retention rates also are

higher for students who listed CCNY as their first choice on the CUNY admissions application.

Predictive Modeling

In order to better understand the influence of various factors on retention behavior,

holding all other factors constant, S&K focused in on students with term 1 GPAs of 1.5 or

higher, as these students would not have been facing academic dismissal. The model predicting

retention to term 3 for students with at least a 1.5 GPA can be found in Attachment #4. The

statistically significant variables in the model are explained in the table below. Note that

applying for aid as well as levels of grant, need, and unmet need were not statistically significant

drivers in this model. This finding dispels the hypothesis expressed by some on campus that

students not eligible for scholarships, Pell, and TAP leave because they can no longer afford

CCNY.

Although not listed in the table, it is also important to note that students intending to

major in engineering, medicine, biology, and psychology were all more likely to retain to term 3

than other majors, at least among students achieving at least a 1.5 GPA in term 1. Note also that

students in earlier cohorts were all less likely to retain than students in the fall 2009 entering

cohort, holding all other factors constant.

6



Variable

Marginal Effects

Calculations Explanation

Term 1 GPA 0.0583

For every additional point in Term 1 GPA (e.g., 2.5 versus

3.5), retention increases by 5.8%.

HS GPA 0.0027

For every additional 10 points on high school GPA (e.g.,

90 versus 80), a student is 2.7% more likely to retain to

Term 3.

SAT MV score 0.000065

For every 100 points on the SAT, a student is less than

1% more likely to retain.

Out-of-state -0.1351

Freshmen from out-of-state are 13.5% less likely to retain

to Term 3

Students of Color 0.0396

Students of color are 4% more likely to retain than

domestic Caucasians.

Participated in

Athletics 0.1049

Freshman athletes are 10.5% more likely to retain than

non-athletes.

SEEK admit 0.0248

SEEK admits are 2.5% more likely to retain, holding all

other factors constant.

Macaulay Honors 0.0480

Macaulay Honors participants are 4.8% more likely to

retain to Term 3.

CCNY 1st choice 0.0248

Admits who list CCNY first on the CUNY admissions

application are 2.5% more likely to retain to Term 3.

Significant Drivers in Term 3 Retention Model for Freshmen with Term 1 GPAs 1.5+

Clearly the special attention students achieve in SEEK is having a positive influence on

retention, once student quality profile differences are accounted for. This program, therefore,

could serve as a model for other programs intended to support academically at risk individuals.

In addition, involvement in special academic or cocurricular programs, like honors and athletics,

positively influences retention, which suggests that programs which connect students to other

students (such as peer-led team learning) should be expanded.

Note: Some on campus expressed concern that requiring a 3.5 GPA for renewing the

Macaulay Honors scholarship might be having a negative impact on retention of these students.

7



Consequently, S&K examined yield rates by term 1 GPA for Macaulay Honors students versus

all others. As can be seen in Attachment #5, it is the case that retention rates are lower for

Macaulay students with GPAs below 3.0 than for other students with those GPAs; however,

there are few students that fall into those categories, and the opposite is true for Macaulay

students with GPAs of 3.0 to 3.49. Although these students also could be facing the loss of their

scholarship, their retention rates are higher than for other students with similar GPAs.

Another hypothesis mentioned during the site visit was the idea that when CCNY was not

a student’s first choice those students move on to other institutions after establishing a good GPA

at CCNY. That hypotheses is somewhat supported by the fact that retention rates are higher for

those listing CCNY first on their application, which suggests that CCNY needs to continue to

“recruit” students not listing CCNY as a first choice, even after they enroll.

Even in the “achiever” model, term 1 GPA has a substantial impact on retention behavior.

Consequently, academic support services are critical, and should be mandatory for those most at

risk, which is not currently the case except for athletes and students in SEEK and SSSP. In order

to provide a clear definition of those who are academically at risk, S&K next developed a model

to estimate which factors contribute to students achieving a term 1 GPA below 1.5. As can be

seen in Attachment #6 and the table below, high school GPA is the most influential factor in

term 1 performance. Note also that populations required to take advantage of tutoring (athletes

and SEEK students) are less likely to do poorly in their first term, holding all other factors

constant. Although not listed in the table, it is also important to note that students in engineering,

8



medicine, and biology are all approximately 3% more likely to receive a low term 1 GPA than

students in other majors, holding everything else constant.

Variable

Parameter

Estimates Explanation

Need minus grant -0.0023

For every $1,000 in unmet need, students are < 1% less

likely to have a low Term 1 GPA.

High School GPA -0.0099

For every additional 10 points in GPA, students are 9.9%

less likely to have a low Term1 GPA.

Applied for Aid -0.03328

Students who apply for aid are 3.3% less likely to have a

low Term 1 GPA

Male 0.01445 Men are 1.4% more likely to have low term 1 GPAs

Athletic

Participation -0.03803 Athletes are 3.8% less likely to have a low term 1 GPA.

SEEK admit -0.02983

SEEK participants are 3% less likely to have a low term 1

GPA.

Macaulay Honors -0.08378

Macaulay honors students are 8.4% less likely to have a

low term 1 GPA, holding all other factors, such as quality

profile, constant.

CCNY first choice -0.0172

Students listing CCNY first on their CUNY admissons

application are 1.7% less likely to have a low Term 1

GPA.

Significant Drivers Influencing Term 1 GPA Below 1.5

This suggests that freshmen who enter with high school GPAs below 75 and are not

already in SEEK or participating in athletics should be targeted for early intervention, especially

if CCNY was not their first choice institution.

Second to Third Year Retention for Freshman Cohorts

Table Analysis

For this analysis, four cohorts (2005 through 2008) were combined in order to explore

retention to term 5 for those who made it to term 3 by subpopulation. (See Attachment #7a.)

Although living in the residence halls did not positively impact first year retention, dorm

9



residents who make it to term 3 are more likely to retain to term 5 than commuters (86%

retention versus 78%). Term 1 GPA and high school GPA continue to be strongly correlated to

retention, even for students who are still enrolled in term 3. (Note: This would support the

hypothesis expressed by some on campus that the reason so many students are lost after

making it to their second year is that CCNY is slow to dismiss students who don’t perform well

in their first year.)

Students who intended engineering, sciences, and medicine upon entry continue to have

stronger retention rates to term 5 than other majors; however, retention in the social sciences is

lower than average from term 3 to term 5, where it was higher than average between term 1 and

term 3. Humanities majors, on the other hand, are now tied with engineering and the sciences for

the second highest retention rate to term 5. Some on campus hypothesized that retention rates

were negatively impacted when students were unable to enter their desired major. Certainly, as

can be seen in Attachment #7b, retention rates to term 5 are lower for students who are still in

Gateway (undecided) as of term 3. They are particularly low for students who initially intended

to major in engineering but are still in Gateway by term 3. (Note, however, that retention rates

are not as low for students in Gateway to Engineering as they are for intended engineering

majors in Gateway proper.) The low retention rates for Gateway students are not just a function

of performance. As can be seen in Attachment #7c, retention rates from term 3 to term 5 for

Gateway and Gateway to Engineering students are lower than for students in other divisions even

when comparing across similar term 3 GPA bands.

10



As was the case with term 1 to term 3 retention, the retention of students from term 3 to

term 5 does not drop sharply as need or unmet need increase. There are a handful of students

who receive increases to their grants after their first year of enrollment. Retention rates to term 5

are very high for these students; however, term 3 enrollees with similar term 3 GPAs who did not

receive additional funding also have high retention rates. (See Attachment #7d.)

Although it was not possible to compare attempted credits to earned credits, S&K also

examined retention rates by the cumulative earned credits students had achieved by the end of

term 3. One would expect a full-time student to have 36-45 credits by this time, but clearly there

are many students with less than that range accumulated. As was hypothesized by some on

campus, there is a strong correlation between the number of credits earned and retention rates.

(See Attachment #7e.) However, without being able to compare to attempted credits, it is not

possible to know whether this is a function of students having failed to complete courses they

attempted or simply having registered for fewer credits in their first three semesters. Also note

that these cumulative credits include AP credits and any credits transferred in from other

institutions. Consequently, it is also likely that there is a strong correlation between cumulative

credits and performance in high school.

Predictive Modeling

S&K next developed a predictive model to understand the term 3 to term 5 transition.

This model identified the factors influencing retention to term 5 for students with term 3

cumulative GPAs of 2.0 or higher. (See Attachment #8 and the table below.) Interestingly, total

grant was statistically significant in this model, although the impact on probability of retaining

11



was quite small. Although all students in this model have GPAs of 2.0 or higher, term 3 GPA

was still a statistically significant driver. As was the case in the model that predicted retention

from term 1 to term 3, students of color, honors students, athletes, and those listing CCNY first

on their admissions application were all more likely to retain than other students, holding all

other factors constant. In addition, students undecided as to major were less likely to retain than

other majors while engineering, bio-medical, and psychology majors were all more likely to

retain than other majors.

Variable

Marginal Effects


Total Grants 0.004689

For every $1,000 in total grants, students are < 1% more

likely to retain to Term 5.

Term 3 Cumulative

GPA 0.0453

For every additional point in Term 3 cumulative GPA (e.g.,

2.5 versus 3.5), students are 4.5% more likely to retain to

Term 5.

International 0.1042

International students are 10.4% more likely to retain to

Term 5 than domestic Caucasians.


Students of color are 3.8% more likely to retain to Term 5

than domestic Caucasians.

Participated in

Athletics 0.0952

Freshman athletes are 9.5% more likely to retain to Term

5 than non-athletes.

Macaulay Honors 0.1567

Macaulay Honors participants are 15.7% more likely to

retain to Term 5 than all other students.

Term 3 Declared

Major: Undecided -0.0495

Students in an Undecided major are 5% less likely to

retain to Term 5 than students in other majors not listed in

this table.

Term 3 Declared

Major: Engineering 0.0281

Students in the Engineering major are 2.8% more likely to


this table.

Term 3 Declared

Major: Bio-Medical 0.2297

Students in the Bio-Medical major are 23% more likely to


this table.

Term 3 Delcared

Major: Psychology 0.1065

Students in the Psychology major are 10.7% more likely

to retain to Term 5 than students in other majors not listed

in this table.

CCNY 1st choice 0.0248

Students listing CCNY as their first choice on the CUNY

admissions application are 2.5% more likely to retain to

Term 5.

Year: 2005 -0.0356

Students in 2005 are 3.6% less likely to retain to Term 5

than students in all other cohorts.

12



This suggests that connecting a student to a major by term 3 is very important to

enhancing retention.

First to Second Year Retention for Transfer Cohorts

Table Analysis

Unlike for freshmen, first year retention for transfers declined for the fall 2009 cohort,

although retention rates are still stronger than they were for the fall 2005 and fall 2006 transfer

cohorts. (See Attachment #9.) Another difference between freshman and transfer retention

patterns is that, for transfers, there is no difference in first year retention rates for domestic

students of color and Caucasian students. However, as was the case for freshmen, transfer

athletes retain at a higher rate than non-athletes, and term 1 GPA is strongly correlated to

retention.

Transfers interested in engineering, humanities, nursing, and medicine have the highest

first year retention rates. It is also important to note that younger transfers (19 or younger) have

higher retention rates than other transfers.

Incomplete aid filers and transfers with $0 EFCs and EFCs above $45,000 have lower

retention rates than those with EFCs of $1 to $45,000. However, as was the case with freshmen,

transfer retention rates do not decline as unmet need (need after grant) increases. In fact,

retention rates for transfers with unmet need above $8,000 are substantially higher than for

transfers with less unmet need. Again, this suggests that additional investments in financial aid

would not contribute to retention goals.

13



Predictive Modeling

To examine the factors influencing the retention of transfers to term 3, S&K focused on

those with term 1 GPAs of at least 1.75, since less than half of the students with GPAs below

that retained. As can be seen in the table below and Attachment #10, unmet need (defined as

need minus all grant) plays a small role in retention although transfers who apply for aid are 7%

less likely to retain than non-aid filers. This may suggest that concerns about financing influence

transfers more than freshmen, regardless of how well their need is being addressed. The fact that

older transfers are also less likely to retain suggests that life factors (such as financial concerns)

may be influencing transfers more than freshmen.

Variable

Marginal Effects


Term 1 GPA 0.0882

For every additional point in Term 1 GPA (e.g., 2.5 versus

3.5), students are 8.8% more likely to retain to Term 3.

Need Minus All

Grants 0.0070

For every $1,000 in unmet need, students are < 1% more

likely to retain to Term 3.

Applied for Aid -0.0714

Students who apply for aid are 7.1% less likely to retain to

Term 3.

Out-of-state -0.0822

Freshmen from out-of-state are 8.2% less likely to retain

to Term 3


Students of color are 5.1% more likely to retain to Term 3

than domestic Caucasians.

Male 0.0276

Male students are 2.8% more likely to retain to Term 3

than female students.

Intended Major:

Engineering 0.1012

Engineering majors are 10.1% more likely to retain to term

3 than all other majors.

Age: 25 or older -0.0739

Students who are 25+ years old are 7.4% less likely to

retain to Term 3 than students who are < 25 years old.

Significant Drivers in Term 3 Retention Model for Transfers with Term 1 GPAs 1.75+

Similar patterns were seen in the model that estimates which factors contribute to

transfers earning a low term 1 GPA. (See Attachment #11 and the table below.) Transfers who

14



applied for aid were more likely to perform poorly in term 1—just the opposite of what was

found for freshmen. Interestingly, transfers receiving grant assistance were more likely to

perform poorly, while those with more unmet need were less likely to perform poorly. (Note,

however, that neither of these two factors had a large influence on the likelihood of poor

performance.) As was the case with freshmen, athletes and those listing CCNY as their first

choice institution were less likely to have a low term 1 GPA, while biology majors were more

likely to have poor performance in term 1.

Variable

Marginal Effects


Total Grants 0.005839

For every $1,000 in total grants, students are < 1% more

likely to have a low term 1 GPA.

Need Minus All

Grants -0.0084

For every $1,000 in unmet need, students are < 1% less

likely to have a low Term 1 GPA.

Applied for Aid 0.0491

Students who apply for aid are 4.9% more likely to have a

low Term 1 GPA.


Students of color are 6.5% more likely to have a low Term

1 GPA than domestic Caucasians.

Male 0.0220

Male students are 2.2% more likely to have a low Term 1

GPA than female students.

Physician

Assistant -0.0966

Physician Assistant majors are 9.7% less likely to have a

low Term 1 GPA than other majors not listed in this table.

Intended Major:

Biology 0.0843

Biology majors are 8.4% more likely to have a low Term 1

GPA than other majors not listed in this table.

Actual Housing:

Commuter -0.0641

Commuter students are 6.4% less likely to have a low

Term 1 GPA than resident students.

CCNY First

Choice -0.0546

Students listing CCNY as their first choice are 5.5% less

likely to have low Term 1 GPAs.

Participated in

Athletics -0.1190

Students who participate in athletics are 11.9% less likely

to have low Term 1 GPA that students who do not

participate in athletics.

Year: 2006 0.0510

Students in fall 2006 cohort are 5.1% more likely to have

low Term 1 GPAs than students in fall 2005, fall 2008, and

fall 2009 cohorts.

Year: 2007 -0.0500

Students admitted in 2007 fall cohort are 5% less likely to

have low Term 1 GPAs than students admitted in fall

2005, fall 2008, and fall 2009 cohorts.

Significant Drivers Influencing Term 1 GPA < 1.75

15



Recommendations

1.) Recommendation

In order to continue to conduct detailed retention analysis, CCNY needs to begin to

routinely capture key data elements on entering cohorts and store the data in a format

easily accessible for analysis.

Comment:

As was mentioned in the Introduction, there were a number of limitations in the data file

provided to S&K that will need to be addressed if CCNY is to be able to annually examine

retention patterns and determine if intervention strategies are effective. In particular, attempted

hours, transfer GPA, extracurricular participation, participation in academic support services, and

the college to which students transfer should begin to be captured routinely. In addition, the data

need to be organized in a comprehensive retention database for ongoing analysis.

2.) Recommendation

Given that the cocurricular data that were available suggest that involvement with

other students has a positive influence on retention, programs that connect students to each

other, such as the new peer-led team learning initiatives, should be expanded.

Comment:

Programming to connect students is particularly important at institutions with large

commuter populations, where connections that occur in residential halls are limited.

16



3.) Recommendation

CCNY needs to continue to “recruit” students even after they enroll by highlighting

faculty honors, the success of recent graduates, and other points of pride in

communications with current students.

Comment:

Because the retention models found that retention rates are higher for both freshmen and

transfers listing CCNY as their first choice institution on the admissions application, building a

sense of pride in the institution among current students through highlighting CCNY’s academic

strengths and cachet among employers as well as graduate schools is important.

4.) Recommendation

Freshmen who enter with high school GPAs below 75 who are not already in SEEK

should be targeted for required tutoring and mentoring, especially if CCNY was not their

first choice institution or they are in challenging majors.

Comment:

Using these factors, which emerged as significant drivers in the modeling predicting low

term 1 performance, to identify students for early intervention will enable CCNY to have a

greater impact on results than waiting for evidence of poor performance to emerge.

17



5.) Recommendation

The financial aid office should reach out to transfer aid applicants, particularly

those who are 25 or older, to provide additional financial counseling to address concerns

these students may have about financing their education.

Comment:

The amount of need and aid students had did not appear to have much influence on

retention, thus providing additional financial aid per se is not recommended. However, the fact

that applying for financial aid had a negative influence on retention and performance for

transfers (but not for freshmen) suggests that transfer behavior may be being influenced by

concerns about financing. Providing additional financial counseling targeted to these students,

therefore, is a pilot worth testing.

6.) Recommendation

The career services office should conduct targeted outreach to students still in

Gateway or Gateway to Engineering (undecided as to major) by term 3, offering interest

testing and counseling to help them select a major.

Comment:

The model estimating retention to term 5 clearly showed that undecided students are less

likely to continue enrollment than students who have selected a major, holding all other factors

constant. Therefore, more intense work to help them identify their academic interests is

suggested by the data.

18



Conclusion

Although retention analysis and predictive modeling did not suggest that increases to

financial aid would have much of an impact on retention, other targeted initiatives emerged from

the analysis, related to mandating academic support services, connecting students to each other,

helping undecided students select a major, providing financial counseling to transfers, and

continuing to “recruit” students for whom CCNY was not a first choice institution.

KK/JS/DG:sp Attachment

Documents

Table of Contents Introduction 1 Retention Analysis 3...retention through predictive modeling, as will be discussed in the next section of this report. High school GPA is also strongly