Columbia College 2007 General Education Assessment Report

History

Columbia College began administering the Academic Profile (AP) test, a product of the Educational Testing Service (ETS), to graduating seniors in 1997 as a means by which to assess outcomes for the general education curricu lum. The College began administering the AP as a pre-test to first-time day-program freshmen enrolled in INCC 111 and HNRS 110 beginning in the fall semester of 1999. The AP was discontinued by ETS in the summer of 2006 and replaced with the Measure of Academic Proficiency and Progress (MAPP™). All exams administered beginning with the fall semester, 2006, are MAPP. There are two versions of the exam, standard and abbreviated. Columbia College uses the abbreviated version.

Test Information Quoting from the ETS website, "The Measure of Academic Proficiency and Progress (MAPP) test is a measure of college-level reading, mathematics, writ ing, and critica l thinking in the context of the humanit ies, social sciences, and natural sciences. The MAPP test is designed for colleges and universities to assess their general education outcomes, so they may improve the quality of instruction and learning. It focuses on the academic skills developed through general education courses, rather than on the knowledge acquired about the subjects taught in these courses."

AP Result s in Culminating Experience Classes In spring of 2006 the AP was administered to 99 graduating seniors in the Columbia College day program and 93 graduating seniors at AHE sites, including the evening program. Results are show in Table 1 below for institutional comparative data.

Day AHE

1

Raw Raw National Standard

Score +/- Std Dev Score +/- Std Dev Average Deviation Mean Score 451 .07 0.20 439.01 -1 .09 449.24 9.36 Critical Thinking 113.15 0 .25 109.13 -1.34 112.52 2.53 Reading 120.68 0 .26 118.28 -0.59 119.94 2.83 Writing 115.62 0 .07 113.40 -1.11 115.48 1.87 Math 115.05 0.17 111 .56 -1.16 114.60 2 .61 Humanities 116.29 -0 .02 113.57 -1 .13 11 6.35 2.45 Social Sciences 115.48 0.31 112.99 -0.80 114.79 2.26 Natural Sciences 116.38 0.19 113.81 -0.96 115.96 2.25 Sample Size 99 93

Table 1 Comparison of Day and AHE AP Scores with National Averages

Norms were derived from students at baccalaureate liberal arts colleges who had completed more than 90 semester hours. Results for the day program are near national means with all but one area above national averages. AHE scores ranged from 0.63 to 1.23 standard deviations below national norms. Figure 1 below shows the same data graphically.

2

Score Reading \Nriting Math Humanities

Sciences Sciences

0.20 0.25 0.26 0.07 0.17 -0.02 0.31 0.19

-1.09 -1.34 -0.59 -1 .11 -1.16

Figure 1 Comparison of Day and AHE AP Scores with National Averages

MAPP Results in Culminating Experience Classes In fall of 2006 the MAPP was administered to 117 graduating seniors in the Columbia College day program and 73 graduating seniors at AHE sites, including the evening program. One exam from the day program and three exams from t he AHE program were invalidated. Results are show in Table 2 below for institutional comparative data. According to ETS, comparative data exclusive to the MAPP is not yet available. However, "Comparative data from Academic Profile (the former ETS and College Board General Education Test) is comparable to the MAPP test as a result of equating MAPP to Academic Profile. "1 Results are shown in Table 2 for AP institutional comparative data.

Day AHE Raw Raw National Standard

Score +/- Std Dev Score +/- Std Dev Average Deviation Total Score 452.74 0.37 443.90 -0.57 449.24 9.36 Critical Thinking 114.56 0.81 111 .99 -0.21 112.52 2.53 Reading 120.60 0.23 118.62 -0.47 119.94 2.83 Writing 115.79 0.17 114.26 -0.65 115.48 1.87 Math 114.79 0.07 112.32 -0.87 114.60 2.61 Humanities 116.61 0.11 115.90 -0.18 116.35 2.45 Social Sciences 115.52 0.32 113.96 -0.37 114.79 2.26 Natural Sciences 118.03 0.92 115.12 -0.37 115.96 2.25 Sample Size 11 7 73

Day results are uniformly above the national norms with critical thinking and natural sciences approaching a full standard deviation above the average. AHE results range from a best of 0.21 standard deviations below the national average for critical

1 ETS web site at http: /lets .orq/oorta 1/site/ets/men uitem .1488512ecfd5b8849a 77b 13bc39 21 509/?vq nextold - f54aaf5e44df 4010VqnVCM10000022f95190RCRD&vgnextchannel=e23646f1674f4010VqnVCM10000022f95190RCRD

thinking to a worst of 0.87 standard deviations below the national average for math. The same data is shown graphica'lly in Figure 2.

c 0

; C'll

-~ c "E C'll "'0 c J!l U)

Figure 2 Comparison of Day and AHE MAPP Scores with National Averages

Results for Entering Freshmen The MAPP was administered to 158 freshman day students in the fall of 2006. Twenty-five student scores were invalidated leaving a sample size of 133. Results are shown in Table 3.

Raw National Standard Score +/- Std Dev Average Deviation

Total Score 440.11 0.02 439.98 7.39 Critical Thinking 110.67 0.71 109.44 1.73 Reading 116.65 0.10 116.40 2.54 Writing 113.73 0.22 113.33 1.78 Math 112.04 -0.38 112.80 2.00 Humanities 113.96 0.45 113.20 1.69 Social Sciences 112.58 0.42 111 .84 1.76 Natural Sciences 114.00 0.31 113.44 1.82 Sample Size 133

Table 3 Comparison of Entering Freshmen AP Scores with National Averages

As a group, the entering freshman class of fall 2006 scored slightly above the national average in all areas except mathematics. The same data is shown graphically in Figure 3. Norms were derived from students at baccalaureate liberal arts colleges who had no semester hours of cred it.

3

0.80

0.60 c 0 0.40 ;::: 111 ·:;

0.20 CD 0

"E 0.00 111

"0 c -0.20 s

U)

-0.40

-0.60 Total Score

Q itical Thinking

0.71

Reading

0.10

Math Social Natural

Humanities Sciences Sciences

Writing

0.22 -0.38 0.45 0.42 0.31

Figure 3 Comparison of Entering Freshmen AP Scores with Nat ional Averages

Results are positive with the Columbia College entering class scoring above national averages in all areas.

Longitudinal Results Figure 4 compares the performance of the entering freshmen classes for 2005 and 2006. The 2005 class performed better in most areas.

Figure 4 Comparison of Entering Freshmen Scores for 2005 and 2006

Figure 5 compares the performance of day program graduating seniors in 2004, 2005 and 2006. There were 124, 130, and 117 students respectively in each group. Nineteen of 24 indicators were above national averages and no indicator was below national averages more than once. Only AP scores are included for the 2006 data.

4

Figure 5 Comparison of Day Program Graduating Seniors for 2004 - 2006

Figure 6 compares the performance of AHE graduating seniors for 2005 and 2006. There were 227 students in the 2005 group and 93 in the 2006 group. Only AP scores are reported for 2006.

0.00

-0.20 c 0 -0.40 ::= ca

-0.60 > G)

c -0.80 "E! -1.00 ca 't:l c -1.20 .!!! en -1.40

-1.60 Total Oitical

Writing Math Hu .t. Social Natural

mam tes S . I Sc. I I ctences tences Re ding

Score Thinking J a

c 2oos -0.75 -0.79 -r

-0. -0.80 -0.78 -0.73 -0.65 61 -0.71

•2006 -1.09 -1.34 I -0. 59 -1 .11 -1.16 -1.13 1 -o.a_o~_-_o._96__.j

Figure 6 Comparison of AHE Graduating Seniors for 2005 and 2006

Value Added Results were available for 28 students in the day program who took the AP both as freshmen and as seniors. The average improvement in score was 10.11. Using current norms for comparison, the average gain from freshman to senior is 9.26. The improvement in our students is slightly more than one-tenth of a standard deviation better than the national average.

Conclusions

5

The Columbia College day program attracts bright students who, as a group, perform generally better than their peers. This advantage is maintained and perhaps increased slightly during their academic careers at Columbia College. The quality of

students in the day program has improved markedly in t he past few years as admission standards have been raised .

6

Students graduating from the Columbia College AHE program consistently under­perform their peers. The AHE program is largely open admission and many students who graduate from the AHE program have a significant number of transfer hours. However, despite these confounding factors, it is apparent that rigor and content need to be carefully examined across the AHE system. This data need to be placed on the agenda of future faculty integration conferences for dissemination and discussion. AHE must investigate the issue of general education transfer credit among their student population so that this factor may be appropriately considered when evaluating assessment resu lts. AHE should develop plans and measurable goals to narrow the gap between the performance of the ir students and those in the day program . In particular, those sites that consistently under-perform should be identified and targeted for special action .