Assessment Overview - Arapahoe Community College · PDF fileAssessment Overview ... ENG 060 to ENG 090 into the transfer-level ENG 121 course. ... week of the semester and then the

Assessment Overview

Discipline/Program Name English & Reading Developmental Studies Assessment Year 2009-2010

Learning Outcome Outcome Type Methodology n History Benchmark Results

Strength of Results

1. Communication (essay

writing skills) SLO Pre-Post Test 538 8

Student scores increase by +1.5 over the semester.

+1.4, +1.8, 1.6, +2.7 Strong

2. Reading Proficiency Discipline Pre-Post Test 229 N/A

70% proficiency in each skill set

Benchmark met and exceeded in most cases Strong

3. Retention and Success

Rates Other Institutional Research 1,928 2

60% student success rate

ENG 030: 65%, ENG 060: 63%, ENG 090: 59%, REA 030: 82%, REA 060: 69%, REA 090: 72%

ENG : Neutral, REA: Strong

4. n/a (as part of the English

Department, assessment

report, which also

assessed four Learning

Outcomes)

Describe the Learning Outcome That You Have Measured

SLO/GE, Discipline or Other

Pre-Post Test, Judged Competition, Embedded Questions, Rubric Graded Essay

Number of Students Assessed

# of Years This Outcome Has Been Assessed

Measurement Standard

Report the Results of Your Data Analysis

Strong: Exceeds Benchmark Neutral: Meets Benchmark Weak: Misses Benchmark

Recommendation: Complete this Assessment Overview Table after you have completed your Assessment Summary in the following template.

Program / Discipline Assessment Report

Program/Discipline: Liberal Arts and Professional Programs: Developmental English and Reading Responsibility: Jamey Trotter: English Michelle Van De Sande: Reading

Program/Discipline's Mission Statement:

The Developmental Studies Program’s mission in a learning-centered college is to provide basic academic skills to prepare students for

success and retention in transfer core courses and vocational programs. The Developmental Studies Program is committed to improving

student learning through continuous assessment which guarantees student access to quality education, job readiness, and personal

satisfaction. The purpose is simply to “develop” students to at point whereupon they are ready and able to succeed in college transfer

classes.

Program/Discipline's Assessment History:

By using the assessment process as an evaluative technique, how has it previously affected your program's curricula and/or

teaching strategies?

By having a beginning-of-the-semester and an end-of-the-semester in-class essay examination, our Developmental Studies classes

have become more academically rigorous because faculty understand the importance of scores improving over the course of the

semester, as do students.

Over the past five years, the faculty of Developmental Studies has responded to assessment data from September 2004 that found

that students’ skills of summarization were weak. We revised curriculum to increase the number of formal summaries to a minimum

of five and promoted more consistency in grading criteria in all levels of composition instruction. Beginning Fall 2006, ENG 090

courses required a minimum of five summaries, and REA 090 faculty would emphasize summary writing in activities for reading

comprehension.

Developmental and transfer-level course faculty meet together at the beginning and end of each term to score pre- and post-writing

blue book essays. Blue book essays contain a summary as well as a response to a college-level article from the newspaper.

Developmental course instructors are also encouraged to do a mid-semester blue book essay to prepare students for the end-of-

semester post-writing essay.

By using the assessment process as an evaluative technique, what changes to student learning have been noted?

Students have found that they are more confident about their writing and grammar skills after taking developmental

courses. Students improve their blue book diagnostic essay scores from the beginning to the end of the semester,

demonstrating that their skills in reading, summarizing, generating a thesis, organizing and supporting a thesis, and

using correct grammar and mechanics do improve over the course of the semester. Students therefore have concrete

proof that their writing abilities have increased, relating back to point one – confidence in one’s reading and writing

abilities is essential to succeeding in college and indeed life.

Another impact we’ve observed on student learning is that the students understand right up front that the

developmental class they are has high academic standards (by engaging in an essay exam on the first or second day of

the semester).

Students understand they are able to transfer their new ability to write an in-class essay to other classes (e.g. history).

What unintended consequences, if any, have occurred because of the assessment process?

Full-time and adjunct faculty come together at both the beginning of the semester and the end of the semester to

scores the hundreds of Blue Book essays; this provides an exceptional professional development opportunity for both

entities, as well as a greater sense of collegiality within the department. Further, before we score the essays, we spend

lots of time norming, which results in a greater level of equality in grading across the department.

We found that while students may pass a course in the Developmental Studies sequence (ENG 030, 060, 090; REA

030, 060, 090), they may not decide to move on and take the next class in the sequence.

Instructors have found that our students may need to take a lower level Developmental course to prepare them for

ENG060 / REA060 or ENG090 / REA 090; hence, we added ENG030 / REA 030 to our departmental course

offerings.

Instructors have found that our students that fall below the minimum cut-off score in ENG 030 and REA 030 do not

have the ability to benefit in a 030 course. Our department will begin enforcing the cut off score of 25 in these

courses, and we are exploring other options for these students.

Who receives information about your department's assessment and why? (Please note if you plan on altering either of these

items for the coming year.)

We share this report and ask for feedback about our results from all instructors who teach Developmental courses and

with the Chair of the English and Developmental Studies. This has not changed.

Part 1: Previous Academic Year Assessment Summary

Previous Academic Year: Previous Academic Year: 2009-2010

Last year we used three methods to assess our Developmental Studies program:

1. The blue book diagnostic essay

2. Reading pre- and post-test

3. Retention data generated from Institutional Research

The Blue Book Diagnostic Essay

We found that students blue book scores did increase from the beginning to the end of the semester, and we

met our benchmark that student scores would increase by 1.5 points or more on a twelve-point scale.

However, it is perhaps a concern that students student’s scores leaving the lower-level class are NOT equal

to or greater than the students entering the next level (for example, a student leaving ENG 060 would score

equal to or higher than a student entering ENG 090).

Reading Pre- and Post-Tests

The reading pre- and post-tests underwent many changes for REA030 and REA090. For REA060 a pre and

post test has been established that consistently measures each skill set. As of spring 2010, the REA 030 and

REA090 pre and post tests have been established. Because of the nature of the program, we have now

found that the results are consistent. We have developed tests that effectively measures student growth and

skill deficiencies

Retention Data Generated from Institutional Research

Institutional Research captured data to show us how many students passed, failed, were passing and

progressing through our developmental sequence from ENG 030, ENG 060 to ENG 090 into the transfer-

level ENG 121 course. We found that while students might pass ENG 030, ENG 060 and ENG 090, fewer

students went on to enroll in the next course in the sequence. Since retention is one of our foremost goals,

most full-time developmental instructors added into their performance objectives for the year two retention

goals such as using the Early Alert system offered from the Advising office, making out midterm grade

reports for students, and encouraging students to enroll in the next developmental course in the sequence.

Outcome1 #: Communication Outcome Title: In-Class Blue Book Essay Exam, Pre- and Post-Semester

Outcome Type (choose one):

Discipline/Program X SLO/GE Other

If Student Learning Outcome/General Education

(choose one): X Communication Society and

Culture/Diversity and Global Awareness Problem

Solving/Critical Thinking Quantitative Reasoning

Technology Interpersonal Skills/Leadership and

Teamwork Aesthetics Values and Ethics

Information Management Personal Development

and Responsibility

Outcome Description:

Students in ENG 060 and 090 demonstrate the ability to respond to an (persuasive) article by

summarizing the article and then responding to it with a supported opinion.

Benchmark for success 1) Please specify what percentage of the sample size is

expected to meet or exceed your benchmark.

2) What is the rationale for choosing this measure?

Students in each type of class show progress from the beginning (pre) semester scores to the

ending (post) semester scores. In other words, we expect the students’ scores to increase,

demonstrating that learning has occurred. The benchmark is an improvement of one-point-five

points, or more, on a twelve-point scale.

Description of assessment process: 1) What assessment methods were used to measure this

outcome (i.e. pre/post test, portfolio review, etc.)?

2) How do these methods show students are learning?

3) What frequency is this outcome being measured (i.e.:

each semester, yearly, every other year, etc.) and why?

In both the Fall and the Spring semesters, ENG Faculty and Instructors come together in the first

week of the semester and then the second-to-last week of the semester (in conjunction with the

administering of the exam) to score hundreds of the exam booklets. ENG 060 and 090 exam

booklets are mixed together, so there is no differentiating of the classes. We use an 18-point scale

to score the essays, and we spend a lot of time norming the scale before we begin live scoring. All

essays receive two reads by two different people, in order to further standardize scoring. There are

4) How many students made up the sample size?

approximately 550 student scores over the academic year that comprise the sample size. An

increase in scores from the beginning of the semester to the end of the semester demonstrates

student learning.

Results What were the results of the assessment process? (List

results for each method, if more than one were used.)

FA ENG 060 scores: +1.486 from pre to post

SP ENG 060 scores: +1.883 from pre to post

FA ENG 090 scores: + 1.646 from pre to post

SP ENG 090 scores: +2.748 from pre to post

What did the department learn? 1) How did group performance compare to the

benchmark?

2) How does the data compare to the previous year, if

applicable?

3) If multiple measures were used, how do they compare

to each other?

1) Indeed, students’ scores improved from the beginning of the semester to the end, demonstrating

an improved ability to communicate through an improved ability to write an in-class essay exam, a

task that requires the student to read an article, understand it, demonstrate that understanding

through a written summary, and then respond to that article through an argument of their own (that

is appropriately supported).

2) The data from last year shows scores for the post-test for ENG 060 to be slightly higher at 7.32;

last year’s dip in post-060 scores, however, is offset by the increase in ENG 090 scores, which in

the previous year averaged out at 8.49.

3) N/A

Student performance summary 1) Based on the findings, how does the department rate

student performance in regards to this outcome (strong,

weak, or neutral)?

2) How does this assessment affect plans for this coming

year in terms of curricula, teaching strategies, and

assessment methods?

We’re going to classify these results as neutral to strong. The Spring 090 class was particularly

strong at an increase of 2.748, while the other numeric increases in scores are about what we

expect to see. When compared to the previous year’s data, the average scores for post ENG 090

demonstrates growth, from 8.49 in the previous year to 9.681 (averaged out) this year. This

increase in scores demonstrates, perhaps, a reflection in the department’s focus on academic rigor

in ENG 090. We plan to continue the in-class essay exam in our curriculum for the upcoming year

because it seems to be a way to keep our finger on the pulse of our student’s communication skills

(an SLO) in a quantifiable manner.

5.4395.078

7.639 7.3296.925 6.961

9.28510.077

0

2

4

6

8

10

12

FA 09 ENG 060 SP 10 ENG 060 FA 09 ENG 090 SP 10 ENG 090

Pre-Test

Post-Test

Outcome #2: Reading Proficiency Outcome Title: REA 030, 060, & 090 Pre and Post Test


X Discipline/Program SLO/GE Other

If Student Learning Outcome/General

Education (choose one): Communication

Society and Culture/Diversity and Global

Awareness Problem Solving/Critical

Thinking Quantitative Reasoning

Technology Interpersonal Skills/Leadership

and Teamwork Aesthetics Values and

Ethics Information Management Personal

Development and Responsibility


Students in all levels of reading classes are tested in the beginning (pre) and at the end (post) of the

semester via a standardized reading diagnostic that breaks reading skills into various categories

dependent on the level of the reading class.

Benchmark for success 1) Please specify what percentage of the sample size

is expected to meet or exceed your benchmark.


Students should test at 70% or above in each skill area, at the end of the semester. Scores reveal that

most students enter the semester at below 70% proficiency (thus the reason they tested into the class);

therefore, if we can students testing at an average of seventy percent or more, we will demonstrate

appropriate student academic growth.

Description of assessment process: 1) What assessment methods were used to measure

this outcome (i.e. pre/post test, portfolio review)?

2) How do these methods show students are

learning?

3) What frequency is this outcome being measured

(i.e.: each semester, yearly, every other year, etc.)

and why?


1.) In both the Fall and the Spring semesters, reading students are given a pre and post reading

assessment. The pre and post assessments measure the same skill areas.

2.) If the post assessment results show a higher score for the post assessments as compared to the pre

assessment, this is an indication the students have increased their understanding.

3.) This is an assessment that is given each semester.

4.) There are approximately 26-030 students’ scores, 90-060 students’ scores, and 115-090 students’

scores collected each semester during the academic year.

Results What were the results of the assessment process?

(List results for each method, if more than one were

used.)

36

72 73 74 68.583

91 88 86 88

0

20

40

60

80

100

Thesis/Main Idea Details Inferred MeaningVocabulary Total

Spring 2010REA 030 - Pre & Post Tests

Pre Test

Post Test

7568

48

79

38

90

45

6352

63.8

88

69

90

4250

81 82

5649

70.6

0102030405060708090

100

Fall 2009REA 060 - Pre & Post Tests

Pre Test

Post Test

6856 54

36 34

88

63 57 59

85 8096

46 54

81 8068 73

020406080

100120


Pre-Test

Post-Test


benchmark?

2) How does the data compare to the previous year,

if applicable?

3) If multiple measures were used, how do they

compare to each other?

REA 030

Students in Reading 030 showed considerable growth in each assessment area. This was done through

concentrated classroom instruction and homework reinforcement for these areas.

The reading department will continue to emphasize direct instruction and practice for the specific skill

areas measured.

Student Data: 26 students

Accuplacer Reading Comprehension Score Average (25-39): 35.30

Course Grade Average: 3.17 (10-As, 10-Bs, 3-Cs, 1-F, 2 -WD)

Pre-Test Score Average: 68.48/100

Post-Test Score Average: 88.40/100

REA 060

Students in Reading 060 showed considerable growth in all skill areas except Inference; these scores fell

slightly for the post-test, but still remained high. Overall the scores improved compared to fall semester.

The emphasis on improving these scores through professional development workshops targeted toward

specific skills proved successful.

The reading department will continue to work on student success specifically in the following areas:

62 67

16

81

60 5846

62

87 86

6579 73 73

85 80

0102030405060708090

100


Pre-Test

Post-Test

transitions and patterns of organization. This will be done through targeted practice in these areas.



Course Grade Average: 2.46 (13-As, 22-Bs, 19-Cs, 4-Ds, 7-Fs, 26-WD)

Pre-Test Score Average: 58.58/100

Post-Test Score Average: 73.13/100

REA 090

Students in Reading 090 showed considerable growth in all areas except Inferred Meaning. Inferred

Meaning remained about the same for the pre and post tests. The area that showed the greatest growth

was Organization/Transition; however, this will still remain a target area since the post-test score lower

at 65%.

The reading department will continue to focus on student growth for organization/transitions. This will

be done through direct instruction and practice. The department will continue to maintain the student-

centered practices for all other areas to maintain consistent growth.



Course Grade Average: 2.2 (16-As, 42-Bs, 18-Cs, 7-Ds, 23-Fs, 6-WD)

Pre-Test Score Average: 63.58

Post-Test Score Average: 80.44

Student performance summary 1) Based on the findings, how does the department

rate student performance in regards to this outcome

(strong, weak, or neutral)?

2) How does this assessment affect plans for this

coming year in terms of curricula, teaching

strategies, and assessment methods?

1.) The reading department would classify these results as average to strong. The REA030 results

showed a significant increase, REA 060 remained average while REA090 results were very strong. This

increase shows the department’s focus on teaching these skills in a very direct manner and increasing

formative assessment throughout the semester.

2.) The department plans to continue focus on these specific academic skills and to continually measure

progress through formative and summative assessments. We will hold department planning and best

practices workshops for each course prefix. These workshops will help to inform progress and keep all

instructors on the “same page” concerning teaching and assessment practices.

Outcome #: Success Rates Outcome Title: Student Success Rates: Personal Development and Responsibility


Discipline/Program X SLO/GE Other

If Student Learning Outcome/General

Education (choose one): Communication

Society and Culture/Diversity and Global

Awareness Problem Solving/Critical Thinking

Quantitative Reasoning Technology

Interpersonal Skills/Leadership and Teamwork

Aesthetics Values and Ethics

Information Management X Personal

Development and Responsibility


Students demonstrate a growth in Personal Development and Responsibility by succeeding in ENG

030, 060, and/or 090 AND/OR REA 030, 060, and/or 090. In order to succeed in any of these classes,

one must be responsible, and as well, by succeeding in a Developmental Studies class, one

experiences Personal Development.

Benchmark for success 1) Please specify what percentage of the sample size

is expected to meet or exceed your benchmark.


Sixty-percent of students to complete the course with a passing grade. Standard bell curve.

This is in line with the national standards for developmental studies students.

Description of assessment process: 1) What assessment methods were used to measure

this outcome (i.e. pre/post test, portfolio review)?

2) How do these methods show students are learning?

3) What frequency is this outcome being measured

(i.e.: each semester, yearly, every other year, etc.) and

why?


Institutional Research (IR) provides the below data.

Results What were the results of the assessment process?

(List results for each method, if more than one were

used.)

ENG 030

201010 201020 201030 AY Totals

Pass (A,B,C) -- 35 25 60

Not Pass (D, F) -- 13 6 19

Withdrew -- 7 7 14

TOTAL: -- 55 38 93

65% Students successfully complete ENG 030.

ENG 060

201010 201020 201030 AY Totals

Pass (A,B,C) 18 91 96 205

Not Pass (D, F) 3 41 39 83

Withdrew 1 21 16 38

TOTAL: 22 153 151 326

63% students successfully completed ENG 060.

ENG 090

201010 201020 201030 AY Totals

Pass (A,B,C) 58 230 189 477

Not Pass (D, F) 15 111 101 227

Withdrew 10 54 57 121

TOTAL: 83 395 347 825

59% of students successfully completed ENG 090.

REA 030

201010 201020 201030 AY Totals

Pass (A,B,C) -- 26 23 49

Not Pass (D, F) -- 6 0 6

Withdrew -- 3 2 5

TOTAL: -- 35 25 60

82% of students successfully completed REA 030.

REA 060

201010 201020 201030 AY Totals

Pass (A,B,C) 17 74 54 145

Not Pass (D, F) 2 23 12 37

Withdrew 1 7 22 30

TOTAL: 19 104 88 211


REA 090

201010 201020 201030 AY Totals

Pass (A,B,C) 34 160 102 296

Not Pass (D, F) 5 30 38 73

Withdrew 1 22 21 44

TOTAL: 40 212 161 413



benchmark?

2) How does the data compare to the previous year, if

applicable?

3) If multiple measures were used, how do they

compare to each other?

The performance group compared about equally with the benchmark.

Student performance summary 1) Based on the findings, how does the department

rate student performance in regards to this outcome

(strong, weak, or neutral)?

2) How does this assessment affect plans for this

coming year in terms of curricula, teaching strategies,

and assessment methods?

Students entered Reading 030 with an above average Accuplacer score. These students were very

successful in this course and had an average course grade of 3.17. Only 1 student failed the course and

2 students withdrew.

The reading department will work towards maintaining above average success rates by continued

focus on skill development and maintain student engagement through D2L course shells.

Students in Reading 060 enter the course with an average Accuplacer score. They exited the course

with a C average and 4 out of 85 students withdrew from the course.

The reading department will work towards higher success rates for reading 060. There will be efforts

made to help the students that are failing the course to successfully pass. This will be done through

more tutoring efforts, Early Alert, and teacher interventions in the classroom.

Students entered (Spring) Reading 060 with a below average Accuplacer score which could account

for so many students withdrawing from the course. Although the course grade was above average, the

success rate was not very high due to the numerous withdraws.

The reading department will work on targeting students with lower Accuplacer test scores and helping

them gain the necessary skills by additional online practice and Student Success Center and individual

tutoring.

Students entered (Spring) Reading 090 with a below average Accuplacer score; this could account for

the fact that 30 students failed the course and 6 students withdrew. Even though the average post-test

scores improved, there are still too many students that are not successful in Reading 090.

The reading department will focus on student success for Reading 090. This will be done by using the

Student Success Center and identification of struggling students through Early Alert. Focus on

specific skills such as, study and organizational skills, time management, and specific reading

comprehension strategies will help target areas of weakness.

Part 2: Current Academic Year Assessment Plan

Your program may wish to examine how retention differs among sections of an important course or you

may choose to do an analysis of grade inflation across courses within your program/discipline. Such items

aren't truly student outcomes, but they certainly affect learning. As such, these outcomes will be classified

as "Other" in the summary you create next year. (Measuring such outcomes is purely optional.)

Two or more instruments of measuring an objective may provide greater clarity and validity, but only one

is required. The department or program makes the decision. The Program Assessment committee and

deans are available for consultation.

In the past, some programs have been identified purely by prefix or in some cases by the type of section

offered. Sometimes, a very limited pool of students have been available for such a program to assess, or

the program lacks full-time faculty to plan, assess, and report outcomes. If your program has such

difficulties, please contact either the Program Assessment committee's chair or your School's Program

Assessment committee representative. We will work with you to find a solution.

CTE programs with external accreditation may use the accreditation report to in addition or in lieu of

these forms, please contact the Program Assessment committee representative if this format is desired. In

absence of this contact, these forms are expected.

Outcomes are to be measured annually. Exceptions are made with VPI approval for outcomes that clearly

need a less (or more) frequent review.

Outcome minimums

At least two outcomes are to be program/discipline-related.

At least two outcomes are to be General Education in nature. One General Education outcome must be

continued from the prior year to develop a historical trend. General Education outcomes need to be

assessed and reported annually, regardless of the frequency of reporting for other outcomes.

Both outcomes above are classified as "student learning" outcomes, requiring benchmarks and analysis. It

is strongly recommended that you use the table provided in Part 1 of this report for this function.

Definitions and examples of these outcomes are provided in Appendix A at the end of this document. Your

Program Assessment committee is available to assist.

An assessment report is requested annually. Such a report may only consist of a report on General

Education outcomes and a plan summarizing where your program is in an assessment with multi-year

frequency.

Current Academic Year: 2010-2011

Intended Learning Outcomes (only include if they differ from those noted in Part 1)

We intend to measure the same Learning Outcomes.

Assessment Method(s) (only include if they differ from those noted in Part 1)

We intend to use the same assessment methods.

Benchmarks (only include if they differ from those noted in Part 1)

We intend to use the same benchmarks.

Have you submitted a separate budget worksheet? (Choose by bolding; for information about this worksheet, please

refer to the specific budgeting e-mail sent by the committee chairperson.)

Yes No

Please submit this report (including both last year's summary and this year's plan) in a Word document to the Program Assessment

committee chairperson (Cheyne Bamford: [email protected]). If you have any questions about the process, please contact the

chairperson.

Documents

Assessment Overview - Arapahoe Community College · PDF fileAssessment Overview ... ENG 060 to ENG 090 into the transfer-level ENG 121 course. ... week of the semester and then the