Christopher Percy

Assessment Internship

April, 2006

Review of the Literature

Introduction: The Problem and Role of Assessment

The problem of assessment is one which has troubled researchers and

practitioners for many years. It is not hard to understand why assessment is such a

concern; assessment permeates individuals’ lives, and is of central importance in

healthcare, mental health and social services, and education (Taras, 2005). It is because

of this critical importance that assessment methods have been such a source of

controversy in recent years, leading researchers to devote a great deal of time and energy

to the development and implementation of new, competing assessment strategies.

Headlining this conflict is the contrast between conventional summative methods

of assessment, and a learning-based formative approach. Though summative methods

have become the primary means of evaluating students in higher education (Smyth,

2004), this system is not without its detractors. Many complainants believe that

summative assessment, with its focus on memorization, recall and lack of constructive

feedback, has created a situation in which valuable academic and vocational skills are no

longer being taught adequately. In response, many advocate a learning-based approach

such as formative assessment, in which the assessment becomes one of the tools used to

teach students valuable skills such as self-evaluation and criticism, writing skills, and

other abilities.

1

However, controversy still exists within both of these modalities concerning the

depth of material, the ability of faculty to live up to increased responsibilities with the

formative method, and the use or disuse of comprehensive and readily accessible

evaluative criteria. In addition, many questions still exist regarding the value of more

extensive feedback as required by the formative method; are students really interested in

making use of this information, or is it wasted effort on the part of the faculty? Does

feedback create the desired result? And even more importantly, is this truly the role that

assessment should play in education?

In a review essay, the editors of Assessment in Education (2004) discuss some

issues related to the development of the role of assessment in education. They suggest

that, for all the apparent value of classroom learning, this modality is simply insufficient

in terms of meeting students’ educational needs. This necessitates a form of assessment

which is not only summative, but which can be used as a tool for enhancing educational

practices. In a sense, education needs to be seen from a “constructivist” approach, where

learning is seen as an organic, interactive process involving a constant restructuring and

re-evaluation of information not simply as received, but as applied (p.213). In effect, this

defines the assessment problem not as a problem of student or faculty involvement, but as

a problem of the methods and measures themselves being unsuitable for their proscribed

purpose. The chief aim of assessment, according to this review, is to serve the varied and

changing learning needs of the students themselves.

Formative vs. Summative Methods

Before delving into a discussion of assessment methods, it is necessary to ensure

that a proper understanding exists regarding the meaning of these terms, and the

2

distinction between the different types of assessment. Taras (2005) in her review of the

literature describes the difference between summative and formative assessment as being

one of product vs. process; in other words, that summative assessment measures the

cumulative progress of the student with the specific end-product in mind, whereas

formative assessment does the same with more focus on the idea of learning as a process.

In less abstract terms, the difference is that formative assessment requires the use of

active feedback to both point out deficits as well as proscribing the necessary steps to

remedy the problem, whereas summative assessment is strictly a quantitative

computation of material already memorized. Black and William (2003) define summative

assessment as assessment which generates information meant to grade a student on their

performance during a particular class unit, whereas formative assessment is assessment

aimed at improving the skills of students with the cooperation of both students and

faculty.

Black (2000) describes formative assessment as what should be at the forefront of

assessment practices, citing deficits in the actual types of learning required for success in

summative evaluations. Summative evaluations, according to the author, encourage “rote

and superficial learning” (p.408) rather than application, emphasizing grade achievement

over actual learning. Torrance (1993) also suggests that summative assessment is an

assessment of achievement, and formative assessment is meant to assist in the

development of necessary skills. According to Black (2000), the end result of summative

assessment is an increase in pressured study which leads to short-term success, but which

is ultimately detrimental to later application of the information learned. This results in a

3

cumulative volume of information learned, the half-life of which is significantly shorter

than might be desired within a particular field.

Higgins, Hartley and Skelton (2002) suggest rather bluntly that summative

assessment treats students as mere “receptacles” (p.53) of information, whereas formative

assessment elicits the active participation of the learner. Torrance also suggests (1993)

while citing previous studies, that formative assessment, in addition to providing

feedback to students on areas of potential weakness, also aids educators in determining

how to direct their individual classes in such a way that serves the individual needs of the

students. In fact, many mainstream educators have already incorporated elements of

formative assessment into their programs. Torrance cites several reports which indicate

that teachers should take the time to assess and provide feedback to students in such a

way that they develop an understanding of the thinking process of the students as it

happens.

These dichotomies of active vs. passive learning, end-product vs. process, and

information vs. application are the conflicts at the heart of this controversy. Which

assessment strategy is best suited to prepare students for successful outcomes in post-

graduate life? Much of the literature suggests that students’ concerns regarding their level

of preparedness are not being addressed by summative methods alone, and that students

actually desire an assessment plan more in tune with skill development, active learning,

and application (Heylings & Tariq, 2001).

But does this process really work? Torrance (1993) suggests that it does,

providing several theoretical underpinnings for the idea of formative assessment, but in

particular the concept of the teacher as an engager, someone who helps the student to

4

confront and overcome new problems. Using formative assessment, the teacher becomes

much more of a mentor than merely a test-maker, both creating the obstacle and at the

same time helping the student to develop the necessary skills to surmount it. This type of

assessment looks forward in terms of the development of skills for future problems,

rather than merely looking backward in a summative sense.

Smyth (2004) suggests that coursework essays are a valuable opportunity for

educators to engage in formative-type assessment by targeting specific skill deficits while

providing feedback and encouraging self-reflection. Formative assessment in this sense

becomes more than merely a tool for measuring proficiency, but actually becomes a tool

for generating skills in a given area. Smyth’s study involved a group of twelve students

who participated in a “high stakes” (p. 372) assessment plan, in which they submitted

several essays which were evaluated on several criteria, of which the students were made

explicitly aware. The author suggested that the use of clearly defined goals empowered

students to engage in greater self-evaluation and self-motivated skill development even

prior to submitting their essays for review. In this sense, the use of a formative

assessment plan created an environment where students were presented with areas of

deficiency, concrete goals, and the methods to evaluate their own performance. The

author also cited adequate feedback as the backbone of a successful assessment plan.

Wininger (2005) developed a plan which was designed to synthesize formative

and summative methods, called Formative-Summative Assessment (FSA), and conducted

a pair of studies to measure the effect of this design. In the first study, the author made

use of 38 students enrolled in an educational psychology course, consisting primarily of

juniors and sophomores. The materials used consisted of PowerPoint slides containing a

5

mixed-up series of items which were used on exams. Students were provided with their

grade, and then went over the exam as a class. During this process, each item was

examined, and students were encouraged to argue for credit if they could provide a

rationale behind an otherwise incorrect answer. Each item was thus covered, with

students being given the opportunity to explain their answers as well as receive feedback

from the professor. The author reported that the students’ attitude toward this approach

was positive, as indicated through an anonymous five-item survey. The format of this

survey was not described in the report.

The second study conducted by Winingers, and published in the same paper, was

a comparative study consisting of seventy-one students enrolled in two sections of an

educational psychology course. The same professor taught both sections, which were

again dominated by juniors and sophomores. The procedure consisted of a 50-item exam

spanning four chapters of course material. For one section, the FSA method described in

the author’s previous study was used after the exam, whereas for the other section (the

control group) only received a copy of their exam with the correct answers indicated for

any incorrect items. Both sections received about 30 minutes to go over the exam. One

week later an identical exam was administered for extra credit, again to both groups.

The results did indicate a difference between the two groups. On the first test,

both groups scored within a similar range (81.41% for the FSA group and 81.62% for the

control group) (Winingers, 2005). However, when the exam was re-taken a week later,

the FSA group showed a marked improvement over the control group (90.82% vs.

83.72%), indicating that the FSA method facilitated higher re-test scores than merely

handing back a sheet with the correct answers indicated. No random sampling was used,

6

and so no significant changes were found, but the results clearly indicate that the FSA

group had an advantage over the control group. Students’ responses to this method

indicated that they felt better informed about the test, that the ideas and concepts were

clearer and better understood when they were given the opportunity to engage in a

discussion regarding those items, as well as being able to formulate arguments in favor of

an otherwise incorrect answer.

This study underscores the role of feedback and discussion, and it is this portion

of the FSA method that is formative rather than summative. By discussing the material,

and by encouraging students to think creatively about their answers, Winingers also

encouraged a more active participatory role in learning, with an emphasis on

understanding, clarity, and creativity. As can be seen in his study, there is some apparent

value to this method.

Among the two primary types of assessment, summative assessment is often seen

in a negative light as impersonal, inadequate, and destructive (Taras, 2005). This is due to

the relatively one-sided nature of summative assessment: the student takes a test, is

evaluated, and then receives a grade. There is no element of problem identification and

remedy, at least not explicitly. In addition, the monotonous task of completing apparently

non-constructive examinations can breed a certain degree of contempt in students

themselves (Smyth, 2004). Knight (2002) describes summative assessment as assessment

“in disarray” (p.278) due to theoretical, practical, and political conflicts. In the case of

higher education, Knight cites the increased responsibility of universities to set concrete

learning goals, and to hold students and faculty accountable for their progress or lack

7

thereof, as a problem which has led to an increased focus on assessment that focuses

more on information learned than on skills and information applied.

Other problems cited in Knight’s report include a growing distrust within the

educational system, where students are no longer trusted (and often with good reason),

and thus summative assessments are used to monitor and ensure that students are learning

what is required. There is no longer a sense of good faith, where it could be assumed that

“good people” (p.278) would take responsibility for their own learning. This enables

students to be lazy about learning, since they are no longer perceived as being personally

responsible or empowered, the precise opposite of the empowerment and self-reflection

advocated by proponents of formative assessment. In addition, the use of new emergent

assessment methods often creates a miasma of various and only somewhat-compatible

means of assessment, and cost-conscious educators are often forced to find the most

efficient and cost-effective means for educating their students. This leads to assessment

techniques which are quick and efficient, but often of questionable or specious reliability

and validity. In addition, these forms of assessment undermine the importance of self-

reflection and deep learning through a focus on grade achievement and a lack of feedback

dialogue.

This lack of adequate goal-directed feedback often leaves students aware of the

existence of a problem, but with little means to identify and correct it. To combat this

one-sidedness, many educators have promoted formative assessment as an alternative,

using explicit and instructional feedback to identify problems and empower students to

correct them. Other studies suggest that formative and summative assessments are not

mutually exclusive, but can be successfully (and advantageously) integrated into a

8

cohesive and goal-directed course plan which yields information both on student’s

proficiencies, and information on how to improve their deficiencies (Brookhart, 2001).

However, providing adequate feedback necessitates a clear and rigorous

understanding of explicit departmental goals against which to compare student

performance, as well as a substantial body of time to dedicate to intensive feedback.

Torrance (1993) sees this delineation of explicit goals as a critical part of a behavioral

explanation of formative assessment.

The Need For Clearly Delineated Educational Goals As Part and Parcel of Any

Assessment Strategy, and a Caveat

One potential problem related to assessment is the tendency for departmental

goals to be vague or ambiguous, leaving educators without a set of specific criteria on

which to base their assessment of students. This can lead to assessments of skills and

areas of knowledge which may or may not be directly related to the actual knowledge

needed to work within a given field. The training of specific skills, rather than simply

general knowledge within a topic range, has caused a great deal of controversy in recent

years (Avery & Bryan, 2001) as many programs have begun to indicate more vocation-

specific skills as a part of their repertoire. This has largely been the work of educators

recognizing the value of certain skills and emphasizing them as part of their programs;

some examples include oral and written proficiency in English as well as other skills

widely applicable in a number of disciplines (Smyth, 2004).

Avery and Bryan (2001) cite the Dearing Report of 1997 as the basis for the

statement that degree programs should “make explicit the range of skills acquired by

students undertaking it” (p. 169). Taras (2005) goes so far as to define assessment based

9

on concrete and measurable goals, implying that without a specific set of criteria in place,

no accurate assessment is possible. It is the recognition of the gap between performance

and these standards that forms the hinge of formative assessment, enabling the educator

to provide accurate and helpful feedback to the student (p. 471). Torrance (1993) sees the

clear delineation of goals as critically important for the educator and student alike, and

Smyth (2004) sees formative assessment as being inextricably linked to the achievement

of specific course goals. But, as pointed out in Black (2000), these goals cannot be

arbitrarily set, but must be related directly to the needs of the individual students as well,

being relevant, realistic, and also somewhat flexible.

With this in mind, the importance of setting down specific assessment goals and

criteria seems self-evident. By doing so, educators provide a reference point both for

themselves, and also for the students. Based on these criteria, both parties can make

critical evaluations of any areas of deficiency, as well as develop plans to remedy those

areas. But, care must be taken to ensure that the goals are mutual, and that students

understand the rationale behind these criteria. It seems logical to conclude that there

would be initial differences between students and faculty in terms of goals and desires

within a specific course.

However, some studies have found that the desired goals for a given subject of

study do not actually vary much between students and faculty. Heylings and Tariq (2001)

described the development of an implementation scheme synthesizing formative and

summative methods of assessment. Beginning with a two-day residential course centered

around a research project and attended both by students and by prospective employers,

the authors led the group in developing a plan based on skill development and assessment

1

with input from both groups of participants. Particular areas of focus included enhancing

students’ awareness of employers’ requirements, developing an assessment protocol that

facilitated learning, and raising self-awareness concerning the skills developed. Using

this information, the participants then drafted a scheme for monitoring and providing

constructive feedback and reflection on students’ work on a particular research project,

and this scheme was based on the previously elucidated criteria. Interestingly, both

groups expressed similar goals within their schemes.

The focal point of these schemes was a feedback loop fed by constructive and

informative comments on a series of reports submitted by the students. These reports also

contained reflective self-evaluation by each student, and they received further input from

the faculty. Over three years of implementing these schemes as part of this particular

study, the authors have found students’ reactions to this form of assessment to be positive

and empowering (Heylings & Tariq, 2001). However, faculty response was decidedly

more ambivalent, primarily due to the increase in the required amount of work to

maintain an adequate feedback loop. Unfortunately, the authors offered no actual data,

apart from samples of the schemes which were developed.

However, this does not mean that this strategy is without risks or potential

problems. Norton (2004) and Higgins, Hartley and Skelton (2002) raise the concern that

an explicit assessment plan may encourage students to develop strategies based on the

superficial goal of grade achievement, rather than a deeper intensive learning process.

This results in the proverbial double-edged sword: assessment plans, while necessary for

encouraging student understanding of the requirements for their course, “may encourage

over dependence on tutor guidance and a concentration on the mechanics of a task,”

1

rather than encouraging creative thinking and self-criticism (Norton, 2004, p.689). In

other words, students may attempt to deduce the precise requirements for a course or

project, rather than engaging in a creative and insightful learning process.

To overcome this difficulty, Norton (2004) attempted to design a module that

would synchronize learning criteria and assessment criteria, in order to avoid a situation

where students focused only on one or the other. The assessment then, becomes a

measure of the application of the material learned, rather than simply a regurgitation of

memorized data. In this way, Norton hoped to circumvent the issue of students engaging

in shallow learning, and encourage the sort of depth that comes with self-reflection and

experience. To accomplish this, Norton created artificial scenarios for use in a particular

psychology course.

Due to the obvious limitations, such as a lack of training, students could not

practice counseling methods on real patients; nor did the author believe that unleashing

the students on each other was wise. The use of artificial scenarios allowed for the

application of principles in a creative and individual way, with students encouraged to

reflect actively on the solutions which they developed. Students were expected not just to

solve the problem, but to apply the data in a way that helped to identify their own needs

and areas of improvement. In this way, the students were actively involved in their own

learning process, rather than simply submitting to a series of bare-minimums, and they

were expected to see not just the information which they used, but the contingencies

between different theories or ideas as they learned more about them in an active and vital

way. In addition to this, students’ responses were then graded and given back with

substantial feedback concerning the strengths and weaknesses of each individual’s

1

response. Unfortunately, the author provided no data apart from models and sample

responses.

However, this concern does not invalidate the benefits of a clearly delineated set

of departmental or class criteria, both to students and to faculty themselves. Problems

associated with formative assessment, including the amount of time required by faculty,

could be mediated through the application of a certain degree of structure. Norton’s

(2004) concerns about strategy are likely to be a problem with any form of assessment

where students are aware of what they are required to do, including summative and

formative assessment plans; in fact, to circumvent problems associated with clearly

delineated goals, Norton simply suggests an alternative structure. The common thread

running through these studies, though, is the need for structure and goals to provide a

framework within which to guide the learning process of each student. To avoid some of

the problems associated with this system, it is necessary for educators to implement a

feedback system, with both students and faculty participating in a reciprocal, progressive

evaluative process.

The Importance of Reflection and Feedback

In addition to the implementation of specific, explicit goal criteria, many

researchers advocate a student-oriented approach that includes an added focus on student

self-reflection and self-evaluation as part of the learning and assessment process. This is

particularly necessary both for the development of individual critical thinking abilities,

and also to prevent an over-dependence on specific grading criteria as opposed to creative

and critical application of ideas and concepts. To facilitate this process, it is necessary

that students and faculty engage in effective feedback loops which are constructive,

1

carefully planned, and informative (Smyth, 2004). This feedback can be maintained in a

variety of forms (Higgins, Hartley & Skelton, 2002), ranging from interviews, meetings,

peer-reviews, written and oral feedback on assignments, or any other dialogue intended to

facilitate skill development. Even post-test discussions of answers as a class have been

shown to facilitate higher performance on a re-test (Winingers, 2005).

Heylings and Tariq (2001) implemented a self-evaluative component in the

development of their assessment scheme, suggesting that as educators provide more

specific and helpful feedback on students’ progress, students should be motivated to

reflect on their own progress, and ultimately develop effective skills of self-criticism and

evaluative thinking. As stated previously, students and teachers participating in this

particular study had similar goals in mind when developing this scheme, and the

development of self-evaluation skills was a concern for both groups. According to the

authors of this study, feedback given must be concise, timely, accurate, and realistic, as

well as encouraging students to personally reflect on their own work.

The benefits of well-developed self-evaluative skills are quite evident given

concerns regarding the time and effort required by both faculty and students in a

formative assessment plan. Heylings and Tariq (2001) noted that faculty enthusiasm

regarding their scheme dipped during the implementation phase, likely due to the vast

increase in time required. With an increase in student self-assessment skills, students

should gradually begin to take on a greater share of the evaluative load, making

corrections and achieving insight into their own progress with less help from faculty.

Benefits for students would include a decreased sense of reliance on faculty, a greater

1

sense of self-efficacy and empowerment, and the ability to make effective critical

decisions on their own.

In this sense, formative assessment should not only be used as a process aimed

toward students’ growth in one particular subject, but also to aid students in an

understanding of the rationale behind the methods and tools used. Smyth (2004) suggests

that this increased awareness of the purpose of assessment tools is an important part of

the process of formative assessment, and that students must be made aware of the

importance of skills such as critical evaluation. Smyth believes that an understanding of

the purpose of formative assessment empowers students to take advantage of the benefits

of this structure; namely, by engaging in reflection and self-evaluation and leading to

deeper and more creative results.

This process of reflection is facilitated by the feedback provided by faculty.

According to Smyth, this student-faculty feedback loop “encouraged students to focus on

the written feedback provided, and to move on in the learning process by revising their

own performance” (p.373). These evaluative skills, once learned, could then be

transferred to other fields of study as well. This “deep learning” (Higgins, Hartley &

Skelton, 2002, p.54) enables students to think critically and creatively about their work,

and theoretically enhances their overall experience of their education, all hinging on

appropriate, timely, and constructive feedback.

In addition to being an important tool, Higgins, Hartley and Skelton (2002) also

cite students’ desires for increased feedback and dissatisfaction with lackluster criticisms

as a demoralizing experience. The authors claim that students do take notice of feedback

provided by faculty, but are often disappointed by what they perceive as vague, unhelpful

1

quips. In a brief survey, the authors found that 80% of students polled believed that

feedback was important for the development of proficiency in a particular area. Heylings

and Tariq (2001) note the same trend: students desire comprehensive feedback, and as

Higgins, Hartley and Skelton suggest, disappointing feedback contributes to a disaffected

and amotivational overall experience (2002).

In addition, many students see feedback as part of a service which they pay for

through their tuition; in other words, lackluster feedback is seen as lackluster service, a

service which students believe they are entitled to. This increases student disaffection

with both their school and the faculty, which is itself prohibitive of further faculty-student

communication. This contributes to problems such as those suggested by Norton (2004)

and Higgins, Hartley and Skelton (2002), who suggested that students may adopt

strategies which minimize their learning experience, limiting them only to those areas

which are necessary to achieve a desired grade.

But how do students actually use the information provided through feedback and

reflection? Brookhart (2001) queried 50 students in high school anatomy and english

classes regarding their perceptions and attitudes toward the information which they

received both in summative and formative measures. The queries were administered prior

to and after each assessment, regardless of the form. A small number of these students

were also interviewed separately each time, yielding a total of 52 interviews. Interview

questions were designed to gauge students’ sense of self-efficacy, perceived task,

challenges, and effort exerted to complete the task. The study found, through an analysis

of this information, that students who were more successful tended to make use of the

1

information provided through assessment in integrative and formative ways, including

the summative results.

The results of this study seemed to indicate that successfully students already

make use of formative self-reflection and self-evaluation, and that providing these

students with information through feedback would seem to enhance this process.

Although this study dealt primarily with higher-performing students, it does not

undermine the importance of feedback, and highlights the fact that educators’ additional

efforts to provide feedback are not wasted.

A Particular Area of Concern: Writing Skills

Of chief concern to many educators and employers is the ability of students to

effectively communicate their ideas in a clear, concise, and well-written essay form. This

is not an innate skill, nor a skill addressed in any great detail (one can assume) at the pre-

college level. Rather it tends to be a prominent area of deficiency among applicants to

undergraduate programs and an area of increasing concern for professors and guidance

counselors who have grown increasingly frustrated with the apparent inabilities of their

students (Allen, 1984). Ediger (2002) points out that this paucity of writing ability

affects the educators as much as it inhibits the students. In developing measures and tests,

professors need to be aware of the validity of the items used, their wording, and the type

of items selected, and the use of essay items is often an easier route compared to the

construction of an effective multiple-choice test.

However, the grading of these tests is a more complex process than an objective

measure, and this additional, subjective variant is where professors and students are likely

to notice deficiencies of writing ability. The application of ideas in a coherent, clearly-

1

organized form, use of proper grammar and spelling, legibility, and the use of meaningful

content all contribute to the quality of an essay’s content. However, a student who may

know the correct answer intellectually but not be capable of communicating it in an essay

form presents problems for himself and for the grading professor, who may have to

decipher a barely intelligible essay before deriving a grade. It is therefore of benefit to

students and professors alike that strategies aimed at the improvement of writing skills

are pursued and implemented.

Many studies have already been conducted in an attempt to create and evaluate

programs specifically geared toward the detection of writing deficiency and its

subsequent remediation. Writing-across-the-Curriculum (WAC) movements stressed the

need to teach writing skills in all areas of study, but writing problems continue to be a

concern in spite of these programs (Alter & Adkins, 2001), leading researchers to believe

that a more active program which involved the direct targeting of deficient trends was

necessary. At the University of Denver, faculty organized and implemented a pilot

program to see if such additional measures were justified. The program consisted of two

parts: an initial assessment of the writing skills of incoming students, followed by access

to a newly developed writing lab based on the results of the initial query. The assessment

was administered in the fall semester of 1998, and consisted of three parts: recognition of

the problem, selection of a solution, and the effective organization and articulation of that

solution. Each essay was then evaluated based on six criteria: diction, sentences,

paragraphs, general organization, mechanics, and usage. Each essay was then assigned a

grade from 4 to 1 (4 = A, 3 = B, 2 = C, and 1 = D) by each of two judges, with a

combined score then being the final grade (8-7 A, 6-5 B, 4-3 C, 2-1 D). The results,

1

scores, and comments attached to each essay were left in the new lab, so that only

students who came to the lab had access to their results.

The results indicated that more than 1/3 of 124 incoming students performed

below a “passing” grade required for their level (in this case, incoming MSW students),

and students showed particular problems in the area of writing mechanics and clarity of

expression. However, the study found that of the students most in need of remedial help,

only 57% ever accessed the writing lab for guidance (Alter & Adkins, 2001) while 43%

never used the facility. Reasons given for this lack of interest ranged from feelings that

their writing was on par with their peers to the convenience of the hours available. Most

students felt that bad grades would motivate them to use the lab, but otherwise did not

indicate an interest in improving their writing.

This lack of interest in remedying a clear deficiency in one’s writing ability may

partially explain students’ ambivalent attitudes toward writing centers and other non-

class-related tools. However, this lack of motivation can carry over to the classroom as

well. Learning to write well is a difficult and often arduous task incorporating intense

focus, exposure to personal criticism and requiring a strong motivational factor on the

part of the student involved. Allen (1984) found that among other factors, the vagueness

of evaluative criteria and personal apprehensions about their own skills often prevent

students from taking courses in which their writing abilities will be actively tested.

Additionally, many professors are reluctant to spend the time and energy necessary to

provide the requisite feedback and timely review of papers needed for students to take

remedial action; in fact, many are skeptical that these measures are even effective.

1

Allen devised an abnormal psychology course in which the chief focus of

evaluation would be the ability of students to write a series of papers along a personalized

schedule. Immediately upon learning that their writing would be evaluated heavily, 12 of

the original 21 enrolled students dropped the course, and six more enrolled. Of the final

15 students, 11 fulfilled the criteria and produced useable data. Grading was determined

by a series of written reports which, depending upon the length and content, could be

turned in for either 2, 4 or 6 points each. The 2-point papers consisted of a brief statement

of two opposing sides in a controversy related to psychology; the 4-point papers were

slightly longer and required gathering additional information from outside sources; the 6-

point papers were a full-fledged research report incorporating 6 to 10 outside citations.

These papers were structured hierarchically based on the amount of skill necessary to

write in a particular style. Students were required to begin with 2-point reports, and were

limited to 2 6-point reports in the semester. Each student also agreed to an individualized

schedule of due dates, and the author agreed to return papers within three days of

submission.

The purpose of this structure was to provide many opportunities for students to

receive feedback on their papers, with enough time available for critical evaluation and

re-submission after corrections were made (Allen, 1984). Among the criteria evaluated,

students were encouraged to focus on teaching the reader, as well as adopting and

defending one particular viewpoint as their own. This involved the evaluation of concepts

critically in order to develop a compelling defense. At the end of the semester, the papers

were collected and submitted to a panel of graduate students, who then devised 16

evaluative criteria and rated the papers according to these. The results showed that on the

2

whole, the students exhibited a marked improvement in their writing abilities and many

rated their own progress as significant.

Despite this improvement, there are several problems with this strategy which are

prohibitive. First, the focus on writing resulted in an immediate turnover of students.

Secondly, the course-load was very involved and time-consuming for both the students

and the professor. Additionally (although not mentioned by the author), the immediate

turnover of students presents a problem for educators who want to provide broad-

sweeping solutions for writers within a particular department; these programs will be of

little use to students who are unwilling to participate.

Concerns related to writing proficiency within a particular field of study have led

some instructors to organize courses specifically geared toward the development of

writing skills within a particular discipline, and in some cases these courses are a

prerequisite for admission into the program. Goddard (2003) created a semester-length 3-

credit course in writing for psychology students, with an explicit focus on writing with a

professionally acceptable grasp of APA style. The author suggests that the ability to write

well within a discipline should be a critical focus of students’ training in that particular

area. Therefore, a course was implemented on two consecutive semesters, during which

time 29 students were enrolled. All of these were current prospective psychology majors

who had completed a freshman composition course, as well as at least one other

psychology course.

The course itself, “Writing in Psychology,” focused on improving grammar,

adherence to APA formats, and revisions based on feedback from the professor. The

assignments consisted of a case report, a report of a study, an abstract, and a larger

2

literature review, arranged in a progressive format. In addition to being completed in a

specific order, the literature review was an ongoing project which took up the entire

semester as students learned the skills necessary for such a project. The instructor/author

took an active role in providing feedback on both stylistic and grammatical aspects of the

assignments, indicating errors and areas in need of attention. Students were also given a

pre- and post-course grammar and APA style test. The post-course means on these

measures (26.19 and 23.85, respectively,) both were markedly improved over the pre-

course means (22.93 and 20.74), indicating that the course and feedback provided were

helpful to the students (Goddard, 2003).

Problems cited with this course format, however, include the time-consuming

nature of a feedback-oriented writing class, as was also seen in the case of Allen (1984).

The common problems associated with these two plans are the time required of students

and professors, and whether or not such courses should be required for all students within

the major. Goddard (2003) cites the relatively high time requirements as a possible reason

why institutions would elect not to offer such a course as a requirement. However, other

programs and disciplines have created classes that are program prerequisites.

Faculty at Northwestern University became concerned about these writing

problems, specifically deficits among prospective journalism majors. To combat this

problem, the faculty developed a Language Skills Diagnostic Test (LSDT), which was

administered as a pre-requisite for their media writing course, which is required by all

majors. Also, an additional, elective course, “Literacy Skills for Journalists” was

designed to provide remedial help for students who failed the LSDT. It is the authors’

belief that WAC programs are too broad-sweeping to truly serve the needs of remedial

2

students (Brocato, Furr, Henderson, & Horton, 2005), and therefore a plan to (a) identify

students who need help and (b) provide them the means of obtaining that help was

designed.

To test the efficacy of this plan, the LSDT was administered to 175 students

between the spring of 2001 and 2003. The results indicated that 57% of these students did

not perform well and were in need of remedial attention. These students then took the

“Literacy Skills for Journalists” course, and all students who initially failed the LSDT

and then took the course subsequently passed their second trial of the LSDT. However,

little other data on this test is available.

The structure of this program is such that, although the additional writing course

is an elective, it is essentially required either directly or indirectly by way of passing the

LSDT. This method of assessment and intervention bypasses students’ lack of motivation

by forcing them to participate in a remedial program until they display a certain level of

proficiency in writing.

Summary

In summary, it is clear from this brief review of the literature that the case of

assessment is far from being laid to rest, as even advocates of similar ideas continue to

find points of contention or concern. But the common threads which bind these ideas

together continue to exhibit relevance: the need for assessment which addresses the needs

of students, which is clearly delineated and based on mutual goals, which provides

informative and helpful feedback, and which aids in the development of skills. Whatever

the final design, it seems clear based on the literature that any successful assessment plan

must contain and address these four points. Educators should not react quickly against the

2

idea of providing more informative feedback, and students should not shy away from the

reception of constructive criticism and the development of responsible self-evaluation.

The question posed at the beginning of this review was, “what role should

assessment play in education.” The answers seem quite clear: to aid faculty and students

in cultivating an open, goal-directed and reciprocal system of communication through

which students’ needs, deficiencies and strengths can be identified, addressed, and

adequate solutions developed. Through this process, students will develop an empowered

sense of self-criticism, and gradually assume a greater critical role with their own work

and a stronger personal stake in the development of their abilities.

2

BULLETS

Smyth (2004) contends that summative methods of assessment have become the

primary method employed by educators in higher education.

The editors of Assessment in Education (2004) suggest that, for all the apparent

value of classroom learning, this modality is simply insufficient in terms of

meeting students’ educational needs.

Taras (2005) in her review of the literature set out to define “summative” and

“formative” assessment, relating the latter to feedback and the need for clearly

defined and delineated goals.

Black and William (2003) define summative assessment as assessment which

generates information meant to grade a student on their performance during a

particular class unit, whereas formative assessment is assessment aimed at

improving the skills of students with the cooperation of both students and faculty.

Black (2000) describes formative assessment as what should be at the forefront of

assessment practices, citing deficits in the actual types of learning required for

success in summative evaluations.

2

Torrance (1993) suggests that summative assessment is an assessment of

achievement, and formative assessment is meant to assist in the development of

necessary skills.

Higgins, Hartley and Skelton (2002) suggest rather bluntly that summative

assessment treats students as mere “receptacles” (p.53) of information, whereas

formative assessment elicits the active participation of the learner.

Torrance also suggests (1993) that formative assessment aids educators in

determining how to direct their classes in a way which serves the individual needs

of the students.

Heylings and Tariq (2001) suggest that students actually desire an assessment

plan more in tune with skill development, active learning, and application.

Smyth (2004) suggests that coursework essays are a valuable opportunity for

educators to engage in formative-type assessment by targeting specific skill

deficits while providing feedback and encouraging self-reflection.

Winingers (2005) found that students who engaged in a formative-summative

assessment (FSA) plan performed better on a re-test when compared with a

control group.

Knight (2002) describes summative assessment as assessment “in disarray”

(p.278) due to theoretical, practical, and political conflicts.

Avery and Bryan (2001) cite the Dearing Report of 1997 as the basis for the

statement that degree programs should “make explicit the range of skills acquired

by students undertaking it” (p. 169).

2

Taras (2005) suggests that without a specific set of criteria in place, no accurate

assessment is possible.

Smyth (2004) sees formative assessment as being inextricably linked to the

achievement of specific course goals.

Black (2000) states that these goals cannot be arbitrarily set, but must be related

directly to the needs of the individual students as well, being relevant, realistic,

and also somewhat flexible.

Heylings and Tariq (2001) described the development of an implementation

scheme synthesizing formative and summative methods of assessment, and

reported that students and faculty/employers expressed similar goals and desires.

Higgins, Hartley and Skelton (2002) and Norton (2004) raise the concern that an

explicit assessment plan may encourage students to develop strategies based on

the superficial goal of grade achievement, rather than a deeper intensive learning

process.

Norton (2004) attempted to design a module that would synchronize learning

criteria and assessment criteria, in order to avoid a situation where students

focused only on one or the other.

Smyth (2004) states that it is necessary that students and faculty engage in

effective feedback loops which are constructive, carefully planned, and

informative.

This feedback can be maintained in a variety of forms (Higgins, Hartley &

Skelton, 2002).

2

Higgins, Hartley and Skelton (2002) claim that students do take notice of

feedback provided by faculty, but are often disappointed by what they perceive as

vague, unhelpful quips.

Brookhart (2001) found that successful students made use of information

provided in feedback in an integrative and creative manner which was beneficial.

Ediger (2002) states that variables in writing ability present a difficulty for

professors when grading essays.

Alter and Adkins (2001) found that 1/3 of incoming students were deficient in

their writing abilities, and yet relatively unmotivated to pursue extracurricular

help.

Allen (1984) constructed a writing-based curriculum based on repeated

submission of papers and constructive evaluative feedback, resulting in

substantial improvements. However, this plan was very time-consuming.

Goddard (2003) developed a class called “Writing in Psychology” in order to

teach general writing skills as well as particular discipline-specific styles, but felt

that the level of time needed to run the class would be prohibitive in terms of

making it a requirement.

Brocato, Furr, Henderson, & Horton (2005) designed a program in which students

were screened for proficiency in writing, and then placed in a remedial program

before they qualified for their particular discipline.

2

REFERENCES

Allen, G. (1984). Using a personalized system of instruction to improve the writing skills

of undergraduates. Teaching of Psychology, 11(2), 95-98.

Alter, C. & Adkins, C. (2001). Improving the writing skills of social work students.

Journal of Social Work Education, 37(3), 493-505.

Avery, S. & Bryan, C. (2001). Improving spoken and written English: From research to

practice. Teaching in Higher Education, 6(2), 169-182.

Black, P. (2000). Research and development of educational assessment. Oxford Review

of Education, 26(3&4), 407-419.

Black, P. & Williams, D. (2003). ‘In praise of educational research’: Formative

assessment. British Educational Research Journal, 29(5), 623-637.

Brookheart, S. (2001). Successful students’ formative and summative use of assessment

information. Assessment in Education, 8(2), 153-169.

Ediger, M. (2002). Problems in grading based on testing university students. College

Student Journal, 36(1), [Electronic version].

2

Editors (2004). Review essay: Attending to students’ learning needs using assessment.

Assessment in Education, 11(2), 213-226.

Goddard, P. (2003). Implementing and evaluating a writing course for psychology

majors. Teaching of Psychology, 30(1), 25-29.

Heylings, D. & Tariq, V. (2001). Reflection and feedback on learning: A strategy for

undergraduate research project work. Assessment and Evaluation in Higher

Education, 26(2), 153-164.

Higgins, R., Hartley, P. & Skelton, A. (2002). The conscientious consumer:

Reconsidering the role of assessment feedback in student learning. Studies in

Higher Education, 27(1), 53-64.

Knight, P. (2002). Summative assessment in higher education: Practices in disarray.

Studies in Higher Education, 27(3), 275-286.

Norton, L. (2004). Using assessment criteria as learning criteria: A case study in

psychology. Assessment and Evaluation in Higher Education, 29(6), 687-702.

Smyth, K. (2004). The benefits of students learning about critical evaluations rather than

being summatively judged. Assessment and Evaluation in Higher Education,

29(3), 369-378.

Taras, M. (2005). Assessment—summative and formative—some theoretical reflections.

British Journal of Educational Studies, 53(4), 466-478.

Torrance, H. (1993). Formative assessment: some theoretical problems and empirical

questions. Cambridge Journal of Education, 23(3) [Electronic version].

Winingers, S. (2005). Using your tests to teach: Formative summative assessment.

Teaching of Psychology, 32(3), 164-166.

3