National Art Education Association

Curriculum Projects: Reflections on Their Evaluation and ContinuanceAuthor(s): Jerome J. HausmanSource: Art Education, Vol. 24, No. 8 (Nov., 1971), pp. 24-27Published by: National Art Education AssociationStable URL: http://www.jstor.org/stable/3191611 .

Accessed: 15/06/2014 17:20

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

National Art Education Association is collaborating with JSTOR to digitize, preserve and extend access to ArtEducation.

http://www.jstor.org

This content downloaded from 185.2.32.46 on Sun, 15 Jun 2014 17:20:08 PMAll use subject to JSTOR Terms and Conditions

PROJECTS: Itlelie1(iolms <(liTheir EiU.lu ll mion l ( oniiiwuiwc Jerome J. Hausman

Recently, I moved into a new house. It was a wearing experience-especially as I sought to gather together my books, periodicals, and various and sundry reports, documents, letters, etc. One is soon impressed with how deeply the life of a professor can become enmeshed in a mass of paper. Moving also provided an occasion for sorting and reflecting upon each and every document to be moved. One by one, I went through xeroxed pages, mimeographed statements, dittoed reproductions, and carbon copies.

My work in the past ten years has brought me in contact with various educational projects of a comprehensive nature involving groups of people seeking to facilitate or encourage educational change-NDEA Institutes, regional laboratories, Title I and III projects of the Elementary and Secondary Education Act, and, more recently, Triple T Projects, development efforts of state education departments, and state arts councils. It would seem that each and every project was planned and encapsulated in paper (lots of it) which I dutifully stored and then boxed for the movers to be taken to my new abode where the material now rests gathering a new layer of dust. But the act of reflecting upon the countless manhours consumed in thinking about, writing, and reproducing the various proposals, progress reports, and final documents still weighs heavily upon my consciousness.

One word that comes to the fore in each project proposal or report is "evaluation." Somehow in the liturgy of proposal or report writing there is a compulsive need to make mention of "evaluation." Indeed, there is more than the mere mentioning of the term. More often than not, there is a detailed description of an evaluation methodology involving variously: questionnaires, interviews, rating scales, observations, tape recordings, photographic or filmed sequences, or just plain people (usually called "consultants") coming in to offer their evaluations. But irony of ironies, with all that was written, the one gnawing and persisting problem that remains for me relates to the broad issue of evaluating what has been done. What has been the effect of the project? What are the strengths and weaknesses? What have we learned that can be generalized? These and other common sensical questions stay with me as lingering and unanswered concerns.

Let me digress briefly. It is a truism that a curriculum project's evaluation depends upon the point of view, means, and criteria of the evaluator. In one fashion or another, we all engage in processes of evaluation in relation to schools-in making value judgments about people, objects, or events. Students for example, have their own very reliable, informal

24

system for evaluating teachers, courses, and other aspects of the educational program. In a similar fashion, parents and others in the community develop their evaluations of the educational program. Of course, there are teacher and administrator evaluative concerns with accompanying criteria (sometimes explicitly stated, but always implicitly there).

One of the characteristics of projects such as NDEA Institutes or Title III programs has been widespread involve- ments of diverse persons in evaluating the planning, operation, and/or outcomes of the project. It sometimes appears that everyone was able to get into the "evaluation act." Indeed, different evaluations seemed to exert differing forces upon the program-depending on when it was done and who was doing it. For example, unfavorable evaluations by a funding agency led to one set of circumstances; an unhappy parent, to another.

Without wanting to become too specific and detailed, I would observe that many of the comprehensive arts curriculum type projects (that I reviewed) have run their alloted time and have not been renewed. For example, countless Title III Projects designed to develop imaginative solutions to educational problems are not now being funded. I would venture the guess that the staff evaluations of these programs recommended "continuance" in virtually all cases. Yet, for one reason or another, funds were not found to enable sustaining the work in these directions. To analyze the reasons for this is a rather complicated affair; it would take us into the realm of politics, persuasion, and the priorities of educational policies. It would also take us into some of the problems of how projects are conceived and the means by which professional evaluation is being carried on. This, incidentally, would be the subject for a book rather than a journal article.

I would not attribute the failure of project continuances to a single cause; nor would I want to defend any particular project effort. Doubtless, some of the projects did not deserve to be continued, others have been continued in other forms by having staff members incorporated into other units of the school system. The concerns to which I wish to address myself are those that pertain to what I perceive as inadequate evaluation and continuance strategies. I want to mention some of the problems and circumstances that have led to a condition in which persons like myself are left somewhat puzzled and frustrated by great promises and expectations but little in the way of recognized achievement and insights as to new ideas and directions.

This content downloaded from 185.2.32.46 on Sun, 15 Jun 2014 17:20:08 PMAll use subject to JSTOR Terms and Conditions

Somehow, in the life of a project, its leadership needs to conceive of its operation and evaluation in a comprehensive manner, not losing a sense of its totality. To do so there must be a clear and strong sense of what the project seeks to accomplish and how these accomplishments can properly relate to the diverse interests (students, teachers, administrators, parents, professional groups) that are involved. Planning a curriculum project requires that purposes and actions be clearly linked. This is no easy matter!

A key difficulty often times stems from the very requirements for gaining project approval (from a governmental source, a foundation, or a Board of Education). "Selling the project" requires some degree of persuasion. This is frequently the case when numerous proposals are "competing" for financial support. To attract attention, many educational problems then are described in "crisis" terms. Words are used to project the urgency of the work to be done as well as the enthusiasm and commitment of the persons involved in the undertaking. Often times there is great ease and fluency in expressing these points; the problem, of course, is that the difficulties in

generating educational change are a different order and magnitude. Developments set forth on a typed page then have to gain translation in the realities of a classroom. Saying it is one thing; doing it, another.

Once one enters the arena of project competition there are a number of other pitfalls. Apart from exhortations, there is the need to convince others that those proposing to do the work have the competencies to do so. Once again, the persuasive language we use becomes our "tool and trap." Frequently, essentially simple and direct words are shunted aside by seemingly specialized terms. Thus, "what you want the kids to learn" becomes "instructional objectives or seeking behavioral outcomes"; or "how teachers organize the time of the school day" becomes "creating time modules in response to perceived educational needs"; or "a brief conversation with a parent" becomes "soliciting community involvement in relation to educational strategies." Stated simply, specialized language becomes an instrument to convince others of specialized competencies. Of course, the two do go together on many occasions; but, such is not always the case.

Another pitfall can be found in the observation that statements outlining more generalized goals are politically useful. Also, one can make note of a seeming cultural abhorence of silence and simplicity. All too many projects are launched from a base of words and promises that must

necessarily pay attention to a perceived first priority concern: getting funded! In one sense, evaluation of a proposal writer's efforts has a simple criterion: does the project get started? If so, he has succeeded! But, what about the actual operation and evaluation of the project that follows? Ay, "there's the rub."

In all too many instances, those engaged in comprehensive curriculum projects are faced with another dilemma once their effort is underway. Having been launched in an aura of promise, theirs is a challenge to begin working with teachers, students, parents, and others in ways specified by the proposal. Theirs is also the concern for providing data to others as to progress being made, accomplishments, and other outcomes. It is painful to observe how many projects have begun with a sense that they must immediately offer evidence to justify their existence and continuance. Hence, some of the same pitfalls pertain to preserving a project as gaining initial approval. The difficulty, of course, is that these pitfalls are compounded by the insistent demands of ope and accountability.

Once begun, many projects face an increasing deman f information about their work (reports, preparation of materials for visitations, and other descriptive data). Ironic increasing the information output does not always contribut to clearer communications at the operational level of th school. Indeed, some have observed that increasing required information loads may contribute to failures in communica- tion.1 Compounding the problem further, the press for more specialized knowledge and information may satisfy a smaller clientele; while, on the other hand, there are mounting cries for relevance and more widespread involvements in formulating educational policies.

A final point in my listing of pitfalls is the often confused substitution of measurement reports (description) r f .o evaluation. The result has been an avalanche of chian;d tables that may warm the hearts of the psychometriesearch establishment but chills the enthusiasm of more common, ; sensical viewers who seek insights into the improvement !f education. As Elliot Eisner stated: "Statistical significanis. not the same as educational significance." Lest I be misuie stood, I would hasten to add my voice in behalf of develodIn measures appropriate to the issues and priorities at hand. What I lament are situations in which items are counted jus because they are quantifiable; or the reduction of complex and subtle distinctions to simple arithmetic categories. We had best be careful that policy decision-making is not being

This content downloaded from 185.2.32.46 on Sun, 15 Jun 2014 17:20:08 PMAll use subject to JSTOR Terms and Conditions

District of Columbia New Hampshire North Carolina New Mexi co

| 'Mississi ppi 9

\ ..d,: ,

4 f

I -

op 1 144W fA ,o 7.dor I "r

,

U

14I o - L ". .* it

I) x, . .

IL

n -v; I > *

/fr /?. __ ^~~~~~~~~~~

# ), 4 -

I

This content downloaded from 185.2.32.46 on Sun, 15 Jun 2014 17:20:08 PMAll use subject to JSTOR Terms and Conditions

made in one room while decimal points are being carried to six places in another.

Thus far I have dwelled upon a less than enthusiastic appraisal of our performance in evaluating and maintaining confidence in comprehensive curriculum projects in the arts. Let me turn now to some base line considerations that I feel ought to guide our efforts.

Evaluation needs to be seen as integral to program planning and operation. To paraphrase Bloom, Hastings, and Madaus,2 evaluation is a method for acquiring and processing evidence needed to improve teaching and learning; it requires a great variety of evidence; it seeks to clarify educational goals and determine the extent to which students are developing in relation to these goals; it is a means to assess the teacher- learning process; and finally, it provides a means for comparing alternative educational practices. Few would disagree with these points.

An effective evaluation effort needs to be undertaken from multiple points of view and at differing points in time. Evaluation should be part of the on-going operation of a pro- gram; it also requires the distance and detachment for an outsider's view of what has happened.

Overall, evaluation of curriculum efforts should pay atten- tion to the processes of the program, its outcomes at pre- determined intervals, an even larger view of the directions and values that underly the work being done. When we speak of the task of evaluation, it is important to be aware of the differing concerns and constituencies that must be brought into relationship. There are differing audiences to which an educational program is accountable. To be sure, there is the same general reference point for a program: its teachers, students, facilities, etc. What is important to keep in mind, however, are the various criteria and differing modes of evaluation and reporting that have to be applied.

My other generalizations follow from the various observa- tions already made:

1. A program should not promise more than it can produce; evaluation efforts should be developed from a program rationale that has translation in the realities of a school.

2. There should be specifications as to program priorities and time commitments for achieving particular educational goals; evaluation, in turn, should be responsive to these priorities.

3. Evaluators should look beyond the limits of immediately perceived issues to larger contexts. It is here that we need to

beware of local enthusiasm or lengthy testimonials-in short, the limits of our own rhetoric and biases.

4. Data presented should be appropriate to the issues at hand. We have to avoid trivialization. There's no need to present masses of facts and information concerning points of little consequence.

5. Necessarily, evaluation of educational projects should be carried forth with knowledge and understanding of decision- making, time schedules, and priorities. It can be pretty academic to read an evaluation report and its recommendations for a project that has long since closed down for want of an adequate means for its being continued.

Recently I read of Forrester's Law, a maxim asserting that in complicated situations, efforts to improve things often tend to make them worse. Perhaps this is a corollary of such well known formulations as Parkinson's Law or the Peter Principle. What I do think to be useful is the observation that there is a growing disillusionment for well intentioned bureaucratic attempts to change complex institutions. Perhaps this helps account for the more recent popularity of Ivan Illish and others who call for a dismantling of the "educational establishment." Simplicity as an antidote to bureaucratic complexity has a very compelling "ring."

Clearly there is merit to ideas that would bring the numbers and complexities of school operations within manageable limits. But let us not be deluded into any naive assumptions that smaller units are necessarily more efficient and effective. The challenge ahead involves learning how to relate our ideas, actions, and bases for evaluation. It is only then that we can hope and expect to have curriculum projects of sustained impact.

Jerome J. Hausman is professor of art education, New York University, New York, N.Y.

This paper is adapted from an earlier version presented at the National Art Education Association 1 1th Biennial Conference, Dallas, Texas, April 1971.

REFERENCES

I am indebted to Professor Robert Stake, University of Illinois, for this point made at a lecture at New York University, Fall, 1970.

2Handbook on Formative and Summative Evaluation of Student Learning. New York: McGraw-Hill Book Co., 1971.

27

This content downloaded from 185.2.32.46 on Sun, 15 Jun 2014 17:20:08 PMAll use subject to JSTOR Terms and Conditions