Evaluation and Practical Deliberations in Education

Evaluation and Practical Deliberations in Education By Philip G. Smith

I. PROPAEDEUTICS

If evaluations and practical deliberations are to be justified they must first be understood. In order to understand what is claimed, asserted, or decided by such deliberations, they need to be connected to some theory of valuation-some way of understanding what is meant, in various situations, by “good” and “right.” Prelimi- nary explication of certain key concepts of a theory of valuation should, therefore, precede our discussion of the structures of evaluative and practical deliberations. The following three ideas seem to be the minimum necessary for theoretical understanding of the context in which evaluative and practical deliberation are embedded.

(1) The first idea concerns the nature and function of normatives. There is a widespread tendency, even among philosophers, to equate normative and prescriptive. A normative assertion is thus believed to be directly connected to action (or doing) in contrast to description, which is thought to be connected to knowing (or thinking or believing) and disconnected from doing. The gap between thought and action IS then supposed to be bridged by some form of command, resolution, or other emotive or prescriptive assertion based upon some person’s desires or feelings.

Actually, normatives are directly connected to meaning and through meaning, indirectly connected to action. Their essential function is deontic4.e. to make explicit requiredness or obligation. This is why a normative science such as logic or metaethics neither describes how persons actually do reason or act, nor prescribes specific reasoning or actions. They do explicate the principles, standards, and rules that govern thinking rightly and doing rightly. To understand and acknowledge such principles, standards, and rules is to accept the obligations they impos€+logical oughts and moral oughts. It is only within a system of meaningful obligation that evaluations can be justified and that prescriptions and intentions (first person practitions‘) can be warranted.

The domain of normatives is much more extensive than generally recognized. It covers not only valuational discourse and action but also descriptive discourse and its related activities. Indeed, all reasoned discourse is norm governed: by linguistic syntactic norms because it is discourse, by logical norms because it is reasoned. And when the discourse is about something and has some point, it is governed by semantic and pragmatic conventions and distinctions that require that words be put together in accord with certain ways rather than others.

Without such linguistic and logical requiredness, meaning would largely evapo- rate, leaving only a residue of rudimentary exclamation, pointing, and gesturing. Without valuational normatives, (principles, ideals, standards, rules) valuational and practical deliberation and action would disintegrate into rudimentary approach and avoidance behavior.

(2) The second idea concerns the primacy of the valuing activity, as the central focus for theory of valuation, as opposed to the traditional emphasis upon so called value properties. For example, the traditional intrinsic-extrinsic distinction now becomes a useful way of naming two distinctive postures toward valued objects. To value intrinsically is to value something for its own sake, in its own terms, and without regard for its actual or potential relationships with anything else. The object of such intrinsic valuing is called, figuratively, an intrinsic value. To value extrinsically is to value something because of its actual or potential relationships with something else. Such an object is said to have extrinsic value. And, of course, the same object may be valued in either posture and in different postures on different occasions.

Philip G. Smith is Professor of Philosophy of Education at Indiana University-Bloomington and is a past president of the Philosophy of Education Society.

1. See Hector-Neri Castaneda, Thinking and Doing (Dordrecht, Holland: D. Reidel Publish- ing Co., 1975), p. 155: “. . . the view that intentions are first-person practitions doesnot reduce them to commands to oneself, or to first-person prescriptions.”

279 VOLUME 28, NUMBER 4

2 80 EDUCATIONAL THEORY

It is not the case, however, that the valuing activity creates value or bestows value upon the object, nor is it the case that the best evidence that something is valuable is that someone values it. To assert something is valuable becomes a warranted assertion only as a result of an evaluation which involves inquiry and assessment of the actual and potential relationships. Since such relationships are matters of contingent fact, the inquiry and assessment are basically empirical. Thus to assert that something is valuable involves an empirical claim about contingent relationships. The forms that such relationships may take determine the structures of evaluation.

To assert that something is intrinsically valuable is a category mistake. For in order to focus upon relationships, one must change from the intrinsic posture and consider something more than the object in and of itself. While valuing intrinsically, one may engage in phenomenological description of the object or in introspective description of self. One may describe, even analyze, what is valued, explaining what particular features one finds especially interesting or attractive. Or one may construct a comparative preference rating, indicating how strong the valuing of this object is compared to the valuing of other objects. But neither phenomenological description nor introspection constitutes evaluation. They do not assess the value of the intrinsically valued object.

(3) The third idea concerns the removing of valuing from the so-called “affective domain.” It is not the case that an essential or defining characteristic of valuing is the presence of some attitude, emotion, desire, enjoyment, want, felt need, or other affective state. If valuing actually were essentially a matter of affect, then the study of valuing could be relegated to humanistic psychology and theory of value would be of no greater or special concern to philosophy than is psychology in general.

But this could hardly be the case. In our experience we can note a difference between merely liking, desiring, or enjoying something, and valuing something; i.e., prizing, cherishing, caring for it because we believe either, that as a unique particular it is worthy of our esteem or, that when considered in relation to other values, it has features, qualities, properties that are “good making”-that make it worthy of being valued. A cognitive state is, thus, one essential ingredient of valuing and this is why valuing is dispositional, proactive, and generally more enduring than our affective states, which are reactive and comparatively transitory. If we value pure, clear water, we do not cease to value it whenever we happen not be be thirsty.

It is true, of course, that very frequently, pleasant affective states are present as personal epiphenomena when we are consciously aware of valuing something. This is, perhaps, most noticeable while valuing intrinsically. But such positive affects are not invariantly associated with valuing. A diabetic may value insulin but not particularly enjoy it, and a person may value honesty and feel obligated to tell the truth even when it is extremely unpleasant to do so.

But valuing involves more than cognition. If we must analyze human personality and character into parts, faculties, mechanisms, or domains, then a suitable name for another essential element in valuing is what the Greeks called nisus, or in Latin, conatus, or in traditional moral philosophy, ‘‘will.” Here is the engine, within human personality, that powers valuing, commitment, dedication, perseverance, and the acceptance of obligation and duty. Plato saw this force as an integral part of ra- tionality and even today when we encounter a person in whom this force is underde- veloped or impaired, we say that he lacks integrity.

11. STRUCTURES OF EVALUATION

(1) Perhaps the most common form of evaluation, whether in education or else- where, is an assessment of the extent to which an evaluatum embodies or exemplifies some ideals, standards, or rules. The officials of a basketball game constantly monitor the action looking for examples of rule violation. When they observe an example of traveling, charging, goal tending, etc., a whistle is blown and a prescribed penalty is imposed. In many situations the standards or rules are not so carefully articulated. When an instructor examines student essays, the logical structure is the same as that used by the basketball official but, typically, the normatives (i.e., standards and rules)

FALL 1978

EVALUATION AND PRACTICAL DELIBERATIONS 281

that specify what is to count as good, bad, right, or wrong, have not been made fully explicit. This does not mean that the instructor’s grading is necessarily arbitrary, but it does make it very difficult to check on either the reliability of his judgments or the validity of his implicit standards.

From these two cases of assessment of exemplified value we may also note the difference between an assessment according to rules and an assessment according to standards. Rules either prescribe or proscribe, and what is exemplified is either rightness or wrongness. Standards usually permit gradations-assessment of the degree or amount of exemplified goodness-and hence, grading, comparisons, and rank ordering of evaluata. Moreover, quite typically, assessments are made in terms of multiple standards or rules so that some system of priorities or weights (whether in additive relation or in interactive complementary relations)2 is necessary in order to reach a conclusion concerning the overall merit or value, or the right thing to do, all things considered.

For example, in grading student essays, there is frequently a complex system of requiredness operating implicitly in addition to some objectives, standards, or rules that have been made explicit. The instructor may announce certain requirements: topic area, minimum and maximum length, evidence of knowledge of certain materials, or of additional reading, or of creative use of materials, or of analysis, or logical arguments, etc. In addition, considerations of composition, style, significance of the content may also play a part in reaching a conclusion as to the overall merit or quality of the essay. Then, in assigning “a grade” to the essay, the judgment may be further complicated by a consideration of “normal expectations” for students working at that educational level. When all elements (including weights and interactive complementary relations) of such a normative system are made fully explicit, and all relevant empirical data can be specified (not necessarily quantified) then, in principle, a grade may be determined by calculation, and, hence by computer program. Otherwise, judgments are required. In making such judgments, what is at issue is not merely estimating, rather than measuring or specifying exactly, nor guessing or conjecturing about unknowns, but expertise in making a holistic qualitative assessment in terms of a normative system much of which is tacit. In short, computing without a fully explicit program for doing s0.3 The notion. in education, of expert-proof evaluations is even more absurd than teacher-proof curricula.

(2) A second form of evaluation is based on an assessment of the extent to which an evaluatum is valuable as a means or instrument in the production of a valuable end. While we usually think in terms of cause-effect, we need not limit instrumental value to a strict sense of cause. Anything that is a catalyst or facilitating in relation to a valuable, or that is a medium, vehicle, or channel to its attainment, has instrumental value. This is why teaching may be viewed (and evaluated) as instrumental to learning without necessarily regarding teaching as directly causative of learning. On the other hand, not everything that has utility has instrumental value. “Utility” is the neutral or descriptive name for the actual or potential empirical relationship that is investigated in an assessment of instrumental value. But to claim that something is instrumentally valuable is to couple the empirical claim to a valuational claim concerning the end.

In education, perhaps the two most common uses of this evaluational structure are the assessment of teaching materials in relation to desirable learning and the assessment of learning in relation to various kinds of occupational competence. But, of course, in all practical deliberation, projection of probable consequences frequently plays a crucial role. Such projections are in the form of an assessment of means-ends relations and since we would not bother to do it unless we believed some

2. See David Hawkins, The Science and Ethics of Equaliiy (New York: Basic Books, 1977); esp. ch. 3, pp. 43-68.

3. All judgments (valuational or otherwise) are subjective in the same ontological sense that all perceiving, thinking, and knowing are subjective. Objectivity in inquiry, judgment, and decision (valuational or otherwise) is a methodological matter. One may strive for objectivity when dealing with any kind of material. The problem is to develop methodological controls that increase reliability without sacrificing validity.

VOLUME 28, NUMBER 4

282 EDUCATIONAL THEORY

ends were more valuable than others, the structural form is an evaluation of potential instrumental value.

It is this structural form of evaluation that lends itself most readily to the use of standard research designs and statistical and other quantitative techniques. It is not surprising that persons trained in educational research tend to view this structure as the form for evaluation in education. Moreover, since the use of such techniques make the conclusion appear to be the result of calculation rather than judgment, evaluations in this form are sometimes thought to be “more objective.” But, of course, the series of interlocking decisions required for the construction of an appropriate design, the selecting of suitable instruments and techniques, and the decisions concerning adequacy of samples, levels of confidence-to say nothing of interpreta- tion of data-usually require expert judgment before we get to the tip of the iceberg that is available for public inspection.4

(3) The third form of evaluation is based on an assessment of the evaluatum as a contributory part of a larger, valuable whole.5 This form of relationship is much more intimate and organic than merely means to end. For example, a graduate student may take a job washing dishes as a means to obtaining his education. But i f he is fortunate he may become a graduate assistant and not only gain the needed financial assis- tance but discover that the work experience makes an important contribution to the end itself-to his education.

While the contributive, exemplifying, and instrumental relationships are contingent and are what they are regardless of our desires or wishes, nevertheless, in order for value to be realized, in the sense of recognized, enjoyed, or capitalized, we must become aware of such possibilities. For example, i f the graduate student fails to see his assistantship duties as being of contributory value-if he views them as merely a means of paying his tuition-then their contributory potential is unlikely to be fully actualized. There is nothing especially strange about this. We sometimes do not know a good thing when we see it nor hear opportunity when it knocks. But it seems likely that in eudcation we have been especially insensitive to the contributive relationship. We have emphasized the instrumental value of school assignments, lessons, and even of learning and have introduced artificial motivations in an attempt to generate the effort needed for sustained movement toward distant goals.

When this fails, as it so often does, we frequently attempt to make these “instrumental” lessons and tasks interesting in their own right, in the hope that they will be valued intrinsically. Certainly some students do sometimes find some subjects and some of the activities involved in school learning to be of intrinsic value. They are prized and enjoyed just for their own sake. When this happens it is indeed a felicitous state of affairs. But it is not surprising that we are unable to make everything that is required in school intrinsically interesting to all students at all times. We know that what is in the best interest of students is not to be judged merely in terms of students’ interests.

An intrinsic-instrumental dichotomy of values is unnecessarily restrictive for making and justifying curriculum (e.g., poetry or push-pin), for assigning lessons and tasks, and for motivating students. If teachers and students were to focus upon the contributive relationship of parts to larger wholes, then while it would certainly not solve all our problems, the parts would participate in whatever value (and interest) the

4. See Richard Rudner, “The Scientist Qua Scientist Makes Value Judgements” in H. S. Broudy, R. H. Ennis, and L. I. Krimerman, eds., Philosophy of Educational Research (New York: John Wiley, 1973), pp. 524-30. Also see the much longer statement by Michael Scriven, “The Exact Role of Value Judgments in Science,” in J. Blaney et al., eds., Program Development in Education (Vancouver: University of British Columbia Press, 1974). The reader should note that Scriven’s theory of values is not in complete agreement with the three ideas expressed in the propaedeutics section of this paper.

5. It is clear that John Dewey was aware of this form of relationship but, unfortunately, he did not give it a distinctive name, viewing it merely as a more desirable special case of instrumental value. C. I. Lewis, however, devoted more attention to and suitably named this relationship, and in the second half of this century, contributory value has become widely recognized as a distinct structural form.

FALL 1978 VOLUME 28, NUMBER 4


whole was believed to have. Moreover, since the part-whole relationship actually does exist in many situations, failure to recognize it is a bar to adequate or justified evaluative conclusion and practical decision.

We have identified three ways in which an evaluatum may be empirically related to established valuables: (1) as an exemplification, (2) as an instrumentality, (3) as a contributory part. To evaluate is to assess the merit or worth of something in terms of its relationship to: (1) established ideals, standards, or rules, or (2) an end or goal of established value, or (3) a valuable whole. We, thus, have three distinct structural forms of evaluative inquiry, judgment, and decision.

111. POINT OF VIEW AND SITUATIONS

Both evaluations and decisions of practice are made in sctual (hence particular) situations and from one or more points of view. While we may value something intrinsically, when we evaluate the goodness or rightness of an object or an action, we do not attempt to assess goodness or rightness simpliciter, but the merit or worth of something in situ, as seen in the light of some particular set or sets of considerations. For example, a text-book selection committee does not try to determine which one of the available books is “best” in some ultimate, abstract, or intrinsic sense but which is best in their actual situation from the standpoint of instructional effectiveness, programmatic contribution, cost effectiveness, etc.

It is generally thought that in any complex society there are a number of basic points of view (or ”realms of value” or “universes of normative discourse”)? e.g., legal, aesthetic, economic, moral, plus, of course, the ubiquitous point of view of self-interest or prudence. In addition, any special interest group or institution, such as education, will have one or more special points of view that become a consideration when evaluations or practical decisions are made. For example, in education the legal and moral points of view are always relevant and typically so are other basic points of view, such as intellectual, economic, and aesthetic. But what makes educational evaluations and decisions peculiarly educational, are considerations from its special interest points of view-e.g., professional, programmatic, instructional effectiveness.

Points of view are defined either by literal definition (e.g., legal) or by constitutive principles or rules (e.g., moral), or by some theoretical or analytical taxis of substantive standards (e.g., aesthetic or professional).7 Such defining or constitutive normatives should not be confused with substantive standards or rules that specify what is to count as good or bad making, right or wrong making characteristics, or what is to count as a valuable goal, outcome, culmination, or state of affairs. For example, the Golden Rule may be taken as constitutive of a moral point of view. It thus stands in contrast to, say, the Ten Commandments (or, at least the last eight) which are substantive generalizations concerning what is to count as morally wrong.

In an educational setting, if a professional point of view is defined in terms of, say, three categories, (1) the substance and logic of what is being taught, (2) the psycho- logical and pedagogical aspects of the teaching activities, and (3) the institutional and managerial rules and constraints of the situation, then within each category there may be established rules and standards that indicate what is to count as marks of professional competence. When teaching is evaluated without regard to these categories (e.g., evaluated only from the point of view of effectiveness), the result says nothing at all about professional competence, just as a decision about a course of action without concern for the Golden Rule (e.g., when self-interest is the only concern) says nothing at all about the morality of the action.

6. See Paul W. Taylor, Normative Discourse (Englewood Cliffs, N.J.: Prentice-Hall, 1961), ch. 12.

7. Thomas Green, for example, has analyzed what is involved in school teaching into three broad categories. See his, The Activities of Teaching (New York: McGraw-Hill, 1971). These categories may be taken as defining or constituting, within education, a professional point of view. With suitable standards within each category, one could then evaluate teaching from a professional point of view. See Philip G. Smith, “Structure of Evaluation in Education,” (an unpublished paper presented at A.E.R.A., San Francisco, 1976).

VOLUME 28, NUMBER 4

284 EDUCATIONAL THEORY

It is widely agreed, however, that any action that is at all likely to have a significant impact upon the well-being of any person ought to be considered from a moral point of view. Failure to do so is itself considered to be an immoral act.B Thus it is said that the moral imperative is categorical in contrast to other forms of requiredness which are conditional. No one has the right to be amoral! One wonders, of course, how anyone gains the right to be illogical, how it is right for any citizen to disregard the law;S or what is the justification for evaluating teaching and instructional materials without regard for a professional point of view.

Iv. JUSTIFYING NORMATIVE FRAMEWORKS

Whenever a framework of standards or goals is proposed as a result of analysis, or even when established de jure or de facto, there is always someone who asks “But how do we know that this is really justified?” The answer, of course, is that we do not “really know.” Normative frameworks are endlessly corrigible, just as any other cognitive construction. When such a framework is rationally developed it is properly accorded a prima facie validity. The reason for this is not to protect the status quo but to make possible a reasoned and orderly process of correction. What is needed in order to initiate the correctional process is not just a general or vague doubt but a reasonable doubt about something specific.

Reasonable doubts may be formulated as a result of either further analyses or empirical evidence that runs contrary to the assumptions or expectations that are built into the framework. Frequently, the most feasible technique is to conduct so- called pilot studies that are deliberately designed to provide a more severe or critical test than would be encountered under normal conditions. The point of all this is that there is no way to warrant the merit or worth of a normative framework other than by evaluation. And the structural forms of evaluative inquiry remain the same regardless of content or level.

Take, for example, a set of instructional objectives designed for a course. Once established, such objectives become an important part of the normative framework by which various teaching activities and materials and student activities and learning are evaluated. Reasonable doubts about the merit of these objectives will take the form of reasearchable questions about whether or not (1) these objectives properly exemplify the higher level objectives (aims and aspirations) of the program or (2) using these objectives is a reasonably effective and efficient way of moving toward the goals of the program (including a consideration of side-effects or other unintended outcomes), or (3) these objectives are a contributory part of the overall purpose or end-in-view of the program. Reasonable doubts about the larger program goals will in turn be studied in terms of still higher level considerations-the purpose of the particular institution, the role of education in a free society and, eventually, the contribution of such an education to a life that is good on the whole. What is unreasonable is to attempt to answer questions on different levels at the same time or to try to use the lever of doubt without someplace to stand.

In any undertaking, such as education, in which we have a vast array of accumu- lated experience, there is much that has been so vindicated that only under very unusual conditions would reasonable doubts arise. For example, who would question that children ought to be taught the three R’s? At any given time, much that is in the curriculum is secure enough to permit terminating judgments concerning the value of alternative lower order means and ends. But at whatever level reasonable doubts arise, there appears to be no alternative to persistent analysis and orderly empirical inquiry into the actual and potential contingent relationships between what is prob- lematic and what is settled in an actual situation.

8. More accurately, it is morally wrong to act as if one were amoral. 9. This is not to say that civil disobedience or other deliberate breaking of a law is always

wrong.

FALL 1978


v. THE ROLE OF EVALUATIONS IN PRACTICAL DECISIONS

It is sometimes said that evaluation consists of developing data that are useful to decision makers. While this is obviously not a proper definition of “evaluation” it does, nevertheless, call attention to the fact that professional evaluators are, typically, employed to provide answers to questions of the form “What is the relative merit or value of this or that (from such and such points of view)?” rather than to answer the question, “All relevant things considered, what ought one to do in a situation such as this?” This is the question that must be answered by the “decision maker.”

The structures of practical decision are the same as the structures of evaluation; they involve the same combination of data and normative networks (requiredness) that enable orderly inquiry and judgment to result in defensible but defeasible conclusion. The conclusion of a practical deliberation is that an action or course of action is the right thing to do, all things considered. But this is just another way of saying that this action is the best thing to do in the situation that prevails. The alleged break between deontological and axiological theories is thus bridged.

The essential difference between practical deliberation and evaluation is that the data that enter into practical deliberation are evaluative rather than empirical; they are the result of evaluations rather than observations. The reason why this must be so is because there would be no point in deciding that one thing rather than another is the right thing to do unless the alternatives were seen as having differential value. A judgment as to what is right or wrong is thus a compound evaluative judgment-a judgment as to what is the best thing to do in the light of all the relevant evaluative data and the obligations that exist in the actual situation.

In education, the evaluative data supplied by evaluators to decision makers are seldom sufficient for the “all things considered” decision that is required. In addition to the evaluative conclusions concerning the merit (from various points of view) of materials, methods, programs, the decision maker will project the probable value of consequences of alternative actions from several points of view (e.g., political, public relations, staff morale) and then place all of these evaluative data in the context of the normative requirements and constraints of the situation (e.g., legal, moral, professional). He thus decides what is the right thing to do in a situation such as that.

It is an interesting question as to whether or not decision makers in education would actually make better decisions if they had a clearer understanding of the nature and structure of evaluative and practical deliberation. But since no such claim for practical benefit is thought necessary to justify doing philosophy, I have stopped short of making any such claim as a justification for my inquiries.

VOLUME 28, NUMBER 4

Documents

Evaluation and Practical Deliberations in Education