ED 073 448

AUTHCRTITLE

INSTITUTION

PUB BATENOTEAVAILABLE FROM

EDRS PRICEDESCRIPTORS

MENT RESUME

CS 000 430

Ruddell, Robert E., Ed.Accountability and Reading Instruction: CriticalIssues.National Council of Teachers of English, Urbana,

7351-

National Council of Teachers of English, 1 KenyonRoad, Urbana, Ill. 61801 (Stock No. 00037, $1.50nonmember, $1.35 member)

NF -$0.65 HC-$3.29Behavior; *Behavioral Objectives; Behavior Patterns;curriculum; Curriculum Development; *EducationalAccountability; *Performance Contracts; PerformanceFactors; Reading; *Reading Instruction; *ReadingTests

ABSTRACTThe intent of the Commission on Reading of the

National Council of Teachers of English in developing this group ofarticles was to provide insight into critical issues related toaccountability and reading instruction. The initial presentation byJames Laffey develops a brief historical analysis of educationalaccountability, followed by a discussion of the alternatives andproblems that must be understood in developing an accountabilitysystem. Richard Hodges identifies the nature and sources ofbehavioral objectives as he examines assumptions related to goalsinstruction, measurement, and the content and methodology oflearning. Kenneth Goodman's paper on testing concerns reading tests,design problems in constructing reading tests, and abuses of tests.Advantages as well as limitations and disadvantages of performancecontracting are discussed by Mary Galvan. GAlvan concludes herdiscussion by identifying needs and making recommendations useful indeveloping curriculum-oriented guidelines for schools interested in-performance contracting. The concluding discussion by Harold Herberidentifies a range of critical issues based on the precedingdiscussions. (Author/WR)

CD

w

accounza iLicv anyRE Inc irili:RUCCiOn

CRIFJCAL

ROBERT B. RUDDELL, EDITORUNIVERSITY OF CALIFORNIA, BERKELEY

FILMED FROM BEST AVAILABLE COPY

U.S. DEPARTMENT OF HEALTH.EDUCATION & WELFAREOFFICE OF EDUCATION

THIS DOCUMENT HAS BEEN REPRO-DUCED EXACTLY AS RECEIVED FROMTHE PERSON OR ORGANIZATION ORIG-INATING IT POINTS OF VIEW OR OPIN-IONS STATED DO NOT NECESSARILYREPRESENT OFFICIAL OFFICE OF EDU-CATION POSITION OR POLICY

NATIONAL COUNCIL OF TEACHERS OF ENGLISH1111 KENYON ROAD,. URBANA, ILLINOIS 61801

PERMISSION TO RI PRODUCE I HIS COPSRIGHTED NIA TFRIAT HAS SEEN (MANTEL)RtNat 1 Council o

Teachers of Englishto ERIC AND ORGANIZATIONS OP, RATINGUNDER AGREE,VENT5 WITH THE U S OFFICEDE EDUCATION FURTHER REPRODUCTIONOUTSIDE THE ERIC SYSTEM REIMIRE5 PERMISSION OF IHE COPYRIGHT OWNER

NATIONAL COUNCIL OF TEACHERS OF ENCLISft

COMMISSION ON READINGRobert B. Ridden, University of California, Berkeley, DirectorHelen G. Bacon, University of California, DavisCharlotte K. Brooks, District of Columbia Public SchoolsCharles R. Cooper, Stale University of New York at Buffalo:Margaret Early, Syracuse UniversityThontas J. Edwin-cis, Stith: Univoixity of New York at BuffaloCarolyn Wicker Field, Free Libi-ary of Philadelphia, PennsylvaniaMar Calvin, Texas Education AgencyKenneth S Goodman, NVayne State UniversityDoris V. Gunderson, U.S. Office of EducationHarold L. Ilet-ber, Syracuse UniversityRichard E. Hodges, University of ChicagoWilliam A. Jenkins, Portland State UniversityJames L. Laffey, Indiana UniversityKenneth W. Lund, Scott, Foresman and CompanyJanet Schalk University of California Laboratory School, BerkeleyRichttrd L. Venesky, University of Wisconsinflohert A, Bennett, San Diego Unified Sehocl District; cx officioHobert F. Hogan, NCTE Executive Secret try; ex officio

EDITORLAL BOARD Richard Corbin, Charlotte S. Hock Owen Thomas,Richard Lloyd-jones, Roy C. O'Donnell, Robert F. Hogan, Paul O'DeaSTAFF DESIGNER Norma Phillips Meyers

ISBN 0-8141-0003._Library of Congress Catalog Caircl Number 73-79000NCTE Stock Number 00037

Copyright 0 1073 by the National Council of Teachers of EnglishAll rights reserved. Printed in the United Suites of America

FOREWORD

The education profession is presently confronted with anunparalleled demand to account for student achievement andteacher performcmce, Systems and management models fromspace age technology are being applied directly in the formula-tion and evaluation of reading and language curricula, The"scientific management" offered by a range of publishers throughinstructional systems, management systems, and performance-based contracts creates at least the impression that pedagogy israpidly being transformed into educational technology. Manystate legislatures have completed, or are in the process of com-pleting, legislation which will "insure" that adequate evaluationof the educational product will take place. Our 'accountabilityera" of the 1970s is in many ways similar to education's "efficiencyera" of the early 1900s. Teachers and administrators are intenselyconcerned about key issues related to instructional objectives,testing, cost accounting, and about the way in which these con-eepts Nvill affect instruction and more importantly the learner.

From the time of the creation of the Commission on Readingby the National Council of Teachers of English in 1970, its mem-bership has been intensely concerned about issues related toaccountability and reading instruction. This is reflected in Com-mission-sponsored meetings at the November, 1971 Las VegasNOTE Convention and the May, 1972 Detroit IRA Convention.The discussions that follow derive from these meetings and pro-

iii

iv/FOREW0110

vide a basis of understanding while raising critical issues relatedto accountability and reading instruction.

The initial presentation by James Laffey develops a briefhistorical anitlysis of educational accountability followed by adiscussion of the alternatives and the crucial 7obleras which mustbe understood in developing an accountability system. Threeissues introduced in Laffey's paper 'behavioral objectives

and performance contractingare discussed in depth byRichard Hedges, Kenneth Goodman, and Mary Ga lvan.

Richard Hodges identifies the nature and sources of be-havioral objectives as lie examines underlying assumptions relatedto goals of instruction, measurement, and the content and meth-odology of learning. He then considers the consequences thatderive logically from the assumptions. His critical analysis of theformulation of objectives provides important cautions and guide-lines for reading instruction.

T1u penetrating paper on testing by Kenneth Goodman con-cerns reading tests, statistical fallacies in reading tests, designproblems in constructing reading tests, and Arises of tests, Goodhman poses critical questions test-makers must consider if theirinstruments are to become valid and move beyond sophisticatedtest theory. Finally, he speculates on the nature of future readingtests, giving special consideration to the evaluation of readingachievement "as it really occurs in natural language."

Mary aary Ca Ivan's discussion provides n overview of recentdevelopments in performarce contracting, ranging from concernswith the early Texarkana project to insights derived from herperformance-contracting work in the Texas Education Agency.Advantages as well as severe limitations and disadvantages ofperformance contracting are discussed. The consideration givento legal questions deserves careful attention. Galvan concludesher discussion by identifying needs and recommendations usefulin developing curriculum-oriented guidelines for schools inter-csted in performance contracting.

The brief concluding discussion by Harold Herber identifiesa range of critical issues based on the preceding discussions. Al=though the intent of these summary issues relates to reading in-staction, the reader will quickly recognize their general applica-bility. Herber's call for a re-examination of our priorities in light

FOnEW0111)/v

of egress of accountability deserves careful _I ught an ,n-sideration by all members of the professim

The intent of the Commission in developing this discussionwas to provide insight into critical issues related to accountabilityand reading instruction- Hopefully, these papers will prove tobe o f d ecision-mal:ing value as educators at all levels carefullyexamine the vast array of theoretical and applied problems thatmust be considered in an "accountability cra:'

Robert 13. Ruddell, DirectorCommission on ReadingNational Council of Teachers Englisl

La3LE OF conzEnn

Foreword

ACCOUNTABILITY: A BREE F JUST( PY A I DANALYSISJames L. Laffey, Indian( Univc ily

SOME ASSUMPTIONS ABOUT BEHAVIORALCH3jECTIVES AS RELATED TO READINGIlichard E. Hodges, University of Chicago

TESTING IN READING: A GENERAL CRITIQUEKenneth S. Goodman, Wayne, Slate Univowity

PERFORNIANCE ,CONTRACTING: A ONCE= VERMary Galvan Texas Education Agency

ACCOUNTABILITY: A SUMMARY STATEMENTHarold L. Herber, Syracuse University

A BRIEF HISTORY AND ANALYSIS

An article by Stanley Elam, "The e of AccountabilityDawns in Texarkana," leaves the reader with the distinct im-pressions that (1) accountability is a commonly understood con-cept and (2) it is a relatively new educational concept.' Bothideas are erroneous.

The purpose of tin; paper is to trace the roots of modernconcepts of ,,ccountability to an earlier period in educational his-tory known as the "efficiency era," to discuss the origins of morerecent ideas of accountability, and to analyze some problemsrelevant to Loth versions of accountability.

Early Origins of Accountability

The "efficiency era" hi education began around 1900 andended about 1925, It was an age when scientific managementoffered itself as the panacea for solving all the problems of theschools. Even though scientific management failed in this en-deavor, as many educators would have predicted, the years andexperiences did point out one of the hard realities educators haveto face. Schools exist in a cultural context, and often the culturalcontext dictates how the schools operate.

1 Stanley Elora, "The Age of Accountability Dawns iri Texarkana," PhiDelta Kappan 51 (June 1970), 509-14.

1

vAccouN r LITY: nisTonv AN ANALYSIS

The cultural context of the liciency era" was that of busi-ness and industry. Raymond E. Callahan states that "the rise ofbusiness and industry to a position of prestige and influence[resulted in] America's subsequent saturation with business andindustrial values and practices." 2 And as business and industry'spolicies and leaders began to exert themselves as a map,- culturalinfluence, it became apparent that educators and school adminis-trators Were in extremely vulneratile positions. Within this cul-tural setting, the "efficiency expert- entered the arena to savethe schools from their _own inefficiencies.

In 1911, Frederick Taylor, an industrial engineer; began toe.-:pound a system of scientific management. In fact, because ofAmerica's concern for efficiency, Taylor became prominent na-tionally. This in turn led Taylor to pronounce that his principleshad universal applieabilitu; for he said "his principles could beapplied with equal force to all social activities:

thethe manage-

ment of homes; the management _of our farms; the managementof the business of our tradesmen." 3

Due to Taylor's influence, and the influence of others likehim, the remaining years of the second decade of the twentiethcentury. were devoted to criticizing the

offor their ineffi-

'ncies and asking why the schools of America were not asefficient as business and industrial organizations. One laymanclarified the issue by stating that "if they [the schools] were

theefficient as businessthat

industry, then they could provide. thepublic with results _that could readily be seen and measured," 4The response by educators to these critical observations led tothe exploratory development of many standardized evaluationforms and tests.

Possibly one of the most important single events during thistime was the hiring of a school administrator named FrankSpaulding. Spaulding began his career as superintendent ofschools in Newton, Nlassaelmselts--a "burial ground for superin-tendents." The school board in Newton was notorious for itsconcern with spending. In other words, the dollar was the eduea-

Raymond F.. callahan, Education and the Cull of Efficiency (Chicago:University of Chicago Press, 1062), p. 5.

3 Ibid., p. 43.Ibid., p,

jAMES L. LAD FEY/3

tfoual criter;on in Newton. And it is possible to suggest thatwith the hiring of Frank Spaulding by the Newton schools, educa-tional accountability had its start.

For the school year 1011-1912, Spaulding initiated a costaccounting system in the Newton schools. lie reported to a group

f schooLadministrutors and discussed not only per pupil expendi-ture for the school year but also per unit costs of specific subjectsand percentage. ,ofthe total budgets invested

thespecific subject

areas. In fact, his analysis -led Callahan to the following con-clusions:

Spaulding's conception of scientific management obviouslyamounted to an analysis of the budget. By a study of local considerations he meant a study of the -r-pupil costs and pupil-recitation costs. Ilis scientific determination of educational valueturned out to be a determination of dollar value. His decisionstai what should be taught were inadc not on educational, outon financial grounds. This was not the first time nor was it tobe the last. But this occasion was particularly unfortunate be-cause it was presented to leading administrators from all overthe nation by one of their leaders and because it clothed thisbusiness philosophy and practice Willi the mantle of science!,

While this event relates to the origin of modern concepts ofaccountability, other signify ant event; occurring during that erareflect an entire range of activities, influencing school account-ability not only then but now.

Year Event Related to accountability of1911 Educational cost accounting School administration

recommended1912 Briefing of school superinten- School administi-ation

dents on use of school survey1912 Measurement of educational Student-Teacher

efficiency (adaptation of F.Taylor method to education)

1915 Student efficiency test Student-Teacher1910 Report of the Gary or Platoon School administration

Planmore efficient use of .school space

G ibid., p. 73.

1 : 11 1STORY AND NAL YSIS

1J1; Teacher efficiency record School administration- Teacher1923 [itemise in class size to de- School adminisiration

crease school expenses rec-ommended

1';)52 Advent of school public rela- Scl,oladministratioiltine;

Each event influenced a different aspect of school over:ation: 1911budget or financial record keeping; 1912--schoolsurveys as a technique for reporting school effectiveness; 1912and 1915--precise measurement as a method for determining.,student learning; 1916more effective use of school space; 1917early attempts to assess teacher eflectiyeness;

and 1925of class size to decrease per pupil costs; and 1925admin--ativc reporting to influence community support for schools.

Recent Origin-- of Accountability

While it is not always possible to relate acts of individualsdirectly to a set of given results, recent events concerned witheducation and the schools suggest that our political and educa-tional leaders arc primarily responsible for the renewed hitcresiin accountability. A number of spokesmen and leaders in govern-ment agencies iraVC recommended that accountability and per-formance.contracts be awarded to contractors willing to negotiatesuch contracts. Lcssinger and Allen stale that educators shouldbe "required to describe and measure the behavior expected ofeach student upon completion of the program they propose forfunding- " This Phi Delta ....rjappan article is one product of earlierwork clone by Lessinger who at one time was affiliated with theU.S. Office of Education. Aineh of Lcssinger's work was preparedfor the specific purpose of renewing or developing ideas relatingto accountability in education.

Political leaders have been influential also in stimulating in-terest in modern concepts of accountability. In his 1969 "Stateof Education" address, President Bichard M. Nixon stated that

Leon NI. Lessinger :Ind D. NI. Allen, "Perform:1mo Prosposnls for Edu.cationnl Funding: A Nev., Approach to Federal ResourceDelta Kappart 51 (November 1969), 136-37.

JAMES I.. I. \ FP'El',

there "n corresponding need in the school systems of thenation . . to begin responsible open measurement of how wellthe educational process is working. It matters very little howmuch a school building costs; it matters a great deal how mucha child in that building learns. . ." He continues, "We have, asa nation, too long avoided thinking of the productivity of schools."He also pointed out that although we are spending more oneducation in this country than in the entire rest of the world(65 billion dollars), we are not getting a significant return onthat investment.

Although President Nixon did mention that we spend 65billion, dollars in education, he failed to mention that we spendless than two per cent of our national budget on education. Sec-ondly, he failed to mention that the high cost of educationthe United States is related to the high standard of living in thisaffluent society. So although there were 6 billions spent oneducation, relatively speaking (i.e. relative to the gross nationalincome) this is a very small sum. The most serious error in thePresident's message probably was the statement, "We have, as anation, too long avoided thinking of the productivity of theschools." The history of American education in the twentiethcentury is filled with the concern of educators for the "produc-tivity of the schools,"

Another possible reason for the renewed interest in account-ability, is that it is viewed once more by educators as a way toanswer critics of the schools. If any institution has been society'sscapegoat, it has been the school. Schools have been and arc ex-tremely vulnerable to public criticism. Since they arc supportedand controlled loc. Ily, public criticism and pressure can beexerted in various ways.

One impact on the curriculum has been to re-introducescientific management techniques as a means of eliminating wasteand of improving the efficiency of the schools. Consequently, anumber of program evaluation Jechniques (e.g., PERT, CTPP)were promoted as unique efforts in scientific management Ofschools and school-related organizations. An analysis of themodels for efficient management promoted in the past two

7 callahnn, Edin, the Cult of Efficiency.

c /ACttc U TAISILITY: HISTORY ND ANALY

decades and of the scientific management techniques introducedinto the schools in the 1920s reveals some parallels. According toCallahan, the essentials of the scientific management methodapplied in the 1920s were (1) the measurement and comparisonof results; (2) the analysis and comparison of the conditionsunder which given results arc securedespecially the means andtime employed in given results; and (3) the consistent adoptionand use of those means that justify themselves most fully,by theirresults, abandoning those that fail to so justify thcinselves.8An examination of these- three principles of the scientificmethod reveals the kernel thoughts of current accountability con -cc ts and also of scientific management techniques, i,e,, (1)ineasurementwhich first requires a definition of the objectivesto be measured; (2) analysis of the learning conditions; and (3)modification or adoption of only those means that are efficient oreffective.Also in these kernel thoughts are sonic ideas presented byStephen Barre,. Ho discusses five different approaches to account-ability and states, "The focus heie is on accountability for effec-tive use of resources." 9 Specific proposals include articulation ofgoals, introduction of output-oriented management methods andmost importantlyregular comprehensive evaluation of newand ongoing programs.

The earlier concepts restricted accountability to the internaloperations of the schools. Newer concepts provide for input fromexternal agents. 13arro's article also highlights this difference. Es-sentially, if the schools were to change under the earlier concepts,teacher or principal behavior within the schools had to change,Newer concepts focus on changes that may (1) emphasize in-ternal changes in the school system; (2) create external evalua-tions or educational audits; (3) initiate performance incentivesfor school personnel; (4) make it possible through performancecontracts for agencies and institutions outside the schools to enterthe schools; (5) cause a shift in the decision-making powers fromthe central administration of large school units to individualschool districts or principals; or (6) result in the development oft' lhkl., p, 08.

Stephen Burro, "An Approach1st Jim Public Sehonlq," Phi MN Developing Accountability Measurestitan 52 ( December 1070), 197,

JAMES L. LAFFEY/7

alternative edncational syste Thew six allornatives seem toreveal the malordifferences between the older and newer con-cepts of accountability. Modern _accountability concepts are mhucmore comprehensive and provide for many more alternatives.

Problems with Accountability

One accountability problem, which vas identified earlier inthe century and still remains, is that of measurement. Educatorsin the earlier part of the century recognized the difficulties ofmeasuring educational achievement. One comment made, in 1013is appropriate today: If scientific measurement is to be accom-plished, we must have units or scales of measurement which willenable us to make measurements which are verifiable by otherobservers. We may not hope to achieve progress except as suchmeasuring sticks arc available or may be derived." 1e What is upparent in this statement is that educators realized the necessity forvalid and reliable measuring instruments oven then.

More recently, there has been a growing dissatisfaction withthe technical development of modern standardized tests11 Thisis especially true in reading, where standardized tests are inap-propriate for evaluating the reading behavior which should beevaluated in any program holding students or teachers account-able. Earlier educators saw the need for developing appropriatetests. Modern educators, after having evaluated valid and reliablestandardized tests, are calling for new kinds of tests, Glaser andNitko suggest that new kinds of tests be developed to measureinstructional outcomes. "Tests which arc used for making instruc-tional decisions demand special elaracteristicscharacteristicsthat are different from the mental test model that has been suc-cessfully applied in aptitude testing work." 3 2 They go on to state

1° Callahan, Education and time Cult of Efficieneth p 101.

'' See Roger Farr, Beading: What Can Be Measured (Newark, Del.;ERIC/CR1ER and International Rending Assecitition, 1909); also Stephen13. Klein, The Uses and Limitations of Standardized Tests in Meeting theDemands for Accountability,- UCLA Evaluation Comment 2 (January1970), 1 -7.

" Robert Glaser and Anthony j. Nitko, "Measurement in Learning andInstruction," Edueatlonal Measurement, ed. Robert L. Thornlike (lAraNh

ingten, D.C. American Council on Education, 1071), p, 002.

8/ACCOUNTABILITY: HISTORY AND ANALYSIS

that special types of criteria need to be developed, Of significanceare (1) the creation of items from stated objectives; (2) thecreation of interpretive materials for such tests in terms of testcontent and criteria for performance as well as references tonorms for other test-takers; and (3) the extensive application oftest performance to domains of content from which the test itemswere sampled. In essence, modern educators are calling for ed.telion.referenced tests which interpret an individual's perform-ance with respect to a defined behavioral criterion and which arenot limited to a comparison with the performance of other in-dividuals. In addition,_there is a need for other newer methodsof measuring student behavior related to the affective domain.Finally, there is a need for personnel education that will preventmisadministration, incorrect scoring, and misinterpretation of testresults.

The focus for modern evaluation schemes related to account-ability is broader than earlier attempts at accountability. Teacherbehavior was measured by rating sheets; imineipal behavior, byrating scales." Little information was gathered on more complexaspects of student, teacher, or administrative behavior. Conse-quently, early efforts at accountability were less than effective.The instruments used to observe and rate teachers and principalswere neither valid nor reliable. Reality has not changed drasticallyin fifty years; theory has."

A concern not mentioned in the earlier history of accounta-bility but significant to modern education is curriculum articula-tion. In a number of current accountability sistems (such as the.Banneker School in Gary, Indiana), curricular articulation wasnot considered initially, Only after the Indiana State Departmentof Public Instruction intervened did the contractor attempt tojustify and articulate the curriculum already implemented in theschool. In the Texarkana Project, a lack of articulation or evenof teacher cooperation was noted when some students performedsuccessfully in the reading part of the curriculum (as indicatedby teacher grades) but failed in the regular curriculum of the

"Callahan, Education and the Cult of Ellicionctl."N. L. Cage (ed.), Handbook of Research on Teaching Chicago:

Band McNally, 1005).

JA X11 LAFtervio

English classes. Roth reading and English were snpposed to bepart of the same curriculum. more important than the gradingpractices in the curriculum are the teaching and the learningwhich should take place from one grade level to the next. It isnot apparent that articulation was identified as a problem inearlier accountability programs. In today's Program, there is atleast the awareness that it may be a problem.

It should be noted, however, that the issue of cuvriculuinarticulation in accountability is more serious in some fields thanin others. For example, because there is no scientifically basedreading curriculum, Articulation is less critical in reading thanin some other areas. The curriculums of the most widely usedinstructional approach (basal and basic readers) are developedby authors and publishers. The skill sequence and content areusually determined by what logically and pedagogically appearsbest. The language used is often not determined on the basisof what is known about children's language usage but by whatappears pedagogically wise. Probably the most critical aspect ofthis issue is whether or not the content of the material is palatable.Will the children read and enjoy the material? Will_ the materialin some way parallel the life experiences of the .children? Is thematerial in line with what we know about children's interest andtastes?

In addition to measurement and articulation, another con-sideration slighted was the long-term effects of the instructionalsystem. In sonic current accountability projects, contractors re-fused to negotiate a delayed testing program to determinewhether the student gains were permanent or whether they woulddisappear after six months. Short-term gains are characteristic

education. It se= s essential that in any ethicationzil program,attention be given to the long-range effects. Nevertheless, it seemsthat modern educators are not as concerned with this problemas they should be. Otherwise, accountability contracts would re-quire that the contracting agency be responsible for long-rangeresults as well as short-term results. The contracts Vvould neverbe for only one, year. Only long-term contracts allow the schoolsto evaluate accountability as a mechanism of change.

Long-range planning relates to a fourth problem: the prob-lem of attaining affective goals in accountability contracts: Former

JO/ACC:OW.0'; BILITY: HISTORY A\ 1-

Coremissioner of Education, Jaws Allen, made issue of the affec-tive dernai., in his Bight to Bead speech. His main goal for the1970s lecluded not only the skill to know how to read but also

desiv to read. From the descriptions of some modern ac-countability contracts, it is apparent that their contractors havegiven this some thought. However, only measures of cognitiveskills were wed to evaluate the Texarkana Project, Methods andtechniques focussing on the affective domain were not apparent`in earlier attempts to deal with accountability. In more recenttimes, it seems that educators are knowledgeable about account-ability but fail to give due consideration to the assessmeat ofaffectin reading instruction at least. This must change,

An aspect of accountability focussed differently today isteacher impact. From one viewpoint, accountability may requirethe development of new relationships with the teachers, teachers'unions, and other organizations. From a second viewpoint, ac-countability may need close observation and control. Will in-dividual teachers still be able to practice the art of teaching?Or will teaching become skill-oriented drudgery? Although inearlier times teachers' unions werq not a concern, the "art" ofteaching was, Today the two concerns should be combined.Teachers' unions will play a role in negotiating accountabilitycontracts; therefore, they should reflect a concern for the art ofteaching as well as the skill of teaching.

Involving the community in the educational decision-makingprocess is also a problem in the accountability process. Therehave been a number of effective innovations lost to the schoolsbecause of insensitivity to the local community. Today parentswant to particip,tte in itny decision-mtiking process that ziffectsthe school life of their child. In earlier times, as well as today,the community stimulated the growth of accountability systems.It is not quite clear what role the community will play beyondthis in modern accountability programs.

A final and probably the most crucial problem related toaccountability is evaluation. At the present time, different viewsof accountability. suggest different approaches to evaluatingschool programs. Some are concerned with student achievement.Others focus attention on teacher behavior, Still others are eon-corned with administrative behavior. Whatever the dimensions

JAMES L. LAFFEY/11

of accountability are, the evaluation plan must correspond to theaccountability plan, and a sufFicient number of trained personnelmust be available to implement the evaluation system. In earlierattempts at accountability, comprehensive evaluation plans wereunknown. Today they ai considered vital to the success of aprogram.

Sum y

In this paper, concepts of accountability were related to anearlier efficiency iTa in education, described in terms of the. morerecent origins of todav's accountability, and anal' zed in regardto problems relevant to both older and newer ideas of account-ability. Obviously, aceouratibility heeds to be defined eil gyrationally to meet the contracts and demands of the governing agency.The realities and failures of accountability in the past will enableeducators to avoid those failures in the future.

You may perceive that I have a positive attitude towardcountability. I assure you that I do. But only because I think thataccountability can have a positive impact on the process ofeducating children. And because I believe that the only wayaccountability as such will have a positive impact is for educa-tors to reject cynical and skeptical attitudes toward accountabilityand to seize it as an opportunity to hold the public accountablefor the resources and support needed by the schools.

bOIME anunnionl.130UZ 3E1-aViORLIL O3iECtiVE

3ELACE ZO

In this short paper, I shall sot forth some views about be-havioral objectives, some principal assumptions that underliethem, some statements of their asserted purposes, and some issuesin relation to behavioral objectives as they pertain to reading.Before doing so, however, it is useful to place this brief discussionof behavioral objectives into its larger educational context.

As a social institution, the education enterprise has always beenliable to society for its successes and failures in educating thenation's children and youth; "accountability" is not a recent intro-duction into American public education. Yet, today, during a timewhen national and international turmoil is causing us to lookdeeply at our traditional values, even greater demands are beingmade than has historically been the case for the nation's schoolsto reaffirm their credentials as society's formal educationalagencies. "Performance contracts," "voucher plans," and the likeare all symptomatic of pervading social forces which bring pres-sure upon school systems, their personnel, and their instructionalprograms to demonstrate both efficiency and effectiveness ineducating the youngin short, to be accountable.

The larger ( and more significant) question of "education forwhat?" cannot be examined in the limited space availiible here.For present purposes, we shall assume that education _may beconstrued as Cl change, in behavior toward some acceptable goal.

13

11/ASSUNIPTIONS ABOUT BEHAVIORAL ()MEC -IVES

The sot of goals for which education is responsible includes theaegitisitioa of basic understandings of the world in which welive, basic competencies for effective participation in the largersociety, _aad the basic values upon which the society rests, Inshort, education is the process for the transmission of the premisethat certain knowledge, skills, and values are deemed neees-smy for individual fulfillment and for the maintenance of thesocial order. It is these educational outcomes that are subject to_ 'cost accounting' ..and give particular support to the notion ofbehavioral objectives.

What Are Behavioral OWN., es?

Behavioral objectives are claimed by their proponents to bea technical device which can improve the effectiveness of school-ing. Tangible evidence that objectives are in fact being reachedis provided in the form of observed behavior manifested by astudent in relation to _a particular objective; the statement of theubjectiv, identifies what behavior a student should demonstrateif the objective is achieved: Here is an example of a behavioralobjective in reading as applied to a population of pupils:

To inemise the reading achievement and skills of disadvantagedlirst-grade students as measured by the total seores'obtained onthe Stimford Reading Achievement Test, Primary Battery 1, inwhich the following is obtained: (a) a 25% decrease in thenumber of students entering the second grade who are one-halfor more years behind grade level than was the case for theprevious year, and (b) an average of .3 increase in the gradelevel achievement of students entering the second grade overthose of the preceding year,1

We should remind ourselves that educational objectives arcvalue tudgments about the ends of education; thus behavioralobjectives also are value judgments of ends that are stated interms of observable pupil behaviors, They are viewed as a meansof identifying goals and of describing the outcomes of instruction

IL NicAshan, Writing lichaviorai Objectives: A New App(New York: Harper & Row, Publishavg, 1070), p. O7,

IIICHARD E. I IODGES/15

_rms of performances that children should have as the result.heir participation in an instructional activity._According to this view, any Nvorthwhilo eth t to improve

educational programs _must include proper identification anddescription of spOcific behavioral objectives if the program is tobe evaluated, For, unless those obje&ives unique to a givenarea of study are clearly stated, the pupil, the teacher, and theprogram planner will not know what is to be done, how it is tobe done, or when the goal has been accomplished:'' Behavioralobjectives are seen, in short, as a significant way of helping every-onewho is involved in an educational activity to clarify what thegoals of that activity are and what criteria to employ in determin-ing if those goals are attained. The proponents of behavioral ob-jectives tacitly assume that instructional goals can be stated interms that command agreement as to their rc ferenne by all whoIINC

r Sources Giving l ise to Behavioral Objectives

It is significant to point out that the impetus for behavioralobjectives has emerged from outside the particular subject fieldsto which they are generally applied. In the main, the behavioralobjectives movernent represents an amalgam of systems _analysts,behavioral psychologists, and measurement theorists who, fromtheir respective vantage points, require that outcomes of an ac-tivity be measurable. In turn, these forces have had a significantimpact upon curriculum decision-making in recent years, wherethe criteria of efficiency and effectiveness arc yardsticks for eval-uating instruction. There are constant pressures to give priorityto those educational goals that can be most ,ly andciently attained, to do that "which u:orke

Views of Knowledge and the Learner

Further embedded in the issue of behavioral objectives area and what we learntosubstantive matters pertaining to

P.

ASSUMPTIONS ABOUT BEHAVIORAL OBJECTIVES

the _ Aogics and philosophies that guide us. Behavioral objec-tives arise part out of, and are clearly nourished by, stimulus-response cmmectionist learning theory in which learning isviewed mechanistically and where precise statements of fact, care-ful observation, and limited induction are accepted as paradigmsof knowledge and thus ar paradigms of human behavior. Thisframework, of course, contrasts with those views of learningwhich accommodate the imaginative, the intuitive, and "dis-covery" tis fundamental attributes in the construction of 'knowl-edge.

Accordingly, from the behaviorist's view, knowledge is pre-sumed to be certain and absolute as opposed to uncertain andrelative; it is also presumed that knowledge is impersonal ( thesame for all) as opposed to the view that experienc4is trans-formed into personal schema which further guide an individual'sinteraction with the world about him. From the behavioral view-point the. child is regarded largely as the recipient of knowledge,rather than being seen as an active participant in its creation.

A Brief Critique of Behavioral Objectives

I have briefly reviewed some underlying aspects of behavioralobjectives which lend perspective to a _more practical examinatibnof them. Three underlying features of behavioral objectives thathave been pinpointed are (1) there is an assumed consensusabout the goals of instruction; (2) an emphasis is placed on theseinstructional activities that "work" and are measurably workable;and (3) what is learned and h ©w it is learned are regarded es-sentially as the same for all.

The consequences of these points in respect toofdevelopment and teaching method raise a number of significant

questions. First, from the behavioral objectives viewpoint, it canbe argued that only that which can be tested is that which canbe stated objectively; the inference is that only what can hetested shall be taught. Given this argument, the curriculum de-velopees task and the teacher's responsibilities are clearly de-lineated, if not prescribed, When this mandate is applied inextreme form, a premium is put on the teaching of information,

R CHARD E. HODGES/17

on rote recall of definitions, rules, and principles, and on thesolution of problems with only one solution. (Hew can divergentthinking outcomes be stated in behavioral terms: as, for example,"How many things can you do with a paper clip?'") Learningthus becomes a "planned happening." The adherence to be-havioral objectives necessarily precludes the belief that learningand knowing can have tacit (personal) dimensions as well asexplicit ones ;'

Second, there is a belief that observed performance in aninstructional setting is conclusive evidence that the pupil hasmastered a behavior which has been taught. But, children (andadults) sometimes behave as though they have learned when theyhave not; while, more significantly, they sometimes learn withoutovertly displaying behaviors that would give testimony to thelearning. Coupled with the realization that students have differentcapabilities for learning and different attitudes toward learning,it seems presumptuous to apply the same educational standardsupon which behavioral objectives generally are based to all chil-dren in all circumstances:

Moreover, teachers also differ in their abilities to teach andin their styles of teaching; they bring to an instructional setting arange of competencies and backgrounds which parallels that oftheir students. Where behavioral objectives structure the child'slearning situation, they in turn constrict the teacher's range ofinstructional alternatives. Both teacher and pupil can becomeconstrained by the planned happening.

Third, there is a belief that we either know or can readilyidentify the educational objectives toward which we ought tostrive. Any curriculum that is used by real people will have out-comes that cannot be anticipated. Of those outcomes that can bereadily identified, fewer still are readily translatable into be-haMoral terms. A danger exists that a curriculum becomes onlythose outcomes that can be readily specified. As Atkins has soclearly said:

3 James B. MacDonald and Bernice J. Wolfson, "A Case Against Be-havioral Objectives," Elementary School Journal 71 (December 1070),119-28.

4 Harry S. Mandy. "Can Research Escape the Dogma of BehavioralObjectives?" School Review 70 (November 1970), ,13-56.

18 A 'U fP'1'IONS ABOUT 13EllAVIORAT, ORJECTIVES

. _it is difficult to resist the assumption that those attributeswhich we can measure are the elements which we consider mostimportant. . . . The behavioral analyst seems to assume for anobjective to be worthwhile, we must have methods of observingprogress. But worthwhile goals come first, not our methods forassessing progress toward these goals. Goals are derived from ourneeds and from our philosophies. They are not and should notbe derived primarily from our measure. It borders on the ir-responsible for those who exhort its to state objectives in behav-ioral terms to avoid the issue of determining worth, inevitablythere is an implication of worth behind any act of measurement.What the educational community poorly realizes at the momentis that behavioral goals may or may not be worthwhile. Theyare articulated from among the vast library of goals because theyare stated relatively easily. Let's not assume that what wecan presently measure necessarily represents our most importantactivity.

Behavioral Objectives and Beading

So far I have made a brief explication of behavioral objec-tives without

hasparticular reference to reading. My reason

for doing so has been to keep the focus of our attention on thematter of bobaolorai objectives as the basis of our discussion andto avoid the possible diversion of our attention to theories ofreading, a significant issue in itself. In order now to put readinginto the context of behavioral objectives, I will assert that readingis a twofold process that involves the transformation of writtenforms of language to meaningthat is, the decoding of and thecomprehension of the printed message.

Aspects of the decoding process are, of _course, more suitableto direct observation than arc aspects of. the comprehensionprocess. For, while an individual's abilities to decode may bemanifested through various word recognition tasks, the observa-tion of his abilities to comprehend is loss certainly grounded. Doesnot the meaning of a work of literature, for example, becomeknown only to the individual who interact with it?

j. Myron Atkins, "Behavioral Objectives in Curriculum Design: ACautionary Note," Sete a Teacher 35 (May 1968), 27311

E, HODGES 19

Measuring a. ides toward reading is an even more elusiveprocedure. It seems clear that - -in respect to readingdecoding,comprehension, and attitudes can be cast into behavioral termsonly at

ofcost of decreasing levels of certainty as to the relia-

bility of the measurement as one progresses from skills to under-standing to affect.

A Summation

I should like, to clew with some observations that 1 thinkshould be considered regarding behavioral objectives uI6-a-cisreading. First, I believe that there is an ever present tendency toregard the demonstration of reading skills which can most easilybe observed (for the most part, the decoding skills) as the read-ing process. The ability to read is not simply the stun of wordrecognition skills; knowing is more than the sum of the processeswhich d to it.

Second, how the child is perceived, how and what he learns,and how schooling should be structured, as viewed through thetemplate of behavioral objectives, warrant careful consideration.A rigid adherence to behavioral objectives, if not prescribingones view of reading, at least puts blinders on the viewer andnarrows his range of visible alternatives.

Third, there is a growing fetish among proponents of be-havioral objectives to regard the process of defining behavioralobjectives as an end in itself and to assume that there are simpleprocedural solutions to the many complex problems of education.Behavioral objectives, of themselves, are neither good nor bad,but can be either, by those who use therm

For reading in particular, behavioral objectives ()light to beviewed in terms of their limited utility, particularly in respectto those skills where they have the most utility. As to the highercognitive functions involved in reading comprehension, as wellas in the affective -aspects of reading, their limitations should berecognized.

Reading researchers, developmental psychologists, andilin-guists are now exploring .new frontiers of knowledge about learn-ing, language, and reading and raising significant points about the

20/ASSUMPTIONS ABOUT BEHAVIORAL OBJ ECTIVES

efficacy of the assumptions and theory underlying behavioralobjectives. Perhaps one of the principal but unforeseen outcomesof the behavioral objectives movement is that it causes us to askourselves what we truly believe about the nature of the childand the function of schoolinq,

REFERENCES

Aerasian, Peter V. "Behavioral Objectives and the Teaching of En-glish." English Jounuil 60 (April 1971), 495-99,

Guth, Hans P. The Monkey on the Bicycle: Behavioral Objectives andthe Teaching of English.- English Journal-59 (September 1970),785-92.

Hersh, Richard H., and Cohen, Stuart J. "A Case Against a CaseAgainst Behavioral Objectives.- Elementary School Journal 71

May 1971) , 430-37.Krathwohl, David R. "Slating Objectives Appropriately for Program,

for Curriculum, and for Instructional Materials Development."Journal of Teacher Education 10 (March 1963), 83-92.

Nlaxwell, John, and Tovatt, Anthony (eds.). On Writing BehavioralObjectives for English. Urbana, National Council of Teach-ers of English, 1970.

Purvcs, Alan C. -Of Behaviors, Objectives, and English." EnglishJournal 59 (September 1970), 793-97,

zE5rme m READirlf:A GENERAL CRITIQUE

Never in the history of education have reading tests enjoyedas much status as they do today. They provide a data base usedincreasingly as a means (often the sole means) of evaluatingpupil progress, teacher effectiveness, and program success. Theyare used in research studies to compare methods and materials.They are linked by law in several states with special or basicstate support for the schools. Schools and school systems arepublicly compared on the basis of rankings of pupil populationson reading tests. Election campaigns often center on pupil per-formance on reading tests. Publishers and private contractors aresometimes paid on the basis of student performance on readingtests.

It is always desirable to re-evalnate the evaluators we usefrom time to time. With so many crucial educational decisionsbeing based on reading tests, this re-evaluation becomes urgent.

Uses of Rending Tests

Above, reference has been made to some of the current usesof reading tests. Only two basic uses of reading tests are legi-a-mate. They are as follows:

(I) To measure the effectiveness with which any person uses21

2-2 /TESTING IN READING: A GENERAL CRITIQUE

reading to comprehend written language. Within this, the twomain concerns are (a) flexibility in comprehending a wide rangeof materials and ( b) degree of proficiency as compared to otherreaders or as compared to sonic absolute scale of proficiency incomprehending written language.(2) To diagnose the strengths and weaknesses of readers asan aid to planning instruction which will help to make themmore elfectii:v.

Testing for each purpose will vary depending on the theoryof the reading process, and of reading acquisition which the testeruses. In sonic cases, readiness tests will be used if the tester be-lieves that there are nonrcading tasks which must be mastered asprerequisites tr successful acquisition of reading.

A major weakness of current reading tests is a failure toarticulate views of the reading process and of learning to readas a basis for building the tests, subtests, and test items.

Tests are often built on eclectic traditions of what is im-portant in reading and learning to read. These are sometimesderived directly from popular instructional reading programs; butjust as often the instructional programs derive their rationale fromthe tests on the theory that if something is commonly tested itmust be important. This misuse of tests results in a self-justifyingcycle which institutionalizes tradition.

That cycle tends to block progress in improvement of read-ing instruction through the application of new insights from re-search, theory, and practice. The tester says we must test whatis being taught; and the teacher says we must teach what is beingtested. Innovative programs are judged on the basis of perform-ance by pupils on traditional tests which incorporate the samefaults that the new programs seek to overcome in old programs.

Since tests grow from tradition rather than articulated theory,they develop subtests with large areas of overlap, while leavinggaping holes that are not tapped at

The successful reader is treated as a posser of bundlesof skills rather than as a user of written language. Traditionalsemilogical, sequencing criteria and hierarchical arrangementsarc imposed on these skills which are isolated, for ease in testing,out of any context of language use which they may have.

In the absence of a strong base in reading theory, current

KENNETH S. GOODMAN/23

reading tests substitute sophisticated test theory. Surrounded bynorms, percentiles, measures of significance and other statisticalarmor, the tests give an impression of scientific validity whichconceals their hollow cores.

Tests, any tests, will produce statistical results with popula-tions that take them, These results c in be mathematically maMp-ulated. By adjusting the test items on the basis of the statisticsthey prodlice, one may achieve neater statistical patterns. But infact one may never draw conclusions whose significance gobeyond the validity of the assumptions on which the test is based.

Criterion-referenced tests. those which measure achievementof stated goals rather than comparing to a statistical norm, areeven more in need of being rooted in a strong theory. In reality,they tend to be selected skills arbitrarily sequenced.

Statistical Fallacies in Beading Testing

There are a number of key statistical fallacies that are widelyinemporated into justifications for misuses of reading tests. A fewwill be explored here,

Nanning over diverse populations. Sophisticated test theorydictates that norms or percentiles should he developed by admin-istering the test to a structured sample of the general school popu-lation. Care is 1,,' n to include the right proportion of urban,suburban and rural pupils, white, black and other; cast, west,north and south, and so on These national norms or percentilesare then published. The implication is that the test is valid for usenationally. Though test-makers often suggest that schools maywish to use regional or local norms, there is a clear implicationthat individual pupils, classes, schools and districts may be use-fully compared to the national norms.

But now let us introduce just one condition. Suppose that thetasks and questions on the tests are selected so that they favorone group (white, suburban, middle-class eastern pupils) overall others. This could be the result of choosing to deal with ex-periences and concerns more common among the favored group.Differences between groups then would be at least partly the re-sult of the relative relevance of the test and not any actual differ-

in reading effectiveness. Furthermore, simply using local

24/TESTING IN READING: A GENERAL CRITIQUE

norms would not remedy this problem, !inee pupils' scores wouldreflect the degree to which they matched the favored group inbackground. Now add to this proWom regional and social dialectdifferences, and the justification for national use of the testthrough a structured sample is even more highly suspect. Onlyif the test-maker argues that in tact all pupils should be judgedby the degree to which they compare to a high status group issuch norming justifiable, and such a judgment involves valuequestions which cannot be answered statistically.

The importance of small variations in test performance. Thestatistical treatment of raw scores on reading tests makes it pos-sible to equate them to grade placementthe grade of the aver-age pupil who achieved the score in the norming

Since the test must use a limited number of items becauseof time considerations, the differences, particularly at the upperand lower ends of the scale between the average score in twoadjacent grades, may be only a few items. One more questionright can acid several months to the grade equivalence of a pupil.Consider this hi relation to the relevancy questions raised aboveand it is char that a slight bias against a group can explain sta-tistically significant differences in group means,

Sky-hooks, split-halves, etc. To cope with questions of whethertests are testing what they should and doing it consistently, anumber of statistical devices have been employed, One usedparticularly for new tests is to correlate them with other oldertests. If a high correlation is achieved, then validity is assumed.However, if the new test is in fact measuring what the old testdid, then why is a new test needed? And if the new test employsnew insights, why expect it to correlate with the old? This sky-hook method of anchoring tests to each other clearly says nothingabout the extent to which reading is re-Ay being tested, A currentfederally funded project seeks to establish a new test to which allcurrent tests could be correlated. Such a test, appropriately calledan anchor test, would surely anchor reading permanently to thepast. Similarly, using split-half techniques to prove that a test isconsistent within itself proves only a symmetry on whatever biasesare built into the test and does not offer evidence about the valueof the test.

KENNETH S. 0OODNIAN/25

Hard Is Hard But Why Is It Hard?

Test tbemy requires that some items should be missed bymost mils, and some items by a few, with the rest of the itemsranging in-between. Further. the high scorers should be the onesgetting the hard items right, and the low scorers should be theones getting the easy items wrong. Close examination of reading-test items reveals that the items arc often difficult for irrelevantreasons: ambiguity, equally correct wrong alternatives, and soforth. The fact that few people get them right may indicate thatthey are hard, but it may also indicate that they are irrelevant orpoorly written. The fact that the right pupils get them right maydemonstrate more that high scorers are good at thinking like testwriters than that they are better readers. Again, what is importantis that statistical evidence cannot substitute for intrinsic criteriain judging the relevance or difficulty of items.

Related to this statistical fallacy is an artifact that resultsfrom weighting certain items by virtual repetition (a series ofvery similar items). A pupil tends to get all like items right orwrong if in reality his performance reflects knowledge or lackof knowledge. A minor lack becomes magnified into a major weak-ness. An example of this is the syllabication sections of certaintests.

Averaging Ends and Means

A statistical fallacy occurs in many reading achievement testswhen an overall score is calculated which combines scores on"skill" subtests with those on comprehension. Since skills areostensibly the means by which comprehension (the end productof reading) is achieved, such a score is meaningless.

Counting in Diagnosis

Statistics which produce summary scores are much easier tomanipulate than those which relate to complex phenomena

bydetail. The effective use of diagnostic testing is often defeated bybeing more concerned with quantity of errors or a grade levelequivalence, than with the specific phenomena revealed by per-

26. STING IN READING: A GENERAL CRITIQUE

forniance on the reading tasks involved. Number of readingor ms wrong gives little diagnostic information,

Design Problems in Constructing Reading Tests

As from the statistical fallacies confusing test use andmisuse, there are a number of design problems which test-makershave not yet solved adequately.

Convergence. One of the most difficult design problems inwriting tests is that there must always be a right answer. Thisleads to focus on convergent responseson those which matchthe preconceptions of the test maker. Two groups are hamperedby such tests: culturally divergent groups whose experience doesnot match the testers, and creative thinkers who are able to see"different" relationships. If you march to the beat of a differentdrummer, the test penalizes you.

Pupils who know too much. Multiple choice responses aredesigned to mislead pupils with common misconceptions. Sincea misconception is better than no concept :,t all, pupils arcpenalized for knowing a little and will be wrong more thanchance would predict. Even worse off is the pupil who knowsmore than the test-maker. He will often reject the "right" answerbecause he recognizes it as a misconception or oversimplification.

What they learn vs What they know. In testing comprehen-sion, it is case to end up testing general knowledge. The pupilmay he able to answer the questions without reading the testselection. To overcome this problem requires a measure of priorknowledge or tests on material all pupils lack background for.The latter is a virtually impossible task.

How test-wise are the subjects? Pupils vary greatly in theircontrol of devices for scoring higher on tests. Only some pupilshave learned simple devices like skipping troublesome items,quick identification of tasks, eliminating obvious wrong choicesto narrow the range of possible options, going to questions with-out reading test paragraphs or answers without reading questions,and so forth. There appears to be no way to neutralize fully thiseffect, which is also linked with the pupil's basic desire to dowell (or his indifference).

Honesty. Related to the latter problem is one of honesty.

KENNETH S. GOODMAN/27

This is a complex problem because many pupils avoid usingtechniques which would produce higher scores. They think thatis like cheating. In fact, those who give reading tests often behavehypocritically. Test-makei-s discourage guessing, tell pupils toread each item before looking at the answers, refer to the testswith smaller children as "a game we're going to play." Theytell pupils that the test is only to help them and that the scorethey make is unimportant.

But in fact the score is the only part of the test in which thetest-makers arc interested, and decisions are made on the basisof the test scores, which may well effect the learner adverselyplacement in a low track, for example. The pupil who is honestand trusting becomes a statistical victim.

'low they do us What they do. Since test-making involvescounting right answers, there is a tendency in trying out testitems to ignore the basis on which pupils respond to questions,In many cases, pupils are producing both wrong and right an-swers for the wrong reasons. Subtests turn out to be testing some-thing quite different from what they claim to test. Auditory dis-crimination tests, for example, turn out to be testing largely theability to deal with abstractions. Some pupils on such tests willresort to matching spelling patterns, producing a fair number ofright answers without being able to abstract sounds from soundsequences.

Making the test clear. -Pupils frequently do not understandwhat the task is that a particular subtest requires of them. It isquite likely that this accounts for a considerable amount of thevariation of performance, particularly among younger children.

Distortion of tasks. Finding a format for test items which issuitable for inclusion in a group-administered reading test fre-quently results in a distortion of actual reading tasks. Some ex-amples follow:

(a) Items too short. Research on reading miscues has demon-strated that short items are harder to read than longer onesbecause reading involves building up expectations on the basisof redundanciml A sentence is proportionately harder to read

D. llenosky, "A Psyeholinguistic Description of Oral Reading MiscuesGenerated during the Reading of Varying Portions of Text by SelectedReaders from Grades Two, Four, Six and Eight" (Doctoral dissertation,Wayne State University, 1971).

28 /TLSTIi:G Iti READING A GENERAL CRITIQUE

than a paragraph, a paragraph harder than a page, and anisolated word hardest of all. Since short items predominate intests (words, phrases, sentences), reading test items will beharder than reading stories or other natural materials.(b) -Wards in isolation are patieulinly hard to "read" becausethere arc no grammatical cues from the sentence structure ormeaning cues from the context to help identify the word mean-ing, yet many subtests deal with isolated words.(c) Comprehension questions that can be treated like nonsense.Many questions arc stated in such a way that readers mayanswer them by transforming the question to a statement andsearching the text for a match without necessarily understand-ing. They mimipulate the sentence patterns as if they were non-sense like the jabberwock. Q--What did the momeraths do?AThe momeraths outgrabe.

Abuses of Tests

There are uses of tests which clearly violate the publisher'sadvice on limitations in their use.

One common, recent abuse is giving tests at too frequentintervals. Many requirements for reporting progress in readingas often as once a month arc built into contracts for research ordemonstration projects. In the name of accountability, tests arebeing used to measure small increments of progress which theyare simply not designed to handle. Factors such as regressiontoward the mean (a tendency for high and low scores to movetoward the mean on repeated testing) become very important.Immediate, often temporary, results become more highly valuedthan long-term, permanently held gains.

If jobs, funding, and professional status and pay are madecontingent on pupil performance on reading tests, then the tend-ency to teach to the test and to build curricula around the testwill become a major trend. Instead of the curricular goals beingcentered around effective reading, the girds become performanceon specific tests. Instead of tests functioning as a measure ofachievement, they are turned into ends in themselves. Even ifthey had a sound theoretical base, that would be unfortunate. Intheir current state, it could be tragic. It could lead to a new kindof widespread functional illiteracy.

KENNETH S. COODNIANIN

Another abuse of tests which we have touched on earlieris the use of test scores without close examination of each pupil'stest performance. It is not enough to say Mary Lopez is readingon the 2.2 level. Her responses to subtests and items must heexamined closely so that her strengths and weaknesses are re-vealed, Standardized reading tests are given wholesale to massesof students. But their results must he interpreted for each learnerif they are to be useful in improving that child's reading, Everychild has a right to be treated as an individual and not as a teststatistic.

A related problem is the use of tests as exclusive means cif(1,k-dilating pupils' reading effectiveness, ignoring more extensiveevidence of competence because it is less easily quantifiable.Teachers will frequently tint a low scorer on a standardized testas a poor reader, even though they observe him functioning as anadequate reader every day in class. The quantifiable performanceon tests is so intimidating to the teacher that he will not trusthis own professional judgment.

Often the pupils perform poorly on the test because it isirrelevant to them and penalizes the for linguistic, experiential,and cultural differences (not deficiencies). Instead of rejectingthe test as irrelevant, wholly or partly, teachers and administra-tors accept the test and misjudge the achievement, strengths andweaknesses of the pupils. Programs are then planned to remediatedeficiencies that never really existed.

Other abuses of reading tests occur in evaluating newmethods and instructional materials. Frequently., little eonsidera=tiOn is given to the basic soundness of the method or the materialsor the principles on which the tests are based. Bather they arejudged largely by how well pupils do on pre- and posttests. Whileeffective instruction must ultimately be judged by the learningit produces, progress in improving instruction cannot conic byusing a trial-and-error technique for evaluation. Not all programsare worth trying, nor can the test results be usefully interpretedif the instructional program is not thoroughly analyzed.

If instructional methods and materials are built around tests,it is likely that pupils will improve their performance on thetests. The most extreme version of this is to use the test items asthe instructional program, asking the pupils to respond over andover until they produce right answers all or almost all the time,

}0 /T STING IN HEADING: A GENEli JE

This ability to do well on the test is then assumed to prove learn-ing has taken place because the learner now can produce a testscore characteristic of a proficient reader. Performance is assumedto reflect the reader's competence exactly, no matter how it isproduced.

What is not understood is that all behavior is the end productof a process and that competence is not behavior, but controlover the process. Behavior, in the form of test performance, canhe used to infer what competence exists in reading, but this re-quires an interpretation of the behavior based on under Landinghow reading works.

Deciding, on the basis of unexamined reading-test scores,such vital aspects of the child's future as the class, group, or trackto which he will be assigned is a terrible abuse of reading tests.It jeopardizes the pupil's future and does not even offer a basisfor improving his reading proficiency, since pupils who are verydifferent in reading may achieve similar scores.

Reaching Theory: Key Questions Test !Akers Must Ask

The earlier parts of this paper have portrayed reading testsas rather primitive, eclectic, and atheoretical in all aspects exceptfor their use of sophisticated test theory.

The questions reading test - makers must deal with to producebetter and more useful tests are clear; however, there is no agree-ment yet on the answers.

Major questions that must be answered in building bettertests are (1) what is reading; ( 2) what are the essential_skillsand strategies that a successful reacher must possess; and (3)Nvhat are the purposes and uses of reading? These will be con-sidered in order.

What is Reading?

Elsewhere the author has stated that "Reading is a complexprocess by which a reader reconstructs, to some degree, a messageencoded by a writer in graphic language." 2

2 Kenneth S. Goodman and Olive S. Niles, Beading Process and Program( Urbana, Ill.: National Council of Teachers of English, 1070), P. 5.

KENNETH S. (:()ODNIANia I

Whether one accepts this definition, which carries with itthe concept that reading must result in meaning to he consideredreading, or some other definition, one must still base test construc-tion on some coherent definition.

The following group of related questions must also beanswered:

(1) Can reading skill (fair example, matching letters tot sounds)be separated from the quest for meaning in teaching or testing?(2) At what point can reading as a process be separated fromits uses?(3) Does the reading process necessarily involve oral languageat all or is it entirely a matter of deriving meaning from writtenlanguage?(=i) Is the reading process different at various stages of develop-ment, or is it the same, varying mainly on the effectiveness ofthe reader?(5) Is reading a general ability, or is it one which varies withcontent, interest, or task witl:a each reader depending on hisown background?(6) Is the reading process the same or different across people,languages, cult ires, or orthographies?

Though there are implicit answers to these questions in manycurrent reading tests, it appears that the test-makers have madeassumptions often without considering the issues involved.

What Are the Essential Skills and Strategies that EffectiveReaders Possess?

Reading tests have generally employed subtests to get atwhat are assumed to be essential reading skills and to monitortheir development. To justify such practice, the following ques-tions must be answered:

(1) Can essential skills or strategies be isolated for testing with-out changing their relative values, their basic uses, or the read-ing tasks in which they occur?(2) Are such strategies or skills universal across people, con-texts, purposes, languages, and orthographies?(3) Is there an essential sequence in learning to read; i.e., mustsome skills or strategies be learned before others?(4) How are reading skills or strategies to be understood interms of how language works I, d is used?

32 /TESTING IN READING: A GENERAL, oirriQuE

What Arc he Purposes of Reaclin

Language, including reading, is always a means and neveran end in itself. This is true whether one is talking about theproficient user or one just learning. Meaning, either its expressionor comprehension, is always the end for which language themeans.

Ultimately then, any reading test must measure- the successof the reader in comprehending written language. It is meaning-less to consider performance on skills tests a measure of readingachievement. What counts at all stages of development is whatthe reader understands as a result of reading.

Test-makers must be concerned with the following ques-tions:

(I) What is comprehension; how does it wor, how is itachieved, how varied is it?(2) What different problems face the reader who is readingto acquire knowledge, as compared to one who is reading for amessage already within his conceptual grasp?(3) What role does the reader's background and interest playin successful reading?(4) HOW does critical reading differ from other din

Future Reading Tests

Diagnostic reading tests in the future will need to focus onreading as it really occurs in natural language. This suggests thetype of task now found in informal reading inventories. But thediagnostic test of the future will be designed so that the strengthss well as the weaknesses of learners will be made clear. A shift

will need to be made away from counting errors to analysis ofperformance, to get at the underlying competence.

Achievement tests will need to deal with comprehension ina range of reading situations. They will need to avoid irrelevance.And they will n(vd to get at the reader's ability to use writtenlanguage effectively. Croup tests may well disappear. Theysacrifice too much fe sake of economy of time.

KENNETH S. GOODMAN/33

Still, however, the in, yin improvement needed in the area oftesting is in use. No test, l owever cleverly it is constructed, cansubstitute for the insights professional teachers get from working-closely with children.

?EV-03E1/11XE COrIERACCNIG:A ONCE-OVER

Ileact ous to the performance contract made between theTexarkana schools and the Dorsett Educational Systems in 1969varied nationally from charges that such contracts NV011id de-humanize education to countercharges that education was too im-portant to be left to educators. Whereas school contracts routinelyprovide for services and equipment, easily auditable, the Texar-kana-Dorsett contract called for the delivery of increased per-formance on the part of school children. The Texarkana-Dorsettproduct was to be a group of better readers, a matter far moredifficult to audit.

This first performance contract seemed to threaten the do-main previously reserved exclusively for classroom teachers. T r a -ditionally, the teacher has been considered to be the one mostknowledgeable about what the child needs. It is unfortunate thatall too many teachers are willing to accept a large number offailures as a normal consequence of well-conducted classroomprocedure. School board members, on the other hand, have be-come "less, confident that teachers are dedicated to teaching students . . increasingly aware that they, perhaps even more thanany other group in public education, are being held accountablefor what transpires in schools." School superintendents in major

"School Board MembeN View Performance Cr f hart n Arizoriurc/icr ( January 1971 ), 18.

35

30/PERFORM CONTRAMING: A ONCE-6VEll

cities, such as Nolan Estes of Dallas, have begun stating thatschools are going to have to deliver one year of educationalgrowth for one year of instruction, no matter what it takes.Former Commissioner of Education jams Allen involved thefederal government in another human right: the right of everychild to learn to read to the full extent of his potential, Mount-ing 'iced, increased public concern, conceded failure of thepresent system to produce desired resultsall made a new ap-proach imperative, and performance contracting entered theeducational scene.

Critics of the initial Texarkana Project felt vindicated whenresults of the 1969-70 contract were announced. Though the$105,000 paid to the Dorsett Corporation produced some results,these were declared by many to be invalid. Dr. Robert Kronerof the EPIC. Evaluation Center, Tucson, the final evaluator, -con-cluded in his final report that 30 to -100 per cent of questionsin the tests administered, Jo Texarkana students in. Alay, 1970,were. 'contaminated,' meaning that they had been taught in theclassroom prior to testing. Said Krancrt The teaching of testitems, or closely related test items, has invalidated the test resultsto the extent they cannot be used as a valid measure of achieve-ment,' Loyd. Dorsett replied that only small amount of improperteaching' went on, which meant that only 7 per cent of the testquestions could be considered invalid." 8

Such criticism did not shake Texarkands faith in the principleof performance contracting. Dorsett bid for a second year's con-tract and Was refused. " Ihere are rumors floating that Texarkanamay go to court to recover 5110,000 1.1ectu.ly paid to Doi.sett."4 Acontract was awarded, however, for the 1970.71 school term byTexarkana to Educational Development Laboratories, Inc.

Very siniply stated, a school contracts with industry to as.sumo responsibility for instruction in tt given area, and industrygilarantees increased performance on the part Of the studentsinvolved, Payment to the contractor depends largely on the

Conference 'Iola 'lit Somlleasiern State College, Durant,Oklahoma, September 1070.

11140110m \Vcelis, "Porformat Cunt Pithill or l'anoci. ArtzoroiTeacher (January 1971), 18,

4 p,

MARY CALA -37

amount of success he has, as revealed by objective measurementof student performance. Contract patterns vary from the estab .lishment of special learning centers conducted by employees ofthe contracting firm to the utilization of regular staffs and facilitiesunder the direction of contracted managers. All kinds of fundslocal, state, and federalare being used to pay .for the contracts.All existing contracts provide for a turnkey operation at the endof one or two years usually. Performance contracts, then, are usedto get new means of guaranteeing educational success into thepolitical system:

Undeterred by the questionable results in Texarkana, a signif-ic number of schools all over the country_ entered into per-formance contracting for the 1970-71 school year. 'A list of theschools, contractors, and programs is given below: Of interestto reading specialists is the significant fact that every programon the list has a reading component. It would seem imperative;then, that reading teachers be xvell aware of the nature of per-I ormance contracting.

Far from the simple contract for services or products whichmost attorneys can execute for school districts, performance con-tracts are exceedingly complex. Several management firms arenow engaged in the preparation of contracts and the supervisionof the job to be done.' jack Stenner, director of the MamigementSupport Services, listed the advantages of performance contract-ing as follows:.

L It facilitates the targeting and evaluation of educational pro-grams. : . . It fosters the objective evaluation of educationalresults and also the managerial processes by which theseresults were achieved,

0. Performance contracting for instructional service couldtraduce greater resources and variability into the publicschool sector.

3. The performance contract approach allows a school systemto experiment in a responsible manner with low costs andlow political and social risks.

4, The.right of every child to read at his grade level will un-doubtedly place great burdens upon the schools' limited re-sources. If the Nation's schools are to make this principle areality,_ they might want to consider using performance con.

OPE

.RA

TIO

.NA

L P

RO

GR

A.M

S FA

LL

1.9

70/S

TU

DE

NT

A.C

IIIE

VE

,ME

NT

5

Edu

catio

nal A

gen4

rB

osto

n R

o.xb

ury)

,. 'M

ass.

Col

orad

o, S

tate

of

henr

y C

reek

Den

ver

.En.

giew

ood

Dal

las,

Tex

.D

alla

s', T

ex..

Flin

t, M

ich.

.G

ary,

Ind

..C

alif

.G

rand

Rap

ids,

Mic

h.G

rand

Rap

ids,

Gre

envi

lle, S

.C.

Jack

sonv

ille,

.Fla

.

Oak

land

, Cal

if.

Phila

delp

hia,

Pa.

Prov

iden

ce, R

.I..

Sava

nnah

Ga.

Tex

arka

na, U

SA.

Vir

gini

a, S

tate

of

Nor

folk

Buc

hana

n C

o.D

icki

nson

Co.

Lun

enhe

rgIN

.lech

lenh

org

Co,

Prin

ce E

dwar

d C

o..

Wis

e C

o..

Lca

rrai

ng S

yste

m C

ontr

acto

rE

duca

tiona

l Sol

utio

ns,

Dor

sett

Edu

catio

nal S

yste

ms

Dor

sett

Edu

catio

nal S

yste

ms

Dor

sett

Edu

catio

nal S

yste

ms

'Dor

sett

Edu

catio

nal S

yste

ms

New

Cen

tury

.T

hiok

ol

Dea

ler

for

.E.D

L,.

Mat

eria

lsB

ehav

iora

l Res

earc

h L

abor

ator

ies.

NV

estin

gh.o

use

Lea

rnin

g.W

estin

ghou

se L

earn

ing

CO

ME

SC

OM

ES

Lea

rnin

g R

esea

rch

Ass

ocia

tes

Edu

catio

n So

lutio

nsB

ehav

iora

l Res

earc

h .L

abor

ator

ies

New

'Cen

tury

/Com

mun

icat

ions

Patie

nts

Lea

rnin

g, F

ound

atio

nsE

duca

tiona

l Dev

elop

men

tal

La

horn

to r

ie,s

Lea

rnin

g R

esea

rch

Ass

ocia

tes

Lea

rnin

g R

esea

rch

Ass

ocia

tes

Lea

rnin

g R

esea

rch

Ass

ocia

tes:

Lea

rnin

g R

esea

rch

Ass

ocia

tes;

Lea

rnin

g R

esea

rch

Ass

ocia

tes:

Lea

rnin

g R

esea

rch

Ass

ocia

tes

Lea

rnin

g R

esea

rch

Ass

ocia

tes,

Lea

rnin

g R

esea

rch

Ass

ocia

tes

Stud

ents

Tar

get

Prog

ram

Num

ber

Gra

des

Paym

ent

Rea

ding

Rea

ding

Rea

ding

Rea

ding

Rea

ding

head

ing

& 'M

ath

Occ

. ,Sk

ills

&

Rea

ding

All

subj

ects

Rea

ding

& M

ath.

Rea

ding

& M

ath

Rea

ding

& M

ath

Rea

ding

.R

eadi

ng, M

ath,

Soci

al S

tudi

es&

Sci

ence

Rea

ding

490

6-8

Rea

ding

20,0

001-

2,, 7

-8R

eadi

ng1,

500

2-8

400

11::-

.6

300

6-8

100

6-8

100'

6-8

100

6-8

875

9-12

.S'7

.39-

12

2,16

09

800

K-0

,10

3;)

,-4

400

1-6

600

6-9

480

'6-9

300

Rea

ding

Rea

ding

,T

atls

,&D

ropo

uts

Rea

ding

& M

ath

Rea

ding

& M

ath

Rea

din

g &

Ma

thR

eadi

ng &

Mat

hH

eadi

ng &

Mat

hR

ea d

ing

& M

ath

Rea

ding

,& M

ath

Rea

ding

& M

ath

Geo

rge

R. H

all a

nd J

ames

Snic

ker,

. -T

he R

and/

HE

W S

tudy

of

Perf

orm

ance

Con

trac

ting,

" C

omm

a(F

ebru

ary

1971

). 8

.,

1,00

030

0'7

-12

2.50

01-

950

04-

950

01-

725

01-

7.1

504-

725

04-

625

04-

650

04-

9

$ 89

000

50,0

00

210;

000

640,

000

60,0

0014

3,70

019

1,00

010

0,00

070

,000

80,0

0080

0,00

014

5,00

0

97,0

001.

03,0

00

:212

,500

O 0 O 0 CD

, 7

MARY CALVAN/39

tracting for the development and validation of new _readingprograms.

6, Through the use of the performance contract approach, manyof the previously segregated children will have their academicdeficiencies removed on a guaranteed achievement basiswhile they are attending the newly-integrated schools.

6. The approach creates dynamic tension and responsible histi-tutional change within the public school system through com-petition. Boards of education can finally establish policy andchoose among alternative instructional programs.'

Whether the charges that performance contracting will de-humanize education and is an actual threat to classroom teachersare valid or not is undecided. Certain disadvantages, however, doseem to be apparent. There is the ever-present danger that insome way teachers or contractors will find a way to teach to thetest. Since Texarkana, most contracts include the services of anindependent evaluator and contain intricate provisions to assurethat the nature of the test to be given remains unknown to thecontractor and the instructional staff until it is administered.Dallas, for instance, has awarded its evaluation contract to Educa-tional Testing Service, but plans to bring in two addition it evalua-tors to cheek the validity of the results.

Another disadvantage, as yet unresolved, is the matter ofwho holds copyrights on materials produced or reports publishedabout the project.

A third disadvantage, probably not as critical as the others,is the matter of responsibility. To whom do citizens complain inthe event of problems or dissatisfaction? It would seem that thechief schools officer of the contracting school would still bear theresponsibility, but it is logical to assume that contracted arrange-nicnts would allow for as shift in responsibility.

Fourth, without guidance from a profession of reading spe-cialists, it is possible that a school and a contracting firm mayarrive at some unsound behavioral objectives, and though theobjectives stated in the contract are met, the end results may bedisappointing if not disastrous,

°Jack Steiner, "Perfonnance Contracting for Instruction" (unpublishedmanuscript, WI

40/PERF ANCE CONTRACTINC: A ONCE-OVER

Fifth, the charge has frequently been made that performancecontracting is yet another way of lining industrys pockets withmoney at the expense of schools, teachers, and children,

Sixth, there is the possibility that Robert Stake and JamsWardrop arc correct_ in feeling that progress_ made tnay not beentirely due to the efforts of the contracting firm.

Suppose that three _students were to he tested with a p;millelform immediately after the pretest: The -chances are better than50:50 that

andleast one of the three would have gained a year

or more and appear ready to graduate from the program.Suppose that_100 students were: admitted to contract instructionand pretested. After a period of time involving no training, theywere tested again and the students "gaining" a year were grad-uated. After another period of time without training, another testand another graduation :occur. After

hadfourth such "terminal"

testingcvon though no instruction had_ occurred the chancesare better than 50150 that two-thirds of the students would havegraduated:7

standa rdi ed testing used in many of the contract-ing situations "does not have the necessary content validity forindividual student assessment. For years test authors and testpublishers have cautioned users against using these tests as ding.nostie instruments.. Performance-contract criterion tests should,in effect, be diagnostic tests." 8

Literature concerning performance contracting makes anumber of recommendations to schools who intend to enter intosuch a program. 13ecause of the complexity and the newness ofsuch programs, it is recommended that schools should employthe services of management consultants in both the preparationof the contract and the conduct of the program. Also becauseof the novelty of the idea, schools arc cautioned to -regulate theflow of visitors to the project; in many instances, numbers ofinterested onlookers have seriously limited the success of theprogram.

It is further recommended that the teacher's role in the pro-gram should be clearly defined before instruction begins.

7 Robert E. Stake and James L: Wardrop, "Cain Score Errors in Per-formance Contracting" (unpublished rraumscript).

8 IbitL

MARY CALVAN/41

If, in a teacher's judgment, other materials andthe

aredichted, and the teacher can support the position, the agreementof the project director and the contractor is negotiable. Noteacher has the right, however, to unilaterally intervene withalien procedures and revise asystem which is legally agreed toand by which a contractor will receive reimbursement. Since, inmost eiises, the teachers involved in this kind of operation Will1)0 Volunteers, making this kumm initially, as a condition of par-ticipation, should require no morc than a signed agreement byeach teacher assenting to this requirement of the contract." 9

The literature suggests that the school and contractor woulddo well to involve as large a segment of the community as possiblein the project. Not only will such involvement yovide much-needed public support, it will be invaluable when the project isturned over to the school district to run. .

Finally, the school should engineer lower cost factors byproviding a number of students which would make for the mosteconomical situation. It is further, uneconomical for a school topurchase any nonconsumable materials during the first year ofoperzition; such materials should be provided by the contractingfirm or leased:

A number of legal considerations enter into any program ofperformance contracting. Not the least of these is the relationshipof the contract to local and state education agencies. Before thecontract in Dallas could be approved by the Texas EducationAgency, for instance, a ruling by the state Attorney General hadto be made. The ruling was that contracts could he made andpaid for with state and federal funds; however, all members ofthe contract instructional staff: had to be certified teachers inthe State of Texas, the prograM had to come under the regula-tions of the Minimum Foundation Program and was subject toAgency accreditation railings. It is obvious that contracts willdiffer widely around the country because of varying opinionsabout their legality.

The management consultant, Jack Stenner, posed the follow-ing legal questions as pertinent to the preparation of the Dallascontracts:

n Albert V. Mayrbofer, "Factors in Consider in Preparing PerformanceContracts for Instruction," Educational Technology Clammy 1971), 48.

.12IPERFC I _'E CONTRACTIN(:: A ONCE -OVER

1. If an outside contractor is utilized by the school district forthe provision of instruction and/or services for pupils, willthe school district's statutory obligation for said instructionand services he fully met?

'7). Are there state or federal limit regarding the use ofState, Local, and Federal funds to support all or part of aperformance contracting project?

3. Do existing state plans for various federal programs, thatmay be potential funding sources for the project, permitutilization of outside contractors who will provide all or partof the instruction and/or services included in the project?

4. Will the State Education Agency authorize credit for instruc-tion provided by the contractor in reading or mathematics?

5. Will the State Education Agency permit paraprofessionalstaff members employed by the contractor to perform cer-tain teaching tasks under the direct supervision of a cer-tified teacher?

6. Will the State Educlition Agency permit the teacher in thecontractor's program to be the contractor's employee?

7. Will an outside contractor's instructional personnel stand inloco parentis to the student?

8, Are the contractor employed teachers required to be cer-tified?

9. Does a local district have authority to contract with or enterinto agreements with an agency of the federal government?

10. Does a local district have authority to contract with or toenter into agreement with private concerns for the provisionof instruction?

11. Does a local district have authority to expend local main-tenance funds to pay for part or all of the cast of instrlic-tion provided by a private concern under contract with thedistrict? 10

Three types of negotiations are available to schools in thepreparation of performance contracts. First, the school may decideon one single source of material and method. The advantage ofbeing able to negotiate rapidly with a single company may beoffset by the disadvantage of limiting the scope of the project towhat that particular company can do. A second type of negotia-

Jack Renner, "Educati n Performance Contracting" (unpublishedmanuscript, January 1970).

NIAI GALVA_

Lion is referred to as the modified sole-source method; two ormore conlpanics arc asked to submit bids, and one of the groupis chosen, or several are asked to work in concert with cach.other:The third type, the competitive system, requires far more timeand is complicated, but it does insure the school the opportunityof investigating natty sources of help and choosing the hest alter-n;ltive. In such a system, criteria for the program would bedrxided upon and published by the schools. Bids would then betaken from any number of competitive Companies. Such competi-tion is likely to produce some economy in the program also.

Needs and Recommendations

Because of the importance and frequency of reading pro-grams in performance contracting, the following needs and recom-mendations have .been formulated to provide schools someguidance in their choice of programs.

A list of sound performance objectives related to readingshould be prepared: There is no indication in Are literature thatany professional reading group has done this There is great needfor the development of objectives that are educationally soundand encompass the issues in the teaching of reading. I have readfar too many proposals of reading programs which miss the markto want to leave this to chance, Perhaps a list of alternative objec-tives from which schools could select those most appropriate totheir situation would be helpful:

Though the literature suggests that various methods andmaterials may be used in concert with each other, I skrw no indica-tion that this practice was used widely. I think we must concedethat no one .reading program, however good, is effective with allchildren in a given school. Varying modalities of learning, lin-guistic and experiential resources, means of perception, and in-dividual approaches to learning provide a heterogeneity whichmakes a combination of systems preferable to one. There is sub-stantial need for the identification of materials which woulddemonstrate to schools and teachers, long accustomed to teachingwith one textbook and one method, the value of working withcombinations of systems.

4 PERFORMANCE CONTRA AINC: A ONCE-OVER

Criteria for critiqueing reading systems and materials wouldbe particularly useful to school distriets Special attention shouldbe given to soundness of programs from a linguistic point of view.None of the contracts which I read indicated that the schoolinvolved was even slightly aware that linguistic variety is a factof life and has tremendous implication for the teaching of read-ing. In_ as much as significant work has been completed in thefield of dialect and language differences, this information is erit7ical to the teaching of reading, and such understandings shouldhe incorporated into guidelines for performance-contracted cur-ricula,

accounza mopA SUMMARN' STATEMENT

As one examines the four previous discussions, the clear rela-tionships that exist among them are readily evident. Both costsin education and the apparent unsuitability of present educationalprocesses for significant chunks of our population are raising ques-tions reoarding the efficiency and effectiveness of what is beingdone under the label "education." These concerns about educa-tion are increasing efforts to hold accountable all those whoparticipate in the process and product of education.

Establishing performance objectives allows one to define thetask, to determine if the task has been completed, and to assessthe quality of the response to the task.

Accountability for meeting specific performance objectiveslogically leads to the establishment of performance criteria andthe development of specific measurement devices.

It was inevitable thatin our free enterprise systemsome-one in business would apply fanner Defense Secretary CharlesWilson's "pet -a-bigger-bang-for-the-buck" principle to educationand move unilaterally to establish relationships with school tobring it off.

The previous discussions were developed as a result of sev-eral basic questions central to the topics discussed. In addition;the very treatment of the topics subsequently raised other ques-tions that the serious participant in education must face. Severalof these questions follow:

45

4G /ACCOUNTABILITY: A SUMMARY STATEMENT

1, Is the accountability that is being urged upon us actually tooimpersonal io be of benefit? Arc we holding schools responsi-ble for the high-quality education of our youth rather thanteachers? If so, what kind of accountability is it? Can ac-countability coexist with personal anonymity?Can objectives be established that encompass the total cduea=tional process and product and their effect on people?

3, Is there more to education than performance? Aud can thatadded something be measured? Indeed, can it he taught.

4, Is education debumanizedfor teachers or for studentswhen it is systematized?

5. Are our present means for evaluation in education eitherappropriate or adequate for the task, given the need tomeasure both the process and product of education?

6. is it possible for noneducation people to do a total job ofeducating students?

Before accepting or rejecting any of these factorsaccounta-bility, behavioral objectives, °Valuation, performance contracting--many issues must be explored.

Responding too quickly to the apparent efficiency of per-formance objectives may lead to mechanistic instruction, Reject-ing behavioral objectives in the nine of affective concerns maymask the fact that in too many classrooms there is little concernfor or clarity of objectives related to either cognitive or affectiveaspects of education,

Ready acceptance of the logic of accountability may allowintrusions into education by groups with questionable motivesand dubious qualifications Rejecting the concept of accounta-bility in the name of professionalism may mask inefficiency, slop-piness, unconcern,

Reliance on inappropriate instruments may damage childrenand support inappropriate teaching. Rejection of evaluation be-cause of the inadequacy of instrumentation can mask irresponsibleteaching.

In conclusion, as it considers instruction, the educationalprofession has a tendency to feel itself inadequate to oppose vic-timizing pressures. But, when it comes to salary, demands on per-sonal time, numbers uA students per class, the profession becomesquite militant, Perhaps we need to examine our priorities, take

HAROLD L. -17

the good that is implicit in the factorsaccountability, behavioralobjectives, evaluation,

thecontractingand vigorously'

pursue that good for the sale of our students and their futures.