"NIVERSITY OF HA WAii LIBRAQ

USE OF THE CLOZE PROCEDURE

TO MEASURE READABILITY .Al~D READING COMPREHENSION

IN THE SAMOAN LANGUAGE

A THESIS SUBMITTED TO THE GRADUATE DIVISION OF THE UNIVERSITY OF HAWAII IN PARTIAL FULFILLMENT

OF THE REQUIREMENTS FOR THE DEGREE OF

MASTER OF ARTS

IN ENGLISH AS A SECOND LANGUAGE

DECEMBER 1981

By

Gary L. Bowne

Thesis Committee:

Charles Mason, Chairman Ted Plaister

Danny Steinberg

-

We certify that we have read this thesis and that in

our opinion it is satisfactory in scope and quality as a

thesis for the degree of Master of Arts in English as a

Second Language.

THESIS COMMITTEE

ii

--ACKNOWLEDGEMENTS

Grateful acknowledgement is given to the following

people for assistance with this project:

iii

The late Dr. Ruth Crymes, for encouragement in the early

stages.

Samoan students of Kaimuki High School in Honolulu, who

took a trial version of the multiple-choice reading test.

Staff and students of Leone High School in American

Samoa, for. continued support and cooperation.

Leonard Burns, for invaluable assistance with the computer

at the University of Hawaii.

Evelyn Nakanishi, Department of East Asian Languages at

the University of Hawaii, for expeditious typing.

iv

ABSTRACT

A series of cloze reading tests of varying deletion rates

and a traditional multiple-choice reading test wer~ adminis­

tered to a group of high school students in American Samoa

in order to determine the efficacy of using the cloze

procedure as a measure of readability and comprehension in

the Samoan language.

Results indicated that there were strong correlations

of r = .80 or better between scores on the cloze reading

tests and the multiple-choice reading test. There was a

slight yet statistically significant difference between the

exact word (EW) and acceptable word (AW) scoring systems,

and a significant difference in mean scores across deletion

rates. This latter difference was largely due to a sizeable

drop in mean score between the five-to-one and four-to-one

deletion rates.

It, was concluded that cloze procedure could serve as

a measure of both readability and reading comprehension in

the Samoan language. The difference in scoring methods was

not considered crucial, thus confirming the practicality of

exact word scoring. Both four-to-one and five-to-one were

identified as functional deletion rates.

ACKNOWLEDGEMENTS

ABSTRACT . . . .

LIST OF TABLES

LIST OF FIGURE . .

THE PROBLEM

BACKGROUND .

TABLE OF CONTENTS

REVIEW OF THE LITERATURE

THE HYPOTHESES .

METHOD ....

Cloze reading tests Traditional reading test Test administration . Data analysis .

RESULTS

Hypothesis One Hypothesis Two Hypothesis Three

DISCUSSION . . . . .

Hypothesis One Hypothesis Two Hypothesis Three

CONCLUSIONS

IMPLICATIONS .

APPENDICES . .

BIBLIOGRAPHY

v

iii

iv

vi

vii

1

2

3

16

16

17 18 19 20

24

24 26 29

34

34 37 38

42

44

46

69

vi

LIST OF TABLES

Table 1. Means for Exact Word and Acceptable Synonym Scoring Method . . . . . . . 24

Table 2. Multiple-choice Reading Test Means 25

Table 3. Analysis of Variance on Multiple-choice Reading Test Scores . . . . . . . . 25

Table 4. Correlations between Multiple-choice Reading Test Scores and Cloze Deletion Rate Scores . 26

Table 5. Two-way ANOVA on Deletion Data ........ 27

Table 6. Correlations between Exact Word Scoring Method and Acceptable Synonym Scoring Method for each Deletion Rate . . . . . . . . . . . . . . 28

Table 7. Analysis of Variance and Trend Analyses on Deletion Rate--Exact Word Score Data . . . . . 30

Table 8. t-tests on Cloze Exact Word Scoring Data . 31

-vii

LIST OF FIGURE

Figure 1. Mean Percent Correct Responses at the Various Deletion Rates . 32

THE PROBLEM

Measuring readability is difficult even in a widely

studied language like English. And although there are a

variety of methods available, we have yet to determine the

best system. This thesis addresses the question of how well

current readability assessments used in English would function

in Samoan, a language not at all similar to English.

There is a need for an accurate measure of readability

for materials in the classroom in Samoa, but formal assess­

ments do not often exist in languages with a limited number

of speakers. A more precise determination of readability

levels for materials under development would insure greater

sensitivity to student reading abilities, and would allow for

a more accurate gradation of materials.

Measuring comprehension of written Samoan is an equally

valid concern. Although an abundance of standardized instru­

ments is available in English, no formal instrument exists

for measuring comprehension in Samoan. The preparation of

a more systematic method for measuring comprehension would

allow for the establishment of grade level expectations, the

creation of norms, and the development of an accurate system

of program evaluation.

Currently, no such formal network exists. While it is

true that there are casual and valuable assessments such as

teacher judgement, it seems worthwhile to consider the

development of a more practical and more "standardized"

system. The lack of any concrete method for measuring

2

readability and reading comprehension in Samoan prompted

a research question: what device could serve as an accurate

yet simple measure of both readability and reading comprehen­

sion for texts written in Samoan?

BACKGROUND

American Samoa is an unincorporated U.S. territory in

the South Pacific. The Department of the Interior controls

funds for American Samoa and as a result, education is to

some extent influenced by the federal government. While

locally appointed school officials determine policy for the

most part, the viability of educational programs is often

determined by the availability of federal funds.

Public schools in American Samoa are modeled after the

American system, following a stateside curriculum pattern

and even a northern hemisphere calendar. The bulk of the

teaching materials and books come from the United States, as

do a fair number of contract teachers at both the elementary

and secondary level.

Students in Samoa are taught in the vernacular for the

first three years of school (a policy that is now being

implemented in American Samoa). Thus the introduction of

reading instruction will be in Samoan using Samoan materials.

But such materials are currently few in number and it will

take some time to develop a repertoire of materials for this

program. Measuring readability or comprehension must, of

necessity, be undertaken on the materials that are presently

available.

3

Although predominantly an oral culture, Samoa is

witnessing growth in printed materials in the native language.

Currently, there is little available in the form of

"literature" printed in Samoan, perhaps because the written

tradition is young by western standards. Another factor may

be that the spoken word still carries far more weight than

the written one in this island culture.

While most people in American Samoa can read in their

native language, they have a limited selection when they

choose to do so. There are several dictionaries, a grammar,

and basal readers developed by the Bilingual Education

project of the Department of Education in American Samoa. A

few stories have been translated from English into Samoan

and published in New Zealand. By far the bulk of the

written literature is composed of Bible stories and Biblical

tracts in Samoan (Marsack 1962) .

Th.is situation translates to a system of informal

assessment in the native language. With no substantial

supply of reading materials available, there has been no rush

to create formal instruments in Samoan to measure language

ability, comprehension, or the reading materials themselves.

But with renewed emphasis on the teaching of Samoan in the

primary grades, a more precise system of measurement is

indicated.

REVIEW OF THE LITERATURE

Studies of readability began in earnest in the early .

part of this century. While there are numerous factors

4

involved in measuring readability, most studies address three

things: legibility, reader interest, and ease of under­

standing. The last factor was probably given the most

attention in the majority of studies.

The recent history of readability studies is basically

one of formulas used to predict reading levels. Chall (1958)

reports that there were twenty-nine quantitative studies of

readability from 1922 through 1953, and the "typical product

of each study was a readability formula." From about 1958

to the present, there has been a decline in the use of read­

ability formulas but an increase in the use of such alternative

measures as the cloze procedure.

This thesis will not examine and define all of the

formulas devised in the past fifty years. For those who wish

to ,trace the development and application of these formulas,

see Klare (1963) and Gilliland (1972). It might, however,

prove useful to look at the characteristics of the readability

formulas that emerged during this period.

Each study usually resulted in a formula "based on the

counting and weighting of several significant factors in the

printed matter that would be used to predict the reading skill

needed to understand it" (Chall 1958). Each formula

typically involved a regression equation derived from a

combination of factors which had the highest multiple

correlation with criterion passages.

Although the formulas each weighted different combina­

tions of factors, most were similar in that only a few factors

5

were used as predictors. Ultimately, most of these formulas

had two things in common--namely, the counting and weighting

of vocabulary difference and sentence length. The commonality

of these two factors provided a thread of continuity over

the years.

The late forties and early fifties was the last time

that any significant work was carried out in this area.

Studies by Lorge (1939), Dale-Chall (1948), Flesch (1951),

and Spache (1953) all produced formulas which differed

slightly but which shared the common features of vocabulary

difference and sentence length.

A significant shift in readability studies came about in

1953 with the formulation of the "cloze" procedure by Wilson

Taylor. His work with Gestalt closure theory represented a

significant shift away from the formula school. Taylor's

research produced results that correlated well with existing

formulas for establishing readability levels; moreover, his

was the first such "formula'1 which took the language abilities

of the reader into account.

With the advent of the cloze procedure, research with

readability formulas almost ceased. Predictive formulas

lost some of their universal appeal, even though they still

found applications. The subsequent appearance of psycho­

linguistic theory and its affinity for cloze research ended

the long reign of readability formulas, with very few

exceptions.

6

One of these was Edward Fry (1968) who proposed a

simple formula which took two factors into account, average

word length and average sentence length. His formula was

one of very few to appear at this time, but it has enjoyed

some popularity because of its ease of use. The formula

involves the use of a graph, thereby eliminating complicated

calculations. The Fry formula provides a quick and easy

estimate of readability, and it correlates well with previous

formulas such as Dale-Chall and Flesch. The Fry formula

stands out as a rare exception in a forest of cloze studies

on readability.

Additional research with the cloze procedure by Taylor

(1957) indicated that cloze was more sensitive to the

nuances of style than any predictive formula. Bormuth

(1962) provided additional evidence to support the work by

Taylor, and Weintraub (1968) found cloze to accurately

measure.the readability of English prose (passage difficulty)

better than any current formula. The sensitivity of cloze

to passage difficulty seems well documented.

Whereas readability studies focused primarily on the

use of predictive formulas, the study of reading comprehen­

sion has for the most part dealt with traditional multiple­

choice tests. Cleland (1966) in a review of the history of

reading comprehension, states that the term "comprehension"

did not appear in the literature until 1917.

The study of comprehension has taken the form of

multiple-choice tests which generally measure the effects

7

of comprehension after a person has read a passage. In the

past, the distinction between readability and comprehension

was that readability formulas were used to predict compre­

hensibility, while reading comprehension typically involved

questions to measure understanding after the fact. The first

did not involve the reader; the latter of course did.

There were no radical developments in the study of

reading comprehension from the twenties to the forties.

Research continued with traditional methods, just as the

measurement of readability at this time continued with little

change in methodology. However, as readability formulas

reached their zenith in the forties, so too the study of

comprehension changed at that time.

The emergence of psycholosical testing stimulated the

development of the statistical technique referred to as

factor analysis. This involves identifying the common

elemen~s that account for an individual's score on a given

test, and then attempting to evaluate each of the elements

to determine the relative importance of each variable.

Davis (1944) conducted a factor analysis with proficient

readers in an attempt to identify the crucial elements that

influence reading comprehension. His study produced results

which indicated that comprehension in reading was a skill

composed primarily of two factors, the understanding of word

meaning and the ability to reason verbally.

In a later study, Davis (1972) found that these same

two factors accounted for eighty-nine percent of the variance

8

in reading comprehension. At the same time he listed several

additional factors which relate to cloze procedure: concen­

tration on the literal meaning of a passage, following

structure, and recognizing the mood and literary techniques

of the author. Other factor analyses have revealed different

variables but the two primary factors that commonly emerged

from these studies were a general verbal ability, and the

specific skills of vocabulary knowledge and the ability to

manipulate language (Spache 1963).

Interestingly, research in the areas of readability

analysis and comprehension study seems to have come together

in the mid-fifties. The development of the cloze procedure

and its validation as a readability measure was closely

followed by experimental studies on cloze as a measure of

reading comprehension.

The cloze procedure opened new doors for research on

readin~. The earlier validation of cloze as a measure of

readability prompted research in the comprehension field.

Bornruth (1962) and Rankin (1965) both found cloze to be a

highly efficient and reliable measure of reading comprehen­

sion in their work. They reported that cloze tests produce

good correlations with criterion tests over the same material

and that cloze tests measure comprehension as well or better

than comparable multiple-choice tests. Taylor (1957) found

that cloze test scores served as accurate predictors of

multiple-choice test scores.

9

The validation of cloze procedure as a measure of both

readability and reading comprehension has resulted in a

"merging" of these two concepts, so that many studies fail

to distinguish between the two. Certainly there is a link

between the two, and it is this interrelationship which gives

cloze one of its major strengths--the capacity to match

students and materials.

Earlier work with factor analysis in comprehension

studies led to factor analysis with the cloze procedure.

Weaver and Kingston (1963) carried out a factor analysis of

cloze tests and multiple-choice tests. Subsequent work by

Bormuth (1969) along the same lines demonstrated that the

same factors which account for most of the variability in

traditional comprehension tests also account for most of

the variability in cloze tests. The two factors isolated in

earlier studies, a vocabulary factor and a verbal relation­

ship f~ctor, proved to be the primary factors involved with

cloze procedure.

Research with cloze procedure as a measure of read­

ability and reading comprehension in English prompted studies

in other languages. These studies reported good results in

languages similar to English, such as French (Landscheere

1972), as well as in languages unlike English such as Thai

and Vietnamese (Oller 1972) and in Japanese (Yamada 1979).

It should be noted, however, that the cloze system appears

to work better in languages that have a history of written

literature. This may be due to the fact that for these

languages a standard written form has evolved.

10

Cloze tests have also been used successfully within an

ESL context to determine proficiency levels of high school

and college students (Oller 1973, Stubbs & Tucker 1974). A

modified "matching" cloze test has been used successfully at

lower elementary levels with ESL students in Saipan to test

comprehension in English (Baldauf & Propst 1979). Thus it

appears there is a definite application for the cloze

procedure in second language reading situations.

The use of cloze procedure has stimulated studies on a

number of technical or statistical issues, one of which is

a functional deletion rate for cloze testing. In English

this functional deletion rate is the point below which there

is too little context to successfully cloze a passage. At

deletion rates of less than four-to-one for example, there

is insufficient context from which to draw inferences, so

the successful completion of one item is highly dependent

on the successful completion of a prior item (MacGinitie

1961). (See also Aborn, et al. 1959.) The extreme inter­

dependence of items diminishes scores as well as the useful­

ness of the results.

At deletion rates of four-to-one and larger, the

surrounding context is sufficient to allow closure without

excessive dependency on a prior item. Most research in

English seems to accept that we need about eighty percent

of the context in order to complete the other twenty percent

(Miller & Friedman 1957). The determination of this

functional deletion rate is based on the availability or

11

lack of context, and not simply on the variability of scores.

Another issue related to cloze testing is which scoring

method to use. Studies in English indicate that there is no

significant difference between the exact word (EW) and the

acceptable synonym (AW) method. Rankin (1957), Ruddell (1964),

and Bormuth (1965) have concluded that the exact word method

is preferable in testing situations, and that it provides

all essential information with far less work.

On the other hand, recent research efforts have shown

a distinct trend toward use of non-exact or synonym scoring

methods. The rationale is that by not examining incorrect

responses and synonym generation, we lose valuable information

(Asher, et al. 1976). While cognizant of the fact that exact

word scoring is more practical, this trend seems to advocate

the le~rning aspects of cloze.

The majority of studies in English conclude that the

exact word scoring method is preferable, and that there is

no practical difference in the two methods. As a result,

the exact word method enjoys greater popularity because it

is more efficient.

A related issue is the concern about the system of

word deletion in cloze procedure. A variety of deletion

styles have been used, including lexical, structural, random

(Meredith & Vaughn 1978), and strictly mechanical deletion

(every gth word). The latter has withstood the test of

time, and also produced the highest correlations with

concurrent measures of validity.

12

The success of the cloze procedure in measuring both

readability and comprehension provides a contrast to the

limitations of standard readability formulas. While these

formulas can be applied to English, the application of such

a formula to the Samoan language does not produce acceptable

results. One reason is that in Samoan there has been no

formal study of word frequency, and there are no word lists

based on frequency of use. Secondly, most factors involved

in these formulas are language-specific.

There is one readability formula that could be applied

to Samoan, the Fry formula. This is based on two factors,

average word length and average sentence length, strictly

mechanical items. Although this formula can be applied to

Samoan (in a mechanical sense), the results are extremely

variab~e. For example, a passage in English was judged to be

at roughly the sixth grade level by Samoan readers, and by

standard readability formulas such as Dale-Chall. The

equivalent passage in Samoan, also at about the sixth grade

level, was placed at the twelfth grade level using the Fry

formula.

The reason for such a discrepancy is that in Samoan

every syllable not only has a vowel; but every vowel is also

a syllable. Syllabic reduplication is also a more common

element in Samoan. While the Fry formula is easy to use,

it is not effective for determining readability in Samoan.

13

Although current readability formulas appear to have no

application in Samoan, there is a strong probability that

cloze procedure would work as a measure of readability or

comprehension. Bormuth (1968) has a procedure for assigning

grade placement levels based on students' cloze percentage

scores. The purely mathematical aspects should work for

Samoan materials as well, if cloze procedure proves to be

applicable.

Certainly there are reasons why the cloze procedure may

not work in Samoan. The written literary tradition is

evolving slowly; the culture is basically an oral one. The

system of public education is relatively new, and only

recently have the schools returned to a policy of using the

vernacular for the lower primary grades. Up to now, most

students have had as much experience with beginning reading

in English as they have in Samoan, with mixed results. All

of thes€ factors make for a complicated educational situation

in which cloze procedure may have no application.

Just as there are reasons why the cloze procedure may

not have an application in Samoan, so too there are reasons

why it should. In terms of methodological considerations,

the process of closure should be within the reach of this

group. Research with cloze in English has involved second

grade through college, and although success at lower levels

is sometimes tenuous, it has been shown that students with

basic decoding abilities can manage the cloze task (Gallant

1965).

14

A second factor that favors cloze success in Samoan

is that the relationship between oral and written language

may be a more pronounced one in Samoan than in a language

like English. Samoa as an oral culture places great emphasis

on oral language and since reading is related to oral

experience, pride in the oral culture and narrative experi­

ence should lead to a positive approach to reading.

Oral experience is related to reading mastery and it

appears to be. related to cloze success as well. As reading

skills mature, students usually do better on cloze tests

(Johnson 1979). The more mature reader exhibits a keener

knowledge of the process; not only are the scores higher but

the type of error is usually of a more logical nature (Asher,

et al. 1976).

Information theorists off er various models of language

processing, including several to explain the cloze procedure

(Tuimau 1972). It appears that no one model would serve to

explain this complicated process but it does appear that

language and cloze processing are themselves similar across

languages.

Perhaps because most semantic relationships are binary,

a student's awareness of collocation and coligation becomes

a primary skill in mastery of cloze procedure. At a certain

point in his development, the individual seems to develop a

metacognitive awareness of both the tendency and the require­

ment of certain words to co-occur (Pickering 1977). This

knowledge, coupled with an understanding of the redundancy

15

in language, provides the student with the ability to cloze

a passage.

The processing of written language and spoken language

is distinctly different . In a speaking situation we cannot

go back and instantly replay words, which we can do in writing.

Cloze procedure is similar to reading in that both seem to

involve a horizontal and a vertical component, according to

Weaver (1965). The horizontal aspect involves an analysis

of the structural elements of the language while the vertical

component involves a lexical search for the proper semantic

aspect at any particular blank.

Samoan students quite probably deal with cloze reading

tests in much the same way that other students do. The rich

oral environment may facilitate closure because with little

in the way of printed material in Samoan, there is a good

chance that what occurs in oral narrative patterns will also

occur ~n print. Ruddell (1964) has shown that students cloze

passages with high frequency oral patterns more easily than

those with low frequency oral patterns.

We know that oral experience and prior knowledge both

play major roles in successful reading. Samoan students

must bring these aspects into play just as any other student

would to achieve mature reading. The probability of success

with reading and cloze procedure in Samoan rests to a great

extent on the integration of oral experience and prior

knowledge by Samoan students. With cloze success in Samoan,

we may infer that it measures the same aspects in Samoan

16

that it measures in other languages. If the process of

dealing with cloze is cross-linguistic in nature, then the

product should also be similar.

THE HYPOTHESES

There is presently no formal assessment measure for

readability or reading comprehension in Samoan . Yet it

seems that some method of measuring these aspects would be

desirable for materials currently being used and those under

development at this time. The possibility of finding a

relatively simple method for measuring both readability and

reading comprehension was considered, and the cloze procedure

was thought to be a logical choice for its simplicity, ease

of use, and range of applications. To test the application

of the cloze procedure, the following hypotheses were

formulated:

H1 : Cloze procedure is an effective tool for measuring

reading comprehension and readability in the

Samoan language.

H2 : There is no appreciable difference between exact

word (EW) and acceptable synonym (AW) scoring in

Samoan cloze tests.

H3 : There is an identifiable functional deletion rate

n for cloze tests in Samoan.

METHOD

Subjects: Ninth grade students were chosen as the

sample group for this study. Significantly, ninth grade is

the first year of high school in American Samoan, and by this

17

time students should have developed reading skills which

would allow them to deal with most of the literature that

is available .in Samoan. There is reason to believe that

students in lower grade levels can also handle the cloze

task, but for this exploratory study, mature subjects were

desired.

A second important factor in favor of this level is

that all ninth grade students take Samoan Language and

Culture as a subject, while those in higher grades do not.

Although students in English classes are tracked according

to proficiency levels, Samoan Culture classes are mixed

across levels, resulting in a more random sample.

Further justification for using students from a Samoan

Culture class is the fact that the course is taught in

Samoan and involves both language and culture. Thus, both

the subject matter in the tests and the tests themselves

would cyave greater content and face validity in a Samoan

Culture class than in any other situation.

Cloze reading tests: A series of fifty-item cloze tests

with varying deletion rates were constructed from a passage

taken from Tala O Le Vavau (1976), a book of Samoan legends

and culture. A scarcity of scientific material in Samoan

effectively precludes the testing of technical prose, even

though testing a variety of prose styles would be more useful

for determining the efficacy of cloze procedure in Samoan.

Seven cloze reading tests were prepared, with deletion

rates ranging from nine-to-one (nine words and a blank) down

18

to three-to-one (three words and a blank). In each case,

the blank space was a standard length of ten type spaces.

Testing this extreme of deletion rates represents an attempt

to discover whether there is an optimal deletion rate in

Samoan for measuring readability, as there appears to be in

English.

Using a four-to-one deletion ratio, one passage can

yield five different forms of a cloze test. There is some

justification for testing each form at any particular

deletion rate to obtain a truer measure of readability.

Bormuth (1964) points out that five forms of a cloze test,

all with the same deletion ratio but each beginning with

a different word, can yield five dif~erent means. In such

a case, the mean of the means would give the best indication

of an overall score.

Creating five or more forms of each test passage for

this s~udy would have meant a total of forty-two different

tests, and administration of each form would have reduced

the number of subjects for each test to about six, an

inadequate number for drawing any substantive conclusions.

Seven tests drawn from the same passage assures that most

words in the passage will be deleted at least once.

Traditional reading test: As a cross-validation measure

to determine the relationship of the cloze tests to tradi­

tional reading comprehension testing, a sixty-item multiple­

choice test on the same subject matter was constructed. This

test was based on fifteen short reading passages in Samoan,

19

covered the same type of material and was taken from the

same book as the cloze passages. Each reading passage was

followed by four multiple-choice questions designed to check

the student's ability to find specific information, make

inferences, understand vocabulary, and determine the main

idea. This test was pre trialed on twenty Samoan students

at Kaimuki High School in Honolulu, following which revisions

were made prior to testing in American Samoa.

The ideal situation would have been to administer a

standardized test in Samoan, against which the cloze tests

could be measured. The lack of such an instrument in Samoan

necessitated the use of an original test. A Samoan language

test given to all seventh and eighth grade students in 1976

no longer exists, and the results of this one-time test

have disappeared. With no standardized instrument available

in Samoan, the writing of an original test became the only

reasonable alternative.

Test administration: The cloze reading tests were given

to ninth graders in the fall of 1980. Tests were prepared

in groups of forty-two, with six forms of each deletion rate

within each group. The pre-packaged tests were then distri­

buted randomly in Samoan Culture classes so that in each class

there would be an approximately equal number of each cloze

form. The mix of forms both within and between classes

provided the best opportunity for random sampling.

Students were told that the passages were designed to

measure their reading and understanding of different aspects

20

of Samoan Culture. The entire forty-five minute period was

allowed for completion of the passage. Oral instructions

and a short sample test in Samoan were presented prior to the

actual test to familiarize students with the format and test­

taking procedures.

The multiple-choice reading test was administered to

the same students a month after the cloze tests. Though

taken from the same book, the multiple-choice format presents

the information in an entirely different way, and tests

receptive rather than productive skills. The same format

was used in the administration of the multiple-choice test.

That is, Samoan Culture teachers gave the test to the same

group of ninth graders as those who had previously taken the

cloze tests, but students in this case were allowed two

periods to complete the tests.

Data analysis: The cloze reading tests were scored

using both the exact word (E1d) and the acceptable synonym

(AW) method. The exact word method dictates that only an

exact replacement of the deleted word counts as correct while

in the acceptable word method, a semantically and syntactically

equal word is also counted as correct. Since one aspect of

this study was to determine which scoring method should

prevail, it was necessary to score each test twice.

Analysis of the multiple-choice reading test was

conducted primarily in the interest of discovering what

revisions might be needed to improve the instrument for any

future application. The test was corrected and subjected to

21

item analysis using the biserial r method, in which scores

of the highest 27% and the lowest 27% are correlated (Garrett

1965). Any item with a correlation of r = .20 or higher is

considered valid. Test reliability was determined with the

Kuder-Richardson Formula 21, a measure of interitem

consistency in student performance (Tuckman 1978).

Results from the multiple-choice reading test reveal

that thirteen items on the test had a correlation lower than

r = .20, and at first glance this would indicate a test of

questionable validity. A close examination shows that seven

of these items were "too easy," i.e. more than seventy

percent of the sample responded correctly to the questions.

An ideal difficulty level would occur if fifty to sixty

percent of the sample responded correctly.

Five of the items present a unique case in that these

five were originally valid items. They were marked correctly

but sc9red as incorrect, because the test key was changed

after inadequate analysis of results from the pre-test group.

The incomplete analysis predicated a change in the item

distractors so that five distractors originally labelled as

incorrect were changed to become the "correct" choices, and

the items thought to be correct were changed to serve as

"incorrect" distractors.

An example may provide a clearer explanation. The

Samoan word atamai was used as one of four distractors for

a question. It was not the correct choice, but the pre-test

group almost universally chose this word, which is similar to

22

Hawaiian akamai (wise). Based on these results, the master

test was changed so that atamai would become the correct

choice.

Upon receiving the tests from Samoa, it became evident

that the pre-test group was making choices more closely

aligned to the Hawaiian experience. Students in Samoa, on

the other hand, made choices based on their own cultural

background, which is to say they generally chose the original

proper choice. However, because the test key was changed

after the pre-test group results, they were in effect being

penalized for marking correct answers. Thus in the item

analysis, the multiple-choice test appears to be less reliable

than it really is.

Acknowledging that seven items were too easy and that

five items were reversed (marked correctly but not scored

correctly) partially explains the reliability index of

r = .7i, or r = .75 based on a reliability estimate from a

table of Harris (1969). A correction factor for the

inappropriate items and the mislabelled items would produce

a better reliability coefficient. For a researcher-made

first edition, the test nevertheless appears to be reasonably

valid as a cross-validation measure with the cloze reading

tests.

The BMDS biomedical program of the University of

California (Dixon 1977) was used in conjunction with the

Statistical Package for the Social Sciences (Nie, et al. 1975)

in the computer of the University of Hawaii for the data

analysis in this study.

23

A two-way analysis of variance was carried out on the

scoring data from the seven deletion rates on the cloze

tests to discover whether there is any significant difference

in means across deletion rates, and whether there is any

significant difference between the exact word and acceptable

word scoring methods.

A one-way analysis of variance was conducted on the

multiple-choice reading test to determine any difference

between groups in the sample which might necessitate the use

of the multiple-choice test scores as a covariate in sub­

sequent analyses. Additionally, Pearson product-moment

correlations were computed on both the exact word and

acceptable word scoring data of each cloze deletion rate with

the multiple-choice reading test.

()Qce it was determined that there was only a slightly

significant difference between scoring methods, a one-way

analysis of variance and t-tests were run on the exact word

scoring data to provide information regarding the point of

significant falloff in scores. Subsequent analyses refer

only to exact word data, since results are similar for either

scoring method.

An analysis of trend was run on the exact word deletion

rate scores to determine which curve model would best

describe the pattern of cloze mean scores. A series of

t-tests were then conducted on all deletion rate pairs to

24

detect any slight but possibly significant differences

between non- adjacent deletion rate scores.

RESULTS

Hypothesis One: Cloze procedure can serve as a measure

of readability and reading comprehension in Samoan.

The results of the cloze reading tests shown in Table 1

demonstrate that at larger deletion rates, the students

replaced a majority of the blanks with the exact word

deleted from the text. At lower deletion rates, there were

less than fifty percent exact word replacements. In general,

students who scored in the mid to high range also generated

more synonyms, a trend that coincides with research by

Asher, et al. (1976).

Table 1

Means for Exact Word and

Acceptable Synonym Scoring Method

. Deletion Rate N Exact Word (SD) AcceEtable Word (SD)

9 to 1 35 29.60 (3.96) 29.71 (3.98)

8 to 1 35 30.11 (4.06) 30.23 (4.09)

7 to 1 35 29.89 (4.01) 30.03 (4.03)

6 to 1 35 29.43 (4.14) 29.S4 (4.19)

s to 1 3S 28.86 (4.61) 29.03 (4.64)

4 to 1 3S 22.31 (4.19) 22.46 (4.20)

3 to 1 3S 18.80 (3. SS) 18.86 (3.61)

Overall 27.00 27.12

25

The multiple-choice reading test scores are shown in

Table 2. An inspection of these means reveals little

variation across the seven deletion rates and the result of

a one-way analysis of variance on the means was non-

significant, F (6,238) = 0.31, ns. Since there were no

significant differences among the groups on their reading

test scores, it was therefore not necessary to employ the

multiple-choice reading test as a covariate in the subsequent

analyses (Cohen & Cohen 1976). The summary statistics . for

this analysis of variance are shown in Table 3.

Table 2

Nulc.iple-cho.ice .Read.ing I'esc Neans

Deletion Rate Mean SD N

9 to 1 41.26 6.06 35

8 to 1 40.14 6.33 35

7 to 1 41.43 5.83 35

6 to 1 40.69 6.73 35

5 to 1 41. 77 5.81 35

4 to 1 41.69 6.99 35

3 to 1 40.80 6.48 35

Table 3

Analysis of Variance on

Multiple-choice Reading Test Scores

Source df MS

12.2710

40.10

F

0.31

Probability

Deletion

Error

6

238

ns

26

Correlations between the multiple-choice reading test

scores and the two scoring methods across deletion rates are

shown in Table 4. These correlations demonstrate a positive

relationship between the two types of tests. The relatively

high scores on the cloze tests, and the strong relationship

between the multiple-choice and cloze reading tests justify

acceptance of the first hypothesis.

Table 4

Correlations between Multiple-choice Reading Test Scores

and Cloze Deletion Rate Scores

Deletion Rate Exact Scoring Acce:etable Scoring

r df .E. r df .E. - -M-C test 9 to 1 .84 33 <.001 .83 33 <.001

M-C test 8 to 1 .87 33 <.001 .86 33 <.001

M-C test 7 to 1 .85 33 <.001 .86 33 <.001

M-C test 6 to 1 .88 33 <.001 .87 33 <.001

M-C test 5 to 1 .90 33 <.001 .90 33 <.001

M-C test 4 to 1 . 85 33 <.001 .84 33 <.001

M-C test 3 to 1 .83 33 <.001 .84 33 <.001

M-C test Overall .58 243 <. 001 .59 243 <.001

Hy:eothesis Two: There is no appreciable difference

between exact word and acceptable word scoring in Samoan.

A two-way analysis of variance (7 x 2) was performed

on the cloze test scoring data. The between factor was

deletion rate (9:1, 8:1, 7:1, 6:1, 5:1, 4:1, and 3:1) and

the within or trial factor was type of scoring method (exact

27

word versus acceptable word). The results from this analysis

of variance indicated that the deletion rate factor and the

type of scoring method factor were both significant,

F (6,238) = 43.04, p<.0001, F (1,238) = 33.55, p<.0001,

respectively. The deletion rate by type of scoring method

interaction was non-significant, F (6,238) = 0.41, p = .87.

Summary statistics for this analysis of variance are shown

in Table 5.

Table 5

Two-way Al-IOVA on Deletion Data

Source

Deletion Rate

Error

Type of Scoring

D x T

Error

df

6

238

1

6

238

MS

1444.97

1.84

0.023

0 . 055

F

43.04

33.55

0.41

Probability

< .0001

< .0001

.87

The correlations between the exact word scoring method

and the acceptable synonym scoring method for each of the

deletion rates is shown in Table 6. As expected, and as the

table indicates, each of these correlations is higher than

.99, demonstrating a near perfect relationship between the

two scoring methods.

28

Table 6

Correlations between Exact Word Scoring Method and

Acceptable Synonym Scoring Method for each Deletion Rate

Deletion Rate r - df Probability

9 to 1 .997 33 <.001

8 to 1 .997 33 <.001

7 to 1 .996 33 <.001

6 to 1 .997 33 <.001

5 to 1 .997 33 <.001

4 to 1 . 996 33 <.001

3 to 1 .998 33 <.001

Overall .998 243 <.001

An inspection of the means for the scoring methods

across the deletion rates reveals a similar finding. The

overall mean for the exact word scoring method was 27.00 and

for the acceptable word scoring method 27.12 (see Table 1).

An inspection of the individual cell reveals a similar slight

difference of about .1, a point favoring the acceptable

synonym scoring method across all deletion rates.

Though the type of scoring factor was significant in a

strictly statistical sense, the actual size of the difference

between the two methods (0.12) and the near perfect correla­

tion between the two scoring methods (.998) raises the

question of whether this is a meaningful difference. It does

not appear that such represents a truly meaningful difference

and the subsequent analyses will only be done on the exact

29

word scoring data. The analyses of acceptable word scoring

data reveals identical results to those to be reported on

the exact word scoring method, and this points to acceptance

of the second hypothesis.

Hypothesis Three: There is a definable functional

deletion rate n for use with cloze tests in Samoan.

Table 7 shows the results of a one-way analysis of

variance and trend analysis on the deletion rate factor

versus exact word scoring data. The results of a one-way

analysis of variance indicated a significant effect for the

deletion rate factor, F (6,238) = 43.11, p<.0001. The

deletion rate factor accounted for fifty-two percent of the

variance in the exact word score data.

30

Table 7

Analysis of Variance and Trend Analyses on

Deletion Rate-.- Exact Word Score Data

Source df SS MS F p R2 ----

Deletion Rate 6 4316.496 719.4160 43.1114 <.0001 .521

Cum R 2

linear trend 1 3004.713 3004.713 180.059 <.0001 .363

I

jdeparture from !linear trend 5 1311.783 262.357 15.723 <.01 i

quadratic 1 1124.183 1124.183 67.367 <.0001 .498

departure from quadratic 4 187.600 46.90 2.81 .03

cubic 1 22.671 22.671 1.359 ns .501

departure from cubic 3 164.929 54.976 3.295 .02

quart~c 1 41.509 41.509 2.49 ns .506

departure from quartic 2 123.420 61.710 3.698 .03

quintic 1 96.991 99.991 5.81 <.OS .518

departure from quintic 1 26.429 26.429 1.584 ns

sex tic 1 26.402 26.402 1. 582 ns .521

Error 238 3971.595 16.6874

31

A series of t-tests were performed between adjacent

deletion rates in order to determine the location of the

significant differences between the various rates. There

were no significant differences between the nine-to-one

versus eight-to-one rates (t (238) = 0.53, p = .60), the

eight-to-one versus seven-to-one, (t (238) = -0.23, p = .82),

the seven-to-one versus six-to-one (t (238) = -0.47, p = .64),

or the six-to-one versus the five-to-one rates (t (238) =

0.59, p = .56).

Table 8

t-tests on Cloze Exact Word Scoring Data

9 to 1 vs. 8 to 1 t(238) = 0.53, p = .60 ns

8 to 1 vs. 7 to 1 t(238) -0.23, p = .82 ns

7 to 1 vs. 6 to 1 t(238) = -0.47, p = .64 ns

6 to 1 vs. 5 to 1 t(238) = -0.59, p = .56 ns

5 to r vs. 4 to 1 t(238) = -6.70, p < .0001 significant

4 to 1 vs. 3 to 1 t(238) = -3.60, p = .0004 significant

However, deletion rates five-to-one versus four-to-one,

and four-to-one versus three-to-one were significantly

different from each other, t (238) = -6.70, p<.0001, and

t (238) = -3.60, p = . 0004, respectively . A summary of

these t-tests is presented in Table 8. Figure 1 presents

these results in graphic form across the deletion rates.

32

Figur e l

Me .?.n Percent Corr ect Res pon ses a t the Various Deletion Rates

60 ~

u w ~ ~

0 u 50 ~

z w u ~ w a... 40 z <( w ~

30

o--~__._~~_._~~..i..-~__..~~--'-~~-'-~..-J

9:1 8:1 7:1 6:1 5:1 4:1

DELETION RATE

33

Subsequent t-tests were also performed between all pairs

of deletion rates, and these results provided the same

information. That is, from deletion rates of nine-to-one

through five-to-one there were no significant differences

between any pair. But there was a significant difference

between pairs four-to-one and above, or four-to-one and

below.

It was hypothesized that a quadratic curve model would

best describe the data shown in Figure 1. An analysis of

trend was used to test this hypothesis, and the results in

Table 7 show that a quadratic model accounted for the data

significantly better than a linear model, F (1,238) = 67.36,

p . 0001.

While the linear model accounted for thirty-six percent

of the variance (of a total of fifty-two percent), the

quadratic model accounted for an additional thirteen percent.

A test.for departure from quadratic was also significant,

F (4,238) = 2.81, p = .03, but the amount of additional

variance gained by going beyond the quadratic model is only

two percent. Thus a quadratic model best describes the

relationship between deletion rate and the exact word score

dependent measure (Kirk 1968).

The analysis of variance on the exact word scoring data

demonstrates that there is a significant difference in means

across deletion rates. The subsequent t-tests point out

exactly where that significant difference occurs. The data

leads to a confirmation of the third hypothesis that there is

' ' I

I l

34

a definable functional deletion rate n for cloze tests in

Samoan; however, it is not clear whether this functional

deletion rate is five-to-one or four-to-one, or perhaps both,

depending upon the type of reading passage used.

DISCUSSION

Hypothesis One: Cloze procedure can serve as a measure

of readability and reading comprehension.

The first hypothesis states that cloze procedure can

effectively serve as a measure of readability and reading

comprehension in the Samoan language, and the results

demonstrate that this is so. The mean percentage correct

scores on the cloze reading tests shown in Table 1 indicate

that Samoan students can use cloze procedure successfully

with all but the smallest deletion rates.

The multiple-choice reading test in Samoan was designed

to measure comprehension over the same material as the cloze

reading tests. Results on the multiple-choice reading test

also show clearly that the students performed reasonably

well with this traditional test format. Table 4 displays

strong correlations between the cloze reading tests and the

multiple-choice reading test, an indication that the two

tests are measuring a common ability.

At larger deletion rates, the cloze mean scores cluster

around sixty-percent, which is in fact the most common mean

across the five larger deletion rates. At a four-to-one

deletion rate, the mean score of forty-four percent suggests

a passage that could be handled at the "instructional" level

35

based on criteria cited by Bormuth (1967), and corresponds

to the 75% comprehension score on traditional reading compre­

hension tests.

The mean score of just under thirty-eight percent at a

three-to-one deletion rate indicates that this test rate

was too difficult. But since this latter test was over the

identical passage as the tests which produced sixty percent

means, we can see that the lower score is a reflection of the

diminished context at this smaller deletion rate (Fillenbaum,

et al. 1963).

Cloze functions effectively in languages dissimilar to

English, notably in Japanese (Yamada 1979) and Vietnamese

(Klare, et al. 1971). There is no basis for assuming that

cloze somehow measures something totally different in

Samoan. The high cloze mean scores demonstrate that the

relative lack of an extensive written tradition is not a

varia~le which affects cloze testing.

The mean scores among all groups on the multiple-choice

reading test establish that these students can deal with a

traditional comprehension test. The correlations between

the multiple-choice reading test and the cloze reading tests

signify a close relationship between the two, a fair indica­

tion that both tests are measuring comprehension.

Just as cloze appears to measure understanding of the

material, so too it gives an indication of the readability

of the passage. In general, a greater understanding of the

material being read will generate a higher cloze score.

36

Cloze seems to measure comprehension and comprehensibility

at the same time, which perhaps explains why many people

speak of them as almost the same concept.

A good indication of the relative readability of the

passage is that the students produced mean scores of around

sixty percent over the five larger deletion rates, with

lower means only when the context is substantially reduced.

The larger scores demonstrate that the students have a good

understanding of the material, for if it were too difficult

the scores would be lower, even at larger deletion rates.

Another indication of the readability level of the

passage used for closure is that subjective ratings by Samoan

readers placed it at the upper elementary level. A consensus

of teachers and students who read the undeleted passage was

that Samoan high school students should have no real diffi­

culties with the passage. (Primary school in American Samoa

runs through eighth grade, and high school begins with the

ninth grade.)

Subjective assessment together with the high mean scores

on the cloze reading tests demonstrate the overall effective­

ness of the cloze procedure as a measure of general

readability. Further, the scores on a traditional instrument

(the multiple-choice reading test) together with the strong

relationship between the two types of tests show that cloze

is a valid measure of comprehension in Samoan. The efficacy

of cloze procedure as a measure of both readability and

37

reading comprehension is substantiated by the data, and this

points to a confirmation of the first hypothesis.

Hypothesis Two: There is no appreciable difference

between exact word and acceptable word scoring methods.

The second hypothesis states that there is no appreciable

difference between the exact word (EW) and acceptable word

(AW) scoring methods with cloze tests in Samoan. While

there is a slight difference between the two scoring methods,

the results nevertheless justify acceptance of this hypothesis.

Table 5 presents the results of a two-way analysis of

variance on the scoring methods across deletion rates. The

results indicate a statistically significant albeit minor

difference in scoring systems. Table 1 illustrates that the

difference in total mean score for the two scoring methods

on the cloze reading tests amounts to 0.12, which in practi­

cal terms does not constitute a crucial difference.

C9rrelations between the exact word and acceptable word

scores are shown in Table 6. These correlations are all in

excess of r = .99, nearly perfect. Since both sets of scores

were produced by the same groups, this is not unusual. But

with the total difference in means so small, and the

correlations between scoring methods so high, it becomes

clear that whatever happens with the exact word scores will

also occur with the acceptable word scores.

This duplication of results is exactly what has

transpired. The analyses of variance, correlations, and

t-tests provided exactly the same information with both

38

scoring methods. That being so, the exact word method should

provide all the necessary information with no question about

what constitutes an acceptable synonym.

In English, the bulk of the proof lies with the exact

word method. Studies by Taylor (1957) and Ruddell (1964)

state that exact word scoring provides basically the same

results as acceptable word scoring with far less work. Exact

word scoring is the preferred method in testing situations

to preserve scorer reliability and to provide economy.

Since there is a slightly significant difference in

scoring metho.ds, additional testing may prove useful.

Because a great deal depends on subjective assessment of

synonyms, it would be wise to proceed with caution before

making any conclusions in favor of the acceptable synonym

method, especially since the exact word method seems to do

the job equally well from a practical standpoint. A careful

exami~ation of the data _prompts acceptance of the second

hypothesi~.

Hypothesis Three: There is ·a definable functional

deletion rate n for cloze tests in Samoan.

The third hypothesis states that there is a functional

deletion rate n for cloze testing in Samoan. An analysis

of the data indicates that not only is this true, but the

functional deletion rate in Samoan falls within the range

of those commonly used in English, four-to-one (4:1) and

five-to-one (5:1).

f

I t i ' ' f ,

I

39

The results of a two-way analysis of variance in Table

5 indicate that there is a significant difference in mean

scores across deletion rates. A close examination reveals

that this difference is the result of a sizeable drop in

mean scores between the five-to-one and four-to-one deletion

rates, and a smaller but still significant drop between the

four-to-one and three-to-one rate. This pattern can be

seen quite clearly in Figure 1.

The t-tests shown in Table 8 confirm that there is no

significant difference in means from the nine-to-one through

five-to-one deletion rates. There is only a significant

difference between deletion rates four-to-one and above, or

four-to-one and below.

The data suggests that both a four-to-one and a five-

to-one deletion rate would be functional for cloze tests in

Samoan. Arguments can be given for both deletion rates using

the data from this study. Individual interpretation may

favor one or the other, but an absolute determination cannot

be given based on the administration of one test passage to

one sample group.

Evidence for the four-to-one functional deletion rate

comes from the fact that it falls directly between a deletion

rate that produces inadequate results, and larger deletion

rates which are equally easy to cloze. Further, the use of

larger deletion rates forces the reader to process more text

and use more context to achieve basically the same results,

so in essence this would be less efficient reading.

t

I I

i

I f !

I I I l I

I I I i

I l

40

The designation of a functional deletion rate in

English appears to be based on the amount of context.

MacGinitie (1961) has established the extreme interdependence

of items at deletion rates of less than four-to-one. Other

authorities as well accept that in English we need about

eighty percent of the context in order to complete the

other twenty percent (Bormuth 1967). ·

Figure 1 demonstrates that scores are relatively stable

from deletion rates five-to-one th~ough nine-to-one. The

fact that there is no significant difference in means for

these deletion rates indicates that increased context at

these rates does not produce a concomitant increase in mean

scores, but does require increasing amounts of context to

provide the text base of fifty deletions. This coincides

with results by Aborn and others (1959) which show that

additional surrounding context beyond four words does not

produce any significant gain in score.

We see in Table 8 that moving from a five-to-one

deletion rate to a four-to-one deletion rate does result in

a significant drop in means. Thus we have some evidence that

a four-to-one deletion rate provides less but still sufficient

context. Going from a four-to-one rate to a three-to-one

rate again results in a significant drop in mean score , and

a crucial drop in context.

At the three-to-one deletion rate. the mean percent

correct scores of just under thirty-eight percent fall within

the "frustration" level designated by Bormuth (196 7) , and

41

this would normally mean that the material is too difficult.

But other cloze tests taken from the same passage show higher

means, so the passage itself is not too difficult. Rather,

this narrow deletion rate is too difficult to process because

of the minimal surrounding context.

The reduced context at a three-to-one deletion rate

would dictate a high degree of interdependence among the

items. This means that the successful completion of one item

rests on successful completion of a prior item. The result

is that scores do not reflect the ability of the testee so

much as they reflect a significant element of chance.

An argument for the five-to-one functional deletion rate

for cloze tests in Samoan is that one must reach this point

for there to be no further change. It is true that moving

to a larger deletion rate would produce no significant

change, and that moving to a smaller deletion rate would.

This ~ine of thought relies on a statistical perspective

rather than the contextual bias generally employed with cloze

research in English.

Additional support for the five-to-one functional

deletion rate may be derived from the larger standard

deviation produced at this rate. If greater variability is

a preferred criterion, then five-to-one would stand as a

more useful deletion rate, based on results from this test

administration. Hittleman (1978) concurs that deletion

rates larger than four-to-one provide more variation.

42

The type of prose is a further consideration which

could favor one deletion rate over the other. The test

passage in this study was essentially narrative material on

relatively familiar subject matter. Narrative material

generally produces higher results than technical prose where

function words carry the logic and where fewer synonyms are

used (Smith-Burke, et al. 1978). The use of a technical

passage may show that five-to-one is a functional deletion

rate.

Where there is a desire to measure information gain,

four-to-one may be the functional deletion rate. However,

a five-to-one deletion rate seems to produce greater

variability and thus may prove to be more functional in some

instances.

It is not possible to unequivocally state which of the

two deletion rates is more functional based on the data

from one test. More comprehensive testing should demonstrate

which of the two is a functional deletion rate for cloze

tests in Samoan. The hypothesis nevertheless stands

confirmed in that an identifiable functional deletion rate

for cloze tests in Samoan is clearly in evidence.

CONCLUSIONS

The intent of this project was to determine whether

the cloze procedure works effectively in Samoan as a measure

of both readability and reading comprehension. An analysis

of the data supports the conclusion that cloze does in fact .

measure aspects of readability and reading comprehension in

43

the Samoan language. This parallels the validation of cloze

in languages dissimilar to English.

The strong correlations between the cloze reading tests

and the multiple-choice reading test over the same type of

material demonstrates that cloze tests measure comprehension

as well as readability. That cloze provides essentially the

same information and functions as a global measure of compre­

hension gives weight to the argument that cloze can serve as

an effective instrument in Samoan.

The issue of which scoring method to use can be resolved

on the basis of practicality. There is a statistically

significant difference between the exact word and acceptable

word scoring methods, although this difference is marginal.

The exact word method provides essentially the same informa­

tion and is far more efficient.

The optimal deletion rate appears to be either five-to­

one or. four-to-one, perhaps depending upon the kind or reading

passage being used. The determination of these .rates is based

upon the significant falloff in mean scores at deletion rates

of less than five-to-one. On the one hand the four-to-one

deletion rate uses the least amount of reading context and

appears to be the point of balance between the too easy and

too difficult continuum. On the other hand the five-to-one

deletion rate functions as effectively as deletion rates of

six-to-one through nine-to-one while using correspondingly

less reading context than the higher rates. The same

44

arguments with regard to necessary surrounding context that

apply to English seem to be relevant to Samoan.

This study produced a few salient points about the

measurement of reading in Samoan. Heretofore, measurement

of readability, reading comprehension, and reading skills

has been conducted on an informal basis. The lack of a

repertoire of materials in Samoan has made the need for

creating evaluation instruments less than critical. But the

recent resurgence of efforts to create materials in Samoan

means that some form of evaluation is in order for reading­

related materials. All indications are that the cloze

procedure can serve as an economical, efficient adjunct to

these efforts .

. IMPLICATIONS

A number of issues that evolved from this study warrant

further considerations. The determination that cloze

procequre is a viable measure of reading comprehension and

readability has implications for the Samoan language program

in general and the materials development/reading program in

particular. The existence of an economical evaluation

instrument for both comprehension and passage difficulty of

new texts should prove beneficial.

Several reconrrnendations seem appropriate. It would be

useful to administer cloze reading tests across grade levels

in all high schools and in upper elementary grades throughout

Samoa. The object would be to test various passages at one

particular deletion rate, either four or five to one, with

45

five forms for each test to obtain an average of the five

means. The means may vary across grade levels, but the data

would provide the basis for establishing grade placement

levels for books.

The creation of a standardized instrument in Samoan, a

Samoan language test or reading comprehension test, is

extremely desirable. There must be a way to evaluate student

progress on a yearly basis. Further, a norm-referenced

instrument would ·prove highly useful as a cross-validation

measure for cloze tests and any other tests developed in

Samoa.

Test administration on this scale requires close

cooperation at high levels and more than a degree of persis­

tence. There are priorities to consider, but every school

system needs evaluation procedures to measure its own

success vis a vis the students. Cloze tests could facilitate

the ev.aluation process substantially by virtue of the fact

that they are simple to create, to use, and to correct.

A project of this scope requires careful planning and

execution, but it seems clear that the end results would

more than compensate for the effort. With little else

available in the way of formal assessment mechanisms, the

cloze procedure would appear to have the broadest application

with the lowest cost, in terms of both time and money .

System-wide testing should provide long term benefits, as

more materials and books are produced in the Samoan language.

46

APPENDIX A: INSTRUCTIONS AND SAMPLE TEST

Su'ega fa'ata'ita'i ma ni fa'amalamalamaga atili

FA I AMALAMALAff.AGA

I le fa'ai'uga o le itulau lenei, o 10 1 0 i ai se

fa'aa'oa'oga o se itua'iga su'ega fou. 0 nei su'ega taitasi

sa lolomiina mai i ni palakalafa o se tusi. 0 lo'o iai upu

o lo'o misi i laina ta'itasi.

0 lau vaega e fai, o le tusiina mai lea o le upu o 10 1 0

misi i faiupu taitasi. Tusi i le avanoa o lo'o iai le laina,

i luga o le laina.

1. Tusi na'o le upu e tasi i avanoa ta'itasi.

2. Taumafai e fa'atumu avanoa uma. 'Aua le fefe e mate.

3. Taumafai e sikipi avanoa faigata, ona toe fo'i mai

lea iai pea uma ona fa'atumu isi avanoa.

4. Taumafai e tusi le upu sa'o. E le afaiina pe a le

sa'o le sapelaina, a ia tau fetaui ma le upu sa'o.

5. 0 le tele o avanoa e taliina i upu ua tatou masani

ai, ae o nisi upu e tusiina fa'apea:

*O fainumera e pei - 3,247 po'o le $12.00 po'o 1978

*O nisi upu e ta'ilua, e pei o le

47

SU'EGA FA'ATA'ITA'I

0 101

0 i lalo ifo o le itulau lenei se su'ega fa'ata'ita'i.

Fa'atumu le avanoa i le upu ua e iloa ua le'o fa'atumuina. ·

Toe siaki lau pepa pe a uma, i lou toe vaai lelei i au tali.

0 tali na o lo'o tusiina, fa'au i luga ma lalo i le pito i

lalo o le pepa. Tusi manino mai ia mama.

Su'ega

0 Ioane o le tama i vasega amata i le

kolisi (freshman), ma lo'o maua e ia

fa'afitauli uma pei ona masani ai le

to'atele tamaiti fa'atoa amata. 0 se mea

mautinoa lava, o ia fa'afitauli uma

amata mai a'o le'i alu ese

ia mai lona aiga. E tele sa ia faia e le'i

mana'o

o ai mo le kolisi.

VC1UJ

0

. 8

• +J

f ai aua 0 le a alu

0 . L

.£

C1SC1

vs

0

vn

. 6

·~

. t

r 1.

t r ! 48

APPENDIX B: CLOZE TEST PASSAGE IN ENGLISH

TRANSFER OF LAND

The transfer of land from one f arnily to another is

justified in the following instance: a chief may be

embarrassed by the unexpected arrival of his sister or a

fellow chief and his not having a pig to welcome the visitor.

He reflects which of his neighbors--chief or orator--owns a

big porker or two. He then goes to the owner a~d says,

"Chief, I find myself in a predicament and should like to

have the pig that is wandering about your place." The other

replies, "You have done well to come to me in your trouble.

Send your young men to catch the animal."

After the visitors have departed, the chief discusses

with the members of his family how to repay his friend. They

decide that the chief go to him and speak as follows: "I

have two offers to make to reward you for your service. I

am ready to give you a boat or, if you prefer, you may have

a piece of land. II The other replies, "I thank you for corning

along. Keep the boat; your children may need it to go

fishing. Let me have the piece of land; it will be useful

to my family. Thank you." The land will belong to the new

owner for ever after.

There is another justification for the transfer of

land. Since the days of yore there have been in Samoa expert

bonito fishermen. They are adept in the art of tying the

49

bonito fishhook. The people firmly believe that if the hook

is wrongly tied, no fish will be caught.

If a chief wants to become a bonito fisher, he must

first learn how to tie the hook. He will go to the chief

fisherman and ask him to teach him the art. The fisherman

grants his request and shows him how to fasten the tortoise

shell hook to the shaft of pearl shell. As a result, the

chief catches many bonitos.

The chief then considers what return to make to the

fisherman, whether to repay him in fine mats or pigs. He

realizes that neither would do him much good because, in a

fit of temper, the fisherman might take away from him the

right of tying the hook. A piece of land, on the other

hand, will be a constant reminder of the transaction .

I

I l

r I i ~ i

50

APPENDIX C: CLOZE TEST PASSAGE IN SAMOAN

0 LE FAAFESUIAIGA FANUA

0 le faafesuiaiga o se f anua mai le isi aiga i se isi

e mafai ona faatulagaina faapea. Afai ua tigaina se alii

ina ua omai se malaga a sona tuafafine poo se tamalii, a ua

leai se manu e tai ai se sua i lena malaga, ona filifili

ai lea o ia poo ai se tulaf ale poo se alii o iai se manu

tele lava, pe tasi pe lua.

Ona ia alu· lea i le al ii o iai sana manu ma fai atu,

"Alii e, ua ou le maufautua, o lea ua ou sau ai i lau manu

o 10 1 0 taa i lou pa'epa'e ina ia ou avea e fai ai lou faa-

lavelave. II Ua tali atu le alii, "Ua le lei ua e maliu mai

lou tigaina. 0 mai ni taulelea e pue atu le manu. II

Ua toe f oi atu le malaga i lo latou nuu moni, ona

f ilif ili ai lea 0 le alii ma lona aiga uma i se mea e taui

ai le pule a le alii na aumai ai le manu. Ona latou maua

ai lea o le tonu e faapea, ia alu atu le alii i le matai

ma fai atu ia te ia. "E lua mea nei ua ou sau ma au ou te

folafolaina atu e tali ai lau pule. 0 le vaa, a e le

f inagalo iai ou te avatua le f asif anua e taui ai lou aga­

lelei." Ua tali atu le tulafale, "Ua faafetai maliu mai,

tuu ai pea le vaa, e aoga i ou alo, ae aumai le fasifanua

ua e folafola mai nei e agai atu iai lau fanau. Faafetai

lava." 0 lena fasifanua o le a tumau lava i le aiga e

faavavau.

i

I I I I l

f t ~ !

51

E iai le isi ala e rnaua ai le fanua. 0 le tu rnai

le vavau a Samoa, ua iloga fale tautai o le alo atu. Sa

faufau e i latou pa e sisi ai atu. Sa faapea taofi o Samoa,

a sese ona fau o le pa, ua leai se atu e maua e lea vaa.

Afai o se alii ua manao i le fale tautai, ona manao

foi lea o ia ina ia na iloa faufau o pa e maua ai atu. Ona

fai atu ai lea o ia i le tautai sili, "Ua ou manao ia ou

iloa faufua pa; ia e alofa lava oe le tautai sili, ia

a'oa'o mai ia te au le faiga o lea mea." Ua talia o ia e

le tautai sili. Ona a'oa'o ai lea o ia i le fauga o pa.

Ua tele atu e maua i ana pa e faufau.

Ona f ilif ili ai lea o ia poo le a se mea e aoga e

ave atu i le taupo o ni ietoga poo ni puaa. Ona faapea

ai lea o lona taofi, e leai se e aveatu ai ni ietoga poo

ni puaa i le tautai, aua a ita o ia i se aso, toe fao lea

o le faufau pa. E lelei ona tuu atu ia te ia o se fasi-

fanua. faamanatu ai i le tautai ua tonu lava, ona ua tuuina

mai e ia le faufau ia te au.

r

' I 52

APPENDIX D: f:.fULTIPLE-CHOICE READING TEST IN SAMOAN

r I

I

I

r I ! ' 53

FAATONUGA

0 palakalafa nei e fesoasoani ia te i matou e suesueina ai

pe faapefea le tulaga o lau faitau. I faai'uga uma o vaega

taitasi o loo iai fesili e fa e tauina mai ia te i matou pe

f aapef ea le lelei ona e

1) sueina faamatalaga moni;

2) faia faaiuga;

3) malamalama i upu;

4) sueina le manatu autu.

E sefulu ma le lima vaega i lenei suega. Ia faitau lemu i

vaega taitasi. A uma ona faitau, tali lea o fesili e fa.

A uma loa lea vaega ona faasolo lea se ia ma e'a. A amata

loa ona e galue i le suega ona, 'aua lea ona e toe fesili.

'Aua nei mataina lenei tusi. Ia faatumu le avanoa · i

talaane o le numera e fetaui ma le tali ua e f ilifilia i

le pepa mo tali; faataitaiga:

A B c D

0) 0 © Amata le suega pe f aapea mai le f aiaoga, "Amata".

54

E lua ala o toga: o toga muamua e ta'ua o le laufau (e lavalavaina e le tamaitai i lana faaipoipoga); o toga rnulimuli e ta'ua o le saga. 0 le ie na aumai ai laufau, o le ie lea o le tamasa. 0 le ie ua aumai ai le saga, o le ie lea o le a maua e le tulafale e ana le togafiti i lona fale; a o tulafale uma e tofu lava i latou ma le ta'itolu pe ta'ifa ie toga ma o latou siapo e faalagolago i le tele o siapo o le nunu.

1. 0 le ituaiga toga e lavalavaina e le tamaitai i lana f aaipoipoga o le

1. saga 2. laufau 3. laufala 4. siapo

2. 0 le ala e lavalavaina ai ele tamaitai ona o le

1. mativa 2. tulaf ono 3 . mo'omo'oga 4. aganuu

3. 0 le upu i le palakalaf a e uiga maua

1. ala 2. faapitoa 3. togafiti 4 . tofu

4. 0 le manatu taua o lenei vaega o le .•

1. ituaiga o ietoga 2. tofusia o ietoga 3. lalagaina o ietoga 4. teuina o ietoga

r 55

0 nofoaga pe a fai ni saofaiga. E le so'ona nonofo i le fale, aua ua iloga nofoaga. Afai o se alii, e nofo tonu i le matuatala po o pou o le tala. Afai o se tulafale, e nofo i le pou i le pepe o le fale ma · le atualuma foi. Ua ta'ua o va-i-pou po'o va-i-matai tulafale ua le maua ni o latou pou, aua e gata ia i latou e sili ona maua pou.

5. I lenei palakalafa, 0 ai tagata e le nof o i se pou?

1. · alii 2. f aipule 3 . tulaf ale 4. kovana

6. 0 tagata i le f ono a matai e

1. maf ai ona filifili lona nofoaga . 2. iloa lona nofoaga. 3. le iloa lona nofoaga. 4 . le mautinoa lona nofoaga.

7. 0 le upu i le palakalaf a e uiga rrrua:mua

1. pou 2. nof oaga 3. maua 4 . sili

8. 0 le manatu sili 0 lenei palakalaf a o le

1. nofoaga sao 2. sili o le alii 3. fono a matai 4. pou o le fale

56

0 tulafale sa faia lea filifiliga latou te su'esu'eina lenei tamaitai pe faapefea ona itu, pe tutusa itu e lua, e lelei le itu o le tama, lelei foi le itu o le tina. E faapea foi le aiga o le tamaitai ma lo latou nuu, latou te su'esu'eina le tama poo le alii, pe tutusa ona itu pe leai. Afai ua latou iloa e le tutusa itu o le alii ma lo latou tamaitai, ona te'ena lea o le aumoega. Afai o se tamaitai e le tutusa ona itu, e le mafai ona alu i ai se aumoega a se nuu atoa; e gata lava i le pitonuu e i ai le alii ona o atu e aumomoe i lea tamaitai.

9. Tasi le ala e lelei lava e taofi ai se talanoaga o se faaipoipoga:

1. e le aulelei le tama. 2. e le masani le tama. 3. e lei faa~ua le tama. 4. o le tagataese le tama.

10. 0 le faiva o le aumoega o le

1. tausi le aumoe. 2. vaavaai le aumoe. 3. puipui le aumoe. 4. filifili le aumoe.

11. 0 le upu i le palakalafa e uiga agatonu

1. motu 2. sosoo 3. gata 4. tutusa

12. 0 le manatu sili o lenei palakalafa o le

1. aumoega 2. faaipoipoga 3. faafiafiaga 4. suesueina itu

57

Af ai o se tagata e tautala tu i luma o alii ma tulafale i totonu o se fale, o le faalemigao lea. E tosoina o ia i fafo e ni taulelea ma f asi ia te ia ma tuli ese, 'aua le toe nofo mai i le mea o potopoto ai le nuu. Afai foi o se tagata e inu tu i totonu o se fale, o le faalemigao lea. E faia lava pei ona taua i luga.

13. 0 se tagata e tu ma inu i totonu o le £ale o le

1. ona 2. faalemigao 3. fia inu 4. agamasesei

14. 0 se tagata o loo tauaveina se amoga i luma o se matai e rnafai ona

1. sasaina 2. f asiotia 3. f aif ai 4 . faatelenoaina

15. 0 le upu i le palakalaf a e uiga alualu

1. totolo 2. tamo'e 3. tuli 4. su'e

16. 0 le ulutala sili mo lenei palakalafa o le

1 . Amioga a le Tupulaga 2. Faalagiga o le Matai 3. Finauga ma le Matai 4. Amioga Faalefeagai

r i

58

0 se alo teine o se alii, ua ta'ua foi o le taupou. E f aapotopotoina i ai o teine ma tamaitai o le nuu e fai se fale aualuma. Ua iloga foi le mamalu o lea tamaitai i le toatele o lona galuteine. E i ai foi i le aualuma ni isi taulelea, o tane a isi o le aualuma, e nonofo tumau ai i le ao ma le po. 0 lo latou igoa o palepaleaulama. 0 le galuteine foi o le tamaitai ua iloga ona soaf afine pe toatolu pe toafa. 0 latou na e mulimuli pea i le tamaitai taupou i mea uma e femaliuai ai. A taumafa foi le tamaitai, o le afafine o le tulafale e auauna i ai. Afai foi e alu le nuu i se malaga, ua iloga pea le taupou ma lona aualuma. E teuteu lelei i ie toga. A talimalo foi le nuu, o le taupou foi e pita tele ana teuga.

17. 0 mea ai a le taupou e laulauina e

1. uso o le tulafale 2. tama o le nuu 3. tuagane o le taupou 4. afafine o le tulafale

18. 0 le taupou e le tele se

1. aualuma 2. tof i 3. galuega 4. ietoga

19. 0 le upu i le palakalafa e uiga faaaloalo

1. mamalu 2. teuga 3 . aualuma 4 . galuteine

20. 0 le igoa sili mo lenei palakalaf a 0 le

1. Uo a le Taupou 2. Galuega a le Taupou 3. Aiga o le Taupou 4. Aualuma a le Taupou

59

E ese'ese uiga o tagata e gaoi i maumaga o isi tagata. 0 le isi e gaoi talo, ae fai lelei tiapula ma tuu lelei e toe toto i le maumaga o lea tagata. E faasalaina o ia i sala o le gaoi pe lima sefulu ni talo ae valu sefulu ni afi. 0 le isi tagata ua gaoi e ia le maumaga, ma ave uma talo ma tiapula. E faasalaina o ia i le sala lava o le gaoi e tai tutusa ma le o muamua.

21. 0 ituaiga gaoiga ese'ese, o le fasalaga e

1. ese'ese 2. tutusa 3. manafa 4. tele lava

22. O se solitulafono matautia o se tagata ua avea aa ma

1. ave tiapula 2. laulaau o tiapula 3. faasalalau o tiapula 4. toe toto tiapula

23. 0 le upu i le palakalafa e uiga e le faapenei o le

1. tutusa 2. eseese 3. isi 4. masani

24. 0 le ulutala sili mo lenei palakalafa o le

1. Tagata Gaoi 2. 0 Ituaiga Talo 3. Gaoiga Talo 4. Tiapula ma Talo

60

Afai ua nofo mai le tamaitai i le manaia sa alu i ai lana aumoega, ona la o mai lea i le nuu o le manaia. Ona faasaga atu lea o le nuu o le manaia, ua latou faia se galuega, o le ati o se paepae e faailoga ai le taupou. Ona mafaufau ai lea o se isi tulafale i sana togafiti ina ia na maua se ie o le tamaitai. Afai ua manao o ia i le taua o i le tama o le tamaitai, pe a aumai ona toga, e alu loa le tulafale ma f ai atu i le tama o le manaia, "Alii e, ai e leZei ona awnai o Zou faZeuZu e ufi ai Ze paepae o Ze tamaitai." Ua fai atu le alii, " Ua ZeZei, awnai."

25. E ofoina atu e tulafale mo se ulugalii fou se

1. ulu 2. fiafiaga 3. fale 4. ietoga

26. 0 le upu sili e f aamatalaina ai lenei tulaf ale 0 le

1. matamau 2. matape'ape'a 3. f ealof ani 4. fa'a'au'au

27. 0 le upu i le palakalaf a e uiga f aamanatu

1. aumai 2. faasaga 3. mafaufau 4. faailoga

28. 0 le ulutala sili mo lenei vaega o le

1. Faaiuga o le Faaipoipoga 2. Mauaina o se Ietoga 3. Fauina se Fale 4. Fauina o le Paepae

61

0 aumaga o fanau tama uma a alii ma tulafale. Tiga ona matutua ma ua tata, e le mafai ona avea i latou ma tagata o le aumaga, seiloga ua avatu se puaa e fai ma ana usufono, ona faatoa avea lea ma tagata o le aumaga. Ua faatoa fai lana monotaga. Ua faatoa maua ai sona tufaaga i mea a taulelea pe a fai ni faigaai.

29. A e lei auai le tama i le aumaga, e tatau ona f aia nei mea vagana ai le mea e tasi

1. ia atoa tausaga 2. ia faaipoipo 3. ia ave se puaa 4. ia ta le pe'a

30. I lenei palakalafa, 0 tama 0 loo i le aumaga e

1. aulelei 2. talavou 3. malosi 4. atamai

31. 0 le upu i le palakalafa e uiga faataga

1. maua 2. totogi 3. faatoa 4. avea

32. 0 le ulutala lelei mo lenei vaega o le

1. Taulelea 2. Aumaga 3. Monotaga 4. Tulafale

62

Af ai ua tigaina se alii ina ua o mai se malaga a sona tuafafine poo se tamalii, a ua leai se manu e tai ai se sua i lena malaga, ona filifili ai lea o ia poo ai se tulafale poo see alii o i ai se manu tele lava, pe tasi pe lua. Ona ia alu lea i le al ii o i ai sana manu ma fai atu, 11 AZii e, ua ou Ze maufautua, o Zea ua ou·sau ai i Zau manu o Zoo taa i Zou paepae ina ia ou avea e fai ai Zou faaZaveZave. 11 Ua tali atu le alii, " Ua ZeZei ua e maZiu mai i Zou tigaina. 0 mai ni ta.uZeZea e pue atu Ze manu."

33. 0 le matai e iai malaga (asiasiga) faateitei mai

1. tamalii 2. uo 3. fafine 4. taulelea

34. E le tatau i se matai ona laulau i ni malo lauiloa se

1. ta lo 2. puaa 3 . pilikaki 4. palusami

35. 0 le upu i le palakalaf a e uiga oto

1. filifili 2. fesuiai 3. talimalo 4. fautua

·36. 0 le igoa le lei mo lenei palakalafa o le

1. Meaalofa a le Matai 2. Faafiafiaga a le Matai 3. Fesoasoani a le Matai 4. Faalavelave a le Matai

63

Faapef ea le pule a alii i tagata ma le laueleele o le nuu? E pule i ona lava tagata, e tautua i lona lava alii. 0 latou foi e ta'ua o ona tagata e pule i ai. Ao fanua, e le soona pule, aua e iloga fanua e autu i le suafa. 0 le ua nofo i le suafa, e pule aoao i fanua uma e autu i le igoa. Peitai afai ua tea ma le igoa ona tea foi lea ma fanua uma sa pule i ai. E gata lava se fanua e pule i ai, o se fanua na maua e ona lava lima.

37. 0 tagata i lalo o le puleaga a le matai e taua latou o ona

1. pologa 2. USO 3. auauna 4. uo

38. E ie maf ai e le matai ona

1. tuf atuf a fanua 2. faatau atu laueleele 3. f aataumai f anua 4. ·gaoi f anua

39. 0 le upu i le palakalaf a e uiga faatamala

1. soona 2. taua 3. pule 4. tautua

40. 0 le ulutala sili mo le palakalafa o le

1. Pulega a le Matai 2. Fanua a le Matai 3. Augatupu o le Matai 4. Aiga a le Matai

64

Mo le fasioti tagata, e taui le oti i le oti. E le faatali sei maua le na ia fasioti le tagata, ae tau lava o se e vave maua e le aiga o le tagata ua oti; poo le uso, poo le matai o le aiga, poo sona atalii, e sui ai le ua oti. A ua i ai se togafiti e mafai ai ona ola o le sala; o le ifoga e alu i le aiga poo le itumalo o le tagata ua oti. E ave le pagota ma faapulou i le ietoga, e ta'ua lea ie o le ie o le malo. E mafai lava ona ola ai le pagota; e saoloto foi o ia i mea uma e fealuai ai.

41. E faamaloina se fasioti tagata pe a fai na te

1. alu ese 2. fai se ifoga 3. tautino lona mama 4. tala faaoti

42. 0 le aiga o le ua lavea e mafai ona tuli vag·ana ai le o le fasioti tagata.

1. f aif eau 2. atalii 3 . tuagane 4 . matai

43. 0 le upu i le palakalaf a e uiga ufiufi

1. f aatali 2. f aasala 3. f aapulou 4. faatauga

ff4. 0 le manatu taua o le palakalaf a o le

1. Faasalaga 2. If oga 3 . Finauga 4. Tauimasui

65

Ua ta'ua o le faalemigao le tu i luma o alii. Afai o se e ai tu i luma o se alii pe tautala, e faasalaina lea. Afai foi e amo tu se toi i lumafale o se alii e fasia lea tagata. 0 se tagata alu malaga, a tau atu i se galuega o fai, afai e alu loa, o le faaletea. E fasia tagata faapea. A tau atu i se galuega e fai i le ala, ona faigaluega ai lea, o le faaaloalo sili lea. E faapea foi o se vaa e tau atu o toso, e tatau ona faasega e tosovaa ai. Ua sa foi ona pa'o se galuega o alu se malaga i le ala. A se malaga o vaa e sa ona ulitu lo latou vaa. Ua sa foi ona faaaoga se faamalu poo se pulou lauua i luma o alii, pe fafatu se tama i lumafale o alii.

45. 0 a ituaiga tagata e tu i luma o alii?

1. pisa 2. faalemigao 3. tauaso 4. talavou

46. A e fetaui ma se tagata o loo toso lona vaa, e tatau ona e

1. faapea tulou ma e pasi 2. taofi lau taumafa 3. tuu i lalo lau avega 4. tu ma fesoasoani

47. 0 le upu i le palakalafa e uiga vavao

1. leaga 2. soli 3. faasala 4. sa

48. 0 le manatu taua o le palakalafa o le

1. galuega mo le matai. 2. aai faatasi ma le matai. 3. faaaloaloga mo le matai. 4. femalagaai ma le matai.

66

Af ai ua f ef e le nuu o le alii na ia mulilua ma le tamaitai ina nei tupu se mea mataga, (misa) ona o uma ai lea o le nuu o le alii, o taulelea ma tulafale uma ma alii e fai le ifoga, faatasi ma le na mulilua. E manao le nuu o le alii, latou te o atu i le vaeluapo ma faatatau ia latou taunuu atu i le nuu o le alii, pe'a ta le lima, a o lei ala tagata. Latou te savavali ma le filemu. E leai se isi e tautala, e leai foi se 1s1 e tale. 0 mea e lavalava e le pagota o le ietoga. Latou te ave foi se ietoga e taua o le ie o le malo, e pulou ai le na agasala. Ona latou nonofo ai lea i le malae ma ifo o latou ulu. Latou te punonou pea seia oso le la.

49. E lavalavaina e le mulilua le

1. lau 2. siapo 3. lavalava 4. ietoga

50. 0 le mulilua ma lona nuu e tatau ona taunuu

51.

52.

i le nuu ua ia faatigaina, a e lei feala tagata ina ia

1. taunuu vave 2. Malo lo laititi 3 . f ai if oga 4. f aatali

0 le upu i le vaega e uiga f usu

1. f inau 2. mataga 3. mis a 4. f ef e

0 le ulutala sili mo lenei vaega 0 le

1. Malaga 2. Ifoga 3. Mulilua 4. Faasalaga

67

0 faailoga vaaia o le tofiga ma pule i le nuu. Ua iloga lava le ua tofia i ni isi faailoga. Afai o se tulafale sili, e lua mea e faailoga ai: o se fue afa tele, ai pe valu ni pauna, ma se tootoo pau, ai e valu futu le umi. A lauga le tulafale i se fono tele a le nuu poo.le itumalo, e tu ma le tootoo ma le fue afa. 0 le fue lea ma le tootoo e tuuf aasolo lava i le f anau ma e e au i le igoa. Ua faapea, a tausi i fue afa ma le tootoo, e maua ai le poto o le tama e lana fanau. Ua sili fanau a tulafale i le matapopoto ma le televavave i feau a alii. 0 latou foi e tufaina ava o alii.

53. 0 le talitonuga, a tausi e le atalii fue af a ma le tootoo, na te maua le a le tama.

1. tupe 2. f anua 3. po to 4. f aaaloalo

54. 0 le f ue ma le too too 0 taua.

1. me lei 2. f aailoga 3. laau 4. pule

55. 0 le upu i lenei vaega e uiga f aasoasoa

1. sueina 2. aumai 3. aveina 4. tuf aina

56. 0 le ulutala sili mo lenei vaega o le

1. Mea a le Tupu 2. Atalii o le Matai 3. Faailoga o le Pule 4 . Agatonu i le Fono

So masani le nuu o Amoa i le tapui faititili. Sa tapuai i latou i faititili. Afai o se tagata ua ia ave

68

se mea i le fanua ua uma ona tapuia i le tapui faititili, o se malaia lava o ia pe to'ia i se faititili pe to'ia o latou fanua poo latou fale i se faititili. Afai e to'ia se tagata i se faititili poo so latou fanua, ona iloa ai lea o le tagata lava lea ua na aia le tapui pe ave ese se mea i le fanua ua tapui i le tapui faititili.

57. 0 faatoaga o Amoa o loo puipuia e

1. faititili 2. matagi 3. uila 4. timu

58. 0 lenei atua e mafai ona taia tagata o loo

1. paie 2. gaoi 3. valea 4. ona

59. 0 le upu i le vaega e uiga taisia

1. ootia 2. toia 3. fofoina 4. tapuai

60. 0 le manatu taua o lenei vaega o le

1. atua o le nuu 2. tala o le nuu 3. faomea o le nuu 4. tapui o le nuu

BIBLIOGRAPHY

Aborn, M., Rubenstein, H., and Sterling, T. D. 1959. Sources of contextual constraint upon words in sentences. Journal of Experimental Psychology 'i]_, 3:171.

Asher, S. R., Hymel, S. and Wigfield, A. 1976. Children's comprehension of high-and-low interest material and a comparison of two cloze scoring methods. ERIC ED 134 939.

Baldauf, R. B. and Propst, I. K. 1979. tests for elementary ESL students. 6:683-690.

Matching cloze Reading Teacher 32,

Bormuth, John. 1962. Cloze tests as measures of readability and comprehension ability. (Unpublished doctoral dissertation) University of Indiana.

. 1964. Experimental applications of cloze tests. -----In J. Allen Figurel (Ed.) Improvement of reading through classroom practice. International Reading Association Conference Proceedings 9:303-306. Newark, Delaware: International Reading Association.

. 1965. Validities of grarmnatical and semantic ---c~l-a-s-sifications of cloze test scores. In J. Allen

Figurel (Ed.) Reading and inquiry. International Reading Association Conference Proceedings 10:283-286. Newark, Delaware: International Reading Association .

. 1967. Comparable cloze and multiple-choice -----comprehension test scores. Journal of Reading 1.Q, 2:291-299.

. 1968. The cloze readability procedure. ---E~l-em-entary English 45, 4:429-436.

. 1969. Factor validity of cloze tests as measures ___ o_f_r-eading comprehension ability. Reading Research

Quarterly 4, 3:358-365.

Chall, Jeanne S. 1958. Readability: Appraisal of research and application. Columbus: Onio State University.

Cleland, D. L. 1966. A construct of comprehension. In J. A. Figurel (Ed.) Reading and inquiry. International Reading Association Conference Proceedings 10:59-64. Newark, Delaware: International Reading Association.

Cohen, Jacob and Cohen, Patricia. 1975. Applied multiple regression/correlation analysis for the behavioral . sciences. Hillsdale, N. J.: Lawrence Erlbaum Associates.

70

Dale, R. and Chall, J. S. 1948. A formula for predicting readability. Educational Research Bulletin 27:37-54.

Davis, F. B. 1944. Fundamental factors of comprehension in reading. Psychometrika ~' 3:185-197.

1972. Psychometric research on comprehension in reading. Reading Research Quarterly z, 4:628-678.

Dixon, W. J. (Ed.) 1977. BMD Biomedical Computer Programs. Berkeley: University oT"California Press.

Entin, Eileen B. and Klare, George B. 1978. relationships of readability, cloze and scores on a reading comprehension test. Reading Behavior ..!.Q., 4:417-435.

Some inter­mul tip le- choice

Journal of

Fillenbaum, S., Jones, L. V. and Rapoport, A. 1963. The predictability of words and their grammatical classes as a function of rate of deletion from a speech transcript. Journal of Verbal Learning and Verbal Behavior l, 2:186-194.

Flesch, Rudolf F. 1951. How to test readability. New York: Harper.

Fry, Edward. 1968. A readability formula that saves time. Journal of Reading!.!., 7:512-516.

Gallant, Ruth. 1965. Use of cloze tests as measures of readability in the primary grades. In J. Allen Figurel (Ed.) Reading and inquiry. International Reading Association Conference Proceedings 10:286-287. Newark, Delaware: International Reading Association.

Garrett, Henry E. 1965. Testing for Teachers. 2nd ed. New York: American Book Company.

Gilliland, John. 1972. Readability. London: University of London Press.

Harris, David P. 1969. Testing English as a second language. New York: McGraw-Hill.

Hittleman, D. R. 1978. Developmental reading: a psycholinguistic perspective. Chicago: Rana-McNally.

Johnson, John R. achievement.

1979. Oral language and reading ERIC ED 177 636.

Kirk, R. E. 1968. Experimental design. Belmont, Ca: Brooks, Cole.

71

Klare, George R. 1963. The Measurement of readability. Ames: Iowa State University Press.

Landsheere, G. 1972. Application du cloze test de W. L. Taylor a la langue francais. Scientia Paedagogica Experimentalis ~' 2:207-256.

Lorge, Irving. 1959. The Lorge formula for estimating difficulty of reading materials. New York: Columbia University.

MacGinitie, Walter H. 1961. Contextual constraint in English prose paragraphs. Journal of Psychology 51, 1:121-130.

McKenna, Michael C. 1976. Synonymic versus verbatim scoring of the cloze procedure. Journal of Reading 20, 2: 1"41-143.

Meredith, Keith and Vaughn, Joseph. 1978. Stability of cloze scores across varying deletion patterns. In P. David Pearson and Jane Hansen (Eds.) Reading: Disciplined inquir! in process and practice. Twenty­seventh yearbook o the NationaI"l'leading Conference. Clemson, S.C: National Reading Conference:l81-184.

Miller, G. R. and Coleman, E. B. 1967. A set of thirty­six prose passages calibrated for complexity. Journal of Verbal Learning and Verbal Behavior~' 6:851-854.

Miller, G. A. and Friedman, E. A. of mutilated English texts. J:38-55.

1957. The reconstruction Information and Control l,

Nie, Norman H., Hull, C. Hadlai, Jenkins, Jean G., Stein­brenner, Karin, and Bent, Dale H. 1975. SPSS: Statistical package for the social sciences. 2nd ed. New York: McGraw-Hirr:-

Oller, John. 1973. Cloze tests of second language proficiency and what they measure. Language Learning 23, 1:105-118.

Oller, John W., Bowen, J. Donald, Dien, Ton That, and Mason, Victor W. 1972. Cloze tests in English, Thai, and Vietnamese: native and non-native performance. Language Learning 22, 1:1-15.

O'Reilly, Robert P. and Streeter, Ronald E. 1977. Report on the development and validation of a system for measuring literal comprehension in a multiple-choice format: preliminary factor analytic results. Journal of Reading Behavior~' 1:45-69.

72

Osgood, C. E. 1962. The nature and measurement of meaning. Psychological Bulletin !!2_, 3:197-237.

Pickering, Michael. 1977. Some observations on cloze tests. ERIC ED 140 602.

Rankin, Earl F. 1957. An evaluation of the cloze procedure as a technique for measuring reading comprehension. (Unpublished doctoral dissertation) University of Michigan.

Rankin, E. F. 1965. The cloze procedure--a survey of research. In E. L. Thurston and C. E. Hafner (Eds.) The philosophical and sociological bases of reading. Fourteenth yearbook of the National Reading Conference. Milwaukee, Wisconsin. National Reading Conference: 133-150.

Ruddell, R. B. 1964. The effect of oral and written patterns of language structure on reading comprehension. Reading Teacher]&, 4:270-275.

Singer, H. and Rhodes, A. 1976. Learning from text: a review of theories and research at the high school level. Reflections and investigations on reading. Twenty-fifth yearbook()f the National Reading Conference. Clemson, S.C: National Reading Conference:22-51.

Smith-Burke, Margaret, Gingrich, Patricia S., and Eagleeye, Daniel. 1978. Differential effects of prior context, style, and deletion pattern on cloze comprehension. In P. David Pearson and Jane Hansen (Eds.) Reading: Disciplined inquirt in process and practice. Twenty­·seventh yearbook o tlie Nationa~eading Conference. Clemson, S.C: National Reading Conference:l33-137.

Spache, George. 1953. A new readability formula for primary grade reading material. Elementary School Journal .2l_, 3:410-413.

1963. Toward better reading. Champaign, Ill: Garrard Press.

Stubbs, J. B. and Tucker, G. R. 1974. The cloze test as a measure of ESL proficiency for Arab students. Modern Language Journal 58, 6:239-241.

Stuebel, C. Samoa.

1976. Tala o le Vavau: Myths and Legends of Wellington, NeW-Zealand: Reed.

Sullivan, Rita J. 1976. A comparison of results obtained using the cloze procedure with readability levels using the Dale-Chall formula in selected university textbooks. ERIC ED 136 246.

Taylor, Wilson. 1953. Cloze procedure: a new tool for measuring readability. Journalism Quarterly lQ, 3:415-433 .

73

. 1957. Cloze readability scores as indices of ~~_,...in-.-d~iv-idual differences in comprehension and aptitude.

Journal of Applied Psychology~' 1:19-26 .

Tuckman, Bruce W. 1978. 2nd ed. New York:

Conducting education research. Harcourt, Brace Jovanovich, Inc.

Tuiman, J. Jaap. 1972. Speculation on cloze as a search procedure. In Frank P. Greene (Ed.). Investigations relatin~ to mature reading. Twenty-first yearbook of the National Reading Conference. Milwaukee, Wisconsin. National Reading Conference:74-84.

Weaver, Wendell W. 1965. Theoretical aspects of the cloze procedure. In Eric S. Thurston and Lawrence E. Hafner (Eds.). The philosophical and sociological bases of reading. Fourteenth yearbo~of the National Reading Conference. Milwaukee, Wisconsin. National Reading Conference:115-132.

Weintraub, Samuel. 1968. The cloze procedure. Reading Teacher 21, 6:567-571.

Yamada, Jun. 1979. Cloze procedure and the measurement of reading comprehension in disconnected sentences and continuous passages. The Science of Reading 23, 2:33-43.