T H E SUSCEPTIBILITY OF T H E QUICK TEST TO INSTRUCTIONAL SETS CHRISTOPHER ORPEN

University of Cape T o m , South Africa

PROBLEM To the present time most of the research into the effects of variation in test

instructions and phrasing of questions on Ss’ responses(‘$ 2 n 5 , has been concerned with personality inventories. Little attention has been paid to the possible effect of these variations on responses to aptitude and achievement tests. Since Ss have been shown to be influenced strongly by general social considerations when they answer tests, psychological instruments that are introduced differently (by means of unlike instructions) should, if they inform the Ss that the instruments measure differ- entially socially desirable traits, motivate the Ss to perform differently as well. There seems no reason why this prediction should not hold for intelligence tests as well as personality tests. The present study was designed to assess whether different instructions would influence Ss’ scores on standard intelligence tests.

METHOD Procedure. Forms 1, 2 and 3 of the Ammons Quick Test (QT), which has been

shown to possess a high degree of construct validity among Coloured Ss in South Africa(5), was given to a sample of Coloured high school pupils under three sets of instructions. In the first group, the QT was given as a test of “intelligence,” in the second group as a test of “achievement,” and in the third group under standard in- structions. The three sets of instructions were those used by Yamamoto and Diz- ney‘g), with only minor changes necessary to make them suitable for Coloured Ss in South Africa (e.g., “good grades in school” was replaced by “good marks in school,” “to go to college” by “to go to university”).

Although the Coloured minority group are segregated from the ruling white group and are subjected to a number of discriminatory laws that effectively keep them near the bottom of the social 5 , they do not have close ties with African tribal cultures and are entirely western in their values and living 6 ) .

Hence, i t is reasonable to suppose that, like American schoolchildren, the Ss of the present study would place a high social premium and prestige on intelligence and achievement.

According to the reasoning given earlier, it therefore was predicted that the Ss given the QT under “intelligence” and “achievement” instructions would score more highly than those given the QT under its L‘normal’) or standard instructions (which did not mention the terms “intelligent” or “achievement”). Given that differential prestige is a product of social learning, as claimed by Yamamoto and Dizney‘g), i t also was predicted that the difference in mean scores between the three groups would be greater in the case of older pupils than younger pupils.

Subjects. The Ss were 126 male pupils in their final or sixth year of high school (average age 18.3 years) and 135 male pupils in their second year of high school (average age 15.1 years). The Ss a t both school levels were assigned randomly to one of three groups, given respectively an “intelligence,” an “achievement” and a “standard” set of instructions. The QT was administered to each of the groups by the same teacher-psychologist during regular school periods.

RESULTS AND DISCUSSION The mean scores of the sixth-year pupils on the combined QT were 45. 21 (SD =

4.10) under the intelligence instructions, 44.16 (SD = 3.21) under the achievement instructions and 38.36 (SD = 4.01) under the standard instructions. The mean cores of the second-year pupils on the combined QT were 46.12 (SD = 4.11) under the intelligence instructions, 46.22 (SD = 3.56) under the achievement instructions

THE SUSCEPTIBILITY OF T H E QUICK TEST TO INSTRUCTIONAL SETS 509

and 40.41 (SD = 3.71) under the standard instructions. A two-way analysis of variance that used the method of unweighted means was applied to these scores. This revealed that the effect of instructional sets (different instructions) was highly significant ( F = 14.6; df 2, 3; p < .Ol), but that the effect of age IeveI (years a t high school) was insignificant ( F = 3.2; df 1, 5 ; p > .05). A Scheff6 multiple comparison test applied to the mean scores indicated that at both age levels the intelligence group did not obtain a significantly higher score than the achievement group ( p > .05), but that both these groups obtained significantly higher scores than the stand- ard group ( p < .Ol). The effect of the interaction between age and instructional &t was insignificant (F = 4.21; df 2, 5 ; p > .05).

The results offer partial support for the predictions. The Ss reacted differently to the standard instructional set and to the other two instructional sets of achieve- ment and intelligence. However, they did not react differently to the achievement and intelligence instructional sets. These results suggest that the concepts of intelli- gence and achievement enjoy greater prestige than the central concept used in the standard instructions (L‘a kind of picture game”) and hence probably motivate the Ss to better performance than the latter. The results offer crosscultural support for the general findings reported by Yamamoto and Dizney(9) insofar as they indicate that prestigious terms like “intelligence” and “achievement” can influence how well the Ss perform, The failure to find a significant effect due to age (or school level) may be due to the fact that the prestige of the terms “intelligent” and “achievement” is already well established by the second year of high school, even among these lower- class Coloured Ss.

The finding that the use of terms such as “intelligence” and “achievement” makes a difference to the Ss’ performance raises a serious doubt as to the com- parability of different aptitude tests, since some explicitly use these terms in their titles and instructions, whereas others do not mention these terms at all. The results underlie the importance of employing neutral instructions in aptitude testing and confirm that Ss’ responses are sensitive to subtle social pressures that emanate from the context within which the test is administered.

SUMMARY The Ammons Quick Test of intelligence was given to 126 Coloured males in

their final year of high school and 135 males in their second year of high school in South Africa. In each grade Ss were assigned randomly to three groups that took the test under intelligence, achievement and standard instructional sets. An analysis of variance revealed a significant main effect for instructional set, but not for grade level. The results are taken as an indication of the susceptibility of the Quick test to instructional sets.

REFERENCES 1. EDwAaDs A. L. The Social Desirability Variable in Personalily Assessment and Research. New

2. HILLS, J. R. The influence of instructions on personality scores. J . counsel. Psychol., 1961, 8,

3. MARQUARD, L. T b Peopbs and Policies oj South Aj7ica. London: Oxford University Press, 1962. 4. ORPI.:N, C. Authoritarianism within an “authoritarian” culture. Unpublished doctoral thesis,

5 . ORPEN, C. Social desirability in the Psychological Screening Inventory. J . clin. Psychol., 1971,

6. ORPEN, C. Internal-external control and perceived discrimination in a South African minority

7. ORPEN, C. The Quick Test with Coloureds in South Africa. Psychol. Rep., 1973, 32, 897-898. 8. VAN DER MERWE, H. W. Social stratification in the Cape Coloured community. Sociol. SOC.

9. YAMAMOTO, K. and DIZNEY, H. F. Effects of three sets of test instructions on scores on an

York: ~ r y d e n Press, 1957.

43-48.

University of Cape Town, 1970.

27, 463-465.

group. Sociol. SOC. Res., 1971, 66, 4448.

Res., 1962, 46, 303-311.

intelligence scale. Edu. psvchol. Meas., 1965, 26, 87-94.