Design of an Instrument to Measure Teacher's Attitudes Toward Experimenting

Design of an Instrument to Measure Teacher’s AttitudesToward Experimenting

Larry M. HoveyCollege of EducationTexas Tech UniversityLubbock, Texas 79409

INTRODUCTION

Attitudinal changes are often mentioned as an objective of "newer"science curricular projects. Yet such changes are seldom adequatelyassessed, often because of the lack of valid instruments.Lawrence F. Lowery has developed an instrument, based on

motivational research techniques, which measures underlying attitudesbetter than past instruments. Lowery’s series of tests, Projectjve Testsof Attitudes (PTOA), were designed for use with the elementary schoolstudent. This article describes a modification of the PTOA for adultusage. The modified version was used in a study, conducted at theUniversity of California, Berkeley (3).

MEASUREMENT OF SCIENCE RELATED ATTITUDES

The testing instruments that have attempted to measure sciencerelated attitudes can be divided into four groupings�Thurston Scales,Likert Scales, Semantic Differentials, and Projective Techniques.The first group of instruments are based upon Thurston’s work

(11). With this type of technique the respondent is asked to checkeach statement with which he agrees. A group of judges has preassigneda rating for each statement�the more positive the statement the higherthe rating. The attitudinal score is the sum of the rated values forall the checked statements.The second group of instruments are based upon the Likert Scale,

which is a modification of Thurston’s. With such instruments therespondent is asked to mark each statement along a scale rangingfrom "strongly agree" to "strongly disagree." The subject’s attitudemay be inferred by the degree of his agreement with a series offavorable and unfavorable statements about science, scientists, andthe like.Semantic differentials are based upon Osgood’s work (9). Klopfer

has been one proponent of the semantic differential in science educationand has been instrumental in its development (4),When using a semantic differential instrument, the respondent ranks

a given concept on a series of bipolar and adjectival scales. Eachof these rankings may be thought to represent some component of

175

176 School Science and Mathematics

the respondent’s image of that concept.The most recently developed devices to assess attitudes in science

education are projective techniques. Word associations, sentencecompletions, and thematic apperception pictures are projective tech-niques which have been used for some time in other fields, notablyclinical psychology.Lowery, feeling that existing devices did not reveal underlying

attitudes, was the first to apply projective techniques to scienceeducation. Describing such instruments, Lowery states: ’They havebeen designed to delve beneath the surface of superficial answersin an attempt to discover subconscious motives behind actions�mo-tives which are not revealed through ordinary questionnaires" (6,p. 13).There are only a few studies in science education which have used

projective techniques. Lowery’s research is most notable (8), butPerrodin also used a projective instrument�20 sentence fragmentsintended to reveal the feelings toward science of fourth, sixth, andeighth grade students (10).

RATIONALE FOR USE OF A PROJECTIVE TECHNIQUE

The success of an attitudinal questionnaire is dependent upon thefrankness of the respondent. This frankness, however, is often inhibitedby fear, desire to build one’s ego, misunderstanding, social taboos,and similar obstructions which can lead to a false impression of attitude.One way to overcome these limitations is to use an approach whichis less direct than a questionnaire. A projective approach is less direct,less obvious to the subject and thus helpful in overcoming such barriersas the following:

a. Awareness: People are not always consciously aware of their feelings and attitudes,and thus cannot be accurately assessed by a questionnaire which calls for adirect conscious response.

b. Irrationality: Society places a high premium on rational logical behavior. Question-naires often reflect the "rational" rather than the underlying attitudes of therespondent.

c. Self-incrimination: Much is said in a roundabout way in order to avoid a threatto one’s ego.

d. Politeness: Many subjects say things just to please the researcher. This mightgive some data the researcher is looking for, but it does not get to the subject’strue attitudes.

The above barriers cannot be completely overcome, however,projective techniques do a more effective job than any other instrumentcurrently in use in science education. "In any case, the capacity

An Instrument to Measure Teacher’s Attitude 177

of these devices (projective instruments) to intercept the private,covert, latent, unconscious components of the individual personalitydistinguishes them from most other psychological instruments" (5,p. 42).

There are several advantages in using a projective instrument.Foremost is the fact that if designed properly, projective tests canbypass the "outer defenses" of a subject and assess underlyingattitudes.The subject is never completely aware of the intent of the measuring

device and so it is never completely possible to give the "right"answer�the logical, non-self-incriminating response. Instead, throughassociation and fantasy responses to ambiguous stimuli, superficialfeelings are bypassed and truer underlying attitudes are measured.

Also, because attitudes are not stable, by increasing the numberof ways of assessing them, as with a multiple projective technique,the possibility of getting a truer, more accurate assessment of attitudeis increased.

DEVELOPMENT OF THE PROJECTIVE TESTS OF ATTITUDES�ADULT FORM(PTOA-AF)

The development of the original Projective Tests of Attitudes (PTOA)is described by Lowery (7).The original PTOA contains three themes, "science," "experi-

menting," and the "scientist." Each theme is repeated in a wordassociation, thematic apperception, and sentence completion sectionof the measuring device.For use with adults (teachers), the instruments had to undergo several

modifications. The modifications occurred in several steps.First, this researcher, working with another doctoral study (1), used

the PTOA in its original form with approximately 40 elementarystudents.

Secondly, discussions between Dr. Lowery, Dr. Robert Karplus,Director of the Science Curriculum Improvement Study, and thisinvestigator led to modifications of the PTOA for usage with adults.The instrument was used with nine friends and relatives, then was

tested on 15 elementary school teachers in a pilot study.Finally, the instrument was used in a study of 88 elementary school

teachers who had experience and/or training with the Science Curricu-lum Improvement Study.

After each of the above steps, modifications were made andincorporated in the Projective Tests of Attitudes�Adult Form(PTOA-AF).


RELIABILITY

Reliability may be considered to be the consistency of a testinstrument to rank individuals in the same order every time theyare measured by that instrument. To establish the reliability of thePTOA-AF, 22 teachers not having experience with SCIS, and notpart of the major study, were selected. Four weeks after the initialinterview, these 22 teachers were again interviewed by using theidentical PTOA-AF.The test-retest responses were analyzed in two ways (statistically

and descriptively) by a team of judges�a science educator, anelementary school teacher, and a scientist. For the statistical analysis,the judges, working independently, scored the responses as eitherpositive, neutral, or negative.Using KendalFs Coefficient of Concordance (2, p. 656-658) the

scoring reliability between the three judges was found to be .89�anacceptable level.

After the judges had scored the responses, the subjects were ranked,and a reliability index was calculated by using the Spearman’s RankCorrelation Coefficient.

Following Lowery (6), an acceptable reliability was pre-establishedat .80. The reliability for this study was found to be .90, which waswell above the pre-established limit.A reliability index for the descriptive analysis was established by

categorizing responses and comparing pre and post categories ofresponses for each test of the PTOA-AF.

Categories were established by this investigator examining the bulkdata concerning the "experimenting" theme, and placing the responsesin groupings which seemed to contain common ideas. Such categoriesas "Process of Experimentation," "Materials Used in Experimenting,""Fields of Science," "Reading Aspects," "Relationship to School,"etc. were established.When the word association test responses were placed into catego-

ries, it was found that six of the 22 subjects (27%) gave three responseswhich fell into the identical categories on both tests. (E.g.: for subject"I," his responses on the pretest�"categorizing," "finding," and"discovering"�were all placed in the "Process of Experimentation"category. Likewise, his responses on the posttest�"seeing," "dis-covering," and "observing"�were all classified the same way.)

Six other subjects (27%) had two of their three responses placedin the same categories. Eight subjects (36%) had identical placementsfor one of their three responses. Only two teachers (9%) had nosimilar responses on the pre and post tests.The pre-post categorizing of responses from the thematic appercep-


tion test indicated complete agreement between the tests for 13 ofthe 22 subjects (59%).When the pro and post verbatim responses for the neutral sentence

on the sentence completion test were compared, it was found thatthere was complete agreement between the tests for 12 of the 22subjects (55%).Although the established of a reliability index in this manner is

not as objective as one might wish, 54% of the sample (27% + 27%)on the word association test, 59% of the sample on the thematicapperception test, and 55% of the sample on the sentence completiontest indicate a very strong case for reliability of the PTOA-AF.

VALIDITY

A test instrument is said to be valid if it measures what it is designedto measure.

Validity is usually established in one of two ways�having expertsjudge the instrument’s content to be representative of what is tobe measured, or through comparisons with other measuring devicesthat have been designated as measuring the criteria variable.The usual methods of establishing validity are not appropriate with

projective techniques. First, it is difficult to designate an expert ofprojective techniques, and is almost impossible to establish an objectivehuman standard with which an expert can compare the test instrument.Secondly, part of the original rationale for using a projective devicefor this study was the assumption that previously established moredirect instruments do not adequately measure attitudes.

Thus, neither judges nor other instruments can be used to validatethe projective technique. What can be done, however, is to lookat internal validity. The instrument used for this study has three distinctsections, word association, thematic apperception, and sentencecompletion, all of which have been designed to assess the subject’sattitude toward "experimenting." If the instrument is to be regardedas valid the three tests should reflect similar attitudes for a givenindividual. By comparing the scores on the three sections of the PTOAfor a quarter of this study’s sample, some objective index of validitycan be established.

After alphabetizing the subjects, every fourth set of responses wasselected for establishing a validity index.Seven subjects (32%) had complete agreement between scores for

all three tests. The scores for two of the tests were the same for13 of the 22 subjects (59%). Only two subjects (9%) were in completedisagreement on all three tests.


The scores for 20 of the 22 subjects (91%) were the same or nearlythe same for all three tests. This degree of agreement of scores betweentests suggests that this modified version of the PTOA has a stronginternal validity.

SUMMARYTest instruments used in past studies in science education have

been criticized for superficially measuring attitudes. In an attemptto assess underlying attitudes this study used an indirect projectivetechnique�a modification of Lowery’s Projective Test of Attitudes(PTOA). This instrument consists of three sections, a word associationtest, a sentence completion test, and a thematic apperception test�each designed to assess the subject’s attitude toward a process ofscience�"experimenting."The preliminary concern of this study was to see whether or riot

the PTOA could be adapted for usage with adults. The adult formof the PTOA (PTOA-AF), as used with this study proved to be valuable,as well as provided insight into possible modifications for future usage.The use of the PTOA-AF creates situations somewhat different

from the more common attitudinal testing procedures. First, it isadministered through individual interviews.

Secondly, the data from the PTOA-AF can be analyzed statisticallyand/or descriptively. A greater emphasis is placed on the descriptiveanalysis than with most other instruments.

Finally, the nature of the instrument and the method of analysisrequires somewhat different ways to establish reliability and validityindices. The ways used in this study suggest that strong reliabilityand validity indices have been established for the PTOA-AF.Lowery made a considerable contribution to the assessment of

attitudes in science education by developing the Projective Tests ofAttitudes. The PTOA-AF has indicated some versatility in the originalinstrument and has pointed out one direction for expanding its usage.Future research will hopefully take advantage of these gains and willcontinue to use and expand the PTOA in an attempt to furtherknowledge of attitudes in education.

BIBLIOGRAPHY

1. DEADY, GENE M. ’The Effects of an Increased Time Allotment on Student Attitudesand Achievement in Science." Unpublished Ed.D. dissertation. Berkeley: Universityof California, 1969.

2. HAYS, WILLIAM L. Statistics for Psychologists. New York: Holt, Rinehart andWinston, 1963.

3. HOVEY, LARRY M. "Measuring Science Curriculum Improvement Study Teachers’Attitudinal Changes Toward Science," Unpublished Ph.D. Dissertation. Berkeley:University of California, 1970.


4. KLOPFER, LEOPOLD E. Unpublished manuscript on evaluation in science, 1970.5. LINDZEY, GARDNER, Protective Techniques and Cross Cultural Research. New York:

Appleton-Century-Crofts, Inc., 1961.6. LOWERY, LAWRENCE F. "An Experimental Investigation into the Attitudes of Fifth

Grade Students Toward Science." Unpublished Ed.D. dissertation. Berkeley:. University of California, 1965.

7. LOWERY, LAWRENCE F. "Development of an Attitude Measuring Instrument forScience Education," School Science and Mathematics, 66(5):494-502, 1966.

8. LOWERY, LAWRENCE F. "An Experimental Investigation into the Attitudes of FifthGrade Students Toward Science." School Science and Mathematics, 67(6):569-579,1967.

9. OSGOOD, C. E., G. J. Suci, and P. H. TANNENBAUM. The Measurement of Meaning.Urbana, Illinois: University of Illinois, 1957.

10. PERRODIN ALEX F. "Children’s Attitudes Towards Elementary School Science,"Science Education, 50:214-218, 1966.

11. THURSTON, L. L. "The Measurement of Social Attitudes." Journal of Abnormaland Social Psychology, 26:249-269, 1931.

APPENDIX

PROJECTIVE TESTSOF ATTITUDES

(REVISED ADULT FORM)

General Instructions for Test Administration

"This is not a test, but rather a technique to gain some insight into the instructionalprogram of the local elementary schools. I will show and tell you various things thatwill call for a response from you. There are no right or wrong answers and no nameswill be mentioned in the study."

Word Association Test

Directions: "For the first portion of the interview I will give you a word, andI would like you to respond with the first three words that come to your mind. Forexample, if I said ’singing," what three words come to your mind?" (Encouragethree responses, and use a second word, "running," if you feel it is necessary.)

Stimulus Words:LOOKINGTALKINGREADINGTEACHINGEXPERIMENTINGDEMONSTRATINGDISCUSSINGLEARNINGCOMPUTINGWRITINGLECTURING

Thematic Apperception Test

Directions: "I am now going to show you a series of pictures, and I will ask youquestions about them. Again there are no right or wrong answers. Just freely useyou imagination, and respond as rapidly as possible.


(If the subject is female use the set of illustrations* depicting females. If the subjectsis male use the set of illustrations depicitng males.)

Theme: Reading (Neutral) Picture 1 (Male)Subject: Male

This man is reading a story to some children.1. What is the story about?2. Why did the man select such a story?3. Do the children like the story?

Why?4. Does the man enjoy reading the story to the children?

Why?

A negative picture (Picture 4 is used for every theme) is shown after the neutralillustration.

Theme: Reading (Negative) Picture 4 (Male)Subject: Male

This man does not feel his reading program is successful. Why does he feel thisway?

Theme: Reading (Neutral) Picture 1 (Female)Subject: Female

This woman is reading a story to some children.1. What is the story about?2. Why did the woman select such a story?3. Do the children like the story?

Why?4. Does the woman enjoy reading the story to the children?

Why?

Theme: Reading (Negative) Picture 4 (Female)Subject: Female

This woman does not feel her reading program is successful. Why does she feelthis way?

Theme: Experimenting (Neutral) Picture 2 (Male)Subject: Male

This man is about to gather some things from the storeroom for a science class.1. How will these materials be used in the classroom?

(If the response does not indicate whether the teacher or the students will generallymanipulate the materials, then ask, "Who will use these materials?")

2. What will be the outcome of this lesson?(If the response does not indicate whether the outcome will be successful orunsuccessful, then ask, "Will the outcome of the class be successful or unsuccessful?"

3. How does the man feel about doing this type of science lesson?

^Note: Illustrations are not included here, but copies are available from the author.


Theme: Experimenting (Negative) Picture 4 (Male)Subject: Male

This man does not feel his science program is successful. Why does he feel thisway?

Theme: Experimenting (Neutral) Picture 2 (Female)Subject: Female

This woman is about to gather some things from the storeroom for a science class.1. How will these materials be used in the classroom?

(If the response does not indicate whether the teacher or the students will generallymanipulate the materials, then ask, "Who will use these materials?")

2. What will be the outcome of this lesson?(If the response does not indicate whether the outcome will be successful orunsuccessful, then ask, "Will the outcome of the class be successful or unsuccessful?"

3. How does the woman feel about doing this type of science lesson?

Theme: Experimenting (Negative) Picture 4 (Female)Subject: Female

This woman does not feel her science program is successful. Why does she feelthis way?

Theme: Arithmetic (Neutral) Picture 3 (Male)Subject: Male

This man is planning tomorrow’s arithmetic lesson.1. What will the lesson be?2. What will be the outcome of the lesson?3. How does he feel about doing this type of arithmetic lesson?

Theme: Arithmetic (Negative) Picture 4 (Male)Subject: Male

This man does not feel his arithmetic program is successful. Why does he feel thisway?

Theme: Arithmetic (Neutral) Picture 3 (Female)Subject: Female

This woman is planning tomorrow’s arithmetic lesson.1. What will the lesson be?2. What will be the outcome of the lesson?3. How does she feel about doing this type of arithmetic lesson?

Theme: Arithmetic (Negative) Picture 4 (Female)Subject: Female

This woman does not feel her arithmetic program is successful. Why does she feelthis way?


Sentence Completion Test

Directions: "I am now going to give you parts of sentences, and I would like youto finish the sentences as fast as you can. For example, if I said, ’Bill went , . .’how would you complete the sentence?" (Encourage a response and use another samplesentence, "John likes to . . ." only if you feel it is necessary.)

Partial Sentences:1. Reading is ...2. Most people like to read because ...

3. Other people do not like to read because . . .4. Experimenting in science is ...5. Some people find experimenting enjoyable because . . .

6. Other people usually do not like to experiment because . . .7. The study of arithmetic is ...8. Most people like arithmetic because . . .9. One thing that puts some people against arithmetic is ...

Letter to the Editor

Mr. George G. Mallinson, EditorSchool Science & MathematicsWestern Michigan Univ.Kalamazoo, Mich. 49001

Dear Mr. Mallinson:

I think many readers of your publication would be interested to knowthat there is, in the May 1974 issue of the American Journal of Physics,a beautifully clear and concise description of laying out and building a sundial, which is (to me, at least) very interesting and useful as a student projectin astronomy.

The mathematical development is given in condensed form, and is easilyunderstood by anyone who knows a little elementary spherical trigonometry.

It is, however, not necessary to understand the mathematical developmentin order to build a sun dial. In that case, one merely takes the formulafor calculating shadow angles on faith, and uses ordinary trig tables to dothe calculations.

The title of the article is "The Horizontal Sundial," and the author isProf. Herman Eriichson, of Staten Island (N.Y.) Community College.

Sincerely

IA/L<I/-^/W. K. ViertelProfessor Emeritus of ScienceState Univ. of N.Y.. Canton, N.Y.

Documents

Design of an Instrument to Measure Teacher's Attitudes Toward Experimenting