Evaluation of a hierarchical mapping technique as an aid to prose processing

CONTEMPORARY EDUCATIONAL PSYCHOLOGY 4, 221-237 (1979)

Evaluation of a Hierarchical Mapping Technique as an Aid to Prose Processing

CHARLES D. HOLLEY, DONALD F. DANSEREAU, BARBARA A. MCDONALD, JOHN C. GARLAND, AND

KAREN W. COLLINS Tc~xus Christirrrz University

Seventeen college students were trained on a hierarchical mapping technique designed to facilitate prose processing. This technique required the students to transform te’kff into node (concept)-link (relationship) networks using a set of experimenter-supphed hnks and structures. Following training the students used the mapping strategy in studying a 3000-word passage extracted from a geology textbook. Five days later they were given four types of tests over this article: multiple-choice, short-answer, essay, and a summary-oriented concept cloze. The treatment group’s performance on these tests was compared with a control group (N=21) who used their normal study and test-taking techniques. A Hotelling T2 test indicated that the treatment group significantly outperformed the control group on the dependent measures. Further analyses showed that the major differences between groups were attributable to the concept cloze and essay exams, both of which were designed to assess performance on “main ideas.” This pattern of results suggests that the strategy is valuable in assisting the student in the extraction and retention of main ideas but does not appear to affect the extraction and retention of details. Finally, 2 x 2 factorial analyses of high and low grade point average (GPA) subgroups indicated that the strategy may be more beneficial for low GPA students. Implications of the results are discussed in terms of organization theory and the encoding variability hypothesis.

Dansereau, Long, McDonald, Actkinson, Ellis, Collins, Williams, and Evans (1976) explored the utility of imagery (drawing), paraphrasing, and questioning as techniques for students to use in rerepresenting textbook material. On a delayed (1 week after study) essay test over four IOOO-word passages the order of mean performance from best to worst was: paraphrase, imagery, question-answer, and control students using their own study methods. The performances of the paraphrase and imagery groups were significantly better than those of the control grbup. Studies conducted by other investigators have also indicated positive findings for paraphrasing (Andre & Sola, 1976), imagery/drawing (Kulhavy & Swen- son, 1975; Levin & Divine-Hawkins, 1974; Rasco, Tennyson, & Bout- well, 1973, and self-generated questions (Frase & Schwartz, 1975). In general, research along these lines has used relatively short bodies of prose material (usually 1500 words or less). With this amount of material,

Requests for reprints should be sent to Dr. Donald F. Dansereau, Department of Psychol- ogy, Texas Christian University, Fort Worth, TX 76129.

This research was supported by the Defense Advanced Research Projects Agency, Con- tract MDA 903-76-C-0218.

227 0361-476X/791030227-11$02.00/O Copyright 0 1979 by Academic Press, Inc. All rights of reproducflon in any form reserved.

228 HOLLEY ET AL.

abstraction of an underlying organization may not be as difficult or as critical as it is for longer passages. A number of researchers have suggested that the identification of an organizing framework or schema is necessary for complete understanding, and may aid in the retention of details (e.g., Bransford & Johnson, 1973). Consequently, with longer bodies of material processing strategies that emphasize the identification and utilization of an organizing framework or schema may be more effective.

As they have been employed, the paraphrase, imagery, and questioning techniques appear to have little promise in abstracting and organizing an overall framework. These techniques are focused primarily on the rerep- resentation of small segments of the material (i.e., sentences and para- graphs). A recent study comparing a combined paraphrase/imagery strategy with normal study methods on a 3000-word textbook passage (longer material than typically used in prior experiments) showed no significant differences (Dansereau, McDonald, Collins, Garland, Holley, Diekhoff, & Evans, Note 1). In that same study a planned comparison between performance of a group using a mapping strategy (networking) and performance of a group using their own methods was significant (Holley, Dansereau, Garland, McDonald, & Collins, Note 2); on the portion of the test designed to tap retention of the main ideas the mapping group performed approximately 42% higher than the control group.

The mapping strategy (called networking) used in this study is designed to assist the student in reorganizing passage information based on princi- ples abstracted from relationship-based models of long-term memory (Anderson & Bower, 1973; Bobrow & Winograd, 1977; Qmrlian, 1969; Rumelhart, Lindsay, & Norman, 1972). The student is trained to convert prose into hierarchically organized node-link diagrams (networks) using a set of six experimenter-provided links. The nodes contain paraphrases and images of key ideas and concepts and the links specify the relation- ships between these key ideas (concepts). As regards structural organization, the networking process emphasizes the identification and representation of hierarchies (type/part), chains (lines of reasoning/temporal orderings/causal sequences), and clusters (characteristics/definitions/ analogies). See Fig. 1 for a schematic representation of these three types of structures and their associated links and Fig. 2 for an example of a summary map of a nursing textbook chapter. Application of this technique results in the production of hierarchically structured two- dimensional maps. These maps provide the student with a spatial organization of the information contained in the passage.

Although there is very little direct evidence to support the utility of organizational strategies in the comprehension and retention of naturally occurring prose, there are studies which have demonstrated the impor-

HIERARCHICAL MAPPING AND PROSE PROCESSING 229

Part (of) Link

Type (of)/ Example (of) Link

t 22 private

HIERARCHY STRUCTURES

The content in a lower node is part of the object, process, idea or concept contained in a higher node.

The content in a lower node is a member or example of the class or category of processes, ideas, concepts, or objects contained in a higher node.

Key Words -__

is a part of is a segment of is a portion of

x Words

is a type of is in the category is an example of is a kind of Three procedures

are

leads to Link

Dractice

CHAIN STRUCTURES 9 Words

The object, process, idea, or conceDt in one node leads to leads to or r&ults in the object, process, idea, or concept in another node.

results in causes is a tool of produces

Analogy Link CLUSTER STRUCTURES

The obiect. idea. process, or 9 Words

Characteristic Link

Evidence Link

_ concept in one node is analogous to, similar to, corresponds to, or is like the object, idea, process, or concept in another node.

The object, idea, process, or concept in one node is a trait, aspect, quality, feature, attribute, detail, or characteristic of the object, idea, process, or concept in another node.

Thz object, idea, process, or concept in one node provides evidence, facts, data, support, proof, documentation, confirm- ation for the object, idea, process or concept in another node.

is similar to is analogous to is like corresponds to

x Words

has is characterized by feature is property is trait is aspect is attribute is

3 Words

indicates illustrated by demonstrated by supports documents is proof of confirms

FIG. 1. Link types and structure types.

Dis

cuss

ion

of

Wou

nds

--inc

isio

n (s

harp

cu

tting

in

stru

men

t)

--abr

asio

n (s

crap

ing

or

rubb

ing)

--pun

ctur

e/st

ab

(nai

l, bu

llet)

--lac

erat

ion

(blu

nt

inst

rum

ent)

c

&I na

y oc

cur

in

any

com

bina

tion

blas

ts

and

smal

l bl

ood

vess

els)

gr

ow

alon

g fib

rin

netw

ork

and

grad

- ab

sorb

it

smal

l bl

ood

vess

els

dis-

scar

sh

rinks

FIG

. 2.

Ex

ampl

e of

a n

etw

ork

of a

cha

pter

fro

m

a nu

rsin

g te

xtbo

ok.


tance of organization in a number of related domains. For example, in free recall studies it has been shown that performance varies directly with increases in organizational structure. This occurs when the structure is created by either the experimenter (e.g., D’Agostino, 1969) or the subject (e.g., Mandler, 1968). In experiments using artificial, “name-attribute” prose both input organization (e.g., Friedman & Greitzer, 1972) and the organizational strategies used by the subjects (e.g., Di Vesta, Schultz, & Dangel, 1973; Frase, 1973) have been shown to have an impact on recall. Although pointing out that the results are somewhat inconsistent, Shim- merlik (1978), in her review of the literature on organization theory, has suggested that instructing subjects to reorganize incoming information may lead to better recall than other instructional sets (e.g., Di Vesta ef al., 1973). While such research suggests reorganization of material may im- prove performance, previous studies typically have been limited to word lists and name-attribute prose. The purpose of the present study is to extend this work by providing the subject with a general procedure for reorganizing naturally occurring prose.

Based on prior results (e.g., Dansereau et al., Note 1; Dansereau, Col- lins, McDonald, Diekhoff, Garland, & Holley, Note 3) the expectation for the present study was that networking training would assist students in the assimilation and retention of the main ideas in a 3000-word prose passage in comparison to a control group using their normal techniques. In order to assess this hypothesis four dependent tests were utilized: concept cloze and essay exams to measure comprehension/retention of main ideas, and short-answer and multiple-choice exams to measure comprehension/retention of details.

METHOD

Participants Forty-four students were recruited from a general psychology class. Each student re-

ceived experimental participation credit and a $5 fee. Two intact groups were formed based on scheduling considerations. One of these groups was randomly selected to receive 5.5 hr of networking training (n = 21) while the other set of students served as a no-treatment control group (n = 23). A total of six students failed to complete the dependent measures and consequently were dropped from the experiment (four in the treatment group and two in the no-treatment control group). Mean grade point averages for the networking (n = 17) and control (n = 21) groups were 2.81 and 2.88, respectively. The scores ranged from 1.93 to 3.90 for the former group and from 1.93 to 3.96 for the latter. These data indicate a very slight bias operating to the apparent advantage of the control group.

Procedure The networking group participated in the following sequence of sessions. Session 1: The students received a general introduction to the strategy, training on the

links and structure names, practice on networking sentences, and an overview of networking as a general retrieval strategy.

232 HOLLEY ET AL.

Session 2: The students were introduced to the use of the hierarchical structure in developing a general purpose organizing framework and they were given practice on networking a series of 500- to lOOO-word passages (experimenter-generated networks were provided as feedback during this exercise).

Session 3: The students were first given an additional practice passage exercise and then given the opportunity to practice networking on their regular general psychology textbooks.

Session 4: The students continued to practice on their own materials: they were also provided with a short review lecture on the networking procedure.

Session 5: The students completed a short prestudy questionnaire and spent 1 hr studying a 3000-word passage extracted from an introductory geology textbook (the content of this passage was unrelated to any of the practice materials used in prior sessions). The treatment group students were required to make maps of the material. These maps were collected for subsequent analysis.

Session 6: In this final session, which occurred 5 days after session 5, the students were given 3 min to review the notes they had made during the previous session and then spent a total of 35 min taking multiple-choice (18 questions), cloze (7 questions; students were required to till in important concepts that had been deleted from a paragraph summarizing the article), and short-answer (10 questions) exams over the geology passage. In addition the students were required to summarize the passage in an essay (14 mitt).

The control group participated in two sessions identical to sessions 5 and 6 described above. These students were instructed to use their normal methods in studying and test taking. The decision to use a no-treatment rather than a placebo control group was based on prior research with learning strategy training. Attempts at equating training time by having students practice their own or less effective, competing methods on the training material have led to suppression of mean performance in comparison to “untrained” students using their own techniques (Collins, 1978; Garland, 1977; Long, 1976). Subjective reports from participants in these placebo groups indicate that they do not view the training as meaningful and consequently become frustrated and bored with the tasks. These reactions apparently carry over to the assessment phase leading to the reduction in mean performance (Dan- sereau, Collins, McDonald, Holley, Garland, Diekhoff, & Evans, 1979). It should also be emphasized that the college-age students participating in these experiments have had 12 to 14 years of experience and practice with their own study methods (in particular naturally occurring prose) and can therefore be considered no-treatment controls in name only.

RESULTS AND DISCUSSION

All tests were scored without knowledge of group affiliation. The short-answer, multiple-choice, and cloze tests were scored in accord with predetermined keys. The essay test, which required the students to summarize the article, was scored for completeness and organization of the main ideas by a colleague not otherwise involved with the experiment (criteria for completeness and ideal organization were determined a priori). To assess reliability the essay tests were independently scored by one of the authors. A Pearson product-moment correlation of .86 between the two sets of scores was judged to represent an adequate degree of inter-rater reliability. Raw scores were converted into percentages of the maximum possible on each test.

The data from the four tests were subjected to a principal component factor analysis and the emergent factors were then rotated. The loadings of the two-factor varimax solution are reported in Table 1. The pattern of


TABLE I FACTOR LOADINGS FOR THY FOUR EXAMS AFIER VARIMAX ROTAIION”

Exam Factor I Factor II

“Details” “Main ideas’

Essay Cloze Short answer Multiple choice

.41 .64 .89

.YO .20

.83

” Loadings less than .I7 are not reported.

loadings suggests that the multiple-choice and short-answer tests were highly related to Factor I and that the essay and cloze tests were highly related to Factor II. This provides some validation for the a priori categorization of these tests presented in the introduction. To increase reliability and to facilitate subsequent communication the multiple-choice and short-answer tests were summed to form one dependent measure (labeled “details”) and the cloze and essay tests were summed to form another (labeled “main ideas”).

The notes taken by both groups of subjects during the study session (session 5) were examined by two raters and categorized according to reorganization properties. Specifically, two categories were developed. The first category consisted of those notes which followed the serial pre- sentation of the text material. The second category was comprised of those notes which departed from this linear organization. More refined ratings of reorganization were not feasible due to differences in amount of information present in the notes. Both raters were in total agreement on the categorization of the notes. All of the control subjects’ notes were placed in the first category and all of the treatment subjects’ notes were placed in the latter category.

The means and standard deviations for the networking and control groups on the dependent measures are presented in Table 2. A Hotelling

TABLE 2 MEANS AND STANDARD DEVIA-I-IONS FOR rHE TRY-AIMEN-I- AND CONTROL GROUPS

ON THE DEPENDtNr MEASURES”

Measure

Group “Main ideas” “Details”

Networking .\- 63.21 40.08 (n = 17) su 20.71 10.17

Control s 47.15 39.83 (r1=21) SD 28.07 14.99

‘I Scores are reported in percentages

234 HOLLEY ET AL.

T2 test of the comparison between the two groups on the two dependent measures (“details” and “main ideas”) was significant; T2(2,35) = 2.62, p 4 .05. Univariate tests were conducted in order to isolate the loci of between-group differences. The one-tailed t tests indicated that the networking group significantly outperformed the control group on the “main ideas”-dependent measure; t(36) = 1.91, p d .03. The comparison on the dependent measure to assess details was not significant; t(36) = .06, p s .91.

This pattern of results suggests that networking assists students in ac- quiring and organizing the main ideas, but does not necessarily help in the acquisition of details. The major differences between the groups occurred on the essay and summary cloze tests, both of which assessed retention of the main ideas. No differences occurred on the multiple-choice and short- answer tests which were designed to assess detailed knowledge of the passage material. Since these measures have been shown to be sensitive to treatment effects in prior research (Dansereau, 1978), this lack of sig- nificance further suggests that the differences on the “main ideas” dimension are not caused by placebo factors. The fact that the experimental group scored higher on one but not the other dependent measure argues against a conclusion that this group simply “tried harder.”

In order to determine if the networking treatment differentially im- pacted on students differing in academic achievement the networking and control groups were divided into subgroups of students with high (above median) and low (below median) grade point averages (GPAs; see Table 3). A 2 x 2 factorial analysis of variance (networking/control x high/low

TABLE 3 MEANS AND S.rANDARD DEVIATIONS FOR THE TREATMENT AND CONTROL SUBGROUPS

ON rHE Two DEPENDENT MEASURES” AND GRADE POINT AVERAGE

Measure

Group

Networking Low GPA subgroup (tt =9)

High GPA subgroup (iz=8)

Control Low GPA subgroup (n=ll)

High GPA subgroup (II= IO)

“Main ideas” “Details” GPA

62.53 43.17 2.34 20.34 11.84 .23

63.97 36.61 3.28 21.09 6.26 .35

40.46 34.94 2.45 27.93 13.56 .31

54.51 45.21 3.31 26.32 14.65 .34

‘I Scores are reported in percentages.


GPA) was run on each of the two dependent measures (“main ideas” and “details”) in order to assess the presence of interactions (treatment main effects were not of interest at this point since they had been evaluated in prior analyses). The results of these two-way analyses revealed a significant interaction on the “details” measure; F(1,34) = 4.25, p s .05) while all other possible interactions were nonsignificant. Examination of Table 3 indicates that on this measure networking students with low GPAs substantially outperformed the control students with low GPAs, while the reverse held for students with high GPAs. Although nonsignificant, a different pattern of interaction occurred on the “main ideas” measure. On this measure the two networking groups were equal and both were super- ior to the control group(s); as expected the high GPA control group substantially outperformed the low GPA control group.

One possible explanation of the results is that the high GPA networking students already had effective learning strategies prior to training. Con- sequently, they were probably able to achieve some minor organizing benefits from the new strategy but this effect may have been ameliorated by competition with their typical approaches. Low GPA networkers, on the other hand, apparently not already having effective learning strategies, perhaps found the new technique less interfering with their normal methods, thus leading to improved performance on both dependent measures. Further research will be conducted to shed light on this potential achievement x treatment interaction.

From a theoretical perspective the better performance of the networking group on the “main ideas” factor, coupled with an equivalent performance on the “details” measure, appears to support and expand prior research on reorganization and encoding variability (see Shimmerlik, 1978). The prose passage used in this experiment is sensitive to ecological considerations in terms of its content (explanatory; see Rigney & Munro, 1977) and length (3000 words). While other treatment factors may be operating in addition to reorganization (e.g., increase in mathemagenic activities) the pattern of results on the dependent measures would suggest that the principal effect was attributable to reorganization. Specifically, the treatment group performed better on the “main ideas” dimension. This is consonant with an organization interpretation. Additionally, the fact that the differences between the notes taken by the two groups were based on organizational rather than informational properties also provides convergent support for the foregoing interpretation.

Follow-up research is currently being conducted to determine the long-term effects of the mapping strategy. Specifically, the objective is to determine if the beneficial aspects of the strategy persist in the absence of the conditions present during training. Experiential reports of students exposed to mapping methods, in this experiment as well as other experi-

236 HOLLEY ET AL.

ments (e.g., Dansereau, 1978), suggest that the strategy continues to be viable for those students who persist in applying it outside of the labora- tory situation. Maximum nonlaboratory benefits appear to accrue when the student integrates the mapping strategy with his/her idiosyncratic note-taking method. While these subjective reports are intuitively ap- pealing their degree of correspondence with empirical data (e.g., improved grade point averages) has not, as yet, been established.

REFERENCES ANDERSON, J. R., & BOWER, G. H. Hvmtrfl clssociufirr rnc’rnory. Washington, D.C.:

Winston, 1973. ANDRE, T., & SOLA, J. Imagery, verbatim and paraphrased questions, and retention of

meaningful sentences. Joumul of Educutionul Psyc~holog?;. 1976, 68, 661-669. BOBROW, D. G., & WINOGRAD, T. An overview of KRL, a knowledge representation lan-

guage. Cognitive Scienw. 1977, l(l), 3-46. BRANSFORD, J. D., & JOHNSON, M. K. Considerations of some problems of comprehension.

In W. G. Chase (Ed.), Visual information processing. New York: Academic Press, 1973.

COLLINS, K. W. C‘ontrol (?f‘~j&ti\~e wsponsrs drrring ~crr&rnic. tusks. Unpublished Mas- ter’s thesis, Texas Christian University, 1978.

D’AGNOSTINO, P. R. The blocked-random effect in recall and recognition. JournuI of’ Verbal Leurning und Verbal Brhuvior. 1969, 8, 815-820.

DANSEREALJ, D. F. The development of a learning strategy curriculum. In Harry O’Neil (Ed.), Lcurning strategies. New York: Academic Press, 1978.

DANSER~AU, D. F., COLLINS, K. W., MCDONALD, B. A., HOLLEY, C. D.,GARLAND, J. C., DIEKHOFF, G., & EVANS, S. H. Development and evaluation of a learning strategy training program. Journul of Educufional Psychology, 1979, 71(l), 64-73.

DANSEREAU, D. F., LONG, G. L., MCDONALD, B. A., ACTKINSON, T. R., ELLIS, A. M., COLLINS, K., WILLIAMS, S., & EVANS, S. H. Effective learning strategy training program: Development and assessment. C‘utcrlog of‘ Selwted Documents in Psychology. 1976, 6, 19.

Dl VESTA, F. J., SCHULTZ, C. B., & DANGEL, T. R. Passage organization and imposed learning strategies in comprehension and recall of connected discourse. Memory and Cognirion. 1973, l(4), 471-476.

FRASE, L. T. Integration of written text. Journul of Educutionul Psychology, 1973, 65(2), 252-261.

FRASE, L. T., & SCHWARTZ, B. J. Effect of question production and answering on prose recall. Journal of Educalional Psychology, 1975, 67(5). 628-635.

FRIEDMAN, M. R., & GREITZER, F. L. Organization and study time in learning from reading. Journul oj’Educutionul Psychology, 1972, 63, 609-615.

GARLAND, J. C. The drvelopmc~nt and ussessmcnt of an imugrry bused learning stratrgy program IO improrv the rrrention c!f prose muteriul. Unpublished Master’s thesis, Texas Christian University, 1977.

KULHAVY, R. W., & SWENSON, I. Imagery instructions and the comprehension of text. British Journul of Edwutionul Psychology. 1975, 45, F’t. 2, 47-51.

LEVIN, J. R., & DIVINE-HAWKINS, P. Visual imagery as a prose-learning process. Journal r,j’Reuding Behavior. 1974, 6, 23-30.

LONG, G. L. Thr dcw~lopmrnr und usse.ssment o,f u cognitive proc’ess based learning .strutrgy twining program .for c,nhunc,ing prose comprehension and retention. Unpub- lished doctoral dissertation, Texas Christian University, 1976.


MANDLER, G. Organized recall: Individual functions. Ps~c~honomic~ Scielrc,c,. 1968, 13, 235-236.

QUILLIAN, M. R. The teachable language comprehender. C‘ornnf,rnic,rrtic,~ls of‘rhc Associtr- lion ji)r Computing Mochitzcry. 1969, 12, 4599476.

RASCO, R. W., TENNYSON, R. P., & BOUTWELL, R. C. Imagery instructions and drawings in learning prose. .lou~icl/ ofEdrrccrtio~~cr/ Psyc~holog,v, 1975, 67(5), 188- 192.

RIGNEY, J. W., & MUNRO, A. Ori n~g~ifi~~~ s/rrrfc~gic,sJi~r procc,.\.siug tc’.r/ (Technical Report No. 80). Los Angeles: University of Southern California. Behavioral Technology Laboratories, March, 1977.

RtJMtLHARr, D. E., LINDSAY, P. H., B NORMAN, D. A. A process model for long-term memory. In E. Tulving & W. Donaldson (Eds.), Orgclrrixrtio~ c~f’mc~mr~y. New York: Academic Press, 1972.

SHIMMERLIK, S. M. Organization theory and memory for prose: A review of the literature. Rcvir~t~ ofEduc~/iono/ Rc>cJcrr-ch, 1978, 48(I), 103- 120.

REFERENCE NOTES 1.’ DANSEREAU, D. F., MCDONALD, B. A., COLLINS, K. W., GARLAND, .I., HOLLEY,

C. D., DIEKHOFF, G., & EVANS, S. H. I1c~eloprnc~ir trrid cvdrrtrtio~r oj’crn in~rru~ti~

lruvning sfrrrfe’~~ s.r%teni. Invited address at the Learning Strategies: Measures and Modules Seminar, Carmel, Calif., December, 1977.

2. HOLLEY, C. D., DANSEREAU, D. F., GARLAND, J. C., MCDONALD, B. A., & COLLINS, K. W. An c~\~rlrctrtion oj’t~et~otking us <,,I ir~fbrmr~tirm p,‘oc~c~ssi,rg cipprotrc,h to <.IN.Ts-

YOOVZ pqforrncu?c~c,. Paper presented at the Annual Meeting of the Southwestern Educational Research Association, Austin, Tex., January 20, 1978.

3. DANSEREAU, D. F., COLLINS, K. W., MCDONALD, B. A., DIEKHOFF, G., GARLAND, J. C., & HOLLEY, C. D., I~c~~~rlopmc~nt crtrd ~scsstn~~nt oj’cc cognitively brrscd lecrrn- ing stratc’g.v c~rr~~ic~ulrrm. Paper presented at the Annual Meeting of the American Educational Research Association, Toronto, Canada, March, 1978.