Download pdf - Adaptive Instructional Strategies for Teaching Rules in Mathematics

Adaptive Instructional Strategies for Teaching Rules in Mathematics Steven M, Ross Ernest A, Rakow

Steven M. Ross and Ernest A. Rakow are associate professors in the Department of Foundations of Educa- tion at Memphis State University, Memphis, TN 38152. The authors acknowledge the contribution of Duncan N. Hansen, who directed the research project involving performance of the present studies and par- ticipated in the writing of earlier drafts. Requests for reprints should be sent to Steven M. Ross, Foun- dations of Education, Memphis State University.

These researchers explored ways of adapt ing instruction to individual students. In one experiment, as an incentive for focusing on more difficult learning tasks, they assigned more points to tasks predicted to be difficult for the student. In a second experiment, they varied both incentives and number of examples offered to demonstrate the tasks, again on the basis of pretest scores. Included is a discussion of how the f ind ings- that both techniques are bene f i c i a l - can be appl ied in classroom instruction.

ECTJ, VOL. 30, NO. 2, PAGES 67-74 ISSN 0148-5806

Research on individualized instruction, as exemplified by studies of apt i tude- treatment interactions (ATI), has tended to focus on group variables in attempting to adapt instruction to individuals (for re- views, see Berliner & Cahen, 1973; Cron- bach & Snow, 1977). Since there usually is considerable variation within groups, the potential of such orientations for optimizing learning for individuals appears limited. In addition, the majority of previous studies have based adaptations primarily on trait (pretask) variables, representing general- ized aptitudes or predispositions for learning, to the exclusion of state variables, representing students' needs and interests at the time of instruction (Tennyson & Rothen, 1977; Tobias, 1976). Consequently, these applications have achieved relatively limited empirical success and offer the prac- titioner few suggestions for individualizing instruction in applied contexts.

The objective of the research described in this article was to test an adaptation approach designed to extend the ATI concept by applying instructional variations to individuals as opposed to groups. The context for the experimentation was an introductory mathematics lesson adapted from the beginning module in an undergraduate statistics course using the Keller (1968) PSI Plan (see Kulik, Kulik, & Cohen, 1979; Rob- in, 1976). Compared with lecture-based instruction, PSI (personalized system of instruction) represents a condition of high in-

68 ECTJ SUMMER 1982

structional support by emphasizing the availability of supplementary study material and teacher assistance for those who need them. The problem, however, is that individuals may lack the sophistication or motivation to use these resources appropri- ately on their own, given that learning objectives and study materials are stan- dardized for an entire class, the only adaptation has to do with the pace at which students complete the lesson. With regard to the "sufficiency" of learning, high achievers may tend to receive more support than they need, and low achievers too little.

On the basis of this rationale, we under- took a preliminary study to explore ways of adapting the quality of instructional support given individuals (Hansen, Ross, & Rakow, 1977). Subjects learned a series of 10 math rules under full (individual) adaptation, partial (group-based) adaptation, and several forms of standard instruction. For the full adaptive strategy, scores on a battery of entry measures provided the basis for as- signing subjects to different learning style groups via cluster analysis and then deriv- ing predictions of individual performance, within groups, via multiple regression techniques. Prescriptions specifying the number of supporting examples to be presented for each rule were matched to the predicted scores and refined during instruction on the basis of on-task performance. For the partial adaptation strategy, prescriptions were matched to group predictions only. Results on a cumulative posttest favored full adaptation over partial adaptation and both adaptive treatments over standard instruction.

In the preliminary study, the full adaptive approach provided individualized prescriptions of instructional support; it increased practice opportunities on rules predicted to be most difficult to learn and economized on rules predicted to be easiest. A possible limitation of this strategy is that it provides no control over how students actually use the prescriptions; they may linger on the "restricted" prescriptions associated with easy rules and give only surface attention to "extended" prescriptions for difficult rules. Thus it would seem advantageous to invoke some type of external prompting, such as incentives, to direct students to use the ma-

terial in the manner intended. In theorizing about the motivational influences of classroom incentives, Atkinson and Wickens (1971; also see Kribs, 1974) emphasized the advantages of varying the level of rewards within-task. Such a weighting of different parts of a lesson unevenly would provide information about which parts are more important than others. From an adaptive standpoint, a logical strategy would be to vary conditions so as to offer the highest rewards on materials the individual student is predicted to find the most difficult and the lowest rewards on those he or she is predicted to find the easiest. The expected outcome would be to modify study activity so as to direct the learning emphasis to weak- nesses rather than to strengths. This idea was examined in the present experiments by determining whether adaptive variations in incentives, used separately from and in combination with adaptation of instructional support, had a more positive effect on performance than weighting all lesson objectives the same. The critical feature of the adaptive treatments examined involved their formulation of prescriptions systematically tailored to the inferred needs of individuals.

EXPERIMENT I: ADAPTATION OF INCENTIVES

The purpose of the first experiment was to determine the effectiveness for learning of within-task, individualized adaptation of incentives. The main hypothesis was that, as a function of modifying study behavior in a manner consistent with individual needs, adaptive incentives would enhance overall performance relative to standard incentives. It was also predicted that a highly differentiated adaptive incentive schedule would provide subjects with more information and direction, and thus be more facilita- tive, than would one having less differentiation between values. Comparisons were made between the following five treatment variations: (a) an adaptive strategy that assigned five different levels of incentives across rules (distributive adaptive), (b) an adaptive strategy that assigned three different levels of incentives across rules (modal adaptive), (c)a nonadaptive strategy that

ADAPTIVE INSTRUCTIONAL STRATEGIES 69

assigned the five-level incentive distributions derived for subjects in treatment a to subjects possessing different learning characteristics (distributive nonadaptive), (d) a nonadaptive strategy misappropria- ting the three-level incentives distribution (modal nonadaptive), and (e)a control strategy that assigned a standard incentive level across all rules (standard).

Method

The procedure used to vary incentives in the adaptive treatments was a modification of the one developed by Hansen, Ross, and Rakow and applied in the experimentation reviewed earlier (see Hansen et al., 1977, for a complete description of the methods and statistical analyses). To provide necessary background for understanding the present treatment manipulations, some highlights of that preliminary work are reviewed below.

Adaptive modeling procedure. The adaptive model developed by Hansen et al. was structured to use multiple predictors for two purposes: (a) to classify students into independent-learning-style groups, thus allowing for group-based adaptations, and (b) to generate estimates of individual performances, thus allowing for instructional adaptation within groups. Predictors were selected on the basis of the degree of their relation to criterion performance on the task in question (i.e., 10 instructional rules covering different mathematical operations). Based on a validation sample of 315 subjects, the following predictor set was found to yield the highest multiple correla- tion (total posttest r = .776): locus of control, trait anxiety, math reading comprehension, and a pretest on the material to be learned.

The selected predictor set was administered to students and the resultant scores were subjected to cluster analysis following the procedure of Ward (1963) and Ward and Hook (1963). In analyses conducted on three independent norming samples, similar results, indicating four separate group- ings, were obtained. One group was high in aptitude and ability, and another was low in aptitude and ability, but both were close to the overall means for locus of control and

trait anxiety. The two other groups were about average in aptitude and ability, but one was external in locus of control and high in anxiety, while the other showed the opposite personality characteristics.

Discriminant analysis was used to derive discriminant functions for classifying new subjects into these four groups. In the validation run 91.6% of the norming sample were correctly classified. The final step was generating different prediction equations within each group. This was done by re- gressing posttest performance on the four predictor measures. These regressions were done separately for each rule within each of the groups; thus we produced a unique set of multiple regression equations for each group into which the pretask scores of new students could be entered to predict their performances on each rule.

Participants and design. Participants in the study were 120 undergraduate students. They were administered the test battery, assigned to cluster groups, and randomly assigned to five treatments involving different distributions of incentives. In the distributive-adaptive (five levels) treatment, the 10 rules were rank-ordered on the basis of the individual subject's predicted performance and assigned incentive values such that 0 points each were offered on the two rules having the highest predicted scores, 5 points were offered on the two rules having the next highest predictions, and 10, 15, and 20 points each were offered, respectively, for the remaining three pairs. In the modal-adaptive (three levels) treatment, the same strategy was employed, except that the incentive values applied were 15 points each for the three lowest rules, 10 points each for the four next lowest, and 5 points each for the three highest. In the d is t r ibut ive-nonadapt ive and modal- nonadapt ive treatments, subjects were paired across cluster groups with counterparts from the corresponding adaptive condition (distributive or modal) and administered the identical incentive distributions prepared for those counterparts. In the control treatment, a standard incentive of 10 points was assigned to each of the 10 rules. The design implied a 4 (cluster group) x 5 (treatment) factorial analysis of variance.

70 ECTJ SUMMER 1982

TABLE 1 Nonadjusted Treatment Means and Standard Deviations

Learning Pretest Posttest Gain Time Test Time

Treatment X S,D. Y( S.D. X S.D. ~ S.D. Y( S.D.

Standard 20.95 6.19 23.95 6.48 3.00 2.98 76.80 23.13 27.00 5.59 Modal

Adaptive 20.10 7.99 26.05 8.08 5.95 3.77 98.75 39.00 27.25 10.44 Distributive

Adaptive 22.85 6.83 31.45 4.62 8.60 3.44 79.55 17.77 25.50 8.19 Modal

Nonadaptive 19.00 9.87 22.25 9.47 3.25 8.76 102.10 26.90 24.95 6.25 Distributive

Nonadaptive 17.40 9.63 21.85 9.35 4.45 4.46 91.55 23.42 30.40 10.46

Materials. The instructional materials were the same as those used in the Hansen et al. (1977) experiments. The content consisted of mathematical rules generally taught in introductory algebra and statistics (e.g., fractions, summation, and factorials). Each unit began with a brief rule definition and was followed by six supporting examples. The last section of each unit consisted of an immediate four-item unit posttest.

Lesson achievement was assessed via a 40-item cumulative posttest containing 4 items per rule. The posttest was designed as a parallel form of the pretest used for entry classification.

Procedure. The incentive distributions adminis tered to subjects in the adapt ive treatments were derived using the multiple regression model described earlier. Stu- dents' predicted rule scores were obtained, rules were rank-ordered on the basis of those scores, and incentive values totaling 100 points were determined using the ap- propriate (distributive or modal) schedule.

For pairing subjects in nonadaptive conditions with adaptive treatment counterparts, two basic criteria were used: (a) the two subjects had to be members of different cluster groups, and (b) their total predicted scores had to differ by an average of at least .125 SD. During instruction, nonadaptive subjects worked under the same incentive distribution developed for their adaptive treatment counterparts. The result, following all pairings, was two independent nonadaptive conditions, one incorporating the modal set of incentives and the other incorporating the distributive set.

Experimental sessions were attended by small groups. Subjects were told that each rule would be assigned a certain number of points and that the amount awarded would be contingent on performance on the rule (immediate) posttest. The basic learning re- quirements consisted of the following se- quence of activities for each rule: (a) study of the in t roduc to ry rule defini t ion, (b) study of complete examples and problem solving of incomplete examples, and (c) completion of the rule posttest. After a rule posttest was completed, the proctor collected the answer sheet, scored it, and returned it to the subject with an indication of the number of points awarded. At the completion of all rules, subjects took the 40-item cumulative posttest.

Results

A 4 x 5 analysis of covariance was performed on four outcome variables: (a) incomplete example score, (b) immediate post tes t score, (c) cumulat ive post tes t score, and (d) learning time. Pretest scores served as the covariate and were highly significant in all analyses (p's < .001). Thus, prior achievement was a strong correlate of task performance. Treatment means are summarized in Table 1.

The analysis of posttest scores yielded a significant main effect due to treatments, F(4, 7 9 ) = 13.10, p < .001. Neither the cluster group main effect nor the interaction was significant. Apparently, with pretest differences control led, c luster g roup characteristics in personality and reading


comprehension were relatively unimpor- tant to task performance. Consistent with orientations used in related studies on adaptat ion (Ross & Rakow, 1981; Ross, Rakow, & Bush, 1980; Tennyson & Rothen, 1977), task-specific prior achievement (e. g., pretest) thus emerged as the most powerful predictor of performance. The five treatment means were further analyzed via the Newman-Keuls procedure. Results indi- cated that the distributive-adaptive strategy (Adj. X = 29.24) was significantly superior to all other strategies (p < .05), thus supporting the major a priori hypothesis. The second most effective strategy was the modal-adaptive (Adj. X = 26.02), which was found to be significantly better than the distributive-nonadaptive (Adj. X = 23.88), standard (Adj. X = 23.24), and modal- nonadaptive (Adj. X" = 23.17) strategies. No other treatment differences were obtained.

The only significant finding in the analysis of learning time was the main effect of treatments, F(4, 79) = 3.18, p < .05. Comparisons of treatment means revealed that the longest learning times were associated with the modal-adaptive (Adj. X- = 100.01) and modal-nonadaptive (Adj. X = 99.39) strategies, while the shortest were associated with the standard (Adj. X = 78.86) strategy. Only the differences between these three extremes were found to be significant.

The 4 x 5 analyses of covariance performed on incomplete example scores yielded no significant treatment effects. Analyses of immediate posttest scores yielded significant treatment effects on only two rules. Generally, on-task performances were fairly high and comparable across treatments.

Discussion

On the basis of the findings, the following interpretations can be made regarding the effects of incentives. First, adaptive incentives benefited students by directing their attention and subsequent study activities to more difficult materials. The higher incentive values assigned to harder rules made these rules more attractive from a reinforcement standpoint and suggested to the students that they probably carried more

importance with regard to the overall performance objectives. Second, more varied incentive values had greater impact on the learning process than did schedules offer- ing less variation. This effect is almost cer- tainly attributable to the increased information provided. Third, relative to the baseline established by the s tandard incentives treatment, performance was not found to be hindered by either of the two nonadaptive conditions. The latter result is not surpris- ing, as the nonadaptive distributions were not directly maladapted to students' needs. Rather, they were distributions prepared for counterparts in the adaptive treatments. Thus, while they would not be expected to be particularly helpful to subjects, neither should they be detrimental. The learning time results, though mostly inconclusive, reflected a tendency for students to increase study time substantially on high-incentive rules, but to spend about as much time on low-incentive rules as did subjects in the standard treatment. In any case, the overall increases in learning time seem a small price to pay for the much larger gains in performance.

EXPERIMENT Ih COMBINATION MODEL

Experiment II examined the effects on learning of adapting incentives and instructional support in combination. The advantages of the combination model were hypothesized to lie in its direct matching of incentives with amount of practice. Accordingly, students were offered high instructional support and high incentives on rules for which predicted performances were low, and vice versa.

Design and Procedure

The procedures for group classification and prediction were the same as in Experiment I. A total of 120 subjects were assigned to six treatments formed by crossing three adaptation conditions (individual, group, and nonadaptive) with two conditions of incentives (distributive and standard).

Materials were the same as those employed in Experiment I, except that instructional units were extended so as to include 10 examples. For subjects in the individualized-adaptation (IA) treatment,

72 ECTJ SUMMER1982

the number of examples prescribed on each rule was varied in accord with their predicted rule scores. The adaptation scheme employed is shown below:

Number of Predicted (z) Examples Score Range

10 Less than - 1.375 9 - . 8 7 5 to - 1.375 8 - . 3 7 5 to - . 8 7 5 7 - . 1 2 5 to - . 3 7 5 6 + .125 to - . 1 2 5 5 + .375 to + .125 4 + .875 to + .375 3 + 1.375 to + .875 2 Greater than + 1.375

Thus, the higher the predicted rule performance, the lower the rule prescription. No prescription was permitted to vary outside the range of from 2 to 10 examples.

Half the subjects receiving the IA treatment (total n in IA treatment = 40) were administered differential (five-level) incentives (IA-D), and the other half were administered standard (10 points) incentives (IA-S). The group-adaptive (GA) and the nonadaptive (NA) treatments were formed by the cluster group pairing procedure used in Experiment I. Present use of the procedure involved pairing each subject in the individual-adaptive treatments with two counterparts, one from the same cluster group (for GA) and the other from a mutu- ally exclusive cluster group (for NA). In all cases, the IA member of the triad (IA-GA- NA) completed the experiment first. The GA and NA subjects were then assigned the IA subject's final prescription and were administered examples and incentives accordingly. The result, after all pairings were made, was the establishment of two GA treatments (GA-s tandard and GA- distributive) and two parallel nonadaptive treatments. The learning task and procedures remained unchanged from Experi- ment I.

Results and Discussion

The analytical design was a 4 (cluster group) x 6 (treatment) analysis of covariance (pretest). The analysis of posttest scores yielded a significant treatment main effect, F (5, 96) = 19.85, p < .001. The cluster group main effect and the two-way interaction were not

significant. Analysis of effects for treatments revealed that the IA-D group (Adj. = 30.26) performed at a significantly higher level than did each of the other five groups (p < .05). The only other significant comparisons showed that both the second highest group, GA-D (Adj. X = 24.83), and the third highest group, IA-standard (Adj. X = 24.64), surpassed (p < .05) the lowest group, NA-standard (Adj. ~" = 21.54). GA- standard (Adj. X = 22.81) was fourth highest, and NA-D (Adj. X = 22.58) was fifth. Apparently, the overall treatment effect was mostly attributable to clear superiority of the IA-D (i.e., combination) strategy over the others. No significant differences were obtained for any of the learning time, test- ing time, or immediate posttest comparisons. With regard to incomplete examples, treatments were found to differ significantly (p < .05) on only two rules.

These results offer strong support for the effectiveness of the combined model. Under standard incentives, the individual- adaptive (IA) strategy yielded only 8% to 15% learning advantages over the group- adaptive and nonadaptive treatments, respectively. By comparison, under differential incentives, the IA treatment advantages were 22% and 34%, respectively. Clearly, these results indicate a much stronger adaptation effect when instructional variations encompassed both examples and incentives than when they encompassed examples only.

DISCUSSION Both experiments demonstrate that learning can be improved when incentives are varied within a task in accord with individual needs. Second, a more finely graded incentive distribution worked better than one having less differentiation between values. Third, incentive effects increased as allocation of instructional support became more adaptive. Clearly, optimum uses of instruction adaptation must involve sensitivity to the needs of individuals; results from both experiments provide strong support for this idea.

From a practical standpoint, the adaptation effects obtained clearly would be significant if replicated in actual classroom situa- tions. For example, the two adaptive-


incentives groups in Experiment I averaged about 20% higher scores on the posttest than did the group receiving standard incentives, and about 30% higher scores than did those receiving nonadaptive incentives. To obtain additional insight into the mag- nitude of effect size, we computed the statistic d by dividing the differences between adaptive and control means by the within-group standard deviation (Cohen, 1969; Glass, 1977). This produced indices of .61 and .86 in comparison with standard and nonadaptive incentives, respectively. Differences in that range would be consid- ered relatively large (Glass, 1977).

These results suggest several possible ways of improving applied individualized learning systems. Student self-man- agement has much intuitive appeal, but carries the risk that students may lack the motivation or self-awareness to use instructional resources to their best advan- tage. Given our concern with validating a particular and somewhat sophisticated model, the adaptive strategies employed (involving cluster grouping, discriminant analysis, multiple regression predictions, etc.) were too complex for practical use. The complexity of the approach should not, however, obscure the basic idea supported: Students are motivated by incentives and will orient their study behaviors to the re- ward system offered. The implication for applied instructional design is that rewards should be structured so as to engender the most effective study behaviors for the particular task. The present assumption was that some of the mathematical rules would be familiar and easy for students, while others would be unfamiliar and difficult. Attaching the highest incentives to the difficult rules was intended to orient students to spend more time and effort on the most challenging parts of the lesson. Although some students might do that on their own, the much lower performances obtained under standard incentives suggest that many do not. For some, the "natural" tendency might even be to spend less time with the harder material, hoping that mastery of easier objectives will suffice for a passing grade.

Practical use of differential incentives can range from complex applications, as used

here, to much simpler forms. The trade-off, of course, is between degree of sensitivity desired and practicality. Some options one might consider are as follows:

1. At the most complex level would be the formal predictive models, as repre- sented by the present approach (also see Tennyson & Rothen, 1977). The objective would be to relate lesson achievement to characteristics of students and to express those relationships in a mathematical model. The model would then be used for predicting how new students will perform on different parts of the task. The final step would be distributing incentives in the manner done in this research, that is, higher values for lower predicted scores, and so forth. This approach might be strong on precision and accuracy, but it is clearly weak from a practical standpoint, for it would require a computer and fairly time-consum- ing pretesting on predictor variables.

2. At an intermediate level of complexity would be an approach that distributes incentives based on pretest scores only, with- out the use of a formal mathematical model. Recently, we have experimented with such a strategy using the introductory math module from the PSI statistics course. First, students were pretested on the math rules comprising the module. On the basis of pretest scores, the rules were rank-ordered for the individual from low to high. The highest rule was assigned the lowest incentive value, the next highest rule the next lowest incentive value, and so on. The available incentives were predetermined so that re- gardless of their distribution, the point total for the module remained constant. Use of the strategy required minimal time and effort beyond that for pretesting, but the ina- bility to randomize student assignment to treatments prevented a valid test of effects on achievement. Accordingly, future research on similar applications is strongly encouraged.

3. At the simplest level would be a system that varies incentives based on what is difficult or easy for the class as a whole. For example, in teaching a math unit covering different operations, an instructor may notice that certain operations (e.g., divi- sion) present more difficulty than others.

74 ECTJ SUMMER 1982

The implication from the present research would be to make those rules worth more po in t s than the o thers . Thus s t u d e n t s would have more incentive to learn them. Naturally, this strategy sacrifices the sen- s i t iv i ty that i n d i v i d u a l i z e d a d a p t a t i o n might offer. Its appeal comes from the re- quirement of little preparat ion beyond that for conventional (standard incentives) instruction.

Related investigations (Ross & Rakow, 1981; Ross, Rakow, Bush, & Cervetti, 1980) suppor t addit ional recommendat ions for practice. Specifically, we found that when the math unit was reduced by one-half (from 10 rules to 5), the benefits of differential incentives disappeared. Questionnaire results conveyed s tudents ' feelings that although the material was difficult, they were able to develop full effort to all rules, regard- less of incentive values. In the present task, the increased length of the unit apparent ly made a d d e d incent ives (par t icular ly on harder material) a more crucial factor. The recommendat ion, therefore, is to reserve adaptat ion of incentives for only the more difficult and demanding lessons.

Finally, al though not investigated here in depth, adaptat ion of amount of support is another orientation that instructional de- signers should keep in mind. It, too, can take the three general forms (formal model, informal model, normative) just described for incentives. Based on the results of Exper- iment II, the suggestion is that whatever general form is selected, adapt both incentives and instructional support in combination. The result is systematic matching of orientation to s tudy and amount of practice made available. Further research in a vari- ety of instructional contexts is needed to establish more clearly the practical effects and utility of these adaptive strategies. For now, they suggest the exciting prospect of moving instructional adapta t ion beyond variations in complet ion rates only.

REFERENCES

Atkinson, R. C., & Wickens, T. D. Human mem- ory and the concept of reinforcement. In R. Glaser (Ed.), The nature of reinforcement. New York: Academic Press, 1971.

Berliner, D. C., & Cahen, L. S. Trait-treatment interaction and learning. In F. N. Kerlinger (Ed.), Review of research in education: I. Itasca, Ill.: F. E. Peacock, 1973.

Cohen, J. Statistical power analyses for the behavioral sciences. New York: Academic Press, 1969.

Cronbach, L. J., & Snow, R. E. Aptitudes and instructional methods. New York: Irvington, 1977.

Glass, G. V. Integrating findings: The meta- analysis of research. In L. S. Shulman (Ed.), Review of research in education: V. Itasca, Ill.: F. E. Peacock, 1977.

Hansen, D. N., Ross, S. M., & Rakow, E. Adap- tive models for computer-based training systems (Semiannual report to Naval Personnel Re- search and Development Center). Memphis, Tenn.: Memphis State University, 1977.

Keller, F. S. "Goodbye, t e a c h e r . . . " Journal of Applied Behavior Analysis, 1968, 1, 79-89.

Kribs, H. D. The impact of incentives on information processing of a CAI learning task (Technical Re- port 30). CAI Center, Florida State University, 1974.

Kulik, J. A., Kulik, C. C., & Cohen, P. A. A meta-analysis of outcome studies of Keller's personalized system of instruction. American Psychologist, 1979, 34,307-318.

Robin, A. L. Behavioral instruction in the college classroom. Review of Educational Research, 1976, 46,313-354.

Ross, S. M., & Rakow, E. A. Learner control vs. program control as adaptive strategies for selection of instructional support on math rules. Journal of Educational Psychology, 1981, 73, 745-753.

Ross, S. M., Rakow, E. A., & Bush, A. J. Instruc- tional adaptation for self-managed learning systems. Journal of Educational Psychology, 1980, 72,312-320.

Ross, S. M., Rakow, E. A., Bush, A. J., & Cer- vetti, M.J. Instructional adaptation for self- managed learning systems. Paper presented at the annual meeting of the American Educa- tional Research Association, Boston, 1980.

Tennyson, R. D., & Rothen, W. Pretask and on- task adaptive design strategies for selecting number of instances in concept acquisition. Journal of Educational Psychology, 1977, 69, 586-592.

Tobias, S. Achievement treatment interactions. Review of Educational Research, 1976, 46, 61-74.

Ward, J.H., Jr. Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 1963, 58, 236-244.

Ward, J. H., & Hook, M. E. Application of hierarchical grouping procedure to a problem of grouping profiles. Educational and Psychological Measurement, 1963, 23, 69-81.