JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR

RATE DIFFERENTIAL REINFORCEMENT INMONKEY MANIPULATION

DAVID PREMACK'

UNIVERSITY OF MISSOURI

A set of four manipulanda were presented to four Cebus monkeys, individually, and later inpairs. Step 1 provided an estimate of each S's probability of operating each item, while Step 2determined whether pairing the items would disturb the ordinal relations among individualresponse probabilities. Both procedures provided information necessary for testing the assump-tion that a reinforcer is simply a contingent response whose independent probability of occur-rence is greater than that of the associated instrumental response. Step 3 tested this assumptionby again presenting pairs of items, but with one locked and its operation made contingentupon operation of the free item of the pair. The four Ss differed markedly in the extent towhich the items produced different independent response probabilities, and correspondingly,in the extent to which the contingent pairs subsequently produced reinforcement. Confirma:tion of the present assumptions came primarily from one S, which differed substantially onthe individual items, and showed five cases of reinforcement, all in the predicted direction.Further, reinforcement was shown by an increase in both contingency and extinction sessions.Finally, the response of intermediate probability reinforced the response of least, but not theone of greatest, probability, indicating that a reinforcer cannot be identified absolutely, butonly relative to the base response.

The generalization that, of any two re-sponses, the independently more probable onewill reinforce the less probable one representsthe major assumption of a recent model ofpositive reinforcement (Premack, 1959, 1961,1962). This assumption was tested in the pres-ent experiments with the use of manipulationresponses in monkeys. Basically, a two-stageprocedure was used. First, estimates wereobtained of S's probability of operating eachof four different manipulanda. Second, themanipulanda were presented in pairs-onefree, one locked-with operation of the lockeditem contingent upon operation of the freeitem. The model predicts that the contingencywill increase the frequency of the free re-sponse, provided the independent probabilityof the contingent response is greater than thatof the free response, and that the increase willbe proportional to the independent probabil-ity of the contingent response.

'Portions of these data were reported earlier in asymposium, "Novelty, Curiosity and Exploratory Be-havior," APA, New York, September, 1960, and at theMPA, Chicago, May, 1961. The work was begun duringthe author's tenure as a USPHS Postdoctoral ResearchFellow, and completed with the aid of grant M-3345from the National Institute of Mental Health, andgrant G19574 from the National Science Foundation.

In addition, that reinforcement is 'a rela-tive, not an absolute property, follows fromthe model, as can be shown by applying it tothree responses of different independent prob-abilities. Let A, B, C represent three re-sponses of an organism with independent pro-babilities in descending order. If contingenciesare arranged between all pairs of responses,the model predicts that A will reinforce bothB and C; C will not reinforce either A or B,while B will reinforce C but will not reinforceA. Considered jointly, the reinforcement prop-erties of A and C appear to corroborate thetraditional absolute view: some events are,while others are not, reinforcers (e.g., Skinner,1938, p. 62). But the reinforcement propertiesof B controvert this view for, according to themodel, B both is and is not a reinforcer, de-pending upon the relative probability of thereferent response.

METHOD

SubjectsFour male Cebus monkeys-two cinnamon,

two hooded capuchin-about 31,4 yr old.None had experienced either food-contingenttraining or experimentally-induced food pri-

81

VOLUME 6, NUMBER I JANUARY, 1963

DAVID PREMACK

vation, but all had had extensive previous ex-perience with operant-level manipulation onitems similar to those of the present experi-ments (e.g., Premack and Bahwell, 1959).Throughout the present series, Ss had freeaccess to Purina monkey pellets and water,supplemented by a daily multi-vitamin sand-wich. Testing was daily from approximately12 to 1 pm, and food was replenished dailyat approximately 4 pm.

ApparatusA metal cage (40 x 30 x 27 in.) which both

housed and tested S, and four manipulandawere used. Two normally closed ports in thefront of the cage were equipped to receive anyone or any pair of items; when inserted, thecenter of each item was approximately 16 in.from the floor of the cage. A lever (L), plunger(P), door (D), and horizontally-operated lever(H) were the items used. Constructed of stain-less steel and ball bearing pivots by the Uni-versity Science Instrument Shop, the itemshad operation distances of approximately 21/,to 3%2 in., and force requirements rangingfrom approximately 35 to 55 gm.The principal feature of each manipula-

ndum was a solenoid operated lock that couldbe used to render any item inoperable. Whenpairs of items were given, either one could belocked and its release made contingent uponoperation of the free item of the pair. Opera-tion of the free item released the locked item,while operation of the previously-locked itemrestored its own lock, thereby reinstating theoriginal contingency. Release of the lockeditem required both displacement and returnto resting position of the free item. Previouswork showed that if return-to-resting-positionwas not an explicit requirement, Ss tended tohold the free item in its displaced positionwith one hand while freely operating the "con-tingent" item with the other hand.

ProcedureThe four different procedures used are

described separately in the appropriate resultssection. General procedure was the same inall test phases: either one or both of the panelswere removed from the front of the cage, andeither one or two items inserted and bolted tothe cage. All sessions lasted 1 hr, and all weregiven daily, those within a test phase, andthose between successive test phases.

RESULTS

Independent Rates

In the first test phase the items were pre-sented individually with no restrictions uponresponding. The four items were given to eachS in a fixed and different order, and eight ses-sions were given per item; a total of 32, 1-hrsessions for the set of four items. The intervalbetween sessions on the same item was 96 hr.Previous work (Premack and Bahwell, 1959;Premack and Collier, 1962) has shown that fre-quency of manipulation, in both rats andmonkeys, is largely regulated by intersessioninterval (ISI); a sufficient iSI tends to main-tain total output per item despite indefinitelymany exposures to the item.The separate quadrants of Fig. 1 show the

frequency with which each S operated eachitem on each of the eight sessions. The x axisidentifies the item, the y axis gives frequencyper session, and the separate numbered curvesshow the repeated measures. Marked individ-ual differences may be seen in Fig. 1. Chicko

z0

(I)U)LLI

I

blJ

lJJ

6005004003002001000

800700600500400300200100

WILLY

7

GIMBEL

BIMBO

D L P H D L P HTEST ITEMS

Fig. 1. Total responses per hr for individually pre-sented items, with numbered curves for the eight ses-sions given per item.

82

RA TE DIFFERENTIAL REINFORCEMENT

CHICKO WILLY

% A I'I s . .

W4 6

.C> Go %cr- Ifto, -

BIMBO

5 MIN. INTERVALSFig. 2. Within-session responding for individually presented items. Frequency per 5-min interval averaged over

all eight replications, with item as the parameter.

and Willy depict the two extremes, the formershowing clear and consistent preferences, thelatter essentially no preferences, coupled witha generally low response level.

Figure 2 shows mean response rate per5 min interval averaged over all eight sessions.The characteristic within-session decrementmay be seen, as well as the different decelera-tions of the several items. All Ss except Willy

showed at least one preference, but only inChicko were these large and consistent.Table 1 gives total frequency per replication

averaged over all items; total frequency variednonmonotonically, but did not decline, withreplications. Thus, although responding de-clined within each session (see Fig. 2), the96-hr inter-item interval provided for com-plete between-sessions recovery.

Table 1Total Frequency per Replication for Individually Presented Items

Replications1 2 3 4 5 6 7 8 Total

Chicko 1623 1219 1044 1273 1518 1268 1444 1344 10,733Gimbel 832 942 613 1080 1128 1130 1233 887 7,845Bimbo 641 864 499 613 561 629 783 704 5,294Willy 765 509 315 590 706 508 814 523 4,730

q80706050403020100

80706050403020100

D0----Ha--L

0zw

wUi.

* GIMBEL

83

II

II JRI

a %

iv0 *,& q

p % A *

0-

tL. . . A ---. . a ., -o .40 . . A . .0 i

3 5 7 9 11 I 3 5 7 9 11

DAVID PREMACK

Independent Rates of PairsIn the second test phase, Ss were presented

with pairs of items, again with no restrictionsupon responding. The six pairs generated bythe four items. were presented to each S in afixed and different order, and four sessionswere run per pair, a total of 24, 1-hr sessionsfor the set of six pairs. Testing was daily, butthe interval between sessions on the same pairwas 144 hr, and that between sessions on thesame item was an average of approximately40 hr.The main purpose of this procedure was to

determine whether pairing the items woulddisturb the ordinal relations among the indivi-dual response probabilities, in particular, re-verse their rank order. If pairing were to havesuch an effect, then, because in the contin-gency situation S confronts pairs of items,predictions as to which items will reinforcewhich others would have to be made in termsof the rank order among pairs (for an actualcase, see Brownstein, 1962).

600500o GIMBEL

o 400/0n 300cX 200

Qt 100

H DH PD P500 BI MBO400

300 a a

o 200

,nnlOk4it 4o

*50 LH L D LP

400

3030

0 200L P

700600 CHICKO5000J

1000 L H L D L

80070016001

400WILY

30020010

H HD D P

500 WILLY400300200

500P400

300

200

100

H D H P D PITEMS BY PAIRS

Fig. 3. Total responses per hr for paired items, withnumbered curves for the four sessions given per pair.

Figure 3 shows total responses per sessionmade on each item of a pair on all of the fourreplications. Differences between Ss on in-dividual items reappear. For example, in thecase of Willy, no pair differed appreciably,whereas for Chicko, in five of six pairs, ordinalrelations among members of pairs were con-sistent over all replications. Moreover, the in-consistent pair consisted of the two items thatdiffered least on individual presentation.Although both of the above procedures

raise independent questions, they serve hereas prerequisites for tests of the present assump-tions. To determine whether a more probableresponse will reinforce a less probable onerequires being able to differentiate betweenmore and less probable responses. Ideally, itrequires ordering a large set of responses interms of their independent probabilities. Thefailure of all but Chicko to approximate thisrequirement reflects a principal difficulty ofthe procedure-finding items on which Ss willdiffer. This difficulty is instructive, however,for although there are few rules to guide thesearch, there is the suggestion that S may itselfbe a critical variable. If this is even approxi-mately true, important practical consequencesmay follow. Subjects that differ little in pro-bability of responding to the constituents oftheir environment will, according to thepresent model, be proportionately limited asto their reinforcement possibilities. For ex-ample, differences among the four Ss on thepresent set of items lead to the prediction that,when the items are paired contingently, Willywill show few if any reinforcement effects,Chicko a number of such effects, and the othertwo Ss an intermediate number; the numberbeing proportional to the number of differ-ences shown among the individual responseprobabilities.

Contingency TestsIn this procedure the items were again

presented in pairs, but now one item waslocked and its operation made contingentupon operation of the free item. A continuousreinforcement schedule was used which, atthe same time, ruled out storage or multipleresponses on the contingent item. That is, atleast one free response was required betweenany two contingent responses.The six pairs of items generated 12 possible

contingency pairs, since in this procedure the

84

RATE DIFFERENTIAL REINFORCEMENT

same item can serve both as a free item on oneoccasion and as a contingent item on another.Chicko and Gimbel were tested on nine, andBimbo and Willy on five of the 12 possiblepairs. For each pair, a block of four daily con-

tingency sessions was given, followed by a

block of four daily extinction sessions. In ex-

tinction, the contingent item was removed,the port was closed as in the original pro-

cedure, and only the previously free item was

given. On five of the nine pairs on whichChicko was tested, the contingent responsehad a higher independent rate than the freeresponse, and these five cases were replicatedcompletely, including the alternating blocksof contingency and extinction sessions. Overthe course of the present procedure, Chickoreceived 112 daily, 1-hr sessions.

Maintaining a 24-hr ISI over as many as

four 1-hr sessions, as in the contingency pro-

cedure, typically produces a cumulative de-crement in total manipulation responses persession. Thus, any increments produced bycontingency training occurred despite an ISIwhich by itself would be expected to producea lower independent rate than that of the96-hr ISI of procedure 1.

Contingency ResultsThe outcome for the four Ss differed in a

manner generally predictable from their inde-pendent rate records. Main results are sum-

marized in Table 2, which gives mean fre-quency per session averaged over all of theindependent rate, independent rate of pairs,and contingency sessions, respectively. Thepairs are listed in the order in which theywere presented during contingency training.Chicko, after receiving the nine pairs in theorder shown, was retested on the first fivepairs, again in the order listed.Rather than showing a reinforcement effect

on any of the five pairs, Willy declined on allitems. The over-all decrement would be ex-

pected on the following grounds: with no

countering incremental effect of reinforce-ment, the 24-hr ISI of contingency trainingshould produce a lower rate than the 96-hrISI of the independent rate procedure.Bimbo and Gimbel showed one strong and

two weak incremental effects, respectively.Further, in neither S did any less-probableresponse increase any more-probable response.However, several more-probable responsesfailed to reinforce less-probable ones, mostnotably the P H case in Gimbel. Also, success

and failure of reinforcement bore no evidentrelation to magnitude of the differencesamong pairs shown in the independent rateprocedures.The clearest predictions possible are those

for Chicko, who differed substantially andconsistently on five of the six possible pairs.An immediate indication of the confirmation

Table 2Comparison of Average Frequency per Session of the Free Response Alone (IR),

in a Pair (IR-P), and in the Contingent Pair (C)

Chicko GimbelItem IR IR-P C Item IR IR-P C

P (H) 78 68 243 P (L) 131 40 152P (D) 78 93 214 L (H) 207 200 224L (H) 270 326 342 H (L) 424 414 318P (L) 78 40 246 P (H) 131 89 82D (H) 382 274 467 D (H) 201 259 148L (D) 270 233 245 P (D) 131 78 105H (P) 543 584 382 L (D) 207 119 94D (P) 382 369 298 H (D) 424 230 206H (D) 543 459 424 D (P) 201 115 161

Willy BimboItem IR IR-P C Item IR IR-P C

D (H) 121 63 57 L (D) 178 111 230L (D) 177 124 112 P (D) 85 118 77P (H) 136 109 95 D (L) 175 193 121H (P) 167 147 83 P (H) 85 42 22L (H) 167 100 120 H (L) 225 59 83

85

DAVID PREMACK

of the main predictions is shown in Table 2.In all cases where the independent probabilityof the contingent response was greater thanthat of the free response, there was an increasein the free response (1-5); all other cases led toa decrement (.6-9). In mean frequency persession, increments in the free response rangedfrom 22.2% to 211.5% and decrements from9.2% to 37.4%.

CONTINGENCY EXTINCTION400300O200100400300200

>- 1000 400w 300M 200w 100i 400w 300O

200100600500400300200

ORI* 0

P(D)* 0

0

P(L)

0

L(H)

0.I 23 4

R2.

*00 0

0

0---@ 00 0

00

1 234

RI R2

0 0

0

0 @0 @0>.0

0 0

----.-- -0

0

0 0

1 234 I 2 3 4SESSIONS

Fig. 4. Frequency per session on all five predicted-positive cases, with separate displays for contingencyand extinction, and for replications 1 and 2 (RI, R2).Open letter, e.g., P, indicates free response, and brack-eted letter, e.g., (H), contingent response; dotted linesgive independent rates or operant-levels of free re-

sponses. Data for Chicko.

Frequency per session for all of Chicko'scontingency and extinction sessions for thefive incremental cases on both replicationsis shown in Fig. 4 (omission of the P [L] caseon the first replication was an experimenteroversight). The general trends of these dataare shown in Fig. 5, which gives frequency persession averaged over all five cases, with sepa-rate curves for replications. As shown inFig. 5, the averaged contingency and extinctiongradients differed in the expected manner. Ineight of the nine possible contingency cases,frequency was greatest on either the secondor third session; the relative decrement on thefourth session probably reflects the 24-hr ISIwhich, by itself, typically produces a decre-ment in this kind of responding. Althoughtotal number of contingency responses was

less on the second replication than on the first

(5621 vs. 4756), none of the individual casesin Fig. 4 show any more-probable response tohave lost its power to reinforce a less-probableone.An extinction level above that of the op-

erant-level is shown clearly in Fig. 4 for allP and L cases, but was not found in any caseof D (H). Extinction, even more than con-tingency training, occurred under an adverseISI. Whereas base frequencies are for a 96-hrISI, even-the first of the four extinction ses-sions represents the fifth consecutive exposureto an item at only a 24-hr ISI.

0zw

wLL

400 CH ICKO

300.

200-

_

100O

oLCONTINGENCY

S2 3 4S ESSIONS

* RI0-R2

EXTINCTION

1 2 3 4

Fig. 5. Averaged contingency and extinction gradientsfor all five predicted-positive cases; dotted line givesoperant-level averaged over all free responses. Data forChicko.

Within-session contingency and extinctionresults for Chicko are shown in Fig. 6, whereeach of the nine sections treats the pair ofitems joined by the horseshoe or material con-ditional sign. In the bottom row, for example,P was consistently the free item, while, inseparate series of tests, operation of D, H andL, respectively, were contingent upon opera-tion of P. The x and y axes show, respectively,time and rate per 10 min interval averagedover the four and in some cases eight sessionsgiven per pair.The independent rate protocol for Chicko

permitted three kinds of contingencies: con-tingent response higher than, less than and,in the one inconsistent case, about equal tothe free response. First, contingent higherthan free response produced in all five casesan increment in the free response, shown inPDD, PDH, PDL, LDH, DDH. Second,contingent less than free response produceda decrement in the free response, shown inH D D, H D P, and less clearly in D D P. Third,contingent and free responses about equaltended to produce little or no change, shownin LDD.

RATE DIFFERENTIAL REINFORCEMENT

200 o EXT.150x 0° 0IND.100 HDD50.0

D,HD

L,H

P,H

H,P

6 1 2 3 4 5 6 1 2 3 4 5 610 MIN. INTERVALS

Fig. 6. Within-session responding on contingently-paired items. Frequency per 10 min interval averaged over

all replications, with separate curves showing independent rate, dependent rate (reinforced), and extinction; datafor Chicko.

Two further comparisons shown in Fig. 6deal with the effect of (a) different contingentresponses upon the same referent response,and (b) the same contingent response upondifferent referent responses. The first case,where H, D and L all served to reinforce P,is pertinent to the assumption that the in-crement to the referent response is propor-tional to the independent probability of thecontingent response. This assumption was

only weakly supported. Differences in H, Dand L were reflected in the initial rates of P(mean initial 10 min) but not in total P re-

sponses per session. The second case, where Hserved to reinforce P, L and D, showed thatthe same reinforcer applied to three differentreferent responses did not produce the same

reinforced rate, but rather a rate that was pro-portional to the operant-levels of the referentresponses. Further, since the increments to D,

L and P (reinforced minus operant-level) were85, 62 and 185 responses per hr, respectively,there is the suggestion that independent ratesof contingent and free responses may interactin determining reinforced rate. However, on

this point, the present data cannot do more

than raise the question of whether there is aninteraction or whether operant-level enterssimply as an additive constant in determiningreinforced rate.A last point of major interest is the evidence

contained in Fig. 6 concerning the relativityof reinforcement. As shown in Chicko's datain both Fig. 1 and 3, H was the most probableresponse of the set, P the least, and D inter-mediate. Figure 6 shows that H reinforced allmembers of the set while P reinforced none

and, instead, was itself reinforced by all othermembers. Considered jointly, the results forH and P appear to *ubstantiate the traditional

150100

zw

wILLtr

LDD

0'

P,D

DAVID PREMACK

absolute view: H, which is a reinforcer, rein-forces all responses, while P, which is not areinforcer, reinforces no response. But the re-sults for D controvert this view, for D, whichwas of intermediate probability, reinforcedP but failed to reinforce" H. Thus, dependingupon the relative probability of the referentresponse, D both was and was not a reinforcer.These results do not accord with the trans-situationality assumption which Meehl (1950)has shown to characterize the classical accountof reinforcement. Since the present case ap-pears to represent the first direct test of theassumption, the previous success of the as-sumption may rest largely upon a failure tohave tested it.

DISCUSSIONOperant-level manipulation was used both

as the contingent and instrumental events,underscoring the assumption that any more-probable response will reinforce any less-probable one. Although manipulation provedto be unstable and of low frequency, its maindetriment for the present procedure is thelack of that kind of information which wouldpermit an advance ordering of a set of manip-ulation responses. Thus, the present fouritems were chosen essentially at random, andon this set of items, of four Ss, only oneshowed sizable and consistent differences;another showed no reliable differences; andtwo others, generally small and inconsistentdifferences.

Results for the S that differed consistentlyamong the items substantially confirmed themain predictions. When items were presentedin contingent pairs, no- more-probable con-tingent event failed to reinforce any less-probable free event; further, no more-probable freeevent was reinforced by any less-probable con-tingent event.The S that showed no differences among the

items showed no reinforcement effects, as waspredicted, but this kind of outcome is neces-sarily less instructive than when both positiveand negative effects are obtained. Least in-formative were the two inconsistent Ss; theircontingency results were as indeterminate astheir original differences among the items. Ingeneral, agreement between Ss' individual re-sponse probabilities and the number of con-tingent pairs that subsequently produced rein-

forcement confirmed a practical implicationof the model-Ss that differ little in probabil-ity of responding to the components of theirenvironment will be proportionately limitedin their reinforcement possibilities for theenvironment.

Contrary to what may be suggested by theresponse units used in these experiments, thepresent assumptions place a unique burdenupon definitions of response and responseprobability. At the time of the experiment,the response problem had not been seriouslyconsidered, but a formal response treatmentcan now be provided that both satisfies thelogical requirements of the present assump-tions and suggests why the response units usedwere generally successful.Only two pieces of information are re-

quired by the treatment: (a) operationaldefinition of the behavior in question, and(b) determination of the minimum durationfor which the behavior occurs. Divide experi-ment time into successive time intervals equalto the minimum duration of the behavior;then make a digital, or yes-no, decision foreach interval. That is, simply decide for eachinterval whether or not the operational defini-tion was fulfilled in the interval. Let

Ps ~FO' (1), where Pr is probability of the

response, and Fol and Fpi are, respectively, ob-served frequency of occupied or "yes" in-tervals, and total possible intervals, the latterbeing the ratio of experiment time to mini-mum duration of the behavior. If the responseis defined as the minimum duration for whichthe operationally-defined behavior is observed

to occur, then (1) becomes P_ -Fr where

Fr is observed frequency of the response. Afurther exact alternative is to operate a clockwhenever the operational definition is fulfil-led; then Pr is simply the ratio of observed re-sponse time to possible response time.

Response duration, defined as the periodfor which the micro-switch was activated, wasmeasured for Chicko from Esterline Angusrecords taken in the original sessions for, in-dependent rate. Mean response durations forH, D, L and P were: 1.48, 1.37, 1.40, and 2.00sec, respectively. Although response durationsdiffered for the several items, the differenceswere not so great as to offset the differences infrequency. Thus, estimated mean response

88

RATE DIFFERENTIAL REINFORCEMENT 89

probabilities for H, D, L and P were: .22, .15,.11, and .04, respectively. This is the samerank order as that given by consideration offrequency alone.

REFERENCESBrownstein, A. Predicting instrumental performance

from the independent rates of contingent responsesin a choice situation. J. exp. Psychol., 1962, 63, 29-31.

Meehl, P. E. On the circularity of the law of effect.Psychol. Bull., 1950, 47, 52-75.

Premack, D. Toward empirical behavior laws: I. Posi-tive reinforcement. Psychol. Rev., 1959, 66, 219-233.

Premack, D. Predicting instrumental performance from

the independent rate of the contingent response.J. exp. Psychol., 1961, 61, 163-171.

Premack, D. Reversibility of the reinforcement rela-tion. Science, 1962, 136, 255-257.

Premack, D., and Bahwell, R. Operant-level leverpressing by a monkey as a function of intertest inter-val. J. exp. Anal. Behav., 1959, 2, 127-131.

Premack, D., and Collier, G. Joint effects of stimulusdeprivation and intersession interval: analysis ofnonreinforcement variables affecting response prob-ability. Psychol. Monogr., 1962, 76, 5 (Whole No.524).

Skinner, B. F. The behavior of organisms. New York:Appleton Century Co., 1938.

Received December 18,1961