Behavior Research Methods & Instrumentation1975, Vol. & (2),145·150

SESSION VIICONTRIBUTED PAPERS:

PHYSIOLOGICAL/ANIMAL LEARNINGACQUISITION AND CONTROL SYSTEMS

RUSSELL CHURCH, Brown University, Presider

On-line brain stimulation:Measurement of threshold of reinforcement

GERALDINE CASSENS, CLIFFORD SHAW, KATHRYN EIKE DUDDING, and A. WILLIAM MILLSTufts University, Medford, Massachusetts 02155

This program and interface for a small computer (PDP 8 Ie) performs the following operations:generates and delivers constant current stimulation to the brain of an animal, monitors electricalresistance at the stimulating electrode, and calculates the threshold of reinforcement based upon thefrequency of characteristic pauses in the animal's responding.

Our research deals with the reinforcing properties ofelectrical stimulation of the brain. This on-line programand interface were devised to: (1) provide a versatilesystem for studies involving brain stimulation andmonitoring, (2) provide instantaneous information aboutchanges in electrical resistance in an animal's brain, and(3) measure threshold current for which an animal willreliably press a lever to stimulate his own brain, i.e., athreshold of intracranial reinforcement (ICR).

Previous methods for measuring a threshold of ICRare time consuming and open to ambiguousinterpretation (Valenstein, 1964). One major defect withprevious methods is that below a critical intensity of theelectrical stimulus, the animal's rate of response falters,eventually extinguishes, and the experimenter losescontrol of the animal's behavior. Another difficulty isthat methods utilizing rate of response fail todiscriminate between the effects of an experimentaltreatment, e.g., a psychoactive drug, on motor responsevs. its effect on threshold of reinforcement.

The procedure developed in this laboratory (Cassens& Mills, 1973; Huston & Mills, 1971) obtains a thresholdthat is' independent of wide variations in the animal'srate of barpressing and maintains responding over theentire range of electrical stimuli presented. The animal istrained to respond on two concurrent schedules ofself-stimulation. Reinforcement on one schedule simplymaintains the animal's behavior throughout theexperiment, while the other schedule is used to measure

The establishment of a computerized animal laboratory wassupported by a Faculty Grant from Tufts University to A. W.Mills and NIH Grant 5 ROI EY00121 to Samuel McLaughlin.

the reinforcing effects of a second stimulus. Afixed-ratio (FR) schedule is used to maintain behavior;the animal is reinforced for every nth response with anintracranial stimulus of constant intensity. A continuousreinforcement (CRF) schedule, one in which the animalis reinforced for every response, is superimposed on theFR schedule, and its intensity is varied to study changesin the animal's behavior. As the intensity of the CRFstimulation on the concurrent CRF-FR schedule isdecreased, pauses in barpressing behavior begin to appearafter each FR reinforcement. These arepostreinforcement pauses (PRPs) and are characteristicof responding under an FR schedule (Ferster & Skinner,1957) (see Figure I). We define a pause in responding asan interresponse interval (lRI) 3 or more standarddeviations above the mean IRI at a given current level.tAs the intensity of the CRF stimulation is increased,PRPs begin to disappear. When the CRF stimulation isvaried in descending and ascending series, keeping theFR reinforcement constant, PRPs reliably appear anddisappear as a function of current level of the CRFstimulation. Thus, there is a current level at whichcontrol of the animal's responding switches from the FRschedule to the CRF·FR schedule as shown in Figure 1.

Once the animal is thoroughly trained to respond on aCRF-FR schedule, this method allows one to introduce,vary, or withdraw CRF stimulation without disruptingthe sequence of responding typical of behavior on an FRschedule. This stability of response is especially desirablein studies in which drugs may alter response rate with orwi tho u t affecting the threshold of intracranialreinforcement.

145

146 CASSENS, SHAW, DUDDING, AND MILLS

Rat 3FR 40

5 m:n.

Figure 1. Cumulative record for a rat on a concurrentCRF/FR schedule. This rat received FR reinforcements of120 microA on a schedule of 40: 1. Changesin the CRF intensityare indicated along the base of the curve and are expressed inmicroamperes base to peak. The postreinforcement pauses(PRPs) are long and consistent when the CRF stimulation is low,such as 30 or 3S microA, but disappear when the CRF intensityis raised to 60 or 6S microA. When the PRPs disappear, thebehavior is identical to that of a subject on a simple CRFschedule of equal reinforcements.

The threshold of intracranial reinforcement reportedhere is based upon the relative frequency of PRPs. Asthe current level of the CRF stimulation issystematically increased, the proportion of FRreinforcements that are followed by PRPs decreases. Thenumber of PRPs per FR reinforcement (PRP/FR) iscalculated at each current level. A least-squaresregression analysis is applied to the resulting proportionsas a function of current level to determine the bestfitting line for the data. The threshold of lCR is derivedfrom the computed best fitting line and is the current atwhich the proportion ofPRP/FR is .5.

Manual control of this procedure was tedious,requiring continuous manipulation and observation bythe experimenter. Consequently, the procedure wasautomated. An important consideration in our softwaredevelopment was to maximize the options for theexperimenter for changing stimulus parameters andpresentation. The computerized system performs thefollowing operations: (1) sets the intensity ofstimulation according to one or more reinforcementschedules determined by the experimenter, specifically aCRF schedule, any FR schedule, and/or a concurrentCRF-FR schedule; (2) accepts parameters specified bythe experimenter concerning the range of current levelsto be sampled, the size of the increments within therange, and sets the requisite level of current;(3) generates trains of 100-Hz bipolar square waves of.2 sec duration. (This duration can be varied.);(4) determines the voltage drop across the stimulatingelectrodes, computes, and prints out the electricalresistance of the animal's brain; (5) turns stimulation offand rings a bell if the resistance level is abnormally highor low (indicating an open or short circuit); (6) times theintervals between each of the animal's responses, andcalculates the mean and standard deviation of the IRIsfor each current level; (7) determines whether a pausehas occurred, whether it is a postreinforcement pause

(i.e., h as occurred immediately after the FRreinforcement and is greater than 3 standard deviationsabove the mean IRI) and determines the ratio of PRPsper FR reinforcement at each current level; (8) After theprogram has sampled the predetermined number ofcurrent levels in a method of limits fashion, an equationfor the best fitting line is determined by least-squaresregression analysis utilizing the PRP/FR ratio as afunction of current level. Finally, the program computesthe threshold of ICR based upon the current level atwhich the proportion of PRP/FR is .5.

HARDWARE

The basic hardware consists of a Digital EquipmentCorporation PDP 8/e. The program uses the followingperipherals: an ASR-33 Teletype with low speed papertape reader and punch; a lO-bit A/D converter(AD8/EA); a lO-bit point-plot display control (VC8/E)with VR 14 scope; a programmable real-time clock(DK8/ES); a 12-channel buffered digital I/O (DR8/EA);a DECtape system (TD8/EA); a 24-bit extendedarithmetic element (KE8/E); and an additional 4,096words of core memory. Neither the scope nor theDECtape units are essential to the functioning of theprogram.

The interface equipment between the computer andanimal utilizes a digitally controlled constant currentstimulator which: (a) converts the digital output fromthe computer to an analog signal to the animal via a D/ Aconverter; (b) varies the voltage as a function ofresistance to keep the current constant, using amodification of a design by Wayner, Peterson, andFlorczyk (1972). The circuit holds current constant forloads of less than 2.25 mW; (c) allows simultaneousmonitoring of current and voltage through the use ofindependent operational and FET instrumentationalamplifiers.

The circuit diagram for this interface is shown inFigure 2, and the components for the circuit are listed inTable 1.

Behavioral EquipmentThe experimental apparatus consists of a rectangular

box 10 x 16 x 12 in. with Plexiglas top and Sides,containing a Gerbrands lever. A closed-circuit televisionprovides continuous observation of the animal.

SOFTWARE

The program software was written in PAL 8 andincludes DEC's 23-bit extended arithmetic elementfloating point package (EAEFPP). The program fillsapproximately 3,000 memory locations and requires anadditional 4,OOO-word buffer for data storage whichpermits asynchronous output of the experimental data.A brief description of the program flowchart (Figure 3)follows:

Program parameters are initially input by the

ON-LINE BRAIN STIMULATION 147

+15 vdc

INPUT BUFFER

DAC

16 Rl

J/ll :lON rron

1C2

R5

R4

+ 15 vdc

R5

R8

RIO

R?

-15 vdc

-15 vdc

"'t;:;:-----------4~~ +15 vdc

R4

R5

R5 C2

h...-- Eoe

IN-

sen!RGI

1\2R7

LOAD

RG2

IN+

R6

C2

--

+

R5

R5

~_"""'"I - Trfbo~--+---J

Al

CURRENTMONITOR

1/10 VOLTAGEMONITOR

Figure 2. Circuit diagram of D/A constant current stimulator. Components are listed in Table I.

experimenter from the Teletype (TTY) keyboard,including stimulus range, amplitudes, and ratioschedules. The actual experimental run begins following

an initial barpress by the animal, or by the experimenter(as a priming stimulus) from the TTY.

Central to the program is a continuous loop which

148 CASSENS, SHAW, DUDDING, AND MILLS

services the various peripherals: the AID converter, thescope, the digital output and input device, the clock, andthe TTY. An important portion of the loop is devoted togeneration of the stimulus and to real-time monitoringof the voltage return from the intracranial electrodes.The program monitors voltage return via the AIDconverter concurrently with stimulus generation.Approximately 1,000 voltage samples are read from theAID converter, output to the scope, and summed duringthe stimulus train. This summing is accomplished byusing a 24-bit arithmetic register rather than therelatively slower floating-point pseudoinstructions.

At the conclusion of the stimulus output, the meanAID voltage and the current are converted to a resistancemeasurement (RESIST). When the stimulus is not beinggenerated, the momentary voltage is monitored. Anyresistance or voltage out of the limits set by theexperimenter triggers an alarm which interrupts theprogram, prints out the reason for the interrupt, sets allstimulus output values to zero, and signals the

experimenter by ringing the TTY bell (ALARM). Aswitch register option permits the experimenter tocontinue the program from the point of interrupt if theproblem has been solved.

The continuous loop also allows output of the

Table 1list of Components for O/A Constant Current Stimulator

Al Philbrick/Nexus 1005A2 Philbrick/Nexus FET 4253Cl 50,uFC2 .001 JlFOAC 0/A 371-8 Bit, Hybrid Systems CorporationlC Texas Instruments Quad Bistable Latch, SN 7475NRl 500nR2 lKR3 12K 1%R4 20K 1%R5 10KR6 2K 1%R7 .5M 1%R8 lOOKR9 900 1%RIO 100 1%

experimental data to a ltl-character/sec printer. TheTTY printer frequently lags behind program dataoutput. However, the animal's frequent pauses offerample time to output all the stored data to the printerand the paper tape punch.s The data can be accessed ata later time from the paper tape.

The final function of the loop is to determine theinterresponse interval by reading a I llO-sec softwarecounter immediately following each barpress. This

No

No

No

Calc meanresist {orschedule II:put in bufler

Calc final stat.Put .'flo buffer

Calc mean IRI.SD, PRP. Putin bulle r

Inc IflO secCounter

Set s t irn nag.Calc II store 1--....IRI in buffer

Updatevoltagesum

Terminate.tim

Startclock

Figure 3. Flowchart of program. See text for discussion.

ON-LINE BRAINSTIMULATION 149

Figure 5. Histogram of interresponse intervals (IRIs) at asingle current level. The computer plots the total number ofresponses at each IRI which is expressed in tenths of a second. N= number of responses at a given IRI.

current level. PRP/FR ratios and current levels arefurther stored as sums and sums of squares for use in thestatistical subroutines at the conclusion of the program.

When the program determines that the requirednumber of FRs have been completed at all current levels,the stored sums and sums of squares are used tocalculate the correlation coefficient between currentlevels and the standard deviates of the PRP/FR ratios.The threshold current is determined from the regressionequation and output to the software TTY buffer. Asegment of the printout is shown in Figure 4.

At we conclusion of the experiment, all counters arereset to deliver a CRF/FR schedule of indeterminatelength at the maximum designated current level. Thisextended run provides reinforcement until the animal isremoved from the experimental chamber.

For the postexperimental analyses, all data areaccessible on paper tape.s For example, a histogram of

the interresponse intervals at a single current level isshown in Figure 5.

software counter, which is dependent on the real-timeclock, may be easily set for a precision of up to10 microsec. The accumulated clock counts for a singleIRI are stored with previous IRIs as sums of IRI andIRI 2 for later statistical manipulation. The IRIsfollowing each FR are stored individually as potentialPRPs.

At the conclusion of the requisite number of FRs at agiven current level, the program uses the stored sums ofIRI and IRI 2 to determine whether any of theindividually stored pauses (potential PRPs) aresufficiently deviant to constitute a PRP (i.e., greaterthan 3 SD above the mean). The number of PRPs, meanIRI, and standard deviation of the IRIs are output to thesoftware TTY buffer followed by the PRP/FR ratio and

r".-t 1.~1';.~~IIY I.. H ~J

Cat" L\ill::.'<I;) 1 r r I J.~ H -; :!!"':rii Ul ,; .•~ II Y .~ 1; H = ~Gri,· -t'~rt ,.jCHt:UJL.c: all, Jr ,.°ri",j ?C::rt CJ:i.ri~.~r LC:JgL - J3, ur Si~~S iriUM ~JI{ri~Nr I IJ J ~ .s,, ur UI~C:CIIJ~ CH~~~';~ =.i

d,; ~ J.~.\d .~~ Jr" 1'14.1.-..1 rlJ.II.

~0 21 JI JJ9 ~9 13 13 16 II 13IJ 13 11 14 13 II 11 IJ II I·J

~/U· 131 ,,- 6866 16 II I:> 1 J IJ 13 33 I' 1214 ;:?j I~ 10 I J b 32 II 13 14

:1,/U- 131 ~- 68,~ 16 II 32 I ~ 13 b II 14 131~ 13 II b I J 1 J II IJ I J 13

I\/u" I J1 ,,- :,)8

Y3 14 14 13 II 01 6, 14 14 l aII 12 13 I J II II IJ I ~ 16 10

4,/U- 131 ~- u'lY9 323 1J 3, 11 13 1J I J :21 1413 ~ 13 <4 I J I I 13 14 16 I J

~/U= 136 ,,- o!:s61 IJ 14 41 l3 14 13 12 13 IIII b 2/ II 12 II ., 16 II 11

A/lJa I > .. 6~

62 14 14 • 1 IJII 21 .~

.v

~I ,9 1013 1> 13 32 10 16

oJ- 131 ~- 6966 13 12 I. 12 12 12 16 13 1314 14 16 14 Y 14 9 13 13 16

A/l)a 131 ~- C>d1;4 10 20 13 14 I' II II 14 II

12 88 15 12 I:> 1/ II 14 II 41A/U- 138 ri- 69132 14 16 b 18 14 II 12 II b

13 36 10 14 13 11 24 10 9 9AID- 138 ~- 09131 12 12 12 19 II h II II 15

13 23 II 13 13 13 13 13 10 I"~/1l· 131 ~- 09

/ IUIP~P- 28 Mli:A.~ 1<1 I = 9 3SU-1P<lP riAI W- ".933 H 36.303 J'l

<I- .". til,

y- -.JHX + 2.6;2

('1:: '"HULIJ- 390121 J'l

Figure 4. Segment of printout of experimental results. TheIRis are expressed in tenths of a second. R = resistance inK ohms, M= mean IRI, 3 SD = 3 standard deviations above themean IRI. Input from the experimenter is underlined on theprintout.

I FoIl N

I •2 •3 •

• •5 •

• 37 IJ

• 239 5310 I'SII I"""12 04""13 39104 32IS 2016 12t 7 III. 519 22. •21 322 •23 I2. •25 22' 121 •2. I29 •3. •Jl •32 •33 •3. I35 •3' •31 •

3' •39 •

•• •.1 •• 2 •• 3 ••• 1.5 •

•• •., .•• •.9 •5. •51 •52 •53 •5. •55 •5' •

" .5. •59 •

•• •., .

TOTAL NUM8ER Of RESPONSES

o S 10 I ~ 20 25 30 35 40 <filS 5~.c"200, .... , .... , .... , ...." ..., ...., ...., .... , .... ,.... ,....,....

.....·.·.·

150 CASSENS, SHAW, DUDDING, AND MILLS

EXPERIMENTAL APPLICAnONS

This program and system have been used thus far tomeasure the thresholds of self-stimulation in themidbrain of rats. The brain site under investigationhasbeen an ascending noradrenergic fiber tract whichsupports self-stimulation. Our research plans includemeasuring thresholds of IeR before and afteradministration of psychoactive drugs and lesions whichaffect the levels and/or metabolism of endogenousnorepinephrine. Our experiences with on-lineexperimentation have been rewarding. We anticipatefurther application of this behavioral system to thestudy of the neural mechanisms underlyingreinforcement.

REFERENCESCassens, G., & Mills, A. Lithium and amphetamine: Opposite

effects on threshold of intracranial reinforcement.Psvchopbarmacologta, 1973, 30, 283-290.

Ferster, M. & Skinner, B. Schedules of reinforcement. NewYork: Appleton-Century-Crofts, 1957.

Huston, J., & Mills, A. Threshold of reinforcing brainstimulation. Communications in Behavioral Biology, 1971, 5,331-340.

Valenstein, E. Problems of measurement and interpretation withreinforcing brain stimuluation. Psychological Review, 1964,71,415-437.

Wayner, M., Peterson, R., & Florczvk, A. A constant currentdevice for brain stimulation. Physiology and Behavior, 1972,8,1189-1191.

NOTES1. A previous publication (Cassens & Mills, 1973) arbitrarily

defined a PRP as an interval exceeding 7 sec, because without acomputer we were unable to calculate the mean and standarddeviation of responses in real time.

2. An additional routine, currently in preparation, will permitdirect output of the experimental data in ASCII format toDECtape, while preserving the TTY printout. The ASCII formatwill be compatible with OS/8 BASIC.