Applications of the Matching Law

Behavioral Contrast

• Behavioral contrast: often found "side effect“: original study: Reynolds, 1961– pigeons on CONC schedules of reinforcement with equal

schedules at first

– then, extinguish reinforcement on one alternative

– got HUGE change in responding for non-EXT alternative

• why? Behavioral contrast- changed value of schedule

• also called the Pullman effect!!!!

Behavioral Contrast• Helps explain "side effects" of reinforcement:– e.g.: EXT boy talking to teacher during class, but

then kid talks more to peers

• Why?– P1/P2=R1/R2 100/100 = 100/100

– But then one option goes to EXT

– P1/P2=R1/R2 100/100 = 100/0?

Behavioral Contrast

• Example: boy talking to teacher during class, so teacher puts the talking on EXT– but then kid talks more to peers

– Look at ratios:

P1 = R1

-- --

P2 R1

Behavioral Contrast• lets plug in values:– before, talking to teacher highly valuable: P1/P2 =

100/50

– now: talking to teacher is not valuable: P1/P2 = 1/50

• alternative is much more "preferable" than in original situation– If alter Ro, get similar changes!

Can mathematically predict• Responses

– P1 = staying in seat– P2 = out of seat

• Rewards– R1 = rewards for staying in seat– R2 = rewards for being out of seat– Ro = reward for playing around in seat

• What happens as we vary each of theseP1 = R1/R1+R2+Ro

-----------------------------P2 = R1/R1+R2+Ro

Using the matching law

• Matching law tells us 2 things:– Sensitivity of an animal to changes in reward ratios– Any inherent or incidental bias an animal may have

towards one response alternative

• We can use the GML to examine differences in reward sensitivity and bias – Across organisms– Across conditions– Across internal states

Noncontingent Reinforcement

• Can be a behavioral disruptor OR a technique used to invoke behavior change

• An example: unconditional positive regard = free, noncontingent reinforcement– will reduce frequency of undesired responding

– BUT, will also reduce behaviors that may want!!!

Changing the reinforcer context: 3 alternatives

• Increase/decrease rate of contingent reinforcement (increase R1)

• Increase/decrease rate of concurrently available reinforcement of one alternative (Decrease R2)

• Increase/decrease rate of free, noncontingent reinforcement (Ro).

Dealing with Noncontingent reinforcement (Ro)

• This can be a nice tool:– works well in rich environments where have more opportunity to alter reinforcement rates.

• Not have to add reinforcers, but can alter rate of contingent reinforcement:– DECREASE contingent reinforcement to both alternatives

• Does not alter ratio of reinforcement situation

• Can avoid satiation/habituaton

– INCREASE NCR for non-target response• E.g.: talking out vs. staying in seat

• Give lots of noncontingent reinforcement for any appropriate behavior

• Do not have to alter reinforcement for out of seat

• invokes satiation/habituation- getting alternative sources of free reinforcers, so why work to get it!?!

• allows for contextual changes in reinforcement

Effects of Noncontingent Reinforcement: NCR in Humans: Ecott and Critchfield

• NCR: Response-independent delivery of a reinforcer (noncontingent reinforcement or NCR) – Become widespread intervention in recent years, – along with extinction, differential reinforcement of other behavior (DRO), and differential

reinforcement of alternative behavior (DRA), – Is considered a highly effective and acceptable form of treatment (Carr et al., 2000). – NCR effects appear to be due to both satiation of reinforcer and extinction effects

• NCR has distinct advantages over these other procedures:– NCR is unlikely to produce side effects sometimes observed during extinction

• E.G., response bursting and aggression• Seems to be because NCR does not involve the elimination of access to reinforcers.

– NCR often suppresses problem behavior more rapidly than do DRO and DRA • no behavioral requirement for the delivery of reinforcement.

– NCR schedules are easier to implement than are DRO and DRA schedules • therapists do not have to carefully monitor the occurrence of either problem or alternative behavior• they simply deliver reinforcers at the appropriate time.

Effects of Noncontingent Reinforcement: NCR in Humans: Ecott and Critchfield

• BUT: are situations in which NCR may produce undesirable effects. – Get response suppression of other responses now that there is

noncontingent reinforcement– Subject may discover that reinforcement is free!– How much or how little NCR is required is debated.

• Problems may occur at different stages of treatment: – initial response suppression– maintenance, – acquisition of alternative behaviors, – may require the use of alternative procedures in conjunction

with or in place of NCR.

Ecott & Critchfield study• 6 undergrads; worked up to 10 hours alone in small room with desk, VGA color monitor and mouse

• Reinforcers:– Course credit linked to experimental task that established points as reinforcers– Seconds of participation time could be accumulated during experimental tasks– Messages would indicate when points were awarded

• Task:– Click here to begin– Screen divided into 2 rectangles with prompt above: total seconds earned this session– Counter started at 0– Each screen region contained small colored square that moved around screen at random directions at 0.5 cm/sec– Click on the moving squares registered as responses– Earned points for time spent responding– 8 min sessions, 9-12 sessions per laboratory session

• Schedules:– When red flashing prompt occurred, click inside a box on center of screen caused 20 s to be added to subject’s total– Baseline: VI 10 s VI 30 s, 2 sec COD– Noncontingent reinforcement sessions:

• NCR: reinforcement schedule for alternative behavior remained unchanged, • randomly determined portion of VI 10-s reinforcers that had been contingent on target behavior delivered independent of responding.

– 100% non contingent– 67%– 33%– 0%

– Points for time accrued on their own during this phase (no click to center box needed)

– 3 subjects had ascending series; 3 subjects had descending series

– Used stability criterion

Results:

• Response rates for target and alternative behaviors across conditions:– 4 of 5 participants: as percentage of NCR

increased, • target-behavior rate tended to decrease• Alternative behavior rate increased• 5th subject more ambiguous: showed trend on

ascending series only

– These results similar to prior investigations

Results:

• Alternative reinforcement effects:– Alternative-behavior changes = part of pattern of response

competition that resulted in decreased target behavior

• To show this: examine point deliveries in each condition– Look at engagement in 2 types of behavior

• Location of response that occurred prior to point delivery, regardless of whether was NCR or CR

• When target side converted to NCR: increase rate of point deliveries associated with altnerative bheaivor, decrease target point behavior

• As target side point deliveries converted to NCR, # of CR point deliveries decreased and NCR point deliveries increased

• Opposite for alternative side point deliveries

Deriving a and b

• Calculated reward sensitivity and bias using the GML– Plotted the log reinforcer ratio (Target/alternative) as a

function of the log response ratio (Target/alternative)– Results showed:

• Higher a parameter for all participants when NCR point deliveres considered in analysis than when omitted

• This suggests that NCR point deliveries did contribute to response patterns

• That is: – NCR adventitiously strengthed both target and alternative beahviors,– Reductions in target behavior were related to changes in

reinforcement of alternative behavior

Why is this important?

• NCR effects are important:– May facilitate both target and alternative responding

– But: interplay between the two as use NCR

– Changes in target behavior are affected by changes in reinforcement of alternative behavior, and vice versa

– Shows importance of context

Matching Law in dogs

• Three HVI dogs– Zoomba: Australian Shepherd, age 4, congenitally deaf– Moe: Australian Shepherd, age 3, congenitally deaf, low vision– Keller: Australian Shepherd, age 2, congenitally deaf/blind

• Three NHV dogs– Seamus: age 9, golden retriever, normal senses– Harry: age 9 mos, German Sheperd/Alaskan Klee Kai, normal

senses– Max: age 2, Australian Shepherd, normal senses

Basic set up:• A 4x8 test area was denoted on the middle of the Canine Behavior and Cognition laboratory

floor.

• The test area was sectioned off into two 4’ x 4’ rectangles using tape to delineate each area.

• A chair was placed in each of the two larger rectangles, Two research assistants sat in the chairs, holding a treat cup of dog treats.

• The research assistants delivered treats according to the reinforcement schedule outlined below.

• A third research assistant will sit approximately 5 feet from the platform and recorded the time the dog spent in each of the 2 rectangles, or time away from the test area using a chess clock timer.

Rectangle 1 Rectangle 2

Brief Method• Procedure.

– Dogs participated in 3 to 5 10-minute sessions at each of the four reinforcement schedules. – Each dog was given 10 to 15 minutes to acclimate to the testing area prior to a testing trial.

• Training Trials. – Discrete training trials consisted of the dog being led to each of the two research assistants (seated on the chairs). – When the dog was within the appropriate rectangle, the research assistant provided a food reward – The dog was led to each of the research assistant three times, in random order, such that the dog has the opportunity to sample the available

reinforcement for that session.

• Matching Law Trials: – Each trial session consisted of a 10-min. session in which the dog was provided with two food reward opportunities.– Research assistants signaled the availability of a food reward by stomping their foot on the floor (enabling both NVI and HVI dogs to attend to the

reinforcer cue).– Food reward was given when the dog entered a rectangle area according to the schedules outlined below. – A contingency was in place such that the dog must enter the rectangle area in order to receive the available reinforce.

– The order of the 4 schedules was presented in random order for each of the dogs.• Schedule 1: 1:4 ratio. VI 15 VI 60 sec• Schedule 2: 1:2 ratio. VI 15 VI 30 sec• Schedule 3: 4:1ratio. VI 60 VI 15 sec• Schedule 4: 2:1ratio VI 30 VI 15 sec

• The time spent in each rectangle and the number of reinforcers consumed for that rectangle were recorded for each session and for each dog.

Results• The average time spent each rectangle and the average number of

reinforcers consumed in that rectangle were calculated for each of the four schedules for each dog.

• Using the Generalized Matching Law equation, values of reward sensitivity, bias and goodness of fit (r2) were obtained for each of the 6 dogs. These are shown for each dog in figures 1through 6 below.

• The mean reward sensitivity, bias and goodness of fit (r2) was obtained for the NVH and HVI groups.

• Mann-Whitney U test was used to compare differences in reward sensitivity, bias and goodness of fit (r2) for the two groups.

– significant differences between the groups for reward sensitivity (z=-1.96, p=.05) – not for bias (z=0.66, N.S.) – or goodness of fit (r2) (-1.16, N.S.).

Conclusions• Results showed that HVI dogs were more sensitive to changes in reward than the NHV dogs.

– All the dogs showed good reward sensitivity, that is, their a values approached 1.0.

– The HVI dogs showed values consistently higher than 1.0. however. • dogs spent more time at the better alternative than would be predicted by the scheduled ratio of reinforcers. • That is, they valued the choice with the higher rate of reward as more valuable than predicted. This is called

overmatching.

– Overmatching occurs when the subject appears to be "more sensitive" to the differences in reinforcement

between the two alternatives than when matching occurs.

• Supports hypothesis that HVI dogs should be more attentive and thus more sensitive to changes

in reward ratio than NHV dogs. – HVI dogs should be more attentive because they lack normal vision and hearing input – Hyper-attend to their environment. – Given that the HVI and NVH dogs should not differ genetically, these differences must be due to

differences in experience as a result of the differences in perceptual ability.

• Results also support previous data from out laboratory – HVI dogs attend more to humans for a pointing task, and a bigger/smaller choice task. – HVI dogs are more strongly attached to humans and show greater separation anxiety. – HVI dogs show deficits in social interactions during play both with HVI and NHV dogs.

Applying the GML to Antisocial Behavior

• Negative reinforcement plays important role in development and maintenance of aggressive and antisocial behavior– Preschooler, school aged boy’s aggressive

behavior in interactions with mother– Maintained by TERMINATION of maternal

behavior that otherwise would have caused conflict to continue

– E.g.,: demand for compliance

Applying the GML to Antisocial Behavior

• By adolescence: antisocial behavior well shaped– Preschool years: • coercion • Termination of demands from peers/teachers when

acted aggressively

– School years: behavior continues to be reinforced– By adolescence: peer interactions become most

important, more than teachers

Interesting Change in reinforcement

• Shift in reinforcement ratio– Preschool: Proportion of time allocated to interactions with

peers correlates with proportion of positive peer interactions– School years: begin shift towards relation between time

allocated to interaction with peers becomes correlated with negative peer interactions

– By adolescence: this is negative pattern is well established

• Is a shaping process! Learning to use antisocial behavior via interactions with parents, teachers, peers.

So what does this mean?

• Negative social reinforcement maintains antisocial behavior in a manner that can be described by the matching law– Acting bad gets you out of things

• Positive social reinforcement also maintains antisocial behavior and can be described by the GML– Acting bad gets you things

• These reinforcement contingencies develop over the child’s reinforcement history

McDowell and Caron:GML and Antisocial boys

• Oregon Youth Study– Behavior broken into 2 categories• Rule-breaking• Normative

– The equation accurately described rule-break vs. normative talk of the boys

– Accounted for 87-97% of the variance!

Proportion of rule-break talk as a function of obtainedReinforcement proportion for rule-break talk

Exponents a and b plotted as a function of child deviance quartiles

Conclusions about deviance behavior• GML described the relationship between deviant

behavior and reinforcement well

• Reward sensitivity decreased as child deviance scores increased!– Suggests that the more deviant the boy, the less the

boy preferred positive reinforcement for normative talk– Again suggests that history important: long history of

attention and negative reinforcement for rule break behavior may lessen sensitivity to positive reinforcement as child becomes more deviant

Conclusions about deviance behavior

• Boys’ verbal behavior was strongly biased towards normative talk! Likely due to:– Positive reinforcement for normative talk– Punishment for rule-break talk

• Bottom line: if the pattern of negative reinforcement/punishment can be altered, then this may alter the likelihood of a child developing more antisocial tendencies!

Conclusions: Clinical applications of the GML

• MUST consider broader environmental conceptualizations of problem behavior

• Must account for sources of reinforcement other than that provided by therapist– again- Herrnstein's idea of context of

reinforcement– if not- shoot yourself in the old therapeutic foot

Take Home Message:

• It is the disparity between 2 relative rates of reinforcement that is important

• Must consider Ro

• Context matters!