The Rat Days are Over

Overall, I really enjoyed the training process. Although it was difficult at times to schedule other things around my training schedule, I was actually surprised by how much progress I saw in the relatively small amount of time that Mitzy and I spent training. I was so impressed with Mitzy's rate of learning, especially on her first few days of training. She seemed to very quickly form an association between the response and consequence (reward), and I felt like a very proud psychology student/ trainer/ rat mommy. I was also surprised when I was somewhat disappointed that it was over!

I really liked being able to work with a live animal, which I was initially nervous about--not because I disliked rats, but rather, because I was worried about what kind of conditions we might have to subject the rats to. In general, I think that I had some very misguided notions about what using animal subjects is like. Spending time in the rat room, however, and seeing how they were all treated by Dr. Trench, Devon, and all of the rest of the students, greatly improved my attitude toward using animal subjects. As far as I know, everyone really bonded with their rats, and I think that Dr. Trench and Devon are pretty protective of them :P

Over the course of this project, I continued to look forward to hanging out with Mitzy for a few minutes after each training session, even though she tended to be kind of antzy at this point. I never had any problems with her biting me, and she seemed perfectly comfortable with being picked up and handled. Mitzy was a great subject, and she made my job pretty easy most of the time. I really enjoyed carefully observing Mitzy during shaping to figure out exactly when or what to reinforce, and I appreciated being able to watch her progress on her own for the rest of training. After shaping, I just sat back with a notepad and a pencil and let the magic happen! :)

Love,

Lindsey and Mitzy

PS: In case bar pressing gets a little bit old, here's some inspiration for the next training project:

Running the Rat Race: Mitzy vs. Sniffy

Overall, magazine training and shaping Mitzy were both faster and more enjoyable processes than magazine training and shaping her virtual counterpart, Sniffy. For Sniffy, magazine training took approximately 50 minutes, and shaping required a little more than an hour. For Mitzy, magazine training took very little time (approximately one 30-minute session), and shaping was completed in two full training sessions (so, after the 3rd day). 

However, one difference that I must note in comparing magazine and shaping durations between the two rats is that in Mitzy’s case, I more or less combined the two processes in our first training session. That is to say, by the end of Day 1, I had already begun to shape Mitzy to touch the bar, and I stopped reinforcing behaviors having to do with the hopper. I am glad that Devon recommended this approach, because I believe that it accelerated Mitzy’s rate of learning, and it also allowed both associations (i.e. sound/food and bar press/food) to become strengthened simultaneously.  

Shaping the two rats was very similar in terms of the successive approximations that I reinforced. That is to say, after “practicing” with Sniffy, I used the same procedure to shape Mitzy: reward for sniffing the back corner, reward for rearing above the bar, reward for touching the bar with hand or nose, reward for touching the bar with both hands, etc. Both rats emitted similar behaviors of the response class (i.e. group of movements similar to bar pressing), even though they varied in the time required to form a strong association.

One thing that I could not learn from shaping Sniffy was how to deal with unwanted or “extra” behaviors. Although Sniffy would rear up or sniff the bar every now and then in his VR training, he would only receive a reward for a normal bar press. On the other hand, Mitzy demonstrated a large amount of such behaviors, and they would be reinforced as long as the bar was depressed when she was doing them.

Additionally, Sniffy very rarely became distracted for more than ~20 seconds at a time, so training him did not shed any light on what I should do when Mitzy decided to wander off for 3 minutes at a time. Mitzy’s cumulative records, therefore, display somewhat less steady rates of response than do Sniffy’s. (However, we are also comparing FR and VR schedules, so Sniffy’s should appear more steady).

Related to this is the fact that Sniffy could be trained for 2 hours in a row, whereas Mitzy often lost motivation after ~20 minutes. This, however, did not necessarily impair what I observed in training the rats or what I learned from the process, but it is an important difference in the two processes.

Lastly, a major difference between the two extinction processes was that Sniffy does not feel frustrated as a live rat likely feels after she is no longer rewarded for her instrumental responses. Although I saw somewhat of an extinction burst from Sniffy, his overall extinction results were not caused by any element of frustration as were Mitzy’s.

Overall, I appreciated having the opportunity to practice on Sniffy before training my live rat. The comments in the book and in the program were very helpful, and I learned a great deal from shaping in particular. Seeing Sniffy’s rate of learning in the shaping process allowed me to modify my own technique—how “picky” I should be in reinforcing behaviors and at what rate, how quickly I should administer a food pellet, and so on.

In conclusion, what it really comes down to is this: Mitzy is way cuter than Sniffy.

The End.

Sniffy (AKA Mitzy 2.0)

Meet Sniffy

Magazine Training + Shaping

Goal 1: Magazine train Sniffy. Use a classical conditioning procedure to form an association between a consequence and a secondary reinforcer (some stimulus that it typically neutral). In this case, I reinforced Sniffy with food pellets in such a way that allowed him to learn the association between the food sound and the reward of a food pellet in the magazine.

Procedure: I completed magazine training by delivering a food pellet whenever Sniffy approached the food hopper, allowing him to wander for a bit before giving him the next pellet.

Discussion: At first, Sniffy would not regularly go to the magazine when I administered a pellet. He might continue walking around or exploring whichever part of the box he was in. After some time, I would administer a pellet only when he was very close to the magazine or was heading directly to it, and he began to more quickly check the magazine after hearing the food sound.

Goal 2: Shape Sniffy to produce the desired instrumental response (bar pressing) by reinforcing successive approximations of this behavior.

Procedure: I completed shaping by gradually reinforcing behaviors that resembled or led up to bar pressing. At first, I would administer a pellet when Sniffy approached the back corner (magazine or bar), and as he began to form an association with general behaviors in the back corner, I narrowed the range of behaviors such that he would then have to rear above the bar, and then sniff, touch, press (etc) the bar in order to receive a pellet.

Discussion: Overall, this process took a lot longer than I thought it would, and it required very careful observation on my part. I noticed that if I allowed too much of a delay to occur between the target behavior and the food reward, Sniffy would often reproduce another behavior he had done since then (that wasn’t desired), and the Bar-Sound and Action-Strength measurements would decrease.  

This figure contains the cumulative records for Sniffy's magazine training (first bar, small portion of second bar), and shaping for bar pressing (starting from yellow highlighted "CRF Press Bar" section).


Variable Ratio + Extinction

Goals 3 & 4: Place Sniffy on a Variable Ratio schedule, and ultimately extinguish the behavior through extinction.

Procedure: After Sniffy had formed very strong associations between the bar and the food sound (“Bar Sound” meter) and between bar pressing and the food reward (“Action Strength” meter), I placed him on a variable ratio schedule. I began with VR5 and worked up to a VR50. Then, I simply set up the program to run extinction, and ran it so that Sniffy received no rewards for his bar presses.

Discussion: Sniffy’s rate of learning was very quick using VR schedules. Compared to his behavior in shaping, he took far fewer and shorter breaks (grooming, exploring), and his focus was more or less fixated on the bar throughout the course of training. After his first VR schedule (VR5), he was pressing the bar very continuously and quickly for the rest of training. 

This figure contains the cumulative record for an early part of Sniffy's variable ratio training. This VR5 record demonstrates an increase in the rate of response (graph becomes more steeply sloped).

In extinction, Sniffy initially would press the bar repeatedly, and take his hands off the bar to sniff inside and around the food hopper. Additionally, Sniffy’s rate of bar pressing seemed to increase in the first few minutes of extinction (extinction burst). However, as the association was further extinguished, he would typically press the bar a few times (to no avail), and come off of it to groom himself or sniff another region of the op box. I stopped the session after a 5-minute interval had passed in which Sniffy had pressed the bar only once. 

Extinction: Not Just for Dinos

When Mitzy started the session, she immediately began to press the bar rapidly and persistently (as was reinforced in previous training). Throughout the session, she also demonstrated a great deal of spontaneous recovery of previously-enforced behaviors. As I mentioned before, bar biting was a problem throughout the two weeks of training, but in extinction, she displayed a dramatically greater rate of such behaviors. Mitzy was biting the bar, holding it down, pressing up on it with her nose, and even biting the magazine as she was initially shaped to do in the first days of training.

In addition, she appeared quite agitated at the beginning of both extinction session, jumping and rearing around different areas of the box. However, after some time (~10 minutes), she would begin to more calmly explore the box, stopping to sniff or groom. This suggested to me that the instrumental response was likely on its way to extinction (or at least, was getting closer).

The following video was taken in the 10-15 minute interval on Day 1 of extinction:

Although I did not see full extinction of the behavior, I believe that it would have been (mostly) reached if we extinguished for more than 2 days.

As you can see from the graph, Mitzy showed a dramatic decrease in response for the first 20 minutes of each extinction session, and then experienced a bit of an extinction burst in the 20-25 minute interval. In addition, she began her second day of training with a fairly large number of bar presses, but demonstrated an overall decrease from the previous session.

Extinction Day 1 Cumulative Record

Extinction Day 2 Cumulative Record

Mitzy Results: Weight & Bar Press Graphs

Average Number of Responses per 30 minutes
Shaping, Fixed Ratio, and Variable Ratio Schedules

Weight Chart
Target Weight: 206 grams

Date	Weight (g)	Food (g)
12-Sep	237.0	3.3
13-Sep	225.6	4.0
14-Sep	219.9	5.3
15-Sep	219.0	4.5
16-Sep	214.4	6.8
17-Sep	214.3	4.8
18-Sep	207.9	6.1
19-Sep	205.7	7.2
20-Sep	210.0	6.0
21-Sep	212.7	3.8
22-Sep	209.6	5.0
23-Sep	211.0	3.3
24-Sep	206.6	4.6
25-Sep	206.3	7.0
26-Sep	206.5	6.5
27-Sep	206.9	5.7
28-Sep	201.2	7.9
29-Sep	204.3	7.0
30-Sep	203.6	6.9
1-Oct	201.6	8.4
2-Oct	203.5	6.5
3-Oct	200.5	8.7
4-Oct	202.9	8.8
5-Oct	204.3	Free feed

Trouble in Paradise

Rats Gone Wild!

As noted throughout my posts, one of the problems I had in training Mitzy was "extra" behavior and effort that she put into her bar presses. In early FR training, Mitzy would put increasing effort into each of her presses. For example, in FR3, she would (1) press the bar normally, receiving no food, (2) press and hold the bar for a few seconds, still receiving no food, and (3) press. hold, and bite the bar, finally receiving her food. I was worried that, as this behavior continue to earn her rewards, she would form an association between her "added effort" and the consequence (receiving a pellet). 

After talking to other students, I came to see that variations of this behavior are a common problem in training, and we briefly discussed this in class. Overall, due to Mitzy's gradual increase in response rate and her large number of bar presses per session, I concluded that this was not a major interference in her learning process. However, I think that if I had seen this pattern of behavior sooner (e.g. in a much lower FR schedule), I likely would have switched to manual reinforcement on whatever our current schedule was. This way, I could have reinforced every 3, 5, 7, etc bar presses only, rather than letting the operant box automatically reinforce any behavior that resulted in a bar press. 

Day 13: VR10

10/3/12 
(notes taken on day of training)

-------

Goal:
Attempt a variable ratio schedule for the first time. I hope to see a steadier pattern of responding than demonstrated in FR, as well as a decrease in predictable pauses.

Discussion:

I was somewhat surprised by the results of this schedule. Although Mitzy's response rate was often quite steady, I did not perceive it to be very different from her higher ratio FR schedules. She took several long breaks (e.g. a few minutes at a time) to explore the box, groom, etc. Although she stayed fairly motivated throughout the session and had a very high number of total responses, her cumulative record displays [what seem to be] post-reinforcement pauses.

In addition, I was not sure how to start Mitzy on her first day of a variable ratio schedule. After leaving off on FR12, I was unsure as to whether I should reduce the # of responses required (e.g. move from FR12 to VR5), or simply continue from there (e.g. from FR12 to VR12). After consulting my class notes, I decided that it was not necessary to reduce the ratio dramatically, as she had already learned to press the bar repeatedly for a treat, and had been training for almost two weeks at this point. However, as the results were not what I expected, if I were to try this again, I would probably reduce the ratio when starting a VR schedule.

Overall, Mitzy appeared to do well on the VR schedule, and the number of bar presses was a record high (yay!). I would have liked to spend a few more days on different VR schedules in order to more clearly see a difference in the cumulative records between FRs and VRs, but alas, time for extinction!

Bar Presses: 618
Reinforcements: 61
Run Time: 30 minutes