Sniffy (AKA Mitzy 2.0)

Meet Sniffy

Magazine Training + Shaping

Goal 1: Magazine train Sniffy. Use a classical conditioning procedure to form an association between a consequence and a secondary reinforcer (some stimulus that it typically neutral). In this case, I reinforced Sniffy with food pellets in such a way that allowed him to learn the association between the food sound and the reward of a food pellet in the magazine.

Procedure: I completed magazine training by delivering a food pellet whenever Sniffy approached the food hopper, allowing him to wander for a bit before giving him the next pellet.

Discussion: At first, Sniffy would not regularly go to the magazine when I administered a pellet. He might continue walking around or exploring whichever part of the box he was in. After some time, I would administer a pellet only when he was very close to the magazine or was heading directly to it, and he began to more quickly check the magazine after hearing the food sound.

Goal 2: Shape Sniffy to produce the desired instrumental response (bar pressing) by reinforcing successive approximations of this behavior.

Procedure: I completed shaping by gradually reinforcing behaviors that resembled or led up to bar pressing. At first, I would administer a pellet when Sniffy approached the back corner (magazine or bar), and as he began to form an association with general behaviors in the back corner, I narrowed the range of behaviors such that he would then have to rear above the bar, and then sniff, touch, press (etc) the bar in order to receive a pellet.

Discussion: Overall, this process took a lot longer than I thought it would, and it required very careful observation on my part. I noticed that if I allowed too much of a delay to occur between the target behavior and the food reward, Sniffy would often reproduce another behavior he had done since then (that wasn’t desired), and the Bar-Sound and Action-Strength measurements would decrease.  

This figure contains the cumulative records for Sniffy's magazine training (first bar, small portion of second bar), and shaping for bar pressing (starting from yellow highlighted "CRF Press Bar" section).


Variable Ratio + Extinction

Goals 3 & 4: Place Sniffy on a Variable Ratio schedule, and ultimately extinguish the behavior through extinction.

Procedure: After Sniffy had formed very strong associations between the bar and the food sound (“Bar Sound” meter) and between bar pressing and the food reward (“Action Strength” meter), I placed him on a variable ratio schedule. I began with VR5 and worked up to a VR50. Then, I simply set up the program to run extinction, and ran it so that Sniffy received no rewards for his bar presses.

Discussion: Sniffy’s rate of learning was very quick using VR schedules. Compared to his behavior in shaping, he took far fewer and shorter breaks (grooming, exploring), and his focus was more or less fixated on the bar throughout the course of training. After his first VR schedule (VR5), he was pressing the bar very continuously and quickly for the rest of training. 

This figure contains the cumulative record for an early part of Sniffy's variable ratio training. This VR5 record demonstrates an increase in the rate of response (graph becomes more steeply sloped).

In extinction, Sniffy initially would press the bar repeatedly, and take his hands off the bar to sniff inside and around the food hopper. Additionally, Sniffy’s rate of bar pressing seemed to increase in the first few minutes of extinction (extinction burst). However, as the association was further extinguished, he would typically press the bar a few times (to no avail), and come off of it to groom himself or sniff another region of the op box. I stopped the session after a 5-minute interval had passed in which Sniffy had pressed the bar only once.