ProbReversalTask

Probabilistic Reversal Task

Overview

The Probabilistic Reversal Task configures FED3 to deliver food pellets based on probabilistic reward schedules that switch sides after a set number of trials. One port is designated as the “high-probability” port (e.g., 80%), while the other is the “low-probability” port (20%). After a block of trials, the contingencies reverse, requiring the animal to update its behavior. This task combines features of probabilistic choice and reversal learning, making it conceptually similar to the two-armed bandit task.

Theory

Probabilistic reversal learning tasks are widely used to study cognitive flexibility and decision-making under uncertainty. The animal must learn that one port is more likely to deliver food, but because reinforcement is probabilistic, even the “best” choice sometimes fails. After a set number of trials, the contingencies reverse, and animals must adapt by shifting their responding to the new high-probability port.

This design is closely related to the two-armed bandit task, which also requires animals to balance exploration (sampling both options) against exploitation (sticking with the more rewarding port). The key difference is that in reversal tasks, the high- and low-probability assignments switch after a fixed number of trials, whereas in many bandit tasks the probabilities may drift gradually or remain stationary. Both paradigms are used to measure reinforcement learning, flexibility, and the neural circuits that mediate adaptive behavior.

Considerations

• Probability setting: Default is 80% vs. 20%, adjustable with the probability variable.
• Reversal schedule: Controlled by trialsToSwitch; after this many trials, the high/low assignments swap sides.
• Timeouts: After each poke, a short timeout (trialTimeout) can be enforced to space trials.
• Randomness: Because outcomes are probabilistic, animals must sample multiple times to detect contingencies.
• Analysis: Win-stay/lose-shift patterns, latency to switch after reversal, and bias are common metrics.

Key Features

• Two operant ports with probabilistic reward delivery.
• High-probability side (default 80%) vs. low-probability side (20%).
• After a fixed number of trials, probabilities reverse.
• Cues (tone + lights) on reward, error tones on nonreward.
• Logs all pokes and outcomes for detailed analysis.
• Similar to the two-armed bandit task, but with fixed block reversals instead of drifting probabilities.

Code Highlights

int probability = 80;                  // Probability of reward on high-value side
int trialsToSwitch = 10;               // Number of trials before reversal
int counter = 1;                       // Trial counter
int trialTimeout = 5;                  // Timeout after each poke (seconds)

if (counter <= trialsToSwitch) 
  fed3.activePoke = 1;                 // Left = high-probability
if (counter > trialsToSwitch) 
  fed3.activePoke = 0;                 // Right = high-probability

// Case 1: Left is high-probability, poke left
if (fed3.activePoke == 1 && fed3.Left) {
  fed3.logLeftPoke();                  
  if (random(100) < probability) {     // ~80% chance reward
    fed3.ConditionedStimulus();        
    fed3.Feed();                       
  } else {
    fed3.Tone(4000, 200);              // Error tone on nonreward
  }
  counter++;                           
  fed3.Timeout(trialTimeout);          
}

// Case 4: Right is high-probability, poke right
if (fed3.activePoke == 0 && fed3.Right) {
  fed3.logRightPoke();
  if (random(100) < probability) {     // ~80% chance reward
    fed3.ConditionedStimulus();
    fed3.Feed();
  } else {
    fed3.Tone(4000, 200);
  }
  counter++;
  fed3.Timeout(trialTimeout);
}

Usage

1. Upload this sketch to your FED3 using Arduino IDE.
2. Observe behavior and log file output.
3. Modify probabilities of both side as needed for your experiment.

Logging

• Logs pellet events, poke activity, timeouts, and more depending on sketch.

Full Code

/*
  Feeding experimentation device 3 (FED3)
  Probabalistic reversal task

  Starts with the left poke resulting in a pellet N% of the time and the right poke resulting in a pellet
  100-N% of the time, with N defined by the variable "probability".  
  For instance, if N = 80, a left poke delivers a pellet 80% of the time and a right poke causes a pellet 20% of the time

  After a defined number of pokes the probabilities on the right and left pokes switch.  This is set
  by the variable "trialsToSwitch".

  Code written by alexxai@wustl.edu and meaghan.creed@wustl.edu
  December, 2020

  This project is released under the terms of the Creative Commons - Attribution - ShareAlike 3.0 license:
  human readable: https://creativecommons.org/licenses/by-sa/3.0/
  legal wording: https://creativecommons.org/licenses/by-sa/3.0/legalcode
  Copyright (c) 2020 Lex Kravitz
*/

#include <FED3.h>                                       //Include the FED3 library 
String sketch = "Prob_Reversal";                        //Unique identifier text for each sketch, change string only
FED3 fed3 (sketch);                                     //Start the FED3 object - don't change

int probability = 80;                                   //Probability of pellet on the high-reward poke
int trialsToSwitch = 10;                                //# of trials before probabilities on the pokes switch
int counter = 1;                                        //trial counter variable - don't change this
int trialTimeout = 5;                                   //timeout duration after each poke, set to 0 to remove the timeout

void setup() {
  fed3.begin();                                         //Setup the FED3 hardware, all pinmode screen etc, initialize SD card
}

void loop() {
  ////////////////////////////////////////////////
  // Determine which poke is active based on the trial number. 
  // fed3.activePoke=1 means Left poke is active, fed3.activePoke=0 means Right poke is active.
  ////////////////////////////////////////////////
  if (counter == ((trialsToSwitch * 2)+1)) counter = 1;
  if (counter <= trialsToSwitch) fed3.activePoke = 1;
  if (counter > trialsToSwitch) fed3.activePoke = 0;

  fed3.run();                                           //Call fed.run at least once per loop

  ////////////////////////////////////////////////
  // There are 4 cases we need to account for:
  ////////////////////////////////////////////////

  // Case 1: The *left* poke is the high probability poke and animal pokes *left*
  if (fed3.activePoke == 1 and fed3.Left) {
    fed3.logLeftPoke();                                 //Log left poke
    if (random(100) < probability) {                    //Select a random number between 0-100 and ask if it is between 0-80 (80% of the time).  If so:
      fed3.ConditionedStimulus();                         //Deliver conditioned stimulus (tone and lights)
      fed3.Feed();                                        //Deliver pellet
    }
    else {                                              //If random number is between 81-100 (20% of the time)
      fed3.Tone(4000, 200);                               //Play the error tone
    }
    counter ++;                                         //Regardless of the probabalistic outcome increment the trial counter
    fed3.Timeout(trialTimeout);                         //Time-out between trials
  }

  // Case 2: The *left* poke is the high probability poke and animal pokes *right*
  if (fed3.activePoke == 1 and fed3.Right) {
    fed3.logRightPoke();                                //Log Right poke
    if (random(100) > probability) {                    //Select a random number between 0-100 and ask if it is between 80-100 (20% of the time).  If so:
      fed3.ConditionedStimulus();                         //Deliver conditioned stimulus (tone and lights)
      fed3.Feed();                                        //Deliver pellet
    }
    else {                                              //If random number is between 0-80 (80% of the time)
      fed3.Tone(4000, 200);                               //Play the error tone
    }
    counter ++;                                         //increment the trial counter
    fed3.Timeout(trialTimeout);                         //Time-out between trials
  }

  // Case 3: The *right* poke is the high probability poke and animal pokes *left*
  if (fed3.activePoke == 0 and fed3.Left) {
    fed3.logLeftPoke();                                 //Log left poke
    if (random(100) > probability) {                    //Select a random number between 0-100 and ask if it is between 80-100 (20% of the time).  If so:
      fed3.ConditionedStimulus();                         //Deliver conditioned stimulus (tone and lights)
      fed3.Feed();                                        //Deliver pellet
    }
    else {                                              //If random number is between 0-80 (80% of the time)
      fed3.Tone(4000, 200);                               //Play the error tone
    }
    counter ++;                                         //increment the trial counter
    fed3.Timeout(trialTimeout);                         //Time-out between trials
  }

  // Case 4: The *right* poke is the high probability poke and animal pokes *right*
  if (fed3.activePoke == 0 and fed3.Right) {
    fed3.logRightPoke();                                //Log Right poke
    if (random(100) < probability) {                    //Select a random number between 0-100 and ask if it is between 0-80 (80% of the time).  If so:
      fed3.ConditionedStimulus();                         //Deliver conditioned stimulus (tone and lights)
      fed3.Feed();                                        //Deliver pellet
    }
    else {                                              //If random number is between 80-100 (20% of the time)
      fed3.Tone(4000, 200);                               //Play the error tone
    }
    counter ++;                                         //increment the trial counter
    fed3.Timeout(trialTimeout);                         //Time-out between trials
  }
}