Social Experiments Planning & Timeline Doc

[jkbhagatio]

Overarching questions

In a noisy and changing environment, do mice:

1. Establish a social hierarchy?
   a) Change ranks in the social hierarchy?
   b) Gain resources from foraging consistent with their ranking in a social hierarchy?
2. Exhibit social behaviour?
   a) Interact more or less than expected?
   b) Exert more or less energy / power than expected?
3. Share information that maximises their resources gained from foraging?
   a) Learn to explore until finding the best patch?
4. Compete over resources gained from foraging?
5. Learn to cooperate to maximise resources gained from foraging?

Social_1.0 seeks to answer questions 1-4.

Social1.0 requirements

• N of 3 “good” pairs of mice.
• Online, multi-animal tracking.
• Probabilistic patch (wheel threshold) properties:
  • Exponential distribution:
    • Motivation: The Exponential distribution is the maximum entropy distribution amongst all continuous distributions supported on [0, inf).
      • According to the principle of maximum entropy, if nothing is known about a distribution except that it belongs to a certain class (usually defined in terms of specified properties or measures), then the distribution with the largest entropy should be chosen as the least-informative default. The motivation is twofold: first, maximizing entropy minimizes the amount of prior information built into the distribution; second, many physical systems tend to move towards maximal entropy configurations over time.
    • For L, lambda = 1, offset by 0.75.
      • A lower-bound of 0.75 ensures mice have to spin wheel at least 3 full rotations (which, empirically, is at least 3 seconds) to get a reward.
    • To find an ideal lambda for M, we will try out various lambdas and choose the lowest value that meets some minimum metric of discriminability.
      • This exact metric is TBD, but it will be something like comparing distributions of wheel distance travelled on L and M, and obtaining some test statistic which claims that the probability (p-val) that the two distributions are the same is <= 0.01.
    • Once we have an ideal lambda for M, we will repeat the same process to find an ideal lambda for H.
  • Re-sample patch thresholds, without replacement, each time a total of X (30m?) has been spun on wheels.

Timeline

Individual sessions for determining probabilistic patch Exponential distribution parameters (presocial0.1):

• 5 different mice.
• 3 hour sessions.
• One initial session per mouse where they simply learn to engage with patches?
• First M: lambda = (3? or 4?) + offset = 0.75.
  • For lambda 3 v 1: In ~26% of sampled pairs, the harder patch sample will have a value less than the easier patch.
  • For lambda 4 v 1: In ~7% of sampled pairs, the harder patch sample will have a value less than the easier patch.
• For testing M: Five mice, three lambda values, three sessions per mouse per threshold = 15 total sessions per lambda value, 45 total sessions for M.
• For testing H: same as M, so 90 total sessions (M + H).
• Could potentially run 3 sessions per day on one arena, and have two arenas, so could run 6 sessions per day max.
• To determine which patch is harder patch during any given session: initially (first 30 mins?) keep thresholds equal, then make the patch for which they have greater preference the harder patch.

Social (social0.1)

• Patch distribution protocol)
  • Initially, each patch's threshold-to-reward value will be sampled from either a low (L), medium (M), or high (H) threshold-to-reward distribution. The patches' corresponding distributions will always be different (unique).
  • We will have notions of "trials" (each time a reward is obtained at a patch) and "blocks" (each time the mapping between patch and threshold-to-reward distribution is changed). After each trial, the threshold-to-reward value will be resampled from the patch's corresponding distribution.
  • Once a threshold-to-change value has been reached (which we have currently decided will be based on a measure of cumulative distance spun on all wheels across patches, e.g. 30m), we will enter a new block: the patch-to-reward-distribution mapping will be resampled without replacement to ensure the patch distributions are different (but it is possible the patches will sample from the same distribution they did during the previous block).
  • We may want to have a “hard check” on the sampled patch thresholds during a block such that we ensure at the end of a block the mean threshold-to-reward for the easier patches was actually easier than the hard patches.
    • (The probability for this actually occurring can be determined once we have finalized which lambda values we will use for M and H).

Mice 1 & 2 (individual then social)

• Day 1:
  • Pre-experiment tube test Mouse 1 & 2;
  • Mouse 1 - equal patches (look for initial patch preference)
  • Mouse 2 - equal patches (look for initial patch preference)
• Days 2-4:
  • Mouse 1 - changing patches (H, M, L).
  • Mouse 2 - changing patches (H, M, L).
• Day 5: one day break
• Days 6-8: social: changing patches (H, M, L)

Mice 3 & 4 (completely individual)

• Day 9 : Repeat same procedure for as was done for Mouse 1 & 2 on Day 1
• Days 10-12: Repeat same procedure as was done for Mouse 1 & 2 on Days 2-4
• Day 13: One day break
• Days 14-16: Repeat Days 10-12 procedure

Patch distributions

After presocial, we've settled on the following means and offsets for the exponential distributions we'll use for sampling foraging thresholds (in Meters) for the patches:

• H: 1 +.75
• M: 3 +.75
• L: 5 +.75

[glopesdev]

@jkbhagatio we need a "unique id" for this dataset (i.e. the folder name in ceph, like experiment0.2 etc). Would it be something like social0.1? Also see #143

[jkbhagatio]

The individual sessions we run to collect data in order to parametrize the distributions for the patches I would actually call something like presocial0.1

[jkbhagatio]

• Do tube tests for multiple days to establish hierarchy?
• Other tests for hierarchy:
  • cold floor -> small warm patch

[jkbhagatio]

Rethink experiment with just two cohorts instead of three cohorts. Shorter individual epochs, longer social epochs

[jkbhagatio]

I think there are still four small issues to resolve regarding the Individual Sessions before we get started:

1. Should we have initial "acclimation" sessions for the mice in the individual sessions, before they start the first threshold discrimination sessions?

2. Do we care about the learning curves of the mice? i.e. Not just that they learn to discriminate one patch from another, but how quickly they learn (e.g. over how many "trials" or how much wheel distance spun or over how much time, e.g. by the end of the first 3 hour session, or second session, or third...?)

3. Should a measure of patch preference include pellets delievered, or not? We previously tentatively agreed on not including pellets delivered, and simply using wheel distance spun on a patch, but it might be worth revisiting briefly. See: Patch preference index #144

4. Should the order in which the mice experience different thresholds for individual sessions be sequential, or randomized? We previously tentatively agreed on sequential, but it might be worth revisiting briefly. A couple pros of the sequential approach are we can start analyzing data faster, and the outcome of all sessions on one threshold can inform us as to which threshold to try next; one con is that as we switch thresholds we get systematic "history"/"recency" effects on learning.

   • Sequential example:
     • session1: mouse1 - threshold1
     • session2: mouse2 - threshold1
     • session3: mouse1-threshold2
     • session4: mouse2-threshold2
   • Randomized example:
     • session1: mouse1 - threshold1
     • session2: mouse2 - threshold2
     • session3: mouse1 - threshold2
     • session4: mouse2 - threshold1

[jkbhagatio]

I'm fine keeping what we said for points 3 and 4 (i.e. simply using wheel distance as a measure of preference, and using the "sequential" approach to choosing thresholds for the individual sessions).

Just including them here for completeness and in case anyone else wants to reopen these discussions.

[Dario55]

1. Yes. Both non food restricted and food restricted mice could learn to get food from the wheel within the first few hours with no previous training. Probably is wise to add a session per mouse with 2 identical patches. Not sure whether it should be fixed rate or probabilistic.

[Dario55]

2. Tha will be interesting to look at analisis levele, but don't think we need to change experiment for that

[jkbhagatio]

For 2. I would say that if they are not showing learning (i.e. patch discrimination, however we define that) by the end of the second day, then we should treat the thresholds as too close together.

For the case where thresholds are too far apart: this will be determined by evaluation of the test statistics for whichever statistical test we use to compare distributions for easy and hard patch cumulative wheel distance. E.g. we may say that a p-value on [0.001, 0.1] for the test that the two empirical distributions belong to the same underlying theoretical distribution falls within the "sweet-spot" bounds, and a value less than this indicates the two thresholds are too easily discriminable, while a p-value greater than this indicates the two thresholds are too hard to discriminate.

[jkbhagatio]

Some thoughts on how to decide "learning" of the environment by the mice for Social experiment

• We want to optimize for a certain number of epoch switches per day
  • We want to compare individual vs. social performance for easy patch preference within an epoch
  • For individual sessions, we'll have some criterion (based on e.g. easy patch preference ratio) they must reach for "learning" the environment. After they've learned the environment, we will then run 72 hours of individual foraging. We will then run 144 hours of social foraging.
• Assuming on average we want each mouse to consume 20 pellets before an epoch switch, and mice on average consume 200 pellets per day, then this will give us 10 epoch switches per day. At an average of 375 cm / pellet delivery, this will mean we should set cumulative distance to 15 k cm / switch (375 * 200 * 2 / 10).
  • Given this, this would mean that for individual sessions we want them to "learn" the environment within 15k cm foraging, so one criterion for "learning" could be the first session in which they "learn" the environment in under 15k cm foraging, after which point we proceed to social sessions.

[jkbhagatio]

With more switches per day (the environment more volatile) will they learn faster / explore more?

[jkbhagatio]

We're going ahead with using time blocks (e.g. 2-3 hours sampled from some distribution) with replacement for social sessions.

@jerlich will summarize in more detail some of the models we discussed to predict/analyze behaviour/learning of the environment

• Multinomial regression across blocks (predict coefficients for H, L, M)
  • Compare solo vs social within social sessions
• RNN / Generative model to predict future frames/trajectories given past.
• Softmax model

[jerlich]

Softmax Model

Assumptions:

1. The preference (i.e. time spent at each patch) is a function of the experienced threshold.
2. The overall foraging time may be a function of overall (or max?) patch quality
3. Mice may have spatial biases.

Method:

1. Slice up time into some window (30 min?) for each patch, compute a 3-vector pellets/meter, $\Psi$, and the fraction of time as a 4-Vector T = [non_foraging, A, B, C], where $\sum_T = 1$

Fit the following model:

$T = [P_{NF}, (1 - P_{NF})\cdot P_F]$

$P_{NF} = 1/(1 + exp(\beta_{NF}\sum \Psi + {NF}_0))$

$P_F(i) = {e^{\beta \Psi + \epsilon_i}} / {\sum_j e^{\beta \Psi + \epsilon_j}}$

• $NF_0$ reflects the overall non-foraging time
• $\beta_{NF}$ reflects the sensitivity of non-foraging time to overall patch quality
• $\beta_{\Psi}$ reflects the sensitivity of patch preference to patch quality
• $\epsilon_i$ is a 3-vector reflecting spatial biases to the three patches. $\sum \epsilon = 1$

[jerlich]

For the first pass we can fit each mouse twice: once for solo bouts and again for social bouts. Later, we can add some parameters to see the effect of the location of the other mouse.

[glopesdev]

@jkbhagatio @jerlich @Dario55 we were discussing today a small detail about individual patch logic that is nevertheless impactful and worth clarifying and discussing for this and future tasks.

Currently a patch acts like an "integrate-and-fire" device, accumulating travelled distance until a threshold is reached to deliver a pellet. This threshold can be changed, which is how we build up depletion functions, probabilistic patches and now block switches.

One of the fundamental questions we discussed at the outset of the arena design was: what exactly should happen in the moment the threshold changes? Should the travelled distance immediately reset, effectively aborting the ongoing run? Or should we apply the change only at the next pellet delivery?

Originally we opted for a design where changes are applied at the next pellet delivery, for a couple of reasons:

• Allows experimenters to correct typos when manually changing the threshold (the "one more zero" accident, or the converse "forgot to add a zero" accidentally setting a minimal threshold that immediately delivers a pellet)
• Avoiding distance-dependent skews in the distribution due to asynchronous change. Basically, because the patch environment is shared across subjects, a patch can change asynchronously to pellet delivery for a given subject, e.g. because another mouse got a pellet. If the mouse is sufficiently unlucky, he can get his threshold reset very close to a long-distance pellet, and now might have to arbitrarily work even harder than the hardest threshold in the worst case, since he is now effectively travelling T_hardest + T_hardest total to get to the next pellet.

All the current tasks are built using a modular approach that builds upon the individual patch behavior described above. That means that probabilistic patches get their next threshold sampled at the next pellet delivery. For the social task this means that the distributions get switched immediately upon a block transition, but for individual patches it means that each patch will complete its current pellet run before sampling the new threshold. In other words, the statistics of each patch will change only after the 1^st pellet delivery following block transition.

We wanted to put this out there just so everyone is aware. Changing this would be possible in theory but would cause delays in the current task implementation. This is due to technical reasons in that we need to test that the effect of such hard resets works as intended, and we also need to maintain back-compatibility with previous tasks, but also for the social task we would need to reinterpret the current state flowchart diagram to make sure we understand what we want to do in each type of block transition.

For example, if there is indeed an immediate "global reset" in world state, this will effectively interrupt all ongoing runs at block transition, and in that case we might want to consider whether it is a good idea to align block transition to pellet delivery count, since in that case we are giving individual mice not just the power to "change the world", but also to interrupt everybody else's current runs.

Feedback is welcome as soon as possible, since we are currently in the process of debugging the arena and we would need to estimate how long until experiments can start, etc, which would be impacted if there is an immediate need to change the current behavior.

[glopesdev]

Just to leave my personal opinion here:

I think no matter what we do at block transition, it will be hard to interpret what exact information is perceived by mice in the transition moment. My impression was that analysis of individual blocks were to be made assuming there is some kind of block "steady state" where this washes out. If the size of each block is small enough that these single pellet effects can be disproportionate one way or the other, then there may be a problem with task design already since in my view there is no way to remove the "kink", since no matter what we do the transition will never be synchronized for all mice in the same way.

[jerlich]

We (speaking for the social proposal) never intended for the block change to be aligned on pellet delivery count. This would create (as @glopesdev says) a strange dependency. What we envisioned was figuring out a block duration that was the "right length" for our scientific questions and then using that duration (or duration distribution).

Are you planning to implement a kind of task logic API (similar to what we have in the octagon?). In that case, i would implement two commands /setnextthreshold patch threshold and /resetpatch patch. If one wants to keep the current logic, then they would just send the /setnextthreshold command if one wants to instantly change threshold, they can additionally call the /resetpatch command. (better names can be discussed).

The experiments we proposed, i think, assumed we could change a threshold without pellet delivery. It would simplify analysis, because we could look at patch leaving aligned to block switch. Without this function, we will have to think a bit more about how to align the behavior after a buffered block switch.

I think I agree with @glopesdev that the blocks have to be long enough for the mice to figure out what's going on anyway.

What i vote for is, let's go forward with experiments using the current logic, but put the /patchreset as a feature request.

[glopesdev]

@jerlich Indeed the easiest way to implement this feature as the system currently stands would be to support something like a "patch reset", which is essentially the same as a threshold crossing but without a pellet delivery.

I think just having this extra command should be enough in combination with the option to set the next threshold as it works today. We just would need to make sure we cover all edge cases including alerts and logging to make sure we are not missing anything, which is why introducing this now might create a further delay in development.

In terms of analysis I thought this would work by looking at patch leaving aligned to block switch, as you said, with the extra restriction that we look at patch leaving events following the 1st pellet delivered at each patch.

[jkbhagatio]

I think maybe @jerlich did not comment on how to handle one important aspect of block, regardless of if the blocks switch according to some duration of time, or some number of pellets.

This aspect is this, as @glopesdev stated:

the distributions get switched immediately upon a block transition, but for individual patches it means that each patch will complete its current pellet run before sampling the new threshold. In other words, the statistics of each patch will change only after the 1st pellet delivery following block transition.

To spell out some implications of this, this means for instance that currently all three patches could be on different blocks, if e.g. over two consecutive blocks an animal only samples from one patch and then another patch. I'm still not sure this is the behavior we want, and would be good to discuss this in regards to the type of analysis we want to perform.

[jerlich]

@jkbhagatio i don't think we should be too concerned about that scenario. First, if the blocks are long enough then that's unlikely. Second, as long as we have the data we can deal with these edge cases in the analyses.

[Dario55]

Hi guys, thank you for the feedback. Just to clarify one point for Jeff. At the moment we can decide if block transition happens with pellet count or time. In both cases the number of pellet (or the block duration) is sampled from a random flat distribution of which we set the minimum and maximum value.

[jkbhagatio]

Adding some rough notes from a current meeting:

• Need to check licence for what to do in case of fighting: right now thinking we leave them in unless behavior changes significantly (e.g. not getting much food / water)
• Two options for running two cohorts in parallel:
  • 1. Each cohort runs individually on same arena
  • 2. Each cohort runs individually on different arenas
  • For now we go with option 1 due to current schedule.