Unfortunately, putting together a unified build scheme has proven challenging, because our different compute resources rely on different containerization schemes (i.e. bespoke docker vs apptainer), and subtle bugs have emerged that only effect one build system or the other. We maintain a Dockerfile
for building the docker image and the retinal_rl.def
file to build the apptainer
image, and we've also had success building with conda
on bare metal.
The apptainer
image is self-contained, and building it should immediately allow running the relevant scripts in retinal-rl
, by prefixing them with apptainer exec [image]
. The versions of most pip packages are floating, but we have an environment.yaml
file from a working apptainer
build.
Here are the steps to get a retinal-rl
environment setup in conda
, which should work on bare metal. First install anaconda or miniconda, and then create the environment
conda create --name retinal-rl pip
conda activate retinal-rl
I'm using miniconda
, so some of the following commands might be redundant if you're using anaconda
.
Next, we install pytorch
conda install pytorch torchvision torchaudio cudatoolkit=11.6 -c pytorch -c conda-forge
and then install sample-factory
and vizdoom
pip install sample-factory
pip install vizdoom
Note, you may require sample-factory=1.121.4
on a server. You may also want to downgrade the gym
library with pip install gym==0.25.2
For vizdoom
, pip install
can sometimes fail when run inside a conda
environment. In this case the solution is to build vizdoom
directly by running
conda install -c conda-forge boost cmake gtk2 sdl2
git clone https://github.com/mwydmuch/ViZDoom.git --recurse-submodules
cd ViZDoom
python setup.py build && python setup.py install
We'll also need some other tools and libraries
conda install -c conda-forge matplotlib pyglet imageio
pip install pygifsicle
pip install openTSNE
IPython might also be necessary:
conda install -c conda-forge ipython
The Dockerfile
is a thin wrapper around the berenslab Dockerfile
for the berenslab cluster, but may still serve as a basis for developing a docker
container for other systems. Regardless, after building the image we then create the conda
environment as above.
Now that we have a (hopefully) working environment, we clone the repo
https://github.com/berenslab/retinal-rl.git
There are three main scripts for working with retinal-rl
:
train.py
: Train a model.analyze.py
: Generate some analyses.enjoy.py
: Watch a real time simulation of a trained agent.
Each script can be run by python in python -m {script}
, where {script} is the name of the desired script (without the .py
extension), followed by a number of arguments. Note that train.py
must always be run first to create the necessary files and folders, and once run will permanently set most (all?) of the arguments of the simulation, and will ignore changes to these arguments if training is resumed.
Certain arguments must always be provided, regardless of script, namely:
--env
: Specifies the desired map. This will always have the formretinal_{scenario}
, where scenario is the shared name of one of the.wad
/.cfg
file pairs in thescenarios
directory.--algo
: The training algorithm; for now this should always beAPPO
.--experiment
: The directory under thetrain_dir
directory where simulation results are saved.
The following argument should always be set when training for the first time:
--encoder_custom
: The options aresimple
, which is a small, hard-coded network that still tends to perform well, andlindsey
, which has a number of tuneable hyperparameters.
For specifying the form of the lindsey
network, the key arguments are:
--global_channels
: The number of channels in each CNN layers, except for the bottleneck layer.--retinal_bottleneck
: Number of channels in the retinal bottleneck.--vvs_depth
: Number of CNN layers in the ventral stream network.--kernel_size
: Size of the kernels.
Finally, when training a model there are a number of additional parameters for controlling the reinforcement learning brain, and adjusting simulation parameters. The key ones to worry about are
--hidden_size
: The size of the hidden/latent state used to represent the RL problem.--num_workers
: This is the number of simulation threads to run. This shouldn't be more than the number of cores on the CPU, and can be less if the simulation is GPU bottlenecked.