Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] Better doc for inverse transform semantic #2459

Merged
merged 4 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/source/_static/img/rename_transform.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
17 changes: 17 additions & 0 deletions docs/source/reference/envs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -722,6 +722,9 @@ Since each transform uses a ``"in_keys"``/``"out_keys"`` set of keyword argument
also easy to root the transform graph to each component of the observation data (e.g.
pixels or states etc).

Forward and inverse transforms
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Transforms also have an ``inv`` method that is called before
the action is applied in reverse order over the composed transform chain:
this allows to apply transforms to data in the environment before the action is taken
Expand All @@ -733,6 +736,20 @@ in the environment. The keys to be included in this inverse transform are passed

>>> env.append_transform(DoubleToFloat(in_keys_inv=["action"])) # will map the action from float32 to float64 before calling the base_env.step

The way ``in_keys`` relates to ``in_keys_inv`` can be understood by considering the base environment as the "inner" part
of the transform. In constrast, the user inputs and outputs to and from the transform are to be considered as the
outside world. The following figure shows what this means in practice for the :class:`~torchrl.envs.RenameTransform`
class: the input ``TensorDict`` of the ``step`` function must have the ``out_keys_inv`` listed in its entries as they
are part of the outside world. The transform changes these names to make them match the names of the inner, base
environment using the ``in_keys_inv``. The inverse process is executed with the output tensordict, where the ``in_keys``
are mapped to the corresponding ``out_keys``.

.. figure:: /_static/img/rename_transform.png

Rename transform logic



Cloning transforms
~~~~~~~~~~~~~~~~~~

Expand Down
Loading