[Feature] Support wrapping IsaacLab environments with GymEnv

[yu-fz]

Description

Adds a configuration parameter to GymLikeEnv to specify whether or not actions should be converted to numpy arrays before being passed to the step function of the underlying environment to retrieve the next state. Users can now pass convert_actions_to_numpy=False when wrapping environments that work with torch tensors.

Let me know if a new test is required for this.

Motivation and Context

Previously, it was assumed that the step function argument expected numpy array types on CPU, which makes torchrl incompatible with environments like IsaacLab that operate on torch tensors and on the GPU.

If it fixes an open issue, please link to the issue here.
You can use the syntax close #15213 if this solves the issue #15213
Closes #2369

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

• [] Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• [] Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• [] Example (update in the folder of examples)

Checklist

• I have read the CONTRIBUTION guide (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2380

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 2 Cancelled Jobs, 7 Unrelated Failures

As of commit 73596a8 with merge base 607db8b ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Workflow failed! Resource not accessible by integration
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Workflow failed! Resource not accessible by integration
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 444c3e0a5e4480ff9288951c6c43f0183dd1eab1629f73921393dda089cf4d10 /exec failed with exit code 139
• Wheels / build-wheel-windows (3.8, 3.8) (gh)
  Process completed with exit code 1.

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• Wheels / build-wheel-windows (3.11, 3.11) (gh)
• Wheels / build-wheel-windows (3.12, 3.12) (gh)

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Wheels / build-wheel-windows (3.10, 3.10.3) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• Wheels / build-wheel-windows (3.9, 3.9) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cpu (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda11_8 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda12_1 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda12_4 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Unit-tests on Windows / unittests-cpu / windows-job (gh) (trunk failure)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[vmoens]

LGTM
I guess that then we should also document better how to use the new Isaac API

Ideally, I'd like not to rely on Gym for Isaac and wrap the simulator directly, but first let's focus on making it work like this!