[CI] Resolve DMC and mujoco pinned versions

[vmoens]

Description

Describe your changes in detail.

Motivation and Context

Why is this change required? What problem does it solve?
If it fixes an open issue, please link to the issue here.
You can use the syntax close #15213 if this solves the issue #15213

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Remove all that do not apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• Example (update in the folder of examples)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

• I have read the CONTRIBUTION guide (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2396

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 21 Unrelated Failures

As of commit 6bc1047 with merge base a4bb63b ():

NEW FAILURES - The following jobs have failed:

• Generate documentation / build-docs (3.9, 12.1) / linux-job (gh)
  No files were found with the provided path: /home/ec2-user/actions-runner/_work/_temp/artifacts/. No artifacts will be uploaded.
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda12_1 (gh) (similar failure)
  Unable to find any artifacts for the associated workflow
• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Lint / c-source / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Lint / python-source-and-configs / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-cpu-oldget (3.12) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-gpu (3.11, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cpu (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda11_8 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda12_4 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	64.2125ms	57.7967ms	17.3020 Ops/s	17.0666 Ops/s	$\color{#35bf28}+1.38\%$
test_sync	43.2392ms	31.7305ms	31.5154 Ops/s	31.3602 Ops/s	$\color{#35bf28}+0.50\%$
test_async	61.4242ms	30.2714ms	33.0344 Ops/s	31.5480 Ops/s	$\color{#35bf28}+4.71\%$
test_simple	0.4827s	0.4129s	2.4217 Ops/s	2.3702 Ops/s	$\color{#35bf28}+2.18\%$
test_transformed	0.6319s	0.5681s	1.7603 Ops/s	1.6863 Ops/s	$\color{#35bf28}+4.39\%$
test_serial	1.3091s	1.2476s	0.8016 Ops/s	0.7918 Ops/s	$\color{#35bf28}+1.24\%$
test_parallel	1.1786s	1.1009s	0.9084 Ops/s	0.8936 Ops/s	$\color{#35bf28}+1.65\%$
test_step_mdp_speed[True-True-True-True-True]	0.2038ms	25.0521μs	39.9168 KOps/s	40.2909 KOps/s	$\color{#d91a1a}-0.93\%$
test_step_mdp_speed[True-True-True-True-False]	54.9830μs	14.5306μs	68.8203 KOps/s	70.3870 KOps/s	$\color{#d91a1a}-2.23\%$
test_step_mdp_speed[True-True-True-False-True]	47.2080μs	14.2821μs	70.0178 KOps/s	71.2300 KOps/s	$\color{#d91a1a}-1.70\%$
test_step_mdp_speed[True-True-True-False-False]	33.8830μs	8.3535μs	119.7096 KOps/s	121.8808 KOps/s	$\color{#d91a1a}-1.78\%$
test_step_mdp_speed[True-True-False-True-True]	73.0870μs	26.7640μs	37.3636 KOps/s	37.6954 KOps/s	$\color{#d91a1a}-0.88\%$
test_step_mdp_speed[True-True-False-True-False]	85.2990μs	16.0013μs	62.4950 KOps/s	62.7648 KOps/s	$\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-True-False-False-True]	46.8370μs	15.7947μs	63.3123 KOps/s	63.5483 KOps/s	$\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-True-False-False-False]	37.8710μs	9.8301μs	101.7287 KOps/s	102.9856 KOps/s	$\color{#d91a1a}-1.22\%$
test_step_mdp_speed[True-False-True-True-True]	64.2400μs	28.7444μs	34.7893 KOps/s	35.5508 KOps/s	$\color{#d91a1a}-2.14\%$
test_step_mdp_speed[True-False-True-True-False]	74.5710μs	17.7726μs	56.2665 KOps/s	56.9809 KOps/s	$\color{#d91a1a}-1.25\%$
test_step_mdp_speed[True-False-True-False-True]	0.1917ms	16.7965μs	59.5363 KOps/s	63.4086 KOps/s	$\textbf{\color{#d91a1a}-6.11\%}$
test_step_mdp_speed[True-False-True-False-False]	33.0620μs	9.7792μs	102.2579 KOps/s	102.6499 KOps/s	$\color{#d91a1a}-0.38\%$
test_step_mdp_speed[True-False-False-True-True]	69.9700μs	29.9740μs	33.3622 KOps/s	33.5741 KOps/s	$\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-False-False-True-False]	53.3800μs	19.1873μs	52.1178 KOps/s	52.9241 KOps/s	$\color{#d91a1a}-1.52\%$
test_step_mdp_speed[True-False-False-False-True]	44.4130μs	17.3385μs	57.6752 KOps/s	58.4823 KOps/s	$\color{#d91a1a}-1.38\%$
test_step_mdp_speed[True-False-False-False-False]	42.8700μs	11.3181μs	88.3541 KOps/s	90.4804 KOps/s	$\color{#d91a1a}-2.35\%$
test_step_mdp_speed[False-True-True-True-True]	0.1114ms	28.2986μs	35.3374 KOps/s	35.8484 KOps/s	$\color{#d91a1a}-1.43\%$
test_step_mdp_speed[False-True-True-True-False]	73.6070μs	17.6595μs	56.6269 KOps/s	57.3764 KOps/s	$\color{#d91a1a}-1.31\%$
test_step_mdp_speed[False-True-True-False-True]	47.1280μs	18.6265μs	53.6870 KOps/s	54.8967 KOps/s	$\color{#d91a1a}-2.20\%$
test_step_mdp_speed[False-True-True-False-False]	35.1360μs	11.0036μs	90.8794 KOps/s	91.7455 KOps/s	$\color{#d91a1a}-0.94\%$
test_step_mdp_speed[False-True-False-True-True]	0.1045ms	30.0391μs	33.2900 KOps/s	34.2606 KOps/s	$\color{#d91a1a}-2.83\%$
test_step_mdp_speed[False-True-False-True-False]	56.1740μs	19.1869μs	52.1190 KOps/s	53.2280 KOps/s	$\color{#d91a1a}-2.08\%$
test_step_mdp_speed[False-True-False-False-True]	56.0540μs	19.9715μs	50.0713 KOps/s	51.5733 KOps/s	$\color{#d91a1a}-2.91\%$
test_step_mdp_speed[False-True-False-False-False]	39.1030μs	12.6052μs	79.3327 KOps/s	80.5038 KOps/s	$\color{#d91a1a}-1.45\%$
test_step_mdp_speed[False-False-True-True-True]	3.1182ms	31.9277μs	31.3207 KOps/s	32.0787 KOps/s	$\color{#d91a1a}-2.36\%$
test_step_mdp_speed[False-False-True-True-False]	53.3300μs	20.7945μs	48.0896 KOps/s	49.2231 KOps/s	$\color{#d91a1a}-2.30\%$
test_step_mdp_speed[False-False-True-False-True]	57.4270μs	20.0565μs	49.8591 KOps/s	51.4680 KOps/s	$\color{#d91a1a}-3.13\%$
test_step_mdp_speed[False-False-True-False-False]	43.2910μs	12.6437μs	79.0908 KOps/s	79.4616 KOps/s	$\color{#d91a1a}-0.47\%$
test_step_mdp_speed[False-False-False-True-True]	81.3220μs	32.9645μs	30.3357 KOps/s	31.3896 KOps/s	$\color{#d91a1a}-3.36\%$
test_step_mdp_speed[False-False-False-True-False]	57.8680μs	22.3762μs	44.6903 KOps/s	46.3692 KOps/s	$\color{#d91a1a}-3.62\%$
test_step_mdp_speed[False-False-False-False-True]	59.5910μs	21.3951μs	46.7397 KOps/s	48.5193 KOps/s	$\color{#d91a1a}-3.67\%$
test_step_mdp_speed[False-False-False-False-False]	42.0380μs	13.9595μs	71.6357 KOps/s	73.0193 KOps/s	$\color{#d91a1a}-1.89\%$
test_values[generalized_advantage_estimate-True-True]	11.8444ms	9.5710ms	104.4826 Ops/s	104.5699 Ops/s	$\color{#d91a1a}-0.08\%$
test_values[vec_generalized_advantage_estimate-True-True]	51.6004ms	33.8737ms	29.5215 Ops/s	27.5891 Ops/s	$\textbf{\color{#35bf28}+7.00\%}$
test_values[td0_return_estimate-False-False]	0.2175ms	0.1711ms	5.8460 KOps/s	5.7278 KOps/s	$\color{#35bf28}+2.06\%$
test_values[td1_return_estimate-False-False]	26.1882ms	23.7736ms	42.0635 Ops/s	41.6116 Ops/s	$\color{#35bf28}+1.09\%$
test_values[vec_td1_return_estimate-False-False]	34.7064ms	33.3888ms	29.9501 Ops/s	28.1823 Ops/s	$\textbf{\color{#35bf28}+6.27\%}$
test_values[td_lambda_return_estimate-True-False]	34.4782ms	33.8142ms	29.5734 Ops/s	28.4695 Ops/s	$\color{#35bf28}+3.88\%$
test_values[vec_td_lambda_return_estimate-True-False]	35.2327ms	33.4858ms	29.8634 Ops/s	28.1220 Ops/s	$\textbf{\color{#35bf28}+6.19\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.2762ms	8.1700ms	122.3991 Ops/s	120.6249 Ops/s	$\color{#35bf28}+1.47\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.5258ms	2.0135ms	496.6431 Ops/s	554.2215 Ops/s	$\textbf{\color{#d91a1a}-10.39\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5815ms	0.3532ms	2.8314 KOps/s	2.7731 KOps/s	$\color{#35bf28}+2.10\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	41.8701ms	41.1318ms	24.3121 Ops/s	21.8879 Ops/s	$\textbf{\color{#35bf28}+11.08\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.1344ms	3.0143ms	331.7474 Ops/s	327.8765 Ops/s	$\color{#35bf28}+1.18\%$
test_dqn_speed	5.7691ms	1.2696ms	787.6791 Ops/s	756.3388 Ops/s	$\color{#35bf28}+4.14\%$
test_ddpg_speed	3.6162ms	2.6655ms	375.1712 Ops/s	364.0360 Ops/s	$\color{#35bf28}+3.06\%$
test_sac_speed	10.8861ms	7.9115ms	126.3980 Ops/s	125.0216 Ops/s	$\color{#35bf28}+1.10\%$
test_redq_speed	14.7508ms	12.4270ms	80.4697 Ops/s	79.0601 Ops/s	$\color{#35bf28}+1.78\%$
test_redq_deprec_speed	13.7201ms	12.7416ms	78.4832 Ops/s	77.9191 Ops/s	$\color{#35bf28}+0.72\%$
test_td3_speed	9.0221ms	7.7755ms	128.6091 Ops/s	126.2133 Ops/s	$\color{#35bf28}+1.90\%$
test_cql_speed	36.6728ms	35.0164ms	28.5580 Ops/s	25.8459 Ops/s	$\textbf{\color{#35bf28}+10.49\%}$
test_a2c_speed	8.1653ms	7.1982ms	138.9234 Ops/s	136.5038 Ops/s	$\color{#35bf28}+1.77\%$
test_ppo_speed	8.0424ms	7.4540ms	134.1553 Ops/s	131.6522 Ops/s	$\color{#35bf28}+1.90\%$
test_reinforce_speed	10.8487ms	6.3830ms	156.6665 Ops/s	155.0063 Ops/s	$\color{#35bf28}+1.07\%$
test_iql_speed	32.9445ms	31.4408ms	31.8058 Ops/s	31.0902 Ops/s	$\color{#35bf28}+2.30\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.0401ms	4.8535ms	206.0359 Ops/s	208.5775 Ops/s	$\color{#d91a1a}-1.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.0011ms	0.4724ms	2.1167 KOps/s	2.0875 KOps/s	$\color{#35bf28}+1.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6457ms	0.4492ms	2.2260 KOps/s	2.2006 KOps/s	$\color{#35bf28}+1.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.0982ms	4.7591ms	210.1218 Ops/s	212.0994 Ops/s	$\color{#d91a1a}-0.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.8370ms	0.4698ms	2.1286 KOps/s	2.1137 KOps/s	$\color{#35bf28}+0.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6148ms	0.4459ms	2.2429 KOps/s	2.2385 KOps/s	$\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.8019ms	1.6576ms	603.2926 Ops/s	586.3262 Ops/s	$\color{#35bf28}+2.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.2049ms	1.5850ms	630.8982 Ops/s	622.1308 Ops/s	$\color{#35bf28}+1.41\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.1510ms	4.9040ms	203.9172 Ops/s	201.2166 Ops/s	$\color{#35bf28}+1.34\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8462ms	0.6043ms	1.6547 KOps/s	1.6274 KOps/s	$\color{#35bf28}+1.68\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7648ms	0.5841ms	1.7120 KOps/s	1.7051 KOps/s	$\color{#35bf28}+0.40\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.8436ms	4.9029ms	203.9606 Ops/s	207.3116 Ops/s	$\color{#d91a1a}-1.62\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8007ms	0.4741ms	2.1093 KOps/s	2.0766 KOps/s	$\color{#35bf28}+1.57\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6585ms	0.4531ms	2.2069 KOps/s	2.1985 KOps/s	$\color{#35bf28}+0.38\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.5627ms	4.8480ms	206.2693 Ops/s	209.2299 Ops/s	$\color{#d91a1a}-1.41\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.3450ms	0.4680ms	2.1367 KOps/s	2.0897 KOps/s	$\color{#35bf28}+2.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6437ms	0.4498ms	2.2231 KOps/s	2.2313 KOps/s	$\color{#d91a1a}-0.37\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.4467ms	4.9621ms	201.5261 Ops/s	203.8890 Ops/s	$\color{#d91a1a}-1.16\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.7585ms	0.6042ms	1.6550 KOps/s	1.6068 KOps/s	$\color{#35bf28}+3.00\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	7.3032ms	0.5925ms	1.6878 KOps/s	1.6688 KOps/s	$\color{#35bf28}+1.14\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1225s	6.0590ms	165.0437 Ops/s	152.9706 Ops/s	$\textbf{\color{#35bf28}+7.89\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1160s	16.2291ms	61.6177 Ops/s	75.0008 Ops/s	$\textbf{\color{#d91a1a}-17.84\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	5.3804ms	1.1733ms	852.3198 Ops/s	742.7807 Ops/s	$\textbf{\color{#35bf28}+14.75\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1069s	5.7679ms	173.3733 Ops/s	151.5428 Ops/s	$\textbf{\color{#35bf28}+14.41\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	16.3467ms	12.5241ms	79.8461 Ops/s	76.7122 Ops/s	$\color{#35bf28}+4.09\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	5.3728ms	1.1700ms	854.7100 Ops/s	894.7518 Ops/s	$\color{#d91a1a}-4.48\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1078s	5.8944ms	169.6522 Ops/s	115.7446 Ops/s	$\textbf{\color{#35bf28}+46.57\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	17.0212ms	12.6156ms	79.2669 Ops/s	75.5951 Ops/s	$\color{#35bf28}+4.86\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.9302ms	1.2337ms	810.5500 Ops/s	758.0251 Ops/s	$\textbf{\color{#35bf28}+6.93\%}$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}2$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1074s	0.1070s	9.3490 Ops/s	9.3513 Ops/s	$\color{#d91a1a}-0.02\%$
test_sync	95.1461ms	92.1583ms	10.8509 Ops/s	10.7362 Ops/s	$\color{#35bf28}+1.07\%$
test_async	0.2422s	87.2388ms	11.4628 Ops/s	11.0799 Ops/s	$\color{#35bf28}+3.46\%$
test_single_pixels	0.1209s	0.1193s	8.3850 Ops/s	8.5753 Ops/s	$\color{#d91a1a}-2.22\%$
test_sync_pixels	77.6837ms	76.4972ms	13.0724 Ops/s	13.1197 Ops/s	$\color{#d91a1a}-0.36\%$
test_async_pixels	0.1405s	70.4897ms	14.1865 Ops/s	14.0252 Ops/s	$\color{#35bf28}+1.15\%$
test_simple	0.8670s	0.7871s	1.2704 Ops/s	1.2772 Ops/s	$\color{#d91a1a}-0.53\%$
test_transformed	1.1155s	1.0236s	0.9769 Ops/s	0.9960 Ops/s	$\color{#d91a1a}-1.92\%$
test_serial	2.2725s	2.2104s	0.4524 Ops/s	0.4597 Ops/s	$\color{#d91a1a}-1.59\%$
test_parallel	1.9780s	1.8940s	0.5280 Ops/s	0.5282 Ops/s	$\color{#d91a1a}-0.03\%$
test_step_mdp_speed[True-True-True-True-True]	95.0550μs	37.9953μs	26.3190 KOps/s	25.5075 KOps/s	$\color{#35bf28}+3.18\%$
test_step_mdp_speed[True-True-True-True-False]	46.0220μs	21.3865μs	46.7584 KOps/s	46.2459 KOps/s	$\color{#35bf28}+1.11\%$
test_step_mdp_speed[True-True-True-False-True]	45.9420μs	20.8936μs	47.8614 KOps/s	45.5269 KOps/s	$\textbf{\color{#35bf28}+5.13\%}$
test_step_mdp_speed[True-True-True-False-False]	35.9620μs	12.0602μs	82.9174 KOps/s	82.7192 KOps/s	$\color{#35bf28}+0.24\%$
test_step_mdp_speed[True-True-False-True-True]	68.9140μs	39.3576μs	25.4080 KOps/s	24.4519 KOps/s	$\color{#35bf28}+3.91\%$
test_step_mdp_speed[True-True-False-True-False]	87.9150μs	23.6726μs	42.2430 KOps/s	41.8260 KOps/s	$\color{#35bf28}+1.00\%$
test_step_mdp_speed[True-True-False-False-True]	50.6630μs	23.3853μs	42.7619 KOps/s	41.2312 KOps/s	$\color{#35bf28}+3.71\%$
test_step_mdp_speed[True-True-False-False-False]	34.8620μs	14.4432μs	69.2369 KOps/s	68.6738 KOps/s	$\color{#35bf28}+0.82\%$
test_step_mdp_speed[True-False-True-True-True]	80.1440μs	42.2105μs	23.6908 KOps/s	23.0644 KOps/s	$\color{#35bf28}+2.72\%$
test_step_mdp_speed[True-False-True-True-False]	48.7030μs	26.1594μs	38.2272 KOps/s	38.6682 KOps/s	$\color{#d91a1a}-1.14\%$
test_step_mdp_speed[True-False-True-False-True]	47.6820μs	23.2811μs	42.9532 KOps/s	41.8056 KOps/s	$\color{#35bf28}+2.75\%$
test_step_mdp_speed[True-False-True-False-False]	33.6820μs	14.3110μs	69.8765 KOps/s	69.4152 KOps/s	$\color{#35bf28}+0.66\%$
test_step_mdp_speed[True-False-False-True-True]	74.3640μs	44.5038μs	22.4700 KOps/s	22.3072 KOps/s	$\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-False-False-True-False]	61.1630μs	28.4627μs	35.1337 KOps/s	35.2821 KOps/s	$\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-False-False-False-True]	49.0630μs	25.4407μs	39.3070 KOps/s	38.7597 KOps/s	$\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-False-False-False-False]	35.0420μs	16.6705μs	59.9860 KOps/s	60.3713 KOps/s	$\color{#d91a1a}-0.64\%$
test_step_mdp_speed[False-True-True-True-True]	68.8630μs	41.9550μs	23.8351 KOps/s	23.1666 KOps/s	$\color{#35bf28}+2.89\%$
test_step_mdp_speed[False-True-True-True-False]	43.9220μs	25.8266μs	38.7198 KOps/s	38.4670 KOps/s	$\color{#35bf28}+0.66\%$
test_step_mdp_speed[False-True-True-False-True]	48.9730μs	27.8949μs	35.8488 KOps/s	35.3263 KOps/s	$\color{#35bf28}+1.48\%$
test_step_mdp_speed[False-True-True-False-False]	36.6220μs	16.2076μs	61.6994 KOps/s	61.1570 KOps/s	$\color{#35bf28}+0.89\%$
test_step_mdp_speed[False-True-False-True-True]	71.1850μs	44.0257μs	22.7140 KOps/s	22.3201 KOps/s	$\color{#35bf28}+1.76\%$
test_step_mdp_speed[False-True-False-True-False]	50.4530μs	28.2832μs	35.3567 KOps/s	35.1040 KOps/s	$\color{#35bf28}+0.72\%$
test_step_mdp_speed[False-True-False-False-True]	53.2430μs	29.8265μs	33.5273 KOps/s	32.8349 KOps/s	$\color{#35bf28}+2.11\%$
test_step_mdp_speed[False-True-False-False-False]	41.5020μs	18.3417μs	54.5204 KOps/s	53.6620 KOps/s	$\color{#35bf28}+1.60\%$
test_step_mdp_speed[False-False-True-True-True]	3.9488ms	47.3156μs	21.1347 KOps/s	20.5126 KOps/s	$\color{#35bf28}+3.03\%$
test_step_mdp_speed[False-False-True-True-False]	58.4030μs	30.7210μs	32.5511 KOps/s	32.3754 KOps/s	$\color{#35bf28}+0.54\%$
test_step_mdp_speed[False-False-True-False-True]	57.7130μs	29.6502μs	33.7266 KOps/s	32.4515 KOps/s	$\color{#35bf28}+3.93\%$
test_step_mdp_speed[False-False-True-False-False]	34.4710μs	18.3242μs	54.5725 KOps/s	53.3362 KOps/s	$\color{#35bf28}+2.32\%$
test_step_mdp_speed[False-False-False-True-True]	73.8640μs	48.5062μs	20.6159 KOps/s	20.1096 KOps/s	$\color{#35bf28}+2.52\%$
test_step_mdp_speed[False-False-False-True-False]	54.3530μs	32.6952μs	30.5855 KOps/s	30.5815 KOps/s	$\color{#35bf28}+0.01\%$
test_step_mdp_speed[False-False-False-False-True]	55.6930μs	31.6156μs	31.6299 KOps/s	30.7792 KOps/s	$\color{#35bf28}+2.76\%$
test_step_mdp_speed[False-False-False-False-False]	37.4520μs	20.5717μs	48.6104 KOps/s	48.3397 KOps/s	$\color{#35bf28}+0.56\%$
test_values[generalized_advantage_estimate-True-True]	26.4157ms	24.9311ms	40.1105 Ops/s	41.0920 Ops/s	$\color{#d91a1a}-2.39\%$
test_values[vec_generalized_advantage_estimate-True-True]	90.8383ms	2.7094ms	369.0909 Ops/s	371.0697 Ops/s	$\color{#d91a1a}-0.53\%$
test_values[td0_return_estimate-False-False]	92.0250μs	65.8394μs	15.1885 KOps/s	15.0234 KOps/s	$\color{#35bf28}+1.10\%$
test_values[td1_return_estimate-False-False]	55.9407ms	55.4045ms	18.0491 Ops/s	18.2358 Ops/s	$\color{#d91a1a}-1.02\%$
test_values[vec_td1_return_estimate-False-False]	1.4205ms	1.0949ms	913.3647 Ops/s	919.1273 Ops/s	$\color{#d91a1a}-0.63\%$
test_values[td_lambda_return_estimate-True-False]	92.5278ms	88.7092ms	11.2728 Ops/s	11.4851 Ops/s	$\color{#d91a1a}-1.85\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.4266ms	1.0929ms	915.0078 Ops/s	918.0996 Ops/s	$\color{#d91a1a}-0.34\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	24.9572ms	24.6677ms	40.5389 Ops/s	40.8231 Ops/s	$\color{#d91a1a}-0.70\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9487ms	0.7241ms	1.3810 KOps/s	1.3849 KOps/s	$\color{#d91a1a}-0.28\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7657ms	0.6776ms	1.4757 KOps/s	1.4820 KOps/s	$\color{#d91a1a}-0.42\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5084ms	1.4734ms	678.6954 Ops/s	680.1836 Ops/s	$\color{#d91a1a}-0.22\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.7856ms	0.7040ms	1.4204 KOps/s	1.4519 KOps/s	$\color{#d91a1a}-2.16\%$
test_dqn_speed	7.4149ms	1.3728ms	728.4222 Ops/s	727.5775 Ops/s	$\color{#35bf28}+0.12\%$
test_ddpg_speed	3.0476ms	2.7942ms	357.8842 Ops/s	363.6299 Ops/s	$\color{#d91a1a}-1.58\%$
test_sac_speed	8.1845ms	7.9197ms	126.2674 Ops/s	125.4576 Ops/s	$\color{#35bf28}+0.65\%$
test_redq_speed	12.5845ms	10.3620ms	96.5066 Ops/s	98.8135 Ops/s	$\color{#d91a1a}-2.33\%$
test_redq_deprec_speed	11.1765ms	10.8655ms	92.0348 Ops/s	93.4400 Ops/s	$\color{#d91a1a}-1.50\%$
test_td3_speed	17.2754ms	8.0027ms	124.9575 Ops/s	126.5106 Ops/s	$\color{#d91a1a}-1.23\%$
test_cql_speed	26.6296ms	25.3146ms	39.5029 Ops/s	40.0496 Ops/s	$\color{#d91a1a}-1.37\%$
test_a2c_speed	5.8377ms	5.5412ms	180.4679 Ops/s	181.0168 Ops/s	$\color{#d91a1a}-0.30\%$
test_ppo_speed	6.1698ms	5.8497ms	170.9490 Ops/s	169.5385 Ops/s	$\color{#35bf28}+0.83\%$
test_reinforce_speed	5.1009ms	4.5007ms	222.1871 Ops/s	225.6421 Ops/s	$\color{#d91a1a}-1.53\%$
test_iql_speed	20.4951ms	19.5154ms	51.2416 Ops/s	53.1416 Ops/s	$\color{#d91a1a}-3.58\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.2012ms	6.9090ms	144.7378 Ops/s	150.9502 Ops/s	$\color{#d91a1a}-4.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.9726ms	0.5200ms	1.9229 KOps/s	1.9361 KOps/s	$\color{#d91a1a}-0.68\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7059ms	0.4976ms	2.0097 KOps/s	2.0203 KOps/s	$\color{#d91a1a}-0.52\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.0496ms	6.7016ms	149.2191 Ops/s	152.8323 Ops/s	$\color{#d91a1a}-2.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.1366ms	0.5150ms	1.9416 KOps/s	1.9716 KOps/s	$\color{#d91a1a}-1.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6656ms	0.4944ms	2.0225 KOps/s	2.0540 KOps/s	$\color{#d91a1a}-1.54\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.2451ms	1.9967ms	500.8204 Ops/s	494.1903 Ops/s	$\color{#35bf28}+1.34\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	5.8143ms	1.8898ms	529.1455 Ops/s	541.2387 Ops/s	$\color{#d91a1a}-2.23\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.9977ms	6.8537ms	145.9060 Ops/s	148.1160 Ops/s	$\color{#d91a1a}-1.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.7014ms	0.6711ms	1.4902 KOps/s	1.5084 KOps/s	$\color{#d91a1a}-1.21\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8620ms	0.6407ms	1.5608 KOps/s	1.5648 KOps/s	$\color{#d91a1a}-0.26\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.9564ms	6.7658ms	147.8030 Ops/s	150.7154 Ops/s	$\color{#d91a1a}-1.93\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.8299ms	0.5136ms	1.9470 KOps/s	1.9301 KOps/s	$\color{#35bf28}+0.88\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7566ms	0.4922ms	2.0317 KOps/s	2.0201 KOps/s	$\color{#35bf28}+0.58\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.0351ms	6.7048ms	149.1474 Ops/s	151.8546 Ops/s	$\color{#d91a1a}-1.78\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6148ms	0.5157ms	1.9390 KOps/s	1.9410 KOps/s	$\color{#d91a1a}-0.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	4.4485ms	0.4955ms	2.0184 KOps/s	2.0329 KOps/s	$\color{#d91a1a}-0.72\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.1364ms	6.9408ms	144.0746 Ops/s	147.2802 Ops/s	$\color{#d91a1a}-2.18\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.2161ms	0.6680ms	1.4970 KOps/s	1.5066 KOps/s	$\color{#d91a1a}-0.64\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7808ms	0.6533ms	1.5307 KOps/s	1.5518 KOps/s	$\color{#d91a1a}-1.36\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1408s	8.0096ms	124.8499 Ops/s	124.1720 Ops/s	$\color{#35bf28}+0.55\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	19.5752ms	17.2092ms	58.1085 Ops/s	60.8572 Ops/s	$\color{#d91a1a}-4.52\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.2579ms	1.2541ms	797.3633 Ops/s	835.2991 Ops/s	$\color{#d91a1a}-4.54\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1242s	7.6841ms	130.1387 Ops/s	98.3569 Ops/s	$\textbf{\color{#35bf28}+32.31\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	19.8740ms	16.9889ms	58.8621 Ops/s	61.2938 Ops/s	$\color{#d91a1a}-3.97\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.2050ms	1.2428ms	804.6444 Ops/s	739.8782 Ops/s	$\textbf{\color{#35bf28}+8.75\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1276s	7.9491ms	125.7998 Ops/s	127.2121 Ops/s	$\color{#d91a1a}-1.11\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1373s	19.4543ms	51.4026 Ops/s	60.5445 Ops/s	$\textbf{\color{#d91a1a}-15.10\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	2.6419ms	1.4354ms	696.6494 Ops/s	774.5669 Ops/s	$\textbf{\color{#d91a1a}-10.06\%}$