[BugFix] Fix MARL-DDPG tutorial and other MODE usages

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2373

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 8 Unrelated Failures

As of commit 10f9d9d with merge base 788710f ():

NEW FAILURES - The following jobs have failed:

• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 2f0ad12ce2a8ae4c9edebaf3f854e353dfc109e00b781662de2b4c467db03c18 /exec failed with exit code 139
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128
• Wheels / build-wheel-windows (3.8, 3.8) (gh)
  Process completed with exit code 1.

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Wheels / build-wheel-windows (3.10, 3.10.3) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• Wheels / build-wheel-windows (3.11, 3.11) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• Wheels / build-wheel-windows (3.12, 3.12) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• Wheels / build-wheel-windows (3.9, 3.9) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cpu (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda11_8 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda12_1 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda12_4 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	67.2747ms	58.7490ms	17.0216 Ops/s	17.3161 Ops/s	$\color{#d91a1a}-1.70\%$
test_sync	47.6042ms	31.8441ms	31.4030 Ops/s	29.6871 Ops/s	$\textbf{\color{#35bf28}+5.78\%}$
test_async	60.5524ms	30.3015ms	33.0017 Ops/s	33.0114 Ops/s	$\color{#d91a1a}-0.03\%$
test_simple	0.4752s	0.4103s	2.4371 Ops/s	2.4323 Ops/s	$\color{#35bf28}+0.20\%$
test_transformed	0.6289s	0.5684s	1.7595 Ops/s	1.7403 Ops/s	$\color{#35bf28}+1.10\%$
test_serial	1.3182s	1.2643s	0.7910 Ops/s	0.7880 Ops/s	$\color{#35bf28}+0.38\%$
test_parallel	1.1549s	1.0970s	0.9116 Ops/s	0.9196 Ops/s	$\color{#d91a1a}-0.88\%$
test_step_mdp_speed[True-True-True-True-True]	0.2323ms	25.5490μs	39.1405 KOps/s	40.0999 KOps/s	$\color{#d91a1a}-2.39\%$
test_step_mdp_speed[True-True-True-True-False]	50.6550μs	14.9611μs	66.8398 KOps/s	69.5593 KOps/s	$\color{#d91a1a}-3.91\%$
test_step_mdp_speed[True-True-True-False-True]	33.8040μs	14.8041μs	67.5490 KOps/s	69.9829 KOps/s	$\color{#d91a1a}-3.48\%$
test_step_mdp_speed[True-True-True-False-False]	30.1160μs	8.7617μs	114.1332 KOps/s	120.1131 KOps/s	$\color{#d91a1a}-4.98\%$
test_step_mdp_speed[True-True-False-True-True]	64.1200μs	27.4122μs	36.4801 KOps/s	37.6422 KOps/s	$\color{#d91a1a}-3.09\%$
test_step_mdp_speed[True-True-False-True-False]	42.7600μs	16.5778μs	60.3216 KOps/s	62.6963 KOps/s	$\color{#d91a1a}-3.79\%$
test_step_mdp_speed[True-True-False-False-True]	45.2050μs	16.5016μs	60.6003 KOps/s	63.3468 KOps/s	$\color{#d91a1a}-4.34\%$
test_step_mdp_speed[True-True-False-False-False]	38.6720μs	10.2756μs	97.3178 KOps/s	101.6857 KOps/s	$\color{#d91a1a}-4.30\%$
test_step_mdp_speed[True-False-True-True-True]	67.9780μs	29.2076μs	34.2376 KOps/s	35.4358 KOps/s	$\color{#d91a1a}-3.38\%$
test_step_mdp_speed[True-False-True-True-False]	41.1770μs	18.2801μs	54.7043 KOps/s	56.6653 KOps/s	$\color{#d91a1a}-3.46\%$
test_step_mdp_speed[True-False-True-False-True]	42.0180μs	16.4844μs	60.6634 KOps/s	62.6825 KOps/s	$\color{#d91a1a}-3.22\%$
test_step_mdp_speed[True-False-True-False-False]	30.2770μs	10.2416μs	97.6408 KOps/s	101.3357 KOps/s	$\color{#d91a1a}-3.65\%$
test_step_mdp_speed[True-False-False-True-True]	0.1094ms	30.4604μs	32.8295 KOps/s	33.2023 KOps/s	$\color{#d91a1a}-1.12\%$
test_step_mdp_speed[True-False-False-True-False]	53.8210μs	19.7179μs	50.7154 KOps/s	52.7499 KOps/s	$\color{#d91a1a}-3.86\%$
test_step_mdp_speed[True-False-False-False-True]	53.6800μs	17.9050μs	55.8505 KOps/s	58.0197 KOps/s	$\color{#d91a1a}-3.74\%$
test_step_mdp_speed[True-False-False-False-False]	47.7120μs	11.7497μs	85.1085 KOps/s	89.0378 KOps/s	$\color{#d91a1a}-4.41\%$
test_step_mdp_speed[False-True-True-True-True]	73.3180μs	28.8954μs	34.6076 KOps/s	35.6901 KOps/s	$\color{#d91a1a}-3.03\%$
test_step_mdp_speed[False-True-True-True-False]	51.7170μs	17.9792μs	55.6197 KOps/s	56.6860 KOps/s	$\color{#d91a1a}-1.88\%$
test_step_mdp_speed[False-True-True-False-True]	44.9840μs	18.8976μs	52.9168 KOps/s	53.0921 KOps/s	$\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-True-True-False-False]	32.9010μs	11.4308μs	87.4828 KOps/s	90.8045 KOps/s	$\color{#d91a1a}-3.66\%$
test_step_mdp_speed[False-True-False-True-True]	69.5400μs	30.4781μs	32.8104 KOps/s	33.5946 KOps/s	$\color{#d91a1a}-2.33\%$
test_step_mdp_speed[False-True-False-True-False]	47.4790μs	19.6127μs	50.9874 KOps/s	52.1920 KOps/s	$\color{#d91a1a}-2.31\%$
test_step_mdp_speed[False-True-False-False-True]	47.7390μs	20.4625μs	48.8699 KOps/s	49.0778 KOps/s	$\color{#d91a1a}-0.42\%$
test_step_mdp_speed[False-True-False-False-False]	52.4990μs	12.9350μs	77.3095 KOps/s	79.4505 KOps/s	$\color{#d91a1a}-2.69\%$
test_step_mdp_speed[False-False-True-True-True]	74.1097ms	34.8955μs	28.6570 KOps/s	31.6122 KOps/s	$\textbf{\color{#d91a1a}-9.35\%}$
test_step_mdp_speed[False-False-True-True-False]	45.6950μs	21.0451μs	47.5169 KOps/s	48.2574 KOps/s	$\color{#d91a1a}-1.53\%$
test_step_mdp_speed[False-False-True-False-True]	60.9940μs	20.1901μs	49.5293 KOps/s	49.5262 KOps/s	$+0.01\%$
test_step_mdp_speed[False-False-True-False-False]	34.2850μs	12.8372μs	77.8984 KOps/s	79.4671 KOps/s	$\color{#d91a1a}-1.97\%$
test_step_mdp_speed[False-False-False-True-True]	63.5390μs	33.3039μs	30.0265 KOps/s	30.6300 KOps/s	$\color{#d91a1a}-1.97\%$
test_step_mdp_speed[False-False-False-True-False]	66.3850μs	22.7952μs	43.8689 KOps/s	45.7990 KOps/s	$\color{#d91a1a}-4.21\%$
test_step_mdp_speed[False-False-False-False-True]	53.4310μs	21.5875μs	46.3232 KOps/s	46.8010 KOps/s	$\color{#d91a1a}-1.02\%$
test_step_mdp_speed[False-False-False-False-False]	38.8030μs	14.3211μs	69.8273 KOps/s	71.2847 KOps/s	$\color{#d91a1a}-2.04\%$
test_values[generalized_advantage_estimate-True-True]	9.6174ms	9.3901ms	106.4955 Ops/s	104.3860 Ops/s	$\color{#35bf28}+2.02\%$
test_values[vec_generalized_advantage_estimate-True-True]	40.2101ms	34.4673ms	29.0130 Ops/s	30.2572 Ops/s	$\color{#d91a1a}-4.11\%$
test_values[td0_return_estimate-False-False]	0.2293ms	0.1639ms	6.0996 KOps/s	6.1074 KOps/s	$\color{#d91a1a}-0.13\%$
test_values[td1_return_estimate-False-False]	26.9667ms	23.6939ms	42.2049 Ops/s	40.9055 Ops/s	$\color{#35bf28}+3.18\%$
test_values[vec_td1_return_estimate-False-False]	37.1737ms	35.4734ms	28.1901 Ops/s	30.2232 Ops/s	$\textbf{\color{#d91a1a}-6.73\%}$
test_values[td_lambda_return_estimate-True-False]	37.2733ms	34.1318ms	29.2982 Ops/s	28.2730 Ops/s	$\color{#35bf28}+3.63\%$
test_values[vec_td_lambda_return_estimate-True-False]	38.1629ms	35.4609ms	28.2001 Ops/s	30.1691 Ops/s	$\textbf{\color{#d91a1a}-6.53\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	10.7108ms	8.4156ms	118.8274 Ops/s	117.7910 Ops/s	$\color{#35bf28}+0.88\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.2786ms	1.8131ms	551.5415 Ops/s	503.8362 Ops/s	$\textbf{\color{#35bf28}+9.47\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.6432ms	0.3569ms	2.8018 KOps/s	2.8077 KOps/s	$\color{#d91a1a}-0.21\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	46.8978ms	44.5172ms	22.4632 Ops/s	22.0672 Ops/s	$\color{#35bf28}+1.79\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.1129ms	3.0161ms	331.5575 Ops/s	331.1469 Ops/s	$\color{#35bf28}+0.12\%$
test_dqn_speed	1.7692ms	1.2821ms	779.9609 Ops/s	777.8361 Ops/s	$\color{#35bf28}+0.27\%$
test_ddpg_speed	3.2242ms	2.6908ms	371.6322 Ops/s	373.0700 Ops/s	$\color{#d91a1a}-0.39\%$
test_sac_speed	12.3809ms	7.9569ms	125.6771 Ops/s	126.3016 Ops/s	$\color{#d91a1a}-0.49\%$
test_redq_speed	13.5062ms	12.3758ms	80.8030 Ops/s	81.0673 Ops/s	$\color{#d91a1a}-0.33\%$
test_redq_deprec_speed	14.5111ms	12.5055ms	79.9645 Ops/s	80.1524 Ops/s	$\color{#d91a1a}-0.23\%$
test_td3_speed	8.0829ms	7.8182ms	127.9062 Ops/s	127.8926 Ops/s	$\color{#35bf28}+0.01\%$
test_cql_speed	36.4566ms	34.9589ms	28.6051 Ops/s	28.6807 Ops/s	$\color{#d91a1a}-0.26\%$
test_a2c_speed	7.9155ms	7.2097ms	138.7027 Ops/s	138.4679 Ops/s	$\color{#35bf28}+0.17\%$
test_ppo_speed	9.2315ms	7.4656ms	133.9483 Ops/s	133.5650 Ops/s	$\color{#35bf28}+0.29\%$
test_reinforce_speed	7.1301ms	6.3872ms	156.5629 Ops/s	156.6232 Ops/s	$\color{#d91a1a}-0.04\%$
test_iql_speed	33.3125ms	31.4967ms	31.7494 Ops/s	31.7497 Ops/s	$-0.00\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.1039ms	4.6886ms	213.2851 Ops/s	213.4863 Ops/s	$\color{#d91a1a}-0.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6531ms	0.4693ms	2.1311 KOps/s	2.1252 KOps/s	$\color{#35bf28}+0.28\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7131ms	0.4471ms	2.2366 KOps/s	2.2400 KOps/s	$\color{#d91a1a}-0.15\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.3792ms	4.6263ms	216.1577 Ops/s	220.0287 Ops/s	$\color{#d91a1a}-1.76\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7443ms	0.4644ms	2.1533 KOps/s	1.8556 KOps/s	$\textbf{\color{#35bf28}+16.04\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6452ms	0.4415ms	2.2652 KOps/s	2.2611 KOps/s	$\color{#35bf28}+0.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.2646ms	1.6728ms	597.8037 Ops/s	595.1938 Ops/s	$\color{#35bf28}+0.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.7826ms	1.5850ms	630.9043 Ops/s	623.6158 Ops/s	$\color{#35bf28}+1.17\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.4261ms	4.8181ms	207.5517 Ops/s	208.7249 Ops/s	$\color{#d91a1a}-0.56\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.1357s	0.7151ms	1.3985 KOps/s	1.6457 KOps/s	$\textbf{\color{#d91a1a}-15.02\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8507ms	0.5787ms	1.7281 KOps/s	1.7328 KOps/s	$\color{#d91a1a}-0.27\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.0035ms	4.6787ms	213.7347 Ops/s	212.9718 Ops/s	$\color{#35bf28}+0.36\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7552ms	0.4707ms	2.1247 KOps/s	2.1256 KOps/s	$\color{#d91a1a}-0.04\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6140ms	0.4447ms	2.2486 KOps/s	2.2286 KOps/s	$\color{#35bf28}+0.89\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.9870ms	4.6042ms	217.1908 Ops/s	216.9914 Ops/s	$\color{#35bf28}+0.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.2724ms	0.4647ms	2.1517 KOps/s	2.1493 KOps/s	$\color{#35bf28}+0.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6455ms	0.4454ms	2.2454 KOps/s	2.2486 KOps/s	$\color{#d91a1a}-0.14\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.0491ms	4.8298ms	207.0465 Ops/s	208.0171 Ops/s	$\color{#d91a1a}-0.47\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8862ms	0.6110ms	1.6367 KOps/s	1.6533 KOps/s	$\color{#d91a1a}-1.00\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	7.1226ms	0.5911ms	1.6918 KOps/s	1.6915 KOps/s	$\color{#35bf28}+0.02\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1168s	5.9649ms	167.6488 Ops/s	171.7561 Ops/s	$\color{#d91a1a}-2.39\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1094s	15.4280ms	64.8171 Ops/s	74.2229 Ops/s	$\textbf{\color{#d91a1a}-12.67\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.8017ms	1.1311ms	884.0758 Ops/s	908.0743 Ops/s	$\color{#d91a1a}-2.64\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1042s	5.7205ms	174.8091 Ops/s	177.0459 Ops/s	$\color{#d91a1a}-1.26\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	16.7375ms	12.7550ms	78.4009 Ops/s	77.6565 Ops/s	$\color{#35bf28}+0.96\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	5.9506ms	1.2213ms	818.7873 Ops/s	906.6459 Ops/s	$\textbf{\color{#d91a1a}-9.69\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1049s	5.8402ms	171.2283 Ops/s	129.1005 Ops/s	$\textbf{\color{#35bf28}+32.63\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	16.8766ms	12.8344ms	77.9155 Ops/s	77.1505 Ops/s	$\color{#35bf28}+0.99\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.7612ms	1.2387ms	807.3148 Ops/s	753.1419 Ops/s	$\textbf{\color{#35bf28}+7.19\%}$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1074s	0.1071s	9.3414 Ops/s	9.3666 Ops/s	$\color{#d91a1a}-0.27\%$
test_sync	93.9582ms	91.2740ms	10.9560 Ops/s	10.9001 Ops/s	$\color{#35bf28}+0.51\%$
test_async	0.1705s	88.4482ms	11.3061 Ops/s	11.1792 Ops/s	$\color{#35bf28}+1.13\%$
test_single_pixels	0.1195s	0.1187s	8.4258 Ops/s	8.5281 Ops/s	$\color{#d91a1a}-1.20\%$
test_sync_pixels	85.6325ms	80.8324ms	12.3713 Ops/s	12.9101 Ops/s	$\color{#d91a1a}-4.17\%$
test_async_pixels	0.1408s	71.3221ms	14.0209 Ops/s	13.7407 Ops/s	$\color{#35bf28}+2.04\%$
test_simple	0.7817s	0.7711s	1.2968 Ops/s	1.2858 Ops/s	$\color{#35bf28}+0.86\%$
test_transformed	1.0925s	1.0215s	0.9790 Ops/s	1.0078 Ops/s	$\color{#d91a1a}-2.86\%$
test_serial	2.2632s	2.1984s	0.4549 Ops/s	0.4560 Ops/s	$\color{#d91a1a}-0.24\%$
test_parallel	1.9587s	1.8758s	0.5331 Ops/s	0.5343 Ops/s	$\color{#d91a1a}-0.22\%$
test_step_mdp_speed[True-True-True-True-True]	75.4520μs	38.5813μs	25.9193 KOps/s	26.1243 KOps/s	$\color{#d91a1a}-0.78\%$
test_step_mdp_speed[True-True-True-True-False]	39.3300μs	21.5106μs	46.4888 KOps/s	46.8703 KOps/s	$\color{#d91a1a}-0.81\%$
test_step_mdp_speed[True-True-True-False-True]	46.7510μs	21.2751μs	47.0032 KOps/s	46.9826 KOps/s	$\color{#35bf28}+0.04\%$
test_step_mdp_speed[True-True-True-False-False]	27.9910μs	12.0703μs	82.8480 KOps/s	82.9070 KOps/s	$\color{#d91a1a}-0.07\%$
test_step_mdp_speed[True-True-False-True-True]	58.9020μs	40.8561μs	24.4762 KOps/s	24.8349 KOps/s	$\color{#d91a1a}-1.44\%$
test_step_mdp_speed[True-True-False-True-False]	45.9900μs	23.9337μs	41.7820 KOps/s	42.5080 KOps/s	$\color{#d91a1a}-1.71\%$
test_step_mdp_speed[True-True-False-False-True]	40.0710μs	24.1133μs	41.4709 KOps/s	42.5652 KOps/s	$\color{#d91a1a}-2.57\%$
test_step_mdp_speed[True-True-False-False-False]	37.5100μs	14.3970μs	69.4590 KOps/s	69.1012 KOps/s	$\color{#35bf28}+0.52\%$
test_step_mdp_speed[True-False-True-True-True]	71.5210μs	42.9798μs	23.2667 KOps/s	23.4348 KOps/s	$\color{#d91a1a}-0.72\%$
test_step_mdp_speed[True-False-True-True-False]	44.4010μs	26.3371μs	37.9692 KOps/s	38.7580 KOps/s	$\color{#d91a1a}-2.04\%$
test_step_mdp_speed[True-False-True-False-True]	49.3910μs	24.9985μs	40.0024 KOps/s	42.6010 KOps/s	$\textbf{\color{#d91a1a}-6.10\%}$
test_step_mdp_speed[True-False-True-False-False]	41.5700μs	15.3182μs	65.2820 KOps/s	69.8649 KOps/s	$\textbf{\color{#d91a1a}-6.56\%}$
test_step_mdp_speed[True-False-False-True-True]	76.9220μs	47.0718μs	21.2441 KOps/s	22.0837 KOps/s	$\color{#d91a1a}-3.80\%$
test_step_mdp_speed[True-False-False-True-False]	56.5610μs	30.0769μs	33.2481 KOps/s	35.8532 KOps/s	$\textbf{\color{#d91a1a}-7.27\%}$
test_step_mdp_speed[True-False-False-False-True]	55.2310μs	26.4290μs	37.8372 KOps/s	38.9773 KOps/s	$\color{#d91a1a}-2.93\%$
test_step_mdp_speed[True-False-False-False-False]	33.6010μs	16.6774μs	59.9615 KOps/s	60.0691 KOps/s	$\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-True-True-True-True]	66.8620μs	42.8287μs	23.3488 KOps/s	23.4532 KOps/s	$\color{#d91a1a}-0.45\%$
test_step_mdp_speed[False-True-True-True-False]	47.5100μs	26.2365μs	38.1148 KOps/s	38.5415 KOps/s	$\color{#d91a1a}-1.11\%$
test_step_mdp_speed[False-True-True-False-True]	54.1710μs	28.6209μs	34.9395 KOps/s	35.2252 KOps/s	$\color{#d91a1a}-0.81\%$
test_step_mdp_speed[False-True-True-False-False]	38.4820μs	16.8103μs	59.4874 KOps/s	61.7801 KOps/s	$\color{#d91a1a}-3.71\%$
test_step_mdp_speed[False-True-False-True-True]	65.6200μs	44.8265μs	22.3082 KOps/s	22.3623 KOps/s	$\color{#d91a1a}-0.24\%$
test_step_mdp_speed[False-True-False-True-False]	44.6300μs	28.3226μs	35.3075 KOps/s	35.7289 KOps/s	$\color{#d91a1a}-1.18\%$
test_step_mdp_speed[False-True-False-False-True]	47.2500μs	30.0999μs	33.2227 KOps/s	32.7354 KOps/s	$\color{#35bf28}+1.49\%$
test_step_mdp_speed[False-True-False-False-False]	42.3710μs	19.1106μs	52.3270 KOps/s	54.0498 KOps/s	$\color{#d91a1a}-3.19\%$
test_step_mdp_speed[False-False-True-True-True]	3.8370ms	48.1606μs	20.7639 KOps/s	20.9156 KOps/s	$\color{#d91a1a}-0.73\%$
test_step_mdp_speed[False-False-True-True-False]	55.1310μs	30.9882μs	32.2704 KOps/s	32.7266 KOps/s	$\color{#d91a1a}-1.39\%$
test_step_mdp_speed[False-False-True-False-True]	49.3800μs	31.6714μs	31.5742 KOps/s	32.6828 KOps/s	$\color{#d91a1a}-3.39\%$
test_step_mdp_speed[False-False-True-False-False]	37.1210μs	19.2414μs	51.9712 KOps/s	53.8226 KOps/s	$\color{#d91a1a}-3.44\%$
test_step_mdp_speed[False-False-False-True-True]	70.9610μs	49.7436μs	20.1031 KOps/s	20.0866 KOps/s	$\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-False-False-True-False]	65.6810μs	33.6204μs	29.7439 KOps/s	30.4807 KOps/s	$\color{#d91a1a}-2.42\%$
test_step_mdp_speed[False-False-False-False-True]	52.4510μs	32.4364μs	30.8296 KOps/s	30.8343 KOps/s	$\color{#d91a1a}-0.02\%$
test_step_mdp_speed[False-False-False-False-False]	36.9610μs	21.3435μs	46.8527 KOps/s	48.1144 KOps/s	$\color{#d91a1a}-2.62\%$
test_values[generalized_advantage_estimate-True-True]	26.3478ms	24.8156ms	40.2972 Ops/s	40.2249 Ops/s	$\color{#35bf28}+0.18\%$
test_values[vec_generalized_advantage_estimate-True-True]	87.8579ms	2.6591ms	376.0631 Ops/s	371.2298 Ops/s	$\color{#35bf28}+1.30\%$
test_values[td0_return_estimate-False-False]	92.2820μs	67.7107μs	14.7687 KOps/s	14.9262 KOps/s	$\color{#d91a1a}-1.06\%$
test_values[td1_return_estimate-False-False]	55.8500ms	55.4997ms	18.0181 Ops/s	17.9144 Ops/s	$\color{#35bf28}+0.58\%$
test_values[vec_td1_return_estimate-False-False]	1.3324ms	1.0909ms	916.7079 Ops/s	911.5234 Ops/s	$\color{#35bf28}+0.57\%$
test_values[td_lambda_return_estimate-True-False]	88.6008ms	88.1887ms	11.3393 Ops/s	11.2443 Ops/s	$\color{#35bf28}+0.84\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.3901ms	1.0884ms	918.8064 Ops/s	911.1244 Ops/s	$\color{#35bf28}+0.84\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	25.1795ms	24.9539ms	40.0739 Ops/s	40.1295 Ops/s	$\color{#d91a1a}-0.14\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9354ms	0.7297ms	1.3704 KOps/s	1.3673 KOps/s	$\color{#35bf28}+0.22\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7445ms	0.6790ms	1.4728 KOps/s	1.4665 KOps/s	$\color{#35bf28}+0.42\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5201ms	1.4790ms	676.1483 Ops/s	674.9353 Ops/s	$\color{#35bf28}+0.18\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.7410ms	0.6968ms	1.4352 KOps/s	1.4311 KOps/s	$\color{#35bf28}+0.29\%$
test_dqn_speed	7.0206ms	1.3555ms	737.7202 Ops/s	751.0536 Ops/s	$\color{#d91a1a}-1.78\%$
test_ddpg_speed	2.9395ms	2.7383ms	365.1898 Ops/s	370.7443 Ops/s	$\color{#d91a1a}-1.50\%$
test_sac_speed	8.1868ms	7.8967ms	126.6346 Ops/s	128.7238 Ops/s	$\color{#d91a1a}-1.62\%$
test_redq_speed	11.0835ms	10.0188ms	99.8128 Ops/s	100.3256 Ops/s	$\color{#d91a1a}-0.51\%$
test_redq_deprec_speed	11.1947ms	10.9112ms	91.6488 Ops/s	94.4007 Ops/s	$\color{#d91a1a}-2.92\%$
test_td3_speed	7.9298ms	7.8255ms	127.7877 Ops/s	128.0985 Ops/s	$\color{#d91a1a}-0.24\%$
test_cql_speed	25.5857ms	24.9396ms	40.0969 Ops/s	41.1555 Ops/s	$\color{#d91a1a}-2.57\%$
test_a2c_speed	5.7540ms	5.5139ms	181.3610 Ops/s	189.2780 Ops/s	$\color{#d91a1a}-4.18\%$
test_ppo_speed	6.2136ms	5.9304ms	168.6229 Ops/s	174.6135 Ops/s	$\color{#d91a1a}-3.43\%$
test_reinforce_speed	5.2083ms	4.4050ms	227.0130 Ops/s	236.5066 Ops/s	$\color{#d91a1a}-4.01\%$
test_iql_speed	19.6748ms	18.9392ms	52.8005 Ops/s	55.4049 Ops/s	$\color{#d91a1a}-4.70\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.8648ms	6.6998ms	149.2589 Ops/s	149.3874 Ops/s	$\color{#d91a1a}-0.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7622ms	0.5065ms	1.9742 KOps/s	1.9879 KOps/s	$\color{#d91a1a}-0.68\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6547ms	0.4864ms	2.0561 KOps/s	2.0542 KOps/s	$\color{#35bf28}+0.09\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.9212ms	6.5962ms	151.6017 Ops/s	152.3104 Ops/s	$\color{#d91a1a}-0.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.6778ms	0.5047ms	1.9814 KOps/s	1.9985 KOps/s	$\color{#d91a1a}-0.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6808ms	0.4833ms	2.0692 KOps/s	2.0988 KOps/s	$\color{#d91a1a}-1.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.1076ms	1.9334ms	517.2347 Ops/s	530.8511 Ops/s	$\color{#d91a1a}-2.57\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.0761ms	1.8385ms	543.9307 Ops/s	554.5321 Ops/s	$\color{#d91a1a}-1.91\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.0420ms	6.7809ms	147.4738 Ops/s	146.0539 Ops/s	$\color{#35bf28}+0.97\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.2517ms	0.6541ms	1.5288 KOps/s	1.5305 KOps/s	$\color{#d91a1a}-0.12\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8007ms	0.6301ms	1.5870 KOps/s	1.5904 KOps/s	$\color{#d91a1a}-0.21\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.8027ms	6.6607ms	150.1343 Ops/s	150.1549 Ops/s	$\color{#d91a1a}-0.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.6747ms	0.5189ms	1.9272 KOps/s	1.9865 KOps/s	$\color{#d91a1a}-2.99\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6390ms	0.4963ms	2.0149 KOps/s	2.0783 KOps/s	$\color{#d91a1a}-3.05\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.8057ms	6.5743ms	152.1078 Ops/s	150.6406 Ops/s	$\color{#35bf28}+0.97\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7398ms	0.5006ms	1.9977 KOps/s	2.0103 KOps/s	$\color{#d91a1a}-0.63\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	4.1012ms	0.4839ms	2.0664 KOps/s	2.1068 KOps/s	$\color{#d91a1a}-1.91\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.9073ms	6.7785ms	147.5258 Ops/s	146.1537 Ops/s	$\color{#35bf28}+0.94\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.3006ms	0.6537ms	1.5296 KOps/s	1.5246 KOps/s	$\color{#35bf28}+0.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7927ms	0.6371ms	1.5695 KOps/s	1.5676 KOps/s	$\color{#35bf28}+0.12\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1274s	7.5754ms	132.0054 Ops/s	98.6361 Ops/s	$\textbf{\color{#35bf28}+33.83\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	19.0747ms	16.2600ms	61.5006 Ops/s	61.6117 Ops/s	$\color{#d91a1a}-0.18\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.1203ms	1.2055ms	829.5021 Ops/s	773.9909 Ops/s	$\textbf{\color{#35bf28}+7.17\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1255s	9.9832ms	100.1686 Ops/s	132.1611 Ops/s	$\textbf{\color{#d91a1a}-24.21\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	18.7532ms	16.3174ms	61.2844 Ops/s	61.7559 Ops/s	$\color{#d91a1a}-0.76\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.1453ms	1.2118ms	825.2241 Ops/s	902.7926 Ops/s	$\textbf{\color{#d91a1a}-8.59\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1279s	7.7751ms	128.6159 Ops/s	128.6525 Ops/s	$\color{#d91a1a}-0.03\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	20.0852ms	16.6048ms	60.2236 Ops/s	61.1643 Ops/s	$\color{#d91a1a}-1.54\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	2.3727ms	1.3887ms	720.1020 Ops/s	694.7721 Ops/s	$\color{#35bf28}+3.65\%$