[BugFix, CI] Set TD_GET_DEFAULTS_TO_NONE=1 in all CIs

[vmoens]

This PR

• Adds TD_GET_DEFAULTS_TO_NONE=1 in all CIs to execute tests based on the new TensorDict.get behaviour
• Adds a tests-cpu-oldget CI to check that tests pass (on CPU) when TD_GET_DEFAULTS_TO_NONE=0
• Fixes behaviours for the new behaviour

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2363

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 14 New Failures, 9 Unrelated Failures

As of commit 6038c7d with merge base e76d8cb ():

NEW FAILURES - The following jobs have failed:

• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 4295fd3be7945d90d00bcaf36350f4344c9b4a9279e9b3f4b935fada121ff274 /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t aa7184f24239e57600cb172182a2244254fd416d645a933594b092a506c1dabd /exec failed with exit code 139
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  test/test_cost.py::TestDreamer::test_dreamer_actor[None-False-5-device0]
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  test/test_cost.py::TestDreamer::test_dreamer_actor[None-False-5-device0]
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  test/test_cost.py::TestDreamer::test_dreamer_actor[None-False-5-device0]
• Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh)
  test/test_cost.py::TestDreamer::test_dreamer_actor[None-False-5-device0]
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  test/test_cost.py::TestDreamer::test_dreamer_actor[None-False-5-device0]
• Unit-tests on Linux / tests-cpu-oldget (3.12) / linux-job (gh)
  test/test_cost.py::TestDreamer::test_dreamer_actor[None-False-5-device0]
• Unit-tests on Linux / tests-gpu (3.11, 12.1) / linux-job (gh)
  test/test_cost.py::TestDreamer::test_dreamer_actor[None-False-5-device0]
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  test/test_cost.py::TestDreamer::test_dreamer_actor[None-False-5-device0]
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh)
  test/test_cost.py::TestDreamer::test_dreamer_actor[None-False-5-device0]
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  test/test_cost.py::TestDreamer::test_dreamer_actor[None-False-5-device0]
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128
• Wheels / build-wheel-windows (3.12, 3.12) (gh)
  Process completed with exit code 1.

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda11_8 (gh) (similar failure)
  Unable to find any artifacts for the associated workflow
• Wheels / build-wheel-windows (3.10, 3.10.3) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• Wheels / build-wheel-windows (3.11, 3.11) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• Wheels / build-wheel-windows (3.8, 3.8) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• Wheels / build-wheel-windows (3.9, 3.9) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cpu (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda12_1 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl (pytorch/rl, python packaging/wheel/relocate.py, test/smoke_test.py, torchrl) / upload / wheel-py3_9-cuda12_4 (gh) (trunk failure)
  Unable to find any artifacts for the associated workflow
• Libs Tests on Linux / unittests-robohive (3.9, 12.1) / linux-job (gh) (trunk failure)
  test/test_libs.py::TestRoboHive::test_robohive[franka_slide_random-v3-True-True]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	61.3467ms	59.4679ms	16.8158 Ops/s	17.0436 Ops/s	$\color{#d91a1a}-1.34\%$
test_sync	38.4086ms	34.3249ms	29.1334 Ops/s	30.9975 Ops/s	$\textbf{\color{#d91a1a}-6.01\%}$
test_async	61.4821ms	31.2785ms	31.9708 Ops/s	31.7838 Ops/s	$\color{#35bf28}+0.59\%$
test_simple	0.4954s	0.4215s	2.3725 Ops/s	2.3522 Ops/s	$\color{#35bf28}+0.86\%$
test_transformed	0.6526s	0.5805s	1.7228 Ops/s	1.7214 Ops/s	$\color{#35bf28}+0.08\%$
test_serial	1.3554s	1.2865s	0.7773 Ops/s	0.7955 Ops/s	$\color{#d91a1a}-2.29\%$
test_parallel	1.1809s	1.1086s	0.9020 Ops/s	0.8901 Ops/s	$\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-True-True-True-True]	0.1729ms	24.9502μs	40.0799 KOps/s	39.2931 KOps/s	$\color{#35bf28}+2.00\%$
test_step_mdp_speed[True-True-True-True-False]	50.4650μs	14.3775μs	69.5532 KOps/s	68.7013 KOps/s	$\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-True-True-False-True]	42.5900μs	14.2990μs	69.9351 KOps/s	68.1837 KOps/s	$\color{#35bf28}+2.57\%$
test_step_mdp_speed[True-True-True-False-False]	31.9700μs	8.2875μs	120.6635 KOps/s	117.3605 KOps/s	$\color{#35bf28}+2.81\%$
test_step_mdp_speed[True-True-False-True-True]	70.5420μs	26.6342μs	37.5457 KOps/s	36.9234 KOps/s	$\color{#35bf28}+1.69\%$
test_step_mdp_speed[True-True-False-True-False]	41.9890μs	15.9215μs	62.8083 KOps/s	60.9605 KOps/s	$\color{#35bf28}+3.03\%$
test_step_mdp_speed[True-True-False-False-True]	43.3720μs	15.8307μs	63.1685 KOps/s	61.9676 KOps/s	$\color{#35bf28}+1.94\%$
test_step_mdp_speed[True-True-False-False-False]	34.8560μs	9.8128μs	101.9075 KOps/s	99.1328 KOps/s	$\color{#35bf28}+2.80\%$
test_step_mdp_speed[True-False-True-True-True]	78.1870μs	28.1791μs	35.4873 KOps/s	34.7440 KOps/s	$\color{#35bf28}+2.14\%$
test_step_mdp_speed[True-False-True-True-False]	57.4280μs	17.5704μs	56.9138 KOps/s	54.9957 KOps/s	$\color{#35bf28}+3.49\%$
test_step_mdp_speed[True-False-True-False-True]	45.7960μs	15.7983μs	63.2979 KOps/s	61.8916 KOps/s	$\color{#35bf28}+2.27\%$
test_step_mdp_speed[True-False-True-False-False]	33.2220μs	9.7950μs	102.0928 KOps/s	99.1497 KOps/s	$\color{#35bf28}+2.97\%$
test_step_mdp_speed[True-False-False-True-True]	73.1570μs	29.8524μs	33.4981 KOps/s	33.1349 KOps/s	$\color{#35bf28}+1.10\%$
test_step_mdp_speed[True-False-False-True-False]	51.2960μs	18.9470μs	52.7789 KOps/s	51.3426 KOps/s	$\color{#35bf28}+2.80\%$
test_step_mdp_speed[True-False-False-False-True]	47.9190μs	17.3634μs	57.5925 KOps/s	56.6347 KOps/s	$\color{#35bf28}+1.69\%$
test_step_mdp_speed[True-False-False-False-False]	36.5880μs	11.2770μs	88.6764 KOps/s	85.4756 KOps/s	$\color{#35bf28}+3.74\%$
test_step_mdp_speed[False-True-True-True-True]	67.0460μs	28.6893μs	34.8562 KOps/s	34.6107 KOps/s	$\color{#35bf28}+0.71\%$
test_step_mdp_speed[False-True-True-True-False]	43.8420μs	17.5805μs	56.8813 KOps/s	55.6970 KOps/s	$\color{#35bf28}+2.13\%$
test_step_mdp_speed[False-True-True-False-True]	51.2360μs	18.8613μs	53.0187 KOps/s	53.6156 KOps/s	$\color{#d91a1a}-1.11\%$
test_step_mdp_speed[False-True-True-False-False]	43.1410μs	11.0426μs	90.5581 KOps/s	88.0257 KOps/s	$\color{#35bf28}+2.88\%$
test_step_mdp_speed[False-True-False-True-True]	70.4720μs	29.9815μs	33.3539 KOps/s	33.2093 KOps/s	$\color{#35bf28}+0.44\%$
test_step_mdp_speed[False-True-False-True-False]	89.8780μs	19.0032μs	52.6227 KOps/s	51.2233 KOps/s	$\color{#35bf28}+2.73\%$
test_step_mdp_speed[False-True-False-False-True]	60.0520μs	20.0739μs	49.8160 KOps/s	49.6436 KOps/s	$\color{#35bf28}+0.35\%$
test_step_mdp_speed[False-True-False-False-False]	37.8110μs	12.5189μs	79.8795 KOps/s	77.4389 KOps/s	$\color{#35bf28}+3.15\%$
test_step_mdp_speed[False-False-True-True-True]	3.2807ms	31.5058μs	31.7402 KOps/s	31.3647 KOps/s	$\color{#35bf28}+1.20\%$
test_step_mdp_speed[False-False-True-True-False]	47.2380μs	20.6596μs	48.4035 KOps/s	47.2920 KOps/s	$\color{#35bf28}+2.35\%$
test_step_mdp_speed[False-False-True-False-True]	47.7900μs	20.0392μs	49.9021 KOps/s	49.4762 KOps/s	$\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-False-True-False-False]	47.8590μs	12.5363μs	79.7681 KOps/s	77.0997 KOps/s	$\color{#35bf28}+3.46\%$
test_step_mdp_speed[False-False-False-True-True]	65.8930μs	32.5695μs	30.7036 KOps/s	30.1680 KOps/s	$\color{#35bf28}+1.78\%$
test_step_mdp_speed[False-False-False-True-False]	49.8030μs	21.9917μs	45.4717 KOps/s	44.4177 KOps/s	$\color{#35bf28}+2.37\%$
test_step_mdp_speed[False-False-False-False-True]	55.5740μs	21.0873μs	47.4220 KOps/s	46.4467 KOps/s	$\color{#35bf28}+2.10\%$
test_step_mdp_speed[False-False-False-False-False]	45.1240μs	13.9863μs	71.4984 KOps/s	69.3845 KOps/s	$\color{#35bf28}+3.05\%$
test_values[generalized_advantage_estimate-True-True]	10.1391ms	9.6662ms	103.4529 Ops/s	106.8826 Ops/s	$\color{#d91a1a}-3.21\%$
test_values[vec_generalized_advantage_estimate-True-True]	37.4715ms	33.3847ms	29.9538 Ops/s	27.4477 Ops/s	$\textbf{\color{#35bf28}+9.13\%}$
test_values[td0_return_estimate-False-False]	0.2240ms	0.1658ms	6.0314 KOps/s	5.7707 KOps/s	$\color{#35bf28}+4.52\%$
test_values[td1_return_estimate-False-False]	24.7102ms	23.7917ms	42.0315 Ops/s	42.9802 Ops/s	$\color{#d91a1a}-2.21\%$
test_values[vec_td1_return_estimate-False-False]	50.3987ms	33.9786ms	29.4303 Ops/s	27.3025 Ops/s	$\textbf{\color{#35bf28}+7.79\%}$
test_values[td_lambda_return_estimate-True-False]	38.0913ms	34.4015ms	29.0685 Ops/s	29.3434 Ops/s	$\color{#d91a1a}-0.94\%$
test_values[vec_td_lambda_return_estimate-True-False]	41.1546ms	33.5891ms	29.7715 Ops/s	27.1839 Ops/s	$\textbf{\color{#35bf28}+9.52\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	11.9030ms	8.4164ms	118.8157 Ops/s	123.1300 Ops/s	$\color{#d91a1a}-3.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.1132ms	1.9859ms	503.5605 Ops/s	488.5812 Ops/s	$\color{#35bf28}+3.07\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4378ms	0.3523ms	2.8384 KOps/s	2.7974 KOps/s	$\color{#35bf28}+1.47\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	49.2251ms	47.0588ms	21.2500 Ops/s	20.7870 Ops/s	$\color{#35bf28}+2.23\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.9642ms	3.0327ms	329.7430 Ops/s	326.9817 Ops/s	$\color{#35bf28}+0.84\%$
test_dqn_speed	6.2211ms	1.3009ms	768.6750 Ops/s	786.4911 Ops/s	$\color{#d91a1a}-2.27\%$
test_ddpg_speed	3.1472ms	2.6855ms	372.3763 Ops/s	375.1099 Ops/s	$\color{#d91a1a}-0.73\%$
test_sac_speed	9.3135ms	7.9493ms	125.7978 Ops/s	126.7249 Ops/s	$\color{#d91a1a}-0.73\%$
test_redq_speed	13.2953ms	12.4253ms	80.4807 Ops/s	77.9782 Ops/s	$\color{#35bf28}+3.21\%$
test_redq_deprec_speed	14.0622ms	12.4986ms	80.0089 Ops/s	76.2969 Ops/s	$\color{#35bf28}+4.87\%$
test_td3_speed	9.0853ms	7.8384ms	127.5772 Ops/s	128.4014 Ops/s	$\color{#d91a1a}-0.64\%$
test_cql_speed	36.8611ms	35.3566ms	28.2833 Ops/s	28.3470 Ops/s	$\color{#d91a1a}-0.22\%$
test_a2c_speed	8.3695ms	7.2947ms	137.0865 Ops/s	137.1751 Ops/s	$\color{#d91a1a}-0.06\%$
test_ppo_speed	8.7053ms	7.6605ms	130.5389 Ops/s	131.8397 Ops/s	$\color{#d91a1a}-0.99\%$
test_reinforce_speed	7.6526ms	6.5429ms	152.8372 Ops/s	154.9577 Ops/s	$\color{#d91a1a}-1.37\%$
test_iql_speed	33.5547ms	31.9052ms	31.3429 Ops/s	31.5251 Ops/s	$\color{#d91a1a}-0.58\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.9982ms	4.7678ms	209.7411 Ops/s	203.4550 Ops/s	$\color{#35bf28}+3.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7561ms	0.4779ms	2.0924 KOps/s	2.0639 KOps/s	$\color{#35bf28}+1.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6655ms	0.4563ms	2.1917 KOps/s	2.1930 KOps/s	$\color{#d91a1a}-0.06\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.0164ms	4.6578ms	214.6946 Ops/s	207.8547 Ops/s	$\color{#35bf28}+3.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7263ms	0.4707ms	2.1244 KOps/s	1.7959 KOps/s	$\textbf{\color{#35bf28}+18.29\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6381ms	0.4562ms	2.1921 KOps/s	2.1714 KOps/s	$\color{#35bf28}+0.95\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.2742ms	1.7010ms	587.9054 Ops/s	592.9415 Ops/s	$\color{#d91a1a}-0.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	8.2134ms	1.6194ms	617.5232 Ops/s	628.2092 Ops/s	$\color{#d91a1a}-1.70\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.2109ms	4.9581ms	201.6914 Ops/s	203.9965 Ops/s	$\color{#d91a1a}-1.13\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.6068ms	0.6136ms	1.6299 KOps/s	1.6198 KOps/s	$\color{#35bf28}+0.62\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8054ms	0.5866ms	1.7048 KOps/s	1.7115 KOps/s	$\color{#d91a1a}-0.39\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.3450ms	4.6974ms	212.8857 Ops/s	207.0439 Ops/s	$\color{#35bf28}+2.82\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6170ms	0.4806ms	2.0809 KOps/s	2.0625 KOps/s	$\color{#35bf28}+0.89\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5911ms	0.4505ms	2.2196 KOps/s	2.1902 KOps/s	$\color{#35bf28}+1.34\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.1011ms	4.6983ms	212.8430 Ops/s	207.9656 Ops/s	$\color{#35bf28}+2.35\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.8471ms	0.4815ms	2.0767 KOps/s	2.0978 KOps/s	$\color{#d91a1a}-1.00\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5738ms	0.4505ms	2.2197 KOps/s	2.2344 KOps/s	$\color{#d91a1a}-0.66\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.8356ms	4.8925ms	204.3925 Ops/s	201.7537 Ops/s	$\color{#35bf28}+1.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.1120s	0.7744ms	1.2914 KOps/s	1.6057 KOps/s	$\textbf{\color{#d91a1a}-19.57\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7585ms	0.5839ms	1.7127 KOps/s	1.6848 KOps/s	$\color{#35bf28}+1.66\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1154s	5.9859ms	167.0600 Ops/s	158.5665 Ops/s	$\textbf{\color{#35bf28}+5.36\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	17.2535ms	13.2840ms	75.2788 Ops/s	67.9802 Ops/s	$\textbf{\color{#35bf28}+10.74\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	4.5147ms	1.1854ms	843.5982 Ops/s	819.0435 Ops/s	$\color{#35bf28}+3.00\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1086s	5.8632ms	170.5543 Ops/s	168.8648 Ops/s	$\color{#35bf28}+1.00\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	16.8871ms	13.1521ms	76.0337 Ops/s	76.9835 Ops/s	$\color{#d91a1a}-1.23\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.4875ms	1.3909ms	718.9630 Ops/s	845.1694 Ops/s	$\textbf{\color{#d91a1a}-14.93\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1140s	8.2260ms	121.5651 Ops/s	166.3599 Ops/s	$\textbf{\color{#d91a1a}-26.93\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	17.0926ms	13.4432ms	74.3870 Ops/s	63.6971 Ops/s	$\textbf{\color{#35bf28}+16.78\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.8775ms	1.2689ms	788.0853 Ops/s	790.1952 Ops/s	$\color{#d91a1a}-0.27\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1053s	0.1047s	9.5480 Ops/s	9.4352 Ops/s	$\color{#35bf28}+1.19\%$
test_sync	95.2362ms	92.6349ms	10.7951 Ops/s	10.6674 Ops/s	$\color{#35bf28}+1.20\%$
test_async	0.1733s	87.0373ms	11.4893 Ops/s	11.2240 Ops/s	$\color{#35bf28}+2.36\%$
test_single_pixels	0.1172s	0.1160s	8.6206 Ops/s	8.5609 Ops/s	$\color{#35bf28}+0.70\%$
test_sync_pixels	77.0084ms	76.1421ms	13.1333 Ops/s	13.2469 Ops/s	$\color{#d91a1a}-0.86\%$
test_async_pixels	0.1372s	71.0910ms	14.0665 Ops/s	14.1813 Ops/s	$\color{#d91a1a}-0.81\%$
test_simple	0.7639s	0.7529s	1.3281 Ops/s	1.2932 Ops/s	$\color{#35bf28}+2.70\%$
test_transformed	1.0635s	0.9989s	1.0011 Ops/s	1.0269 Ops/s	$\color{#d91a1a}-2.51\%$
test_serial	2.2264s	2.1456s	0.4661 Ops/s	0.4615 Ops/s	$\color{#35bf28}+1.00\%$
test_parallel	1.9210s	1.8753s	0.5332 Ops/s	0.5363 Ops/s	$\color{#d91a1a}-0.57\%$
test_step_mdp_speed[True-True-True-True-True]	0.1630ms	37.9695μs	26.3370 KOps/s	26.5814 KOps/s	$\color{#d91a1a}-0.92\%$
test_step_mdp_speed[True-True-True-True-False]	81.3210μs	21.2003μs	47.1692 KOps/s	47.8670 KOps/s	$\color{#d91a1a}-1.46\%$
test_step_mdp_speed[True-True-True-False-True]	39.1900μs	21.0912μs	47.4131 KOps/s	47.5373 KOps/s	$\color{#d91a1a}-0.26\%$
test_step_mdp_speed[True-True-True-False-False]	35.8100μs	12.1052μs	82.6090 KOps/s	84.6971 KOps/s	$\color{#d91a1a}-2.47\%$
test_step_mdp_speed[True-True-False-True-True]	62.1910μs	39.5365μs	25.2931 KOps/s	24.9416 KOps/s	$\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-True-False-True-False]	43.0310μs	23.3969μs	42.7408 KOps/s	43.6045 KOps/s	$\color{#d91a1a}-1.98\%$
test_step_mdp_speed[True-True-False-False-True]	40.9010μs	23.5040μs	42.5459 KOps/s	42.2429 KOps/s	$\color{#35bf28}+0.72\%$
test_step_mdp_speed[True-True-False-False-False]	47.4110μs	14.4840μs	69.0416 KOps/s	71.0644 KOps/s	$\color{#d91a1a}-2.85\%$
test_step_mdp_speed[True-False-True-True-True]	67.3410μs	42.5020μs	23.5283 KOps/s	23.7720 KOps/s	$\color{#d91a1a}-1.03\%$
test_step_mdp_speed[True-False-True-True-False]	46.1300μs	25.9171μs	38.5846 KOps/s	39.7193 KOps/s	$\color{#d91a1a}-2.86\%$
test_step_mdp_speed[True-False-True-False-True]	86.8920μs	23.0688μs	43.3486 KOps/s	43.9842 KOps/s	$\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-False-True-False-False]	30.5410μs	14.2987μs	69.9366 KOps/s	70.6885 KOps/s	$\color{#d91a1a}-1.06\%$
test_step_mdp_speed[True-False-False-True-True]	66.7220μs	44.2598μs	22.5939 KOps/s	22.9852 KOps/s	$\color{#d91a1a}-1.70\%$
test_step_mdp_speed[True-False-False-True-False]	43.8010μs	27.8941μs	35.8498 KOps/s	36.0052 KOps/s	$\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-False-False-False-True]	51.0310μs	25.4291μs	39.3250 KOps/s	40.6322 KOps/s	$\color{#d91a1a}-3.22\%$
test_step_mdp_speed[True-False-False-False-False]	35.9910μs	16.3961μs	60.9902 KOps/s	61.8706 KOps/s	$\color{#d91a1a}-1.42\%$
test_step_mdp_speed[False-True-True-True-True]	63.2210μs	41.7965μs	23.9255 KOps/s	24.1088 KOps/s	$\color{#d91a1a}-0.76\%$
test_step_mdp_speed[False-True-True-True-False]	0.2159ms	25.7195μs	38.8810 KOps/s	39.9669 KOps/s	$\color{#d91a1a}-2.72\%$
test_step_mdp_speed[False-True-True-False-True]	0.2304ms	28.2502μs	35.3979 KOps/s	36.5471 KOps/s	$\color{#d91a1a}-3.14\%$
test_step_mdp_speed[False-True-True-False-False]	0.2004ms	16.1837μs	61.7905 KOps/s	62.7008 KOps/s	$\color{#d91a1a}-1.45\%$
test_step_mdp_speed[False-True-False-True-True]	69.1510μs	44.4235μs	22.5106 KOps/s	22.6319 KOps/s	$\color{#d91a1a}-0.54\%$
test_step_mdp_speed[False-True-False-True-False]	48.7610μs	27.8836μs	35.8633 KOps/s	36.4586 KOps/s	$\color{#d91a1a}-1.63\%$
test_step_mdp_speed[False-True-False-False-True]	47.4710μs	29.9515μs	33.3874 KOps/s	33.8122 KOps/s	$\color{#d91a1a}-1.26\%$
test_step_mdp_speed[False-True-False-False-False]	36.9010μs	18.4253μs	54.2733 KOps/s	53.6604 KOps/s	$\color{#35bf28}+1.14\%$
test_step_mdp_speed[False-False-True-True-True]	3.9152ms	47.0730μs	21.2436 KOps/s	21.3903 KOps/s	$\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-False-True-True-False]	55.5710μs	30.3496μs	32.9493 KOps/s	33.3967 KOps/s	$\color{#d91a1a}-1.34\%$
test_step_mdp_speed[False-False-True-False-True]	49.3910μs	30.2896μs	33.0146 KOps/s	33.5753 KOps/s	$\color{#d91a1a}-1.67\%$
test_step_mdp_speed[False-False-True-False-False]	0.1982ms	18.4063μs	54.3293 KOps/s	54.6860 KOps/s	$\color{#d91a1a}-0.65\%$
test_step_mdp_speed[False-False-False-True-True]	0.2307ms	48.3854μs	20.6674 KOps/s	20.6587 KOps/s	$\color{#35bf28}+0.04\%$
test_step_mdp_speed[False-False-False-True-False]	48.1910μs	32.6590μs	30.6195 KOps/s	31.7064 KOps/s	$\color{#d91a1a}-3.43\%$
test_step_mdp_speed[False-False-False-False-True]	56.0320μs	31.8083μs	31.4384 KOps/s	31.5626 KOps/s	$\color{#d91a1a}-0.39\%$
test_step_mdp_speed[False-False-False-False-False]	43.0700μs	20.6784μs	48.3596 KOps/s	49.6377 KOps/s	$\color{#d91a1a}-2.57\%$
test_values[generalized_advantage_estimate-True-True]	23.5982ms	23.0001ms	43.4780 Ops/s	41.3223 Ops/s	$\textbf{\color{#35bf28}+5.22\%}$
test_values[vec_generalized_advantage_estimate-True-True]	96.0361ms	2.8036ms	356.6817 Ops/s	357.9025 Ops/s	$\color{#d91a1a}-0.34\%$
test_values[td0_return_estimate-False-False]	86.7920μs	63.2857μs	15.8014 KOps/s	15.6110 KOps/s	$\color{#35bf28}+1.22\%$
test_values[td1_return_estimate-False-False]	52.8236ms	52.1743ms	19.1665 Ops/s	18.5006 Ops/s	$\color{#35bf28}+3.60\%$
test_values[vec_td1_return_estimate-False-False]	1.3327ms	1.0618ms	941.7635 Ops/s	935.1000 Ops/s	$\color{#35bf28}+0.71\%$
test_values[td_lambda_return_estimate-True-False]	83.5580ms	82.8796ms	12.0657 Ops/s	11.7037 Ops/s	$\color{#35bf28}+3.09\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.4093ms	1.0621ms	941.5723 Ops/s	935.6489 Ops/s	$\color{#35bf28}+0.63\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	23.3688ms	22.9682ms	43.5385 Ops/s	41.7675 Ops/s	$\color{#35bf28}+4.24\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9171ms	0.6928ms	1.4435 KOps/s	1.4153 KOps/s	$\color{#35bf28}+1.99\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.8108ms	0.6472ms	1.5451 KOps/s	1.5237 KOps/s	$\color{#35bf28}+1.41\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5763ms	1.4457ms	691.7272 Ops/s	686.9766 Ops/s	$\color{#35bf28}+0.69\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.8331ms	0.6616ms	1.5116 KOps/s	1.4921 KOps/s	$\color{#35bf28}+1.30\%$
test_dqn_speed	7.8758ms	1.3721ms	728.8168 Ops/s	760.9041 Ops/s	$\color{#d91a1a}-4.22\%$
test_ddpg_speed	2.9950ms	2.7585ms	362.5150 Ops/s	371.9561 Ops/s	$\color{#d91a1a}-2.54\%$
test_sac_speed	9.5728ms	7.9071ms	126.4682 Ops/s	128.7897 Ops/s	$\color{#d91a1a}-1.80\%$
test_redq_speed	11.8269ms	10.1514ms	98.5085 Ops/s	96.6678 Ops/s	$\color{#35bf28}+1.90\%$
test_redq_deprec_speed	11.8365ms	11.2181ms	89.1414 Ops/s	89.8662 Ops/s	$\color{#d91a1a}-0.81\%$
test_td3_speed	8.0970ms	7.7918ms	128.3402 Ops/s	130.0531 Ops/s	$\color{#d91a1a}-1.32\%$
test_cql_speed	26.8586ms	25.2219ms	39.6481 Ops/s	40.1902 Ops/s	$\color{#d91a1a}-1.35\%$
test_a2c_speed	5.8683ms	5.5482ms	180.2384 Ops/s	179.2765 Ops/s	$\color{#35bf28}+0.54\%$
test_ppo_speed	6.1391ms	5.8832ms	169.9763 Ops/s	171.6192 Ops/s	$\color{#d91a1a}-0.96\%$
test_reinforce_speed	5.2409ms	4.5559ms	219.4979 Ops/s	224.0815 Ops/s	$\color{#d91a1a}-2.05\%$
test_iql_speed	19.6907ms	19.2500ms	51.9480 Ops/s	51.3739 Ops/s	$\color{#35bf28}+1.12\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.9058ms	6.6830ms	149.6335 Ops/s	154.4791 Ops/s	$\color{#d91a1a}-3.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.1088s	0.6010ms	1.6638 KOps/s	1.9576 KOps/s	$\textbf{\color{#d91a1a}-15.01\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6889ms	0.4953ms	2.0191 KOps/s	2.0322 KOps/s	$\color{#d91a1a}-0.65\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.8626ms	6.5260ms	153.2328 Ops/s	155.8966 Ops/s	$\color{#d91a1a}-1.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.2236ms	0.5130ms	1.9493 KOps/s	1.9770 KOps/s	$\color{#d91a1a}-1.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7117ms	0.4897ms	2.0421 KOps/s	2.0541 KOps/s	$\color{#d91a1a}-0.58\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.1410ms	1.9276ms	518.7908 Ops/s	519.4664 Ops/s	$\color{#d91a1a}-0.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.0014ms	1.8182ms	550.0064 Ops/s	542.9849 Ops/s	$\color{#35bf28}+1.29\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.0291ms	6.7518ms	148.1076 Ops/s	150.6105 Ops/s	$\color{#d91a1a}-1.66\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0020ms	0.6596ms	1.5162 KOps/s	1.5207 KOps/s	$\color{#d91a1a}-0.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8125ms	0.6389ms	1.5651 KOps/s	1.5727 KOps/s	$\color{#d91a1a}-0.48\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.8381ms	6.6264ms	150.9115 Ops/s	153.5049 Ops/s	$\color{#d91a1a}-1.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.0759ms	0.5163ms	1.9367 KOps/s	1.9476 KOps/s	$\color{#d91a1a}-0.56\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6751ms	0.4955ms	2.0180 KOps/s	2.0210 KOps/s	$\color{#d91a1a}-0.15\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.9407ms	6.5741ms	152.1117 Ops/s	154.6925 Ops/s	$\color{#d91a1a}-1.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1393s	0.7042ms	1.4201 KOps/s	1.9686 KOps/s	$\textbf{\color{#d91a1a}-27.86\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6481ms	0.4904ms	2.0390 KOps/s	2.0469 KOps/s	$\color{#d91a1a}-0.38\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	8.4627ms	6.9290ms	144.3209 Ops/s	150.5704 Ops/s	$\color{#d91a1a}-4.15\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8616ms	0.6636ms	1.5070 KOps/s	1.5174 KOps/s	$\color{#d91a1a}-0.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.3106ms	0.6486ms	1.5419 KOps/s	1.2613 KOps/s	$\textbf{\color{#35bf28}+22.25\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1285s	7.5330ms	132.7496 Ops/s	134.3476 Ops/s	$\color{#d91a1a}-1.19\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	19.1025ms	15.9551ms	62.6758 Ops/s	63.4944 Ops/s	$\color{#d91a1a}-1.29\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.2432ms	1.2653ms	790.3547 Ops/s	799.7528 Ops/s	$\color{#d91a1a}-1.18\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1251s	7.5021ms	133.2962 Ops/s	134.3970 Ops/s	$\color{#d91a1a}-0.82\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1364s	18.3097ms	54.6158 Ops/s	63.9872 Ops/s	$\textbf{\color{#d91a1a}-14.65\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	7.3237ms	1.3603ms	735.1418 Ops/s	795.0032 Ops/s	$\textbf{\color{#d91a1a}-7.53\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1253s	7.6549ms	130.6355 Ops/s	130.7575 Ops/s	$\color{#d91a1a}-0.09\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.3886ms	16.0443ms	62.3276 Ops/s	63.6365 Ops/s	$\color{#d91a1a}-2.06\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.2872ms	1.5056ms	664.2070 Ops/s	663.8550 Ops/s	$\color{#35bf28}+0.05\%$