[Refactor] Deprecate NormalParamWrapper

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2308

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Pending, 1 Unrelated Failure

As of commit 5066ec0 with merge base 94abb50 ():

NEW FAILURE - The following job has failed:

• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t f2db62e89ee5463aafd4fe592c469eb3af0a7301027d7f7ac4675153cb4e4697 /exec failed with exit code 139

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Windows / unittests-cpu / windows-job (gh) (trunk failure)
  test/test_transforms.py::TestActionDiscretizer::test_trans_parallel_env_check[False]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	59.5193ms	58.6612ms	17.0470 Ops/s	17.1092 Ops/s	$\color{#d91a1a}-0.36\%$
test_sync	42.3374ms	31.9010ms	31.3470 Ops/s	30.8822 Ops/s	$\color{#35bf28}+1.51\%$
test_async	75.5890ms	30.2112ms	33.1003 Ops/s	32.3573 Ops/s	$\color{#35bf28}+2.30\%$
test_simple	0.4909s	0.4134s	2.4191 Ops/s	2.3575 Ops/s	$\color{#35bf28}+2.61\%$
test_transformed	0.6348s	0.5757s	1.7370 Ops/s	1.7610 Ops/s	$\color{#d91a1a}-1.36\%$
test_serial	1.3128s	1.2483s	0.8011 Ops/s	0.7864 Ops/s	$\color{#35bf28}+1.87\%$
test_parallel	1.1565s	1.0941s	0.9140 Ops/s	0.8747 Ops/s	$\color{#35bf28}+4.49\%$
test_step_mdp_speed[True-True-True-True-True]	0.1940ms	24.1255μs	41.4499 KOps/s	40.9981 KOps/s	$\color{#35bf28}+1.10\%$
test_step_mdp_speed[True-True-True-True-False]	77.6780μs	14.0206μs	71.3234 KOps/s	69.9380 KOps/s	$\color{#35bf28}+1.98\%$
test_step_mdp_speed[True-True-True-False-True]	41.8280μs	13.7971μs	72.4791 KOps/s	71.7323 KOps/s	$\color{#35bf28}+1.04\%$
test_step_mdp_speed[True-True-True-False-False]	33.1720μs	8.1650μs	122.4747 KOps/s	121.2782 KOps/s	$\color{#35bf28}+0.99\%$
test_step_mdp_speed[True-True-False-True-True]	0.1081ms	25.8451μs	38.6920 KOps/s	38.0733 KOps/s	$\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-True-False-True-False]	46.6670μs	15.5926μs	64.1329 KOps/s	63.0527 KOps/s	$\color{#35bf28}+1.71\%$
test_step_mdp_speed[True-True-False-False-True]	51.6750μs	15.3201μs	65.2737 KOps/s	64.3631 KOps/s	$\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-True-False-False-False]	41.1060μs	9.5520μs	104.6906 KOps/s	103.0699 KOps/s	$\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-False-True-True-True]	60.2920μs	27.6052μs	36.2250 KOps/s	35.4510 KOps/s	$\color{#35bf28}+2.18\%$
test_step_mdp_speed[True-False-True-True-False]	60.9130μs	17.2345μs	58.0231 KOps/s	57.6825 KOps/s	$\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-False-True-False-True]	42.2990μs	15.5551μs	64.2877 KOps/s	63.6105 KOps/s	$\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-False-True-False-False]	36.1170μs	9.6436μs	103.6956 KOps/s	101.6943 KOps/s	$\color{#35bf28}+1.97\%$
test_step_mdp_speed[True-False-False-True-True]	67.1950μs	29.3687μs	34.0499 KOps/s	33.1371 KOps/s	$\color{#35bf28}+2.75\%$
test_step_mdp_speed[True-False-False-True-False]	46.4160μs	18.7722μs	53.2704 KOps/s	53.1977 KOps/s	$\color{#35bf28}+0.14\%$
test_step_mdp_speed[True-False-False-False-True]	51.0050μs	17.0800μs	58.5479 KOps/s	57.8838 KOps/s	$\color{#35bf28}+1.15\%$
test_step_mdp_speed[True-False-False-False-False]	38.0510μs	11.1483μs	89.6996 KOps/s	90.2983 KOps/s	$\color{#d91a1a}-0.66\%$
test_step_mdp_speed[False-True-True-True-True]	64.7110μs	27.8302μs	35.9322 KOps/s	35.5439 KOps/s	$\color{#35bf28}+1.09\%$
test_step_mdp_speed[False-True-True-True-False]	41.4570μs	17.2470μs	57.9810 KOps/s	57.1681 KOps/s	$\color{#35bf28}+1.42\%$
test_step_mdp_speed[False-True-True-False-True]	51.0850μs	18.2760μs	54.7166 KOps/s	51.9655 KOps/s	$\textbf{\color{#35bf28}+5.29\%}$
test_step_mdp_speed[False-True-True-False-False]	34.6940μs	10.9272μs	91.5147 KOps/s	90.1554 KOps/s	$\color{#35bf28}+1.51\%$
test_step_mdp_speed[False-True-False-True-True]	81.2600μs	29.0242μs	34.4540 KOps/s	34.1128 KOps/s	$\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-True-False-True-False]	49.3120μs	18.8854μs	52.9511 KOps/s	53.0107 KOps/s	$\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-True-False-False-True]	52.7980μs	20.1873μs	49.5360 KOps/s	48.7566 KOps/s	$\color{#35bf28}+1.60\%$
test_step_mdp_speed[False-True-False-False-False]	51.5650μs	12.3801μs	80.7748 KOps/s	80.3086 KOps/s	$\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-False-True-True-True]	3.8967ms	30.9925μs	32.2659 KOps/s	31.0675 KOps/s	$\color{#35bf28}+3.86\%$
test_step_mdp_speed[False-False-True-True-False]	51.4550μs	20.2505μs	49.3815 KOps/s	49.2781 KOps/s	$\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-False-True-False-True]	46.9570μs	20.0387μs	49.9035 KOps/s	52.4627 KOps/s	$\color{#d91a1a}-4.88\%$
test_step_mdp_speed[False-False-True-False-False]	31.9390μs	12.3637μs	80.8818 KOps/s	81.8636 KOps/s	$\color{#d91a1a}-1.20\%$
test_step_mdp_speed[False-False-False-True-True]	92.4420μs	31.9937μs	31.2562 KOps/s	31.8615 KOps/s	$\color{#d91a1a}-1.90\%$
test_step_mdp_speed[False-False-False-True-False]	61.3540μs	21.5181μs	46.4724 KOps/s	46.8146 KOps/s	$\color{#d91a1a}-0.73\%$
test_step_mdp_speed[False-False-False-False-True]	70.1100μs	21.1061μs	47.3797 KOps/s	48.9936 KOps/s	$\color{#d91a1a}-3.29\%$
test_step_mdp_speed[False-False-False-False-False]	59.5700μs	13.7608μs	72.6700 KOps/s	74.6654 KOps/s	$\color{#d91a1a}-2.67\%$
test_values[generalized_advantage_estimate-True-True]	11.0288ms	9.3864ms	106.5366 Ops/s	102.0460 Ops/s	$\color{#35bf28}+4.40\%$
test_values[vec_generalized_advantage_estimate-True-True]	37.5237ms	35.6793ms	28.0274 Ops/s	28.1524 Ops/s	$\color{#d91a1a}-0.44\%$
test_values[td0_return_estimate-False-False]	0.2342ms	0.1722ms	5.8082 KOps/s	5.8659 KOps/s	$\color{#d91a1a}-0.98\%$
test_values[td1_return_estimate-False-False]	27.2885ms	23.5235ms	42.5106 Ops/s	41.1189 Ops/s	$\color{#35bf28}+3.38\%$
test_values[vec_td1_return_estimate-False-False]	37.8276ms	35.5327ms	28.1431 Ops/s	28.1477 Ops/s	$\color{#d91a1a}-0.02\%$
test_values[td_lambda_return_estimate-True-False]	38.0143ms	33.8244ms	29.5645 Ops/s	28.6514 Ops/s	$\color{#35bf28}+3.19\%$
test_values[vec_td_lambda_return_estimate-True-False]	38.3904ms	35.6503ms	28.0502 Ops/s	27.9300 Ops/s	$\color{#35bf28}+0.43\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	11.5350ms	8.1793ms	122.2602 Ops/s	117.8315 Ops/s	$\color{#35bf28}+3.76\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.3461ms	2.0307ms	492.4437 Ops/s	507.8029 Ops/s	$\color{#d91a1a}-3.02\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4503ms	0.3573ms	2.7989 KOps/s	2.7559 KOps/s	$\color{#35bf28}+1.56\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	50.0232ms	48.2641ms	20.7193 Ops/s	22.4025 Ops/s	$\textbf{\color{#d91a1a}-7.51\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	4.1141ms	3.0378ms	329.1853 Ops/s	326.4260 Ops/s	$\color{#35bf28}+0.85\%$
test_dqn_speed	1.8029ms	1.3729ms	728.4077 Ops/s	720.3764 Ops/s	$\color{#35bf28}+1.11\%$
test_ddpg_speed	3.7556ms	2.9073ms	343.9584 Ops/s	342.4722 Ops/s	$\color{#35bf28}+0.43\%$
test_sac_speed	10.6448ms	8.4779ms	117.9531 Ops/s	115.1256 Ops/s	$\color{#35bf28}+2.46\%$
test_redq_speed	15.6624ms	13.7815ms	72.5611 Ops/s	71.5106 Ops/s	$\color{#35bf28}+1.47\%$
test_redq_deprec_speed	16.9362ms	13.2548ms	75.4443 Ops/s	71.5093 Ops/s	$\textbf{\color{#35bf28}+5.50\%}$
test_td3_speed	9.0144ms	8.4279ms	118.6540 Ops/s	118.1054 Ops/s	$\color{#35bf28}+0.46\%$
test_cql_speed	39.5668ms	36.7201ms	27.2330 Ops/s	26.8929 Ops/s	$\color{#35bf28}+1.26\%$
test_a2c_speed	8.7339ms	7.4706ms	133.8575 Ops/s	133.8022 Ops/s	$\color{#35bf28}+0.04\%$
test_ppo_speed	8.7883ms	7.7140ms	129.6346 Ops/s	129.3882 Ops/s	$\color{#35bf28}+0.19\%$
test_reinforce_speed	8.1635ms	6.5902ms	151.7411 Ops/s	152.4676 Ops/s	$\color{#d91a1a}-0.48\%$
test_iql_speed	34.0641ms	32.6554ms	30.6228 Ops/s	30.4314 Ops/s	$\color{#35bf28}+0.63\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.5214ms	4.8391ms	206.6500 Ops/s	207.8897 Ops/s	$\color{#d91a1a}-0.60\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6998ms	0.4806ms	2.0807 KOps/s	2.0830 KOps/s	$\color{#d91a1a}-0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7346ms	0.4582ms	2.1822 KOps/s	2.2065 KOps/s	$\color{#d91a1a}-1.10\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.2749ms	4.8672ms	205.4572 Ops/s	208.2311 Ops/s	$\color{#d91a1a}-1.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.2898ms	0.4719ms	2.1192 KOps/s	2.1209 KOps/s	$\color{#d91a1a}-0.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7041ms	0.4540ms	2.2026 KOps/s	2.1922 KOps/s	$\color{#35bf28}+0.48\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.3873ms	1.7027ms	587.3171 Ops/s	583.0057 Ops/s	$\color{#35bf28}+0.74\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.1081ms	1.6116ms	620.5166 Ops/s	617.0233 Ops/s	$\color{#35bf28}+0.57\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.7446ms	5.0120ms	199.5222 Ops/s	196.7061 Ops/s	$\color{#35bf28}+1.43\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9452ms	0.6207ms	1.6112 KOps/s	1.5752 KOps/s	$\color{#35bf28}+2.28\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9481ms	0.5921ms	1.6890 KOps/s	1.6643 KOps/s	$\color{#35bf28}+1.48\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.5752ms	4.8543ms	206.0047 Ops/s	197.1966 Ops/s	$\color{#35bf28}+4.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8070ms	0.4864ms	2.0560 KOps/s	2.0267 KOps/s	$\color{#35bf28}+1.45\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5531ms	0.4576ms	2.1853 KOps/s	2.1452 KOps/s	$\color{#35bf28}+1.87\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.7724ms	4.8434ms	206.4668 Ops/s	201.9991 Ops/s	$\color{#35bf28}+2.21\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.8850ms	0.4760ms	2.1010 KOps/s	2.0377 KOps/s	$\color{#35bf28}+3.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6180ms	0.4551ms	2.1972 KOps/s	2.0790 KOps/s	$\textbf{\color{#35bf28}+5.68\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	8.1812ms	5.1470ms	194.2869 Ops/s	195.1857 Ops/s	$\color{#d91a1a}-0.46\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9038ms	0.6429ms	1.5555 KOps/s	1.5591 KOps/s	$\color{#d91a1a}-0.23\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8028ms	0.6100ms	1.6394 KOps/s	1.6747 KOps/s	$\color{#d91a1a}-2.10\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1369s	8.6721ms	115.3125 Ops/s	117.8260 Ops/s	$\color{#d91a1a}-2.13\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	17.8394ms	12.9470ms	77.2382 Ops/s	77.4138 Ops/s	$\color{#d91a1a}-0.23\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.8724ms	1.0813ms	924.7927 Ops/s	888.3999 Ops/s	$\color{#35bf28}+4.10\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1123s	5.8954ms	169.6225 Ops/s	165.6868 Ops/s	$\color{#35bf28}+2.38\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	17.3978ms	12.8878ms	77.5925 Ops/s	70.5878 Ops/s	$\textbf{\color{#35bf28}+9.92\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	4.6885ms	1.1643ms	858.9083 Ops/s	907.7016 Ops/s	$\textbf{\color{#d91a1a}-5.38\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1097s	6.0021ms	166.6076 Ops/s	164.5490 Ops/s	$\color{#35bf28}+1.25\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1223s	15.1826ms	65.8648 Ops/s	77.3056 Ops/s	$\textbf{\color{#d91a1a}-14.80\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	4.9630ms	1.3475ms	742.1381 Ops/s	728.1222 Ops/s	$\color{#35bf28}+1.92\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1066s	0.1063s	9.4043 Ops/s	9.3433 Ops/s	$\color{#35bf28}+0.65\%$
test_sync	96.2661ms	95.4303ms	10.4789 Ops/s	10.4543 Ops/s	$\color{#35bf28}+0.23\%$
test_async	0.2341s	89.5024ms	11.1729 Ops/s	11.2280 Ops/s	$\color{#d91a1a}-0.49\%$
test_single_pixels	0.1180s	0.1169s	8.5522 Ops/s	8.5604 Ops/s	$\color{#d91a1a}-0.10\%$
test_sync_pixels	76.1508ms	73.9376ms	13.5249 Ops/s	13.1044 Ops/s	$\color{#35bf28}+3.21\%$
test_async_pixels	0.1379s	68.6228ms	14.5724 Ops/s	14.3987 Ops/s	$\color{#35bf28}+1.21\%$
test_simple	0.8648s	0.7876s	1.2696 Ops/s	1.2506 Ops/s	$\color{#35bf28}+1.53\%$
test_transformed	1.0962s	1.0215s	0.9790 Ops/s	0.9847 Ops/s	$\color{#d91a1a}-0.58\%$
test_serial	2.2929s	2.2202s	0.4504 Ops/s	0.4423 Ops/s	$\color{#35bf28}+1.84\%$
test_parallel	2.0312s	1.9745s	0.5065 Ops/s	0.5047 Ops/s	$\color{#35bf28}+0.35\%$
test_step_mdp_speed[True-True-True-True-True]	99.3220μs	36.3516μs	27.5091 KOps/s	27.0616 KOps/s	$\color{#35bf28}+1.65\%$
test_step_mdp_speed[True-True-True-True-False]	0.1104ms	20.5263μs	48.7181 KOps/s	47.4845 KOps/s	$\color{#35bf28}+2.60\%$
test_step_mdp_speed[True-True-True-False-True]	0.1542ms	20.2946μs	49.2742 KOps/s	48.0702 KOps/s	$\color{#35bf28}+2.50\%$
test_step_mdp_speed[True-True-True-False-False]	31.5200μs	11.6346μs	85.9506 KOps/s	84.8336 KOps/s	$\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-True-False-True-True]	0.1108ms	38.0196μs	26.3022 KOps/s	25.6026 KOps/s	$\color{#35bf28}+2.73\%$
test_step_mdp_speed[True-True-False-True-False]	59.8810μs	22.2470μs	44.9500 KOps/s	42.6712 KOps/s	$\textbf{\color{#35bf28}+5.34\%}$
test_step_mdp_speed[True-True-False-False-True]	47.9710μs	21.9700μs	45.5165 KOps/s	43.2025 KOps/s	$\textbf{\color{#35bf28}+5.36\%}$
test_step_mdp_speed[True-True-False-False-False]	36.6400μs	13.5789μs	73.6437 KOps/s	71.1043 KOps/s	$\color{#35bf28}+3.57\%$
test_step_mdp_speed[True-False-True-True-True]	67.3220μs	39.9481μs	25.0325 KOps/s	24.3005 KOps/s	$\color{#35bf28}+3.01\%$
test_step_mdp_speed[True-False-True-True-False]	50.8610μs	24.3726μs	41.0297 KOps/s	39.3283 KOps/s	$\color{#35bf28}+4.33\%$
test_step_mdp_speed[True-False-True-False-True]	43.6500μs	22.0349μs	45.3826 KOps/s	43.3904 KOps/s	$\color{#35bf28}+4.59\%$
test_step_mdp_speed[True-False-True-False-False]	30.1110μs	13.8027μs	72.4494 KOps/s	71.4333 KOps/s	$\color{#35bf28}+1.42\%$
test_step_mdp_speed[True-False-False-True-True]	85.8220μs	42.1924μs	23.7010 KOps/s	23.3410 KOps/s	$\color{#35bf28}+1.54\%$
test_step_mdp_speed[True-False-False-True-False]	45.0610μs	26.6110μs	37.5785 KOps/s	36.6565 KOps/s	$\color{#35bf28}+2.52\%$
test_step_mdp_speed[True-False-False-False-True]	49.9310μs	24.1825μs	41.3522 KOps/s	40.2984 KOps/s	$\color{#35bf28}+2.61\%$
test_step_mdp_speed[True-False-False-False-False]	42.2700μs	15.7467μs	63.5055 KOps/s	62.4376 KOps/s	$\color{#35bf28}+1.71\%$
test_step_mdp_speed[False-True-True-True-True]	0.1971ms	40.2311μs	24.8564 KOps/s	24.3288 KOps/s	$\color{#35bf28}+2.17\%$
test_step_mdp_speed[False-True-True-True-False]	0.1925ms	24.5908μs	40.6656 KOps/s	39.6123 KOps/s	$\color{#35bf28}+2.66\%$
test_step_mdp_speed[False-True-True-False-True]	0.2229ms	26.6730μs	37.4911 KOps/s	36.5505 KOps/s	$\color{#35bf28}+2.57\%$
test_step_mdp_speed[False-True-True-False-False]	98.1610μs	15.4285μs	64.8149 KOps/s	63.6780 KOps/s	$\color{#35bf28}+1.79\%$
test_step_mdp_speed[False-True-False-True-True]	69.6520μs	42.5364μs	23.5093 KOps/s	23.3371 KOps/s	$\color{#35bf28}+0.74\%$
test_step_mdp_speed[False-True-False-True-False]	0.2098ms	26.7241μs	37.4194 KOps/s	36.7021 KOps/s	$\color{#35bf28}+1.95\%$
test_step_mdp_speed[False-True-False-False-True]	49.4110μs	28.8630μs	34.6464 KOps/s	34.1548 KOps/s	$\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-True-False-False-False]	0.1611ms	17.8598μs	55.9917 KOps/s	55.9093 KOps/s	$\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-False-True-True-True]	4.2464ms	45.4508μs	22.0018 KOps/s	21.8380 KOps/s	$\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-False-True-True-False]	58.7410μs	29.2350μs	34.2056 KOps/s	33.7155 KOps/s	$\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-False-True-False-True]	47.4710μs	29.0612μs	34.4102 KOps/s	34.1912 KOps/s	$\color{#35bf28}+0.64\%$
test_step_mdp_speed[False-False-True-False-False]	54.5910μs	17.7601μs	56.3061 KOps/s	55.9531 KOps/s	$\color{#35bf28}+0.63\%$
test_step_mdp_speed[False-False-False-True-True]	70.7720μs	46.7244μs	21.4021 KOps/s	21.4793 KOps/s	$\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-False-False-True-False]	89.3520μs	31.4919μs	31.7542 KOps/s	31.7218 KOps/s	$\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-False-False-False-True]	48.9710μs	30.5925μs	32.6877 KOps/s	32.2621 KOps/s	$\color{#35bf28}+1.32\%$
test_step_mdp_speed[False-False-False-False-False]	46.8610μs	19.6031μs	51.0123 KOps/s	50.5178 KOps/s	$\color{#35bf28}+0.98\%$
test_values[generalized_advantage_estimate-True-True]	25.5960ms	24.2111ms	41.3034 Ops/s	40.8663 Ops/s	$\color{#35bf28}+1.07\%$
test_values[vec_generalized_advantage_estimate-True-True]	92.7786ms	2.7522ms	363.3490 Ops/s	352.2619 Ops/s	$\color{#35bf28}+3.15\%$
test_values[td0_return_estimate-False-False]	89.8930μs	65.1168μs	15.3570 KOps/s	15.3743 KOps/s	$\color{#d91a1a}-0.11\%$
test_values[td1_return_estimate-False-False]	54.4745ms	53.9639ms	18.5309 Ops/s	18.3676 Ops/s	$\color{#35bf28}+0.89\%$
test_values[vec_td1_return_estimate-False-False]	1.2871ms	1.0793ms	926.5261 Ops/s	915.0027 Ops/s	$\color{#35bf28}+1.26\%$
test_values[td_lambda_return_estimate-True-False]	86.6609ms	86.1266ms	11.6108 Ops/s	11.5495 Ops/s	$\color{#35bf28}+0.53\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.2429ms	1.0742ms	930.8927 Ops/s	927.0705 Ops/s	$\color{#35bf28}+0.41\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	24.3280ms	23.9260ms	41.7956 Ops/s	40.6299 Ops/s	$\color{#35bf28}+2.87\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9722ms	0.7189ms	1.3909 KOps/s	1.3717 KOps/s	$\color{#35bf28}+1.40\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.8292ms	0.6663ms	1.5008 KOps/s	1.4787 KOps/s	$\color{#35bf28}+1.49\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.6188ms	1.4600ms	684.9349 Ops/s	679.7104 Ops/s	$\color{#35bf28}+0.77\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.8505ms	0.6790ms	1.4728 KOps/s	1.4043 KOps/s	$\color{#35bf28}+4.87\%$
test_dqn_speed	7.9479ms	1.4384ms	695.1970 Ops/s	694.0519 Ops/s	$\color{#35bf28}+0.16\%$
test_ddpg_speed	3.2032ms	2.9444ms	339.6333 Ops/s	335.1645 Ops/s	$\color{#35bf28}+1.33\%$
test_sac_speed	0.1062s	9.1952ms	108.7525 Ops/s	117.5763 Ops/s	$\textbf{\color{#d91a1a}-7.50\%}$
test_redq_speed	11.8472ms	10.9756ms	91.1108 Ops/s	90.3937 Ops/s	$\color{#35bf28}+0.79\%$
test_redq_deprec_speed	12.1221ms	11.3171ms	88.3619 Ops/s	87.1165 Ops/s	$\color{#35bf28}+1.43\%$
test_td3_speed	8.3992ms	8.3142ms	120.2767 Ops/s	116.1787 Ops/s	$\color{#35bf28}+3.53\%$
test_cql_speed	27.7663ms	26.3347ms	37.9728 Ops/s	34.0097 Ops/s	$\textbf{\color{#35bf28}+11.65\%}$
test_a2c_speed	6.0190ms	5.7393ms	174.2386 Ops/s	171.1350 Ops/s	$\color{#35bf28}+1.81\%$
test_ppo_speed	6.5467ms	6.0754ms	164.5988 Ops/s	161.2187 Ops/s	$\color{#35bf28}+2.10\%$
test_reinforce_speed	4.9044ms	4.6474ms	215.1728 Ops/s	211.0269 Ops/s	$\color{#35bf28}+1.96\%$
test_iql_speed	20.7358ms	20.0143ms	49.9644 Ops/s	48.7838 Ops/s	$\color{#35bf28}+2.42\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.7052ms	6.5337ms	153.0537 Ops/s	149.2710 Ops/s	$\color{#35bf28}+2.53\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.1152s	0.5981ms	1.6720 KOps/s	1.9037 KOps/s	$\textbf{\color{#d91a1a}-12.17\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6749ms	0.4981ms	2.0078 KOps/s	1.9818 KOps/s	$\color{#35bf28}+1.31\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.8742ms	6.4316ms	155.4815 Ops/s	151.3327 Ops/s	$\color{#35bf28}+2.74\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.8100ms	0.5076ms	1.9702 KOps/s	1.9492 KOps/s	$\color{#35bf28}+1.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7020ms	0.4913ms	2.0355 KOps/s	2.0305 KOps/s	$\color{#35bf28}+0.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.1749ms	1.9464ms	513.7737 Ops/s	504.1554 Ops/s	$\color{#35bf28}+1.91\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.2410ms	1.8544ms	539.2622 Ops/s	530.1695 Ops/s	$\color{#35bf28}+1.72\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	8.5169ms	6.7615ms	147.8957 Ops/s	146.2500 Ops/s	$\color{#35bf28}+1.13\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.1340s	0.7770ms	1.2869 KOps/s	1.5061 KOps/s	$\textbf{\color{#d91a1a}-14.55\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8333ms	0.6423ms	1.5570 KOps/s	1.5533 KOps/s	$\color{#35bf28}+0.24\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.7545ms	6.5503ms	152.6644 Ops/s	149.1843 Ops/s	$\color{#35bf28}+2.33\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7106ms	0.5184ms	1.9291 KOps/s	1.9253 KOps/s	$\color{#35bf28}+0.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6936ms	0.5023ms	1.9907 KOps/s	2.0016 KOps/s	$\color{#d91a1a}-0.54\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.7705ms	6.4157ms	155.8678 Ops/s	151.5038 Ops/s	$\color{#35bf28}+2.88\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.0958ms	0.5138ms	1.9464 KOps/s	1.9437 KOps/s	$\color{#35bf28}+0.14\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6886ms	0.4909ms	2.0372 KOps/s	2.0023 KOps/s	$\color{#35bf28}+1.74\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.9270ms	6.6999ms	149.2557 Ops/s	146.9276 Ops/s	$\color{#35bf28}+1.58\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.9494ms	0.6801ms	1.4703 KOps/s	1.4919 KOps/s	$\color{#d91a1a}-1.45\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8444ms	0.6591ms	1.5173 KOps/s	1.5312 KOps/s	$\color{#d91a1a}-0.90\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1507s	8.0784ms	123.7871 Ops/s	122.1472 Ops/s	$\color{#35bf28}+1.34\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	18.3873ms	15.7404ms	63.5308 Ops/s	62.0401 Ops/s	$\color{#35bf28}+2.40\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	6.6274ms	1.3518ms	739.7556 Ops/s	795.4765 Ops/s	$\textbf{\color{#d91a1a}-7.00\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1279s	7.6246ms	131.1537 Ops/s	130.0973 Ops/s	$\color{#35bf28}+0.81\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1395s	18.2733ms	54.7245 Ops/s	61.6684 Ops/s	$\textbf{\color{#d91a1a}-11.26\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.0078ms	1.1186ms	893.9930 Ops/s	797.7798 Ops/s	$\textbf{\color{#35bf28}+12.06\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1297s	7.8363ms	127.6120 Ops/s	127.9071 Ops/s	$\color{#d91a1a}-0.23\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.7539ms	15.8142ms	63.2342 Ops/s	61.2385 Ops/s	$\color{#35bf28}+3.26\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	6.3870ms	1.5135ms	660.7065 Ops/s	656.1810 Ops/s	$\color{#35bf28}+0.69\%$