Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Doc] Better doc for make_tensordict_primer #2324

Merged
merged 2 commits into from
Jul 28, 2024
Merged

[Doc] Better doc for make_tensordict_primer #2324

merged 2 commits into from
Jul 28, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 26, 2024 •
edited
Loading

No description provided.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 26, 2024
@vmoens vmoens linked an issue Jul 26, 2024 that may be closed by this pull request
[BUG] Function make_tensordict_primer Overlooks Batch-Locked Envs #2323
Closed
3 tasks
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jul 26, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2324

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Pending, 1 Unrelated Failure

As of commit e049f47 with merge base c6ef080 (image):

NEW FAILURE - The following job has failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@vmoens vmoens added the documentation Improvements or additions to documentation label Jul 26, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 59.3862ms 58.3959ms 17.1245 Ops/s 17.3478 Ops/s $\color{#d91a1a}-1.29\%$
test_sync 37.9244ms 32.2590ms 30.9991 Ops/s 32.1161 Ops/s $\color{#d91a1a}-3.48\%$
test_async 0.1062s 30.3767ms 32.9200 Ops/s 33.3761 Ops/s $\color{#d91a1a}-1.37\%$
test_simple 0.5052s 0.4256s 2.3495 Ops/s 2.5506 Ops/s $\textbf{\color{#d91a1a}-7.89\%}$
test_transformed 0.6583s 0.5844s 1.7112 Ops/s 1.8053 Ops/s $\textbf{\color{#d91a1a}-5.21\%}$
test_serial 1.3372s 1.2617s 0.7926 Ops/s 0.7944 Ops/s $\color{#d91a1a}-0.22\%$
test_parallel 1.1911s 1.1151s 0.8968 Ops/s 0.8990 Ops/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[True-True-True-True-True] 0.2024ms 24.5378μs 40.7534 KOps/s 41.1964 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[True-True-True-True-False] 39.5440μs 14.2084μs 70.3807 KOps/s 71.8103 KOps/s $\color{#d91a1a}-1.99\%$
test_step_mdp_speed[True-True-True-False-True] 65.5120μs 14.1885μs 70.4796 KOps/s 72.0904 KOps/s $\color{#d91a1a}-2.23\%$
test_step_mdp_speed[True-True-True-False-False] 33.8330μs 8.2643μs 121.0022 KOps/s 124.3486 KOps/s $\color{#d91a1a}-2.69\%$
test_step_mdp_speed[True-True-False-True-True] 86.0010μs 26.1179μs 38.2880 KOps/s 38.8463 KOps/s $\color{#d91a1a}-1.44\%$
test_step_mdp_speed[True-True-False-True-False] 48.8310μs 15.7295μs 63.5746 KOps/s 64.5118 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-True-False-False-True] 77.8250μs 15.5751μs 64.2052 KOps/s 64.9102 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[True-True-False-False-False] 31.5680μs 9.7994μs 102.0466 KOps/s 103.0512 KOps/s $\color{#d91a1a}-0.97\%$
test_step_mdp_speed[True-False-True-True-True] 84.7990μs 27.6977μs 36.1041 KOps/s 36.2966 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-False-True-True-False] 60.3530μs 17.3373μs 57.6792 KOps/s 57.8064 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[True-False-True-False-True] 46.3370μs 15.6772μs 63.7869 KOps/s 64.7028 KOps/s $\color{#d91a1a}-1.42\%$
test_step_mdp_speed[True-False-True-False-False] 34.7150μs 9.7405μs 102.6644 KOps/s 105.2383 KOps/s $\color{#d91a1a}-2.45\%$
test_step_mdp_speed[True-False-False-True-True] 98.1470μs 29.2251μs 34.2171 KOps/s 34.8117 KOps/s $\color{#d91a1a}-1.71\%$
test_step_mdp_speed[True-False-False-True-False] 93.6040μs 18.5540μs 53.8967 KOps/s 53.6169 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[True-False-False-False-True] 72.7700μs 17.1613μs 58.2707 KOps/s 59.1133 KOps/s $\color{#d91a1a}-1.43\%$
test_step_mdp_speed[True-False-False-False-False] 46.5870μs 11.0742μs 90.2996 KOps/s 91.3482 KOps/s $\color{#d91a1a}-1.15\%$
test_step_mdp_speed[False-True-True-True-True] 82.9720μs 27.5713μs 36.2697 KOps/s 36.6207 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[False-True-True-True-False] 47.5390μs 17.4208μs 57.4027 KOps/s 58.6565 KOps/s $\color{#d91a1a}-2.14\%$
test_step_mdp_speed[False-True-True-False-True] 59.0200μs 18.2066μs 54.9253 KOps/s 55.5065 KOps/s $\color{#d91a1a}-1.05\%$
test_step_mdp_speed[False-True-True-False-False] 38.5120μs 10.9379μs 91.4254 KOps/s 92.1960 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[False-True-False-True-True] 79.1380μs 28.9008μs 34.6011 KOps/s 35.1088 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[False-True-False-True-False] 93.4740μs 18.5971μs 53.7719 KOps/s 53.9009 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[False-True-False-False-True] 79.8490μs 19.8206μs 50.4526 KOps/s 51.8774 KOps/s $\color{#d91a1a}-2.75\%$
test_step_mdp_speed[False-True-False-False-False] 38.9520μs 12.3854μs 80.7405 KOps/s 80.8507 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[False-False-True-True-True] 3.7991ms 29.7000μs 33.6701 KOps/s 32.7472 KOps/s $\color{#35bf28}+2.82\%$
test_step_mdp_speed[False-False-True-True-False] 93.4910μs 19.6794μs 50.8145 KOps/s 50.2868 KOps/s $\color{#35bf28}+1.05\%$
test_step_mdp_speed[False-False-True-False-True] 52.9090μs 19.4434μs 51.4313 KOps/s 52.2843 KOps/s $\color{#d91a1a}-1.63\%$
test_step_mdp_speed[False-False-True-False-False] 43.3520μs 12.2671μs 81.5187 KOps/s 80.9043 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-False-False-True-True] 74.6200μs 31.5189μs 31.7270 KOps/s 31.9367 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[False-False-False-True-False] 55.1430μs 21.2769μs 46.9993 KOps/s 47.3949 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[False-False-False-False-True] 61.7250μs 20.7052μs 48.2971 KOps/s 49.1083 KOps/s $\color{#d91a1a}-1.65\%$
test_step_mdp_speed[False-False-False-False-False] 63.3380μs 13.5488μs 73.8075 KOps/s 75.1489 KOps/s $\color{#d91a1a}-1.78\%$
test_values[generalized_advantage_estimate-True-True] 10.6787ms 9.5585ms 104.6194 Ops/s 105.7537 Ops/s $\color{#d91a1a}-1.07\%$
test_values[vec_generalized_advantage_estimate-True-True] 39.3481ms 35.8363ms 27.9047 Ops/s 27.9445 Ops/s $\color{#d91a1a}-0.14\%$
test_values[td0_return_estimate-False-False] 0.2231ms 0.1807ms 5.5333 KOps/s 5.3466 KOps/s $\color{#35bf28}+3.49\%$
test_values[td1_return_estimate-False-False] 26.0715ms 23.5012ms 42.5511 Ops/s 41.4163 Ops/s $\color{#35bf28}+2.74\%$
test_values[vec_td1_return_estimate-False-False] 40.9525ms 35.9745ms 27.7975 Ops/s 27.7976 Ops/s $-0.00\%$
test_values[td_lambda_return_estimate-True-False] 39.5780ms 34.2492ms 29.1977 Ops/s 29.3725 Ops/s $\color{#d91a1a}-0.60\%$
test_values[vec_td_lambda_return_estimate-True-False] 42.0569ms 35.9930ms 27.7832 Ops/s 27.9471 Ops/s $\color{#d91a1a}-0.59\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.6465ms 8.2234ms 121.6041 Ops/s 122.1691 Ops/s $\color{#d91a1a}-0.46\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2940ms 1.9609ms 509.9699 Ops/s 487.5781 Ops/s $\color{#35bf28}+4.59\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5979ms 0.3614ms 2.7670 KOps/s 2.7448 KOps/s $\color{#35bf28}+0.81\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 50.5012ms 48.4200ms 20.6526 Ops/s 19.9768 Ops/s $\color{#35bf28}+3.38\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.0674ms 3.0610ms 326.6918 Ops/s 326.1081 Ops/s $\color{#35bf28}+0.18\%$
test_dqn_speed 2.0124ms 1.3093ms 763.7537 Ops/s 753.9722 Ops/s $\color{#35bf28}+1.30\%$
test_ddpg_speed 4.1366ms 2.7575ms 362.6476 Ops/s 362.4393 Ops/s $\color{#35bf28}+0.06\%$
test_sac_speed 10.5203ms 8.3367ms 119.9511 Ops/s 120.9100 Ops/s $\color{#d91a1a}-0.79\%$
test_redq_speed 14.3579ms 13.2442ms 75.5048 Ops/s 76.9205 Ops/s $\color{#d91a1a}-1.84\%$
test_redq_deprec_speed 15.9078ms 13.6511ms 73.2539 Ops/s 73.9168 Ops/s $\color{#d91a1a}-0.90\%$
test_td3_speed 8.6892ms 8.1712ms 122.3810 Ops/s 121.7355 Ops/s $\color{#35bf28}+0.53\%$
test_cql_speed 38.3415ms 36.2119ms 27.6152 Ops/s 27.1433 Ops/s $\color{#35bf28}+1.74\%$
test_a2c_speed 8.5211ms 7.5306ms 132.7913 Ops/s 130.7075 Ops/s $\color{#35bf28}+1.59\%$
test_ppo_speed 9.3164ms 7.7135ms 129.6420 Ops/s 127.1441 Ops/s $\color{#35bf28}+1.96\%$
test_reinforce_speed 7.6507ms 6.6209ms 151.0363 Ops/s 151.0772 Ops/s $\color{#d91a1a}-0.03\%$
test_iql_speed 33.7241ms 32.5884ms 30.6857 Ops/s 30.8380 Ops/s $\color{#d91a1a}-0.49\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.6590ms 5.0317ms 198.7398 Ops/s 199.7559 Ops/s $\color{#d91a1a}-0.51\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.7987ms 0.4920ms 2.0324 KOps/s 2.0296 KOps/s $\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6635ms 0.4649ms 2.1509 KOps/s 2.1732 KOps/s $\color{#d91a1a}-1.03\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.8559ms 5.0433ms 198.2813 Ops/s 202.4262 Ops/s $\color{#d91a1a}-2.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3982ms 0.4835ms 2.0682 KOps/s 2.1021 KOps/s $\color{#d91a1a}-1.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6630ms 0.4651ms 2.1502 KOps/s 2.1368 KOps/s $\color{#35bf28}+0.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.4906ms 1.7077ms 585.5846 Ops/s 586.3947 Ops/s $\color{#d91a1a}-0.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.1717ms 1.6179ms 618.0666 Ops/s 618.3568 Ops/s $\color{#d91a1a}-0.05\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.5529ms 5.1335ms 194.7973 Ops/s 199.4678 Ops/s $\color{#d91a1a}-2.34\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8032ms 0.6203ms 1.6120 KOps/s 1.5993 KOps/s $\color{#35bf28}+0.80\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7459ms 0.5920ms 1.6891 KOps/s 1.6572 KOps/s $\color{#35bf28}+1.92\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8862ms 5.0043ms 199.8266 Ops/s 199.6231 Ops/s $\color{#35bf28}+0.10\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0177ms 0.4912ms 2.0358 KOps/s 2.0413 KOps/s $\color{#d91a1a}-0.27\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6173ms 0.4624ms 2.1625 KOps/s 2.1151 KOps/s $\color{#35bf28}+2.24\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.4562ms 5.0613ms 197.5769 Ops/s 202.1064 Ops/s $\color{#d91a1a}-2.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.6841ms 0.4801ms 2.0830 KOps/s 1.5690 KOps/s $\textbf{\color{#35bf28}+32.76\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6370ms 0.4626ms 2.1616 KOps/s 2.1730 KOps/s $\color{#d91a1a}-0.52\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 8.0594ms 5.2035ms 192.1796 Ops/s 194.3638 Ops/s $\color{#d91a1a}-1.12\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6747ms 0.6260ms 1.5976 KOps/s 1.5943 KOps/s $\color{#35bf28}+0.20\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8002ms 0.6050ms 1.6529 KOps/s 1.6381 KOps/s $\color{#35bf28}+0.90\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1429s 8.8687ms 112.7563 Ops/s 158.7217 Ops/s $\textbf{\color{#d91a1a}-28.96\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.2606ms 13.1909ms 75.8098 Ops/s 77.2283 Ops/s $\color{#d91a1a}-1.84\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.8342ms 1.1791ms 848.0975 Ops/s 831.8613 Ops/s $\color{#35bf28}+1.95\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1213s 6.1103ms 163.6580 Ops/s 120.5422 Ops/s $\textbf{\color{#35bf28}+35.77\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.5930ms 13.3162ms 75.0966 Ops/s 78.0055 Ops/s $\color{#d91a1a}-3.73\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.1242ms 1.2064ms 828.8902 Ops/s 833.1534 Ops/s $\color{#d91a1a}-0.51\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1239s 6.3106ms 158.4647 Ops/s 157.4294 Ops/s $\color{#35bf28}+0.66\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1380s 15.8317ms 63.1642 Ops/s 77.1443 Ops/s $\textbf{\color{#d91a1a}-18.12\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.8072ms 1.2178ms 821.1192 Ops/s 774.3208 Ops/s $\textbf{\color{#35bf28}+6.04\%}$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1075s 0.1074s 9.3121 Ops/s 8.6123 Ops/s $\textbf{\color{#35bf28}+8.13\%}$
test_sync 95.7106ms 95.0683ms 10.5188 Ops/s 10.5518 Ops/s $\color{#d91a1a}-0.31\%$
test_async 0.2599s 89.9309ms 11.1197 Ops/s 11.1424 Ops/s $\color{#d91a1a}-0.20\%$
test_single_pixels 0.1195s 0.1178s 8.4860 Ops/s 8.4704 Ops/s $\color{#35bf28}+0.18\%$
test_sync_pixels 76.2088ms 74.2845ms 13.4618 Ops/s 13.2639 Ops/s $\color{#35bf28}+1.49\%$
test_async_pixels 0.1297s 68.0015ms 14.7056 Ops/s 14.2634 Ops/s $\color{#35bf28}+3.10\%$
test_simple 0.7820s 0.7774s 1.2864 Ops/s 1.2405 Ops/s $\color{#35bf28}+3.70\%$
test_transformed 1.0972s 1.0270s 0.9737 Ops/s 0.9862 Ops/s $\color{#d91a1a}-1.27\%$
test_serial 2.3305s 2.2566s 0.4431 Ops/s 0.4404 Ops/s $\color{#35bf28}+0.62\%$
test_parallel 2.0245s 1.9573s 0.5109 Ops/s 0.5072 Ops/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-True-True-True-True] 0.1138ms 36.8171μs 27.1613 KOps/s 27.6324 KOps/s $\color{#d91a1a}-1.70\%$
test_step_mdp_speed[True-True-True-True-False] 49.0910μs 20.8125μs 48.0479 KOps/s 47.6663 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-True-True-False-True] 45.7110μs 20.6567μs 48.4105 KOps/s 48.9620 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[True-True-True-False-False] 30.9710μs 11.8640μs 84.2888 KOps/s 84.0330 KOps/s $\color{#35bf28}+0.30\%$
test_step_mdp_speed[True-True-False-True-True] 70.3410μs 38.6999μs 25.8399 KOps/s 26.3201 KOps/s $\color{#d91a1a}-1.82\%$
test_step_mdp_speed[True-True-False-True-False] 54.2110μs 22.8952μs 43.6772 KOps/s 43.3430 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[True-True-False-False-True] 49.2110μs 22.7391μs 43.9771 KOps/s 44.1660 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[True-True-False-False-False] 38.7310μs 14.0804μs 71.0206 KOps/s 70.7044 KOps/s $\color{#35bf28}+0.45\%$
test_step_mdp_speed[True-False-True-True-True] 70.2110μs 40.7406μs 24.5456 KOps/s 24.6153 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[True-False-True-True-False] 51.4110μs 25.3574μs 39.4362 KOps/s 39.1632 KOps/s $\color{#35bf28}+0.70\%$
test_step_mdp_speed[True-False-True-False-True] 47.2210μs 22.4948μs 44.4547 KOps/s 44.1738 KOps/s $\color{#35bf28}+0.64\%$
test_step_mdp_speed[True-False-True-False-False] 39.5110μs 14.0787μs 71.0290 KOps/s 70.8466 KOps/s $\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-False-False-True-True] 81.7420μs 42.8366μs 23.3445 KOps/s 23.4086 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[True-False-False-True-False] 52.6410μs 27.4632μs 36.4124 KOps/s 36.0178 KOps/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[True-False-False-False-True] 50.5500μs 24.4408μs 40.9153 KOps/s 40.5790 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-False-False-False-False] 37.5800μs 16.1810μs 61.8007 KOps/s 60.9312 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[False-True-True-True-True] 74.8120μs 41.0581μs 24.3557 KOps/s 24.6223 KOps/s $\color{#d91a1a}-1.08\%$
test_step_mdp_speed[False-True-True-True-False] 51.7410μs 25.3440μs 39.4571 KOps/s 39.2156 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[False-True-True-False-True] 66.1620μs 27.0481μs 36.9712 KOps/s 36.7833 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-True-True-False-False] 39.7810μs 15.8939μs 62.9173 KOps/s 61.7739 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[False-True-False-True-True] 70.6210μs 42.9894μs 23.2616 KOps/s 23.3384 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-True-False-True-False] 53.5210μs 27.4141μs 36.4776 KOps/s 35.7441 KOps/s $\color{#35bf28}+2.05\%$
test_step_mdp_speed[False-True-False-False-True] 58.5310μs 29.3773μs 34.0399 KOps/s 34.5036 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[False-True-False-False-False] 37.1010μs 17.9990μs 55.5587 KOps/s 54.4375 KOps/s $\color{#35bf28}+2.06\%$
test_step_mdp_speed[False-False-True-True-True] 4.0695ms 45.4554μs 21.9996 KOps/s 22.0242 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-False-True-True-False] 55.7610μs 29.9497μs 33.3894 KOps/s 33.0464 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-False-True-False-True] 57.1910μs 28.8829μs 34.6225 KOps/s 34.3310 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[False-False-True-False-False] 39.9800μs 18.0551μs 55.3859 KOps/s 53.4838 KOps/s $\color{#35bf28}+3.56\%$
test_step_mdp_speed[False-False-False-True-True] 74.9910μs 47.1466μs 21.2104 KOps/s 21.4059 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[False-False-False-True-False] 57.8710μs 31.4935μs 31.7525 KOps/s 31.3332 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[False-False-False-False-True] 59.9610μs 30.8894μs 32.3736 KOps/s 32.2089 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-False-False-False-False] 43.6510μs 20.1203μs 49.7011 KOps/s 49.6607 KOps/s $\color{#35bf28}+0.08\%$
test_values[generalized_advantage_estimate-True-True] 25.7562ms 24.8874ms 40.1810 Ops/s 40.5831 Ops/s $\color{#d91a1a}-0.99\%$
test_values[vec_generalized_advantage_estimate-True-True] 88.9029ms 2.6808ms 373.0231 Ops/s 368.5821 Ops/s $\color{#35bf28}+1.20\%$
test_values[td0_return_estimate-False-False] 95.3420μs 66.4962μs 15.0385 KOps/s 15.2836 KOps/s $\color{#d91a1a}-1.60\%$
test_values[td1_return_estimate-False-False] 56.0804ms 55.9034ms 17.8880 Ops/s 18.3704 Ops/s $\color{#d91a1a}-2.63\%$
test_values[vec_td1_return_estimate-False-False] 1.4010ms 1.0922ms 915.6097 Ops/s 926.0953 Ops/s $\color{#d91a1a}-1.13\%$
test_values[td_lambda_return_estimate-True-False] 89.0105ms 88.6960ms 11.2745 Ops/s 11.5543 Ops/s $\color{#d91a1a}-2.42\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.4063ms 1.0913ms 916.3407 Ops/s 924.7071 Ops/s $\color{#d91a1a}-0.90\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.2420ms 25.1176ms 39.8128 Ops/s 40.0900 Ops/s $\color{#d91a1a}-0.69\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9658ms 0.7251ms 1.3791 KOps/s 1.3775 KOps/s $\color{#35bf28}+0.11\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7713ms 0.6800ms 1.4706 KOps/s 1.5058 KOps/s $\color{#d91a1a}-2.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5117ms 1.4744ms 678.2519 Ops/s 681.2599 Ops/s $\color{#d91a1a}-0.44\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7290ms 0.6887ms 1.4519 KOps/s 1.4722 KOps/s $\color{#d91a1a}-1.38\%$
test_dqn_speed 92.8831ms 1.5081ms 663.0944 Ops/s 729.7799 Ops/s $\textbf{\color{#d91a1a}-9.14\%}$
test_ddpg_speed 3.0014ms 2.7649ms 361.6736 Ops/s 360.6425 Ops/s $\color{#35bf28}+0.29\%$
test_sac_speed 8.3128ms 7.9576ms 125.6662 Ops/s 125.4412 Ops/s $\color{#35bf28}+0.18\%$
test_redq_speed 11.7651ms 10.2124ms 97.9198 Ops/s 97.6367 Ops/s $\color{#35bf28}+0.29\%$
test_redq_deprec_speed 10.9193ms 10.6274ms 94.0968 Ops/s 90.6683 Ops/s $\color{#35bf28}+3.78\%$
test_td3_speed 8.0236ms 7.8752ms 126.9811 Ops/s 126.2729 Ops/s $\color{#35bf28}+0.56\%$
test_cql_speed 26.6279ms 25.3368ms 39.4682 Ops/s 39.0879 Ops/s $\color{#35bf28}+0.97\%$
test_a2c_speed 7.9909ms 5.5064ms 181.6082 Ops/s 181.3022 Ops/s $\color{#35bf28}+0.17\%$
test_ppo_speed 6.3914ms 5.7677ms 173.3802 Ops/s 173.2688 Ops/s $\color{#35bf28}+0.06\%$
test_reinforce_speed 4.5914ms 4.4104ms 226.7363 Ops/s 226.5312 Ops/s $\color{#35bf28}+0.09\%$
test_iql_speed 19.7724ms 18.9078ms 52.8883 Ops/s 51.9272 Ops/s $\color{#35bf28}+1.85\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7898ms 6.6116ms 151.2501 Ops/s 149.7829 Ops/s $\color{#35bf28}+0.98\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5016ms 0.5157ms 1.9393 KOps/s 1.9383 KOps/s $\color{#35bf28}+0.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7294ms 0.4943ms 2.0232 KOps/s 2.0146 KOps/s $\color{#35bf28}+0.43\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.8442ms 6.5486ms 152.7043 Ops/s 152.1971 Ops/s $\color{#35bf28}+0.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9370ms 0.5073ms 1.9713 KOps/s 1.9532 KOps/s $\color{#35bf28}+0.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7308ms 0.4901ms 2.0403 KOps/s 2.0330 KOps/s $\color{#35bf28}+0.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1323ms 1.9718ms 507.1485 Ops/s 506.3003 Ops/s $\color{#35bf28}+0.17\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.0380ms 1.8790ms 532.2061 Ops/s 533.4890 Ops/s $\color{#d91a1a}-0.24\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.9291ms 6.7857ms 147.3690 Ops/s 146.1307 Ops/s $\color{#35bf28}+0.85\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8611ms 0.6613ms 1.5121 KOps/s 1.5013 KOps/s $\color{#35bf28}+0.72\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8222ms 0.6391ms 1.5648 KOps/s 1.5369 KOps/s $\color{#35bf28}+1.82\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7679ms 6.6596ms 150.1590 Ops/s 150.2523 Ops/s $\color{#d91a1a}-0.06\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9712ms 0.5158ms 1.9388 KOps/s 1.9351 KOps/s $\color{#35bf28}+0.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6759ms 0.4930ms 2.0282 KOps/s 1.5702 KOps/s $\textbf{\color{#35bf28}+29.17\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9995ms 6.5369ms 152.9766 Ops/s 150.2508 Ops/s $\color{#35bf28}+1.81\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6397ms 0.5118ms 1.9541 KOps/s 1.9621 KOps/s $\color{#d91a1a}-0.41\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7045ms 0.4905ms 2.0386 KOps/s 2.0370 KOps/s $\color{#35bf28}+0.08\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.8741ms 6.7822ms 147.4444 Ops/s 147.1242 Ops/s $\color{#35bf28}+0.22\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9315ms 0.6637ms 1.5067 KOps/s 1.2100 KOps/s $\textbf{\color{#35bf28}+24.52\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.3213ms 0.6462ms 1.5476 KOps/s 1.5594 KOps/s $\color{#d91a1a}-0.76\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1240s 7.5342ms 132.7278 Ops/s 131.8571 Ops/s $\color{#35bf28}+0.66\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.8612ms 16.4143ms 60.9224 Ops/s 61.2988 Ops/s $\color{#d91a1a}-0.61\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.3471ms 1.2456ms 802.8161 Ops/s 726.6237 Ops/s $\textbf{\color{#35bf28}+10.49\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1265s 7.6243ms 131.1590 Ops/s 132.7684 Ops/s $\color{#d91a1a}-1.21\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1360s 18.7708ms 53.2743 Ops/s 61.2286 Ops/s $\textbf{\color{#d91a1a}-12.99\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.4184ms 1.3560ms 737.4794 Ops/s 842.7107 Ops/s $\textbf{\color{#d91a1a}-12.49\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1254s 7.7974ms 128.2485 Ops/s 129.3754 Ops/s $\color{#d91a1a}-0.87\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.3081ms 16.4506ms 60.7880 Ops/s 61.1869 Ops/s $\color{#d91a1a}-0.65\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.5355ms 1.3799ms 724.6974 Ops/s 661.0872 Ops/s $\textbf{\color{#35bf28}+9.62\%}$

@vmoens vmoens merged commit 0063741 into main Jul 28, 2024
53 of 55 checks passed
@vmoens vmoens deleted the doc-rnn-primers branch August 7, 2024 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] Function make_tensordict_primer Overlooks Batch-Locked Envs
2 participants
@vmoens @facebook-github-bot