Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Partial steps in batched envs #2377

Merged
merged 14 commits into from
Aug 12, 2024
Merged

[Feature] Partial steps in batched envs #2377

merged 14 commits into from
Aug 12, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Aug 7, 2024 •
edited
Loading

@vmoens
Update
271557f 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 7, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2377

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 7 Unrelated Failures

As of commit f56b8e0 with merge base a6310ae (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOB - The following job was cancelled. Please retry:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Aug 7, 2024
@vmoens
[Feature] Partial steps in batched envs
4f8fb45 
ghstack-source-id: d82f1e4e14a3cc9741240b9d7915ad1cd3727caf
Pull Request resolved: #2377
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 7, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 7, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 60.8632ms 58.1405ms 17.1997 Ops/s 16.6836 Ops/s $\color{#35bf28}+3.09\%$
test_sync 39.4636ms 31.8931ms 31.3547 Ops/s 31.0384 Ops/s $\color{#35bf28}+1.02\%$
test_async 60.1505ms 30.6260ms 32.6520 Ops/s 32.8534 Ops/s $\color{#d91a1a}-0.61\%$
test_simple 0.4853s 0.4150s 2.4096 Ops/s 2.3756 Ops/s $\color{#35bf28}+1.43\%$
test_transformed 0.6275s 0.5652s 1.7692 Ops/s 1.7200 Ops/s $\color{#35bf28}+2.86\%$
test_serial 1.3073s 1.2510s 0.7993 Ops/s 0.7846 Ops/s $\color{#35bf28}+1.88\%$
test_parallel 1.2140s 1.1359s 0.8804 Ops/s 0.8983 Ops/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[True-True-True-True-True] 0.2238ms 24.9836μs 40.0262 KOps/s 40.0897 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[True-True-True-True-False] 69.6200μs 14.3203μs 69.8309 KOps/s 69.7123 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[True-True-True-False-True] 44.9840μs 14.3892μs 69.4965 KOps/s 69.8356 KOps/s $\color{#d91a1a}-0.49\%$
test_step_mdp_speed[True-True-True-False-False] 32.9310μs 8.3067μs 120.3844 KOps/s 119.4105 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[True-True-False-True-True] 82.5740μs 26.7878μs 37.3304 KOps/s 37.4864 KOps/s $\color{#d91a1a}-0.42\%$
test_step_mdp_speed[True-True-False-True-False] 68.1480μs 16.0530μs 62.2935 KOps/s 62.7346 KOps/s $\color{#d91a1a}-0.70\%$
test_step_mdp_speed[True-True-False-False-True] 73.2670μs 15.7858μs 63.3481 KOps/s 63.0574 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[True-True-False-False-False] 48.9210μs 9.8137μs 101.8984 KOps/s 102.3150 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[True-False-True-True-True] 68.2080μs 28.5286μs 35.0525 KOps/s 35.3193 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[True-False-True-True-False] 62.9980μs 17.4923μs 57.1681 KOps/s 56.9106 KOps/s $\color{#35bf28}+0.45\%$
test_step_mdp_speed[True-False-True-False-True] 48.0200μs 15.9753μs 62.5965 KOps/s 62.4535 KOps/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[True-False-True-False-False] 51.9280μs 9.8354μs 101.6734 KOps/s 103.1183 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[True-False-False-True-True] 66.5540μs 29.9882μs 33.3465 KOps/s 33.4665 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[True-False-False-True-False] 71.9150μs 19.1854μs 52.1231 KOps/s 52.5365 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-False-False-False-True] 75.8620μs 17.3174μs 57.7454 KOps/s 57.5499 KOps/s $\color{#35bf28}+0.34\%$
test_step_mdp_speed[True-False-False-False-False] 44.2130μs 11.2319μs 89.0324 KOps/s 89.0693 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[False-True-True-True-True] 83.6960μs 28.6492μs 34.9049 KOps/s 35.5143 KOps/s $\color{#d91a1a}-1.72\%$
test_step_mdp_speed[False-True-True-True-False] 57.3570μs 17.5596μs 56.9489 KOps/s 56.9927 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-True-True-False-True] 60.7230μs 18.5614μs 53.8752 KOps/s 54.8193 KOps/s $\color{#d91a1a}-1.72\%$
test_step_mdp_speed[False-True-True-False-False] 34.9960μs 11.0821μs 90.2352 KOps/s 90.4759 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[False-True-False-True-True] 79.8190μs 29.6669μs 33.7076 KOps/s 33.7327 KOps/s $\color{#d91a1a}-0.07\%$
test_step_mdp_speed[False-True-False-True-False] 62.8770μs 19.0216μs 52.5719 KOps/s 52.3985 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[False-True-False-False-True] 69.4290μs 19.9268μs 50.1837 KOps/s 50.4920 KOps/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[False-True-False-False-False] 54.7230μs 12.4891μs 80.0698 KOps/s 79.9532 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-False-True-True-True] 3.5387ms 31.7678μs 31.4784 KOps/s 31.5740 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[False-False-True-True-False] 74.2290μs 20.5596μs 48.6392 KOps/s 48.1955 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-False-True-False-True] 70.1520μs 20.1617μs 49.5990 KOps/s 49.9304 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[False-False-True-False-False] 35.3260μs 12.4739μs 80.1674 KOps/s 79.7047 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-False-False-True-True] 0.1018ms 32.8530μs 30.4386 KOps/s 30.4948 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-False-False-True-False] 51.5860μs 21.6943μs 46.0950 KOps/s 45.4133 KOps/s $\color{#35bf28}+1.50\%$
test_step_mdp_speed[False-False-False-False-True] 71.5340μs 21.3296μs 46.8832 KOps/s 46.8950 KOps/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[False-False-False-False-False] 41.7380μs 13.8134μs 72.3935 KOps/s 71.7756 KOps/s $\color{#35bf28}+0.86\%$
test_values[generalized_advantage_estimate-True-True] 11.2218ms 9.3900ms 106.4957 Ops/s 105.2704 Ops/s $\color{#35bf28}+1.16\%$
test_values[vec_generalized_advantage_estimate-True-True] 37.8157ms 35.4019ms 28.2471 Ops/s 28.4059 Ops/s $\color{#d91a1a}-0.56\%$
test_values[td0_return_estimate-False-False] 0.2268ms 0.1676ms 5.9651 KOps/s 6.0359 KOps/s $\color{#d91a1a}-1.17\%$
test_values[td1_return_estimate-False-False] 44.2346ms 23.6957ms 42.2017 Ops/s 43.1759 Ops/s $\color{#d91a1a}-2.26\%$
test_values[vec_td1_return_estimate-False-False] 37.2828ms 35.5175ms 28.1552 Ops/s 27.9010 Ops/s $\color{#35bf28}+0.91\%$
test_values[td_lambda_return_estimate-True-False] 37.1921ms 33.5917ms 29.7693 Ops/s 29.8992 Ops/s $\color{#d91a1a}-0.43\%$
test_values[vec_td_lambda_return_estimate-True-False] 37.9307ms 35.7284ms 27.9889 Ops/s 28.1731 Ops/s $\color{#d91a1a}-0.65\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.4485ms 8.3192ms 120.2039 Ops/s 121.8417 Ops/s $\color{#d91a1a}-1.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3829ms 1.9325ms 517.4679 Ops/s 557.7811 Ops/s $\textbf{\color{#d91a1a}-7.23\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4231ms 0.3535ms 2.8285 KOps/s 2.7842 KOps/s $\color{#35bf28}+1.59\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 49.8119ms 46.7754ms 21.3787 Ops/s 21.7076 Ops/s $\color{#d91a1a}-1.51\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.0709ms 3.0373ms 329.2383 Ops/s 329.0695 Ops/s $\color{#35bf28}+0.05\%$
test_dqn_speed 6.7965ms 1.2781ms 782.4188 Ops/s 787.4660 Ops/s $\color{#d91a1a}-0.64\%$
test_ddpg_speed 3.5682ms 2.6677ms 374.8505 Ops/s 372.3216 Ops/s $\color{#35bf28}+0.68\%$
test_sac_speed 9.4749ms 7.8877ms 126.7796 Ops/s 124.3936 Ops/s $\color{#35bf28}+1.92\%$
test_redq_speed 14.2535ms 12.6705ms 78.9235 Ops/s 78.5820 Ops/s $\color{#35bf28}+0.43\%$
test_redq_deprec_speed 14.6222ms 12.8074ms 78.0797 Ops/s 78.4427 Ops/s $\color{#d91a1a}-0.46\%$
test_td3_speed 7.9521ms 7.7720ms 128.6678 Ops/s 125.9638 Ops/s $\color{#35bf28}+2.15\%$
test_cql_speed 38.9174ms 35.8615ms 27.8851 Ops/s 28.1006 Ops/s $\color{#d91a1a}-0.77\%$
test_a2c_speed 8.9830ms 7.2435ms 138.0557 Ops/s 137.2199 Ops/s $\color{#35bf28}+0.61\%$
test_ppo_speed 8.8240ms 7.6015ms 131.5538 Ops/s 132.9150 Ops/s $\color{#d91a1a}-1.02\%$
test_reinforce_speed 7.6729ms 6.4488ms 155.0682 Ops/s 150.9942 Ops/s $\color{#35bf28}+2.70\%$
test_iql_speed 33.2770ms 31.7462ms 31.4999 Ops/s 31.2207 Ops/s $\color{#35bf28}+0.89\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.1163ms 4.8470ms 206.3127 Ops/s 212.5227 Ops/s $\color{#d91a1a}-2.92\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8046ms 0.4771ms 2.0960 KOps/s 2.1040 KOps/s $\color{#d91a1a}-0.38\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6929ms 0.4692ms 2.1315 KOps/s 2.2231 KOps/s $\color{#d91a1a}-4.12\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.2867ms 4.8854ms 204.6935 Ops/s 213.8714 Ops/s $\color{#d91a1a}-4.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7538ms 0.4698ms 2.1285 KOps/s 2.1370 KOps/s $\color{#d91a1a}-0.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6015ms 0.4457ms 2.2436 KOps/s 2.2540 KOps/s $\color{#d91a1a}-0.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.3513ms 1.6718ms 598.1617 Ops/s 553.9809 Ops/s $\textbf{\color{#35bf28}+7.98\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.9887ms 1.5812ms 632.4290 Ops/s 614.6850 Ops/s $\color{#35bf28}+2.89\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.9121ms 4.9538ms 201.8668 Ops/s 206.1799 Ops/s $\color{#d91a1a}-2.09\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1379s 0.7099ms 1.4086 KOps/s 1.6462 KOps/s $\textbf{\color{#d91a1a}-14.43\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8403ms 0.5774ms 1.7318 KOps/s 1.7024 KOps/s $\color{#35bf28}+1.73\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.2050ms 5.0248ms 199.0140 Ops/s 208.7380 Ops/s $\color{#d91a1a}-4.66\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5658ms 0.4726ms 2.1158 KOps/s 1.6818 KOps/s $\textbf{\color{#35bf28}+25.80\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6855ms 0.4520ms 2.2124 KOps/s 2.2140 KOps/s $\color{#d91a1a}-0.07\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.2691ms 4.9203ms 203.2414 Ops/s 210.3061 Ops/s $\color{#d91a1a}-3.36\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8550ms 0.4691ms 2.1317 KOps/s 2.1132 KOps/s $\color{#35bf28}+0.87\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5984ms 0.4464ms 2.2402 KOps/s 2.1559 KOps/s $\color{#35bf28}+3.91\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.6834ms 5.1563ms 193.9388 Ops/s 201.1300 Ops/s $\color{#d91a1a}-3.58\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3433ms 0.6052ms 1.6524 KOps/s 1.6283 KOps/s $\color{#35bf28}+1.48\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.3112ms 0.5849ms 1.7098 KOps/s 1.6659 KOps/s $\color{#35bf28}+2.63\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1327s 6.3628ms 157.1636 Ops/s 153.9238 Ops/s $\color{#35bf28}+2.10\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.0314ms 12.9973ms 76.9393 Ops/s 76.1809 Ops/s $\color{#35bf28}+1.00\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.5497ms 1.1626ms 860.1376 Ops/s 851.9255 Ops/s $\color{#35bf28}+0.96\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1094s 8.0247ms 124.6145 Ops/s 165.6228 Ops/s $\textbf{\color{#d91a1a}-24.76\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.0636ms 13.0068ms 76.8829 Ops/s 76.5837 Ops/s $\color{#35bf28}+0.39\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.6437ms 1.1003ms 908.8382 Ops/s 870.0488 Ops/s $\color{#35bf28}+4.46\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1053s 6.1347ms 163.0082 Ops/s 120.4636 Ops/s $\textbf{\color{#35bf28}+35.32\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.0737ms 13.1299ms 76.1618 Ops/s 76.1457 Ops/s $\color{#35bf28}+0.02\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.0559ms 1.2884ms 776.1678 Ops/s 772.3126 Ops/s $\color{#35bf28}+0.50\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 7, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1065s 0.1063s 9.4104 Ops/s 9.3351 Ops/s $\color{#35bf28}+0.81\%$
test_sync 92.3511ms 89.8114ms 11.1344 Ops/s 10.6098 Ops/s $\color{#35bf28}+4.94\%$
test_async 0.2575s 88.7286ms 11.2703 Ops/s 11.0934 Ops/s $\color{#35bf28}+1.60\%$
test_single_pixels 0.1182s 0.1175s 8.5083 Ops/s 8.5233 Ops/s $\color{#d91a1a}-0.18\%$
test_sync_pixels 77.9664ms 76.6016ms 13.0546 Ops/s 12.6249 Ops/s $\color{#35bf28}+3.40\%$
test_async_pixels 0.1417s 71.0136ms 14.0818 Ops/s 13.8908 Ops/s $\color{#35bf28}+1.38\%$
test_simple 0.7575s 0.7563s 1.3222 Ops/s 1.2932 Ops/s $\color{#35bf28}+2.24\%$
test_transformed 1.0888s 1.0244s 0.9762 Ops/s 1.0058 Ops/s $\color{#d91a1a}-2.94\%$
test_serial 2.2596s 2.1838s 0.4579 Ops/s 0.4586 Ops/s $\color{#d91a1a}-0.15\%$
test_parallel 1.9535s 1.8865s 0.5301 Ops/s 0.5334 Ops/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[True-True-True-True-True] 0.1032ms 38.2585μs 26.1380 KOps/s 26.2803 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-True-True-True-False] 81.2520μs 21.2294μs 47.1046 KOps/s 47.2300 KOps/s $\color{#d91a1a}-0.27\%$
test_step_mdp_speed[True-True-True-False-True] 99.6920μs 21.4877μs 46.5382 KOps/s 47.0918 KOps/s $\color{#d91a1a}-1.18\%$
test_step_mdp_speed[True-True-True-False-False] 0.1282ms 12.0871μs 82.7328 KOps/s 83.4421 KOps/s $\color{#d91a1a}-0.85\%$
test_step_mdp_speed[True-True-False-True-True] 71.6710μs 40.3025μs 24.8124 KOps/s 24.9910 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[True-True-False-True-False] 43.0710μs 23.6558μs 42.2729 KOps/s 42.4897 KOps/s $\color{#d91a1a}-0.51\%$
test_step_mdp_speed[True-True-False-False-True] 55.5310μs 23.7993μs 42.0180 KOps/s 42.7543 KOps/s $\color{#d91a1a}-1.72\%$
test_step_mdp_speed[True-True-False-False-False] 31.4600μs 14.3895μs 69.4952 KOps/s 69.9184 KOps/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[True-False-True-True-True] 80.3420μs 42.1520μs 23.7237 KOps/s 23.5873 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[True-False-True-True-False] 56.1210μs 26.1582μs 38.2289 KOps/s 38.8222 KOps/s $\color{#d91a1a}-1.53\%$
test_step_mdp_speed[True-False-True-False-True] 58.9810μs 23.3533μs 42.8204 KOps/s 41.7944 KOps/s $\color{#35bf28}+2.46\%$
test_step_mdp_speed[True-False-True-False-False] 0.1830ms 14.3906μs 69.4898 KOps/s 69.5403 KOps/s $\color{#d91a1a}-0.07\%$
test_step_mdp_speed[True-False-False-True-True] 83.8520μs 44.0528μs 22.7001 KOps/s 22.4340 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[True-False-False-True-False] 0.2150ms 28.4165μs 35.1908 KOps/s 35.5105 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[True-False-False-False-True] 0.2200ms 25.8035μs 38.7545 KOps/s 39.3265 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-False-False-False-False] 45.0810μs 16.6247μs 60.1514 KOps/s 60.8624 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[False-True-True-True-True] 97.5430μs 42.3013μs 23.6400 KOps/s 23.7365 KOps/s $\color{#d91a1a}-0.41\%$
test_step_mdp_speed[False-True-True-True-False] 0.2140ms 25.9465μs 38.5409 KOps/s 38.7714 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-True-True-False-True] 0.1945ms 28.5579μs 35.0165 KOps/s 36.0357 KOps/s $\color{#d91a1a}-2.83\%$
test_step_mdp_speed[False-True-True-False-False] 0.2025ms 16.1112μs 62.0685 KOps/s 61.9868 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[False-True-False-True-True] 0.2495ms 44.9052μs 22.2691 KOps/s 22.3686 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[False-True-False-True-False] 0.2293ms 28.4406μs 35.1610 KOps/s 35.9037 KOps/s $\color{#d91a1a}-2.07\%$
test_step_mdp_speed[False-True-False-False-True] 56.9620μs 30.4312μs 32.8610 KOps/s 33.5739 KOps/s $\color{#d91a1a}-2.12\%$
test_step_mdp_speed[False-True-False-False-False] 40.0020μs 18.5080μs 54.0308 KOps/s 54.3744 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-False-True-True-True] 4.0731ms 46.9934μs 21.2796 KOps/s 21.0363 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[False-False-True-True-False] 57.6620μs 30.9108μs 32.3512 KOps/s 32.8397 KOps/s $\color{#d91a1a}-1.49\%$
test_step_mdp_speed[False-False-True-False-True] 0.1115ms 30.6080μs 32.6712 KOps/s 33.1079 KOps/s $\color{#d91a1a}-1.32\%$
test_step_mdp_speed[False-False-True-False-False] 42.4710μs 18.3469μs 54.5051 KOps/s 53.5464 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[False-False-False-True-True] 86.8720μs 49.1155μs 20.3602 KOps/s 20.5848 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[False-False-False-True-False] 60.7620μs 33.0067μs 30.2969 KOps/s 30.6565 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[False-False-False-False-True] 67.7420μs 32.0265μs 31.2241 KOps/s 31.4488 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[False-False-False-False-False] 40.9220μs 20.6589μs 48.4054 KOps/s 48.2713 KOps/s $\color{#35bf28}+0.28\%$
test_values[generalized_advantage_estimate-True-True] 25.2960ms 24.5324ms 40.7625 Ops/s 41.0157 Ops/s $\color{#d91a1a}-0.62\%$
test_values[vec_generalized_advantage_estimate-True-True] 97.3425ms 2.8493ms 350.9656 Ops/s 367.3899 Ops/s $\color{#d91a1a}-4.47\%$
test_values[td0_return_estimate-False-False] 87.7830μs 64.6725μs 15.4625 KOps/s 15.3467 KOps/s $\color{#35bf28}+0.75\%$
test_values[td1_return_estimate-False-False] 55.7448ms 53.8501ms 18.5701 Ops/s 18.2424 Ops/s $\color{#35bf28}+1.80\%$
test_values[vec_td1_return_estimate-False-False] 1.3939ms 1.0802ms 925.7846 Ops/s 923.6245 Ops/s $\color{#35bf28}+0.23\%$
test_values[td_lambda_return_estimate-True-False] 89.4054ms 86.7889ms 11.5222 Ops/s 11.6016 Ops/s $\color{#d91a1a}-0.68\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.5006ms 1.0891ms 918.1828 Ops/s 923.5451 Ops/s $\color{#d91a1a}-0.58\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 25.4259ms 24.6348ms 40.5929 Ops/s 41.7981 Ops/s $\color{#d91a1a}-2.88\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9581ms 0.7330ms 1.3642 KOps/s 1.4118 KOps/s $\color{#d91a1a}-3.37\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8525ms 0.6638ms 1.5064 KOps/s 1.5042 KOps/s $\color{#35bf28}+0.15\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6555ms 1.4679ms 681.2268 Ops/s 683.3107 Ops/s $\color{#d91a1a}-0.30\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8630ms 0.6772ms 1.4767 KOps/s 1.4723 KOps/s $\color{#35bf28}+0.30\%$
test_dqn_speed 7.4023ms 1.4466ms 691.2715 Ops/s 721.9512 Ops/s $\color{#d91a1a}-4.25\%$
test_ddpg_speed 3.1413ms 2.8425ms 351.7975 Ops/s 356.4058 Ops/s $\color{#d91a1a}-1.29\%$
test_sac_speed 0.1039s 8.9021ms 112.3334 Ops/s 122.6688 Ops/s $\textbf{\color{#d91a1a}-8.43\%}$
test_redq_speed 12.9146ms 10.4173ms 95.9941 Ops/s 96.0764 Ops/s $\color{#d91a1a}-0.09\%$
test_redq_deprec_speed 11.3327ms 10.8713ms 91.9854 Ops/s 86.7566 Ops/s $\textbf{\color{#35bf28}+6.03\%}$
test_td3_speed 8.1533ms 8.0011ms 124.9823 Ops/s 124.9570 Ops/s $\color{#35bf28}+0.02\%$
test_cql_speed 27.0434ms 25.6749ms 38.9485 Ops/s 38.9312 Ops/s $\color{#35bf28}+0.04\%$
test_a2c_speed 5.9046ms 5.6361ms 177.4284 Ops/s 174.8457 Ops/s $\color{#35bf28}+1.48\%$
test_ppo_speed 6.6134ms 5.9721ms 167.4449 Ops/s 166.0015 Ops/s $\color{#35bf28}+0.87\%$
test_reinforce_speed 5.2108ms 4.5646ms 219.0765 Ops/s 216.5422 Ops/s $\color{#35bf28}+1.17\%$
test_iql_speed 20.5105ms 19.6402ms 50.9160 Ops/s 50.2715 Ops/s $\color{#35bf28}+1.28\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.0059ms 6.7200ms 148.8086 Ops/s 149.3373 Ops/s $\color{#d91a1a}-0.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1506ms 0.5253ms 1.9038 KOps/s 1.9211 KOps/s $\color{#d91a1a}-0.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7396ms 0.5015ms 1.9940 KOps/s 1.9951 KOps/s $\color{#d91a1a}-0.06\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.0683ms 6.6465ms 150.4549 Ops/s 151.7289 Ops/s $\color{#d91a1a}-0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.7348ms 0.5198ms 1.9238 KOps/s 1.9513 KOps/s $\color{#d91a1a}-1.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6706ms 0.4993ms 2.0027 KOps/s 2.0215 KOps/s $\color{#d91a1a}-0.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.2016ms 2.0125ms 496.8850 Ops/s 508.3002 Ops/s $\color{#d91a1a}-2.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 5.6587ms 1.9090ms 523.8230 Ops/s 532.9395 Ops/s $\color{#d91a1a}-1.71\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.1549ms 6.8019ms 147.0176 Ops/s 146.1465 Ops/s $\color{#35bf28}+0.60\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1325s 0.7831ms 1.2770 KOps/s 1.4955 KOps/s $\textbf{\color{#d91a1a}-14.61\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8893ms 0.6472ms 1.5450 KOps/s 1.5353 KOps/s $\color{#35bf28}+0.64\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.9650ms 6.6914ms 149.4463 Ops/s 149.0858 Ops/s $\color{#35bf28}+0.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5439ms 0.5228ms 1.9126 KOps/s 1.9168 KOps/s $\color{#d91a1a}-0.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7150ms 0.5043ms 1.9830 KOps/s 1.9806 KOps/s $\color{#35bf28}+0.12\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.1367ms 6.6388ms 150.6304 Ops/s 150.8526 Ops/s $\color{#d91a1a}-0.15\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6907ms 0.5213ms 1.9184 KOps/s 1.9302 KOps/s $\color{#d91a1a}-0.61\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.1108ms 0.5504ms 1.8169 KOps/s 2.0034 KOps/s $\textbf{\color{#d91a1a}-9.31\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.1863ms 6.8666ms 145.6316 Ops/s 145.9825 Ops/s $\color{#d91a1a}-0.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8838ms 0.6750ms 1.4815 KOps/s 1.4816 KOps/s $-0.01\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.5311ms 0.6570ms 1.5221 KOps/s 1.5317 KOps/s $\color{#d91a1a}-0.63\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1326s 7.6741ms 130.3082 Ops/s 97.4203 Ops/s $\textbf{\color{#35bf28}+33.76\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.3764ms 16.0922ms 62.1421 Ops/s 62.5000 Ops/s $\color{#d91a1a}-0.57\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2219ms 1.2716ms 786.4125 Ops/s 727.8055 Ops/s $\textbf{\color{#35bf28}+8.05\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1293s 7.6510ms 130.7019 Ops/s 130.8788 Ops/s $\color{#d91a1a}-0.14\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1409s 18.5313ms 53.9628 Ops/s 62.3500 Ops/s $\textbf{\color{#d91a1a}-13.45\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.4841ms 1.3807ms 724.2854 Ops/s 726.6537 Ops/s $\color{#d91a1a}-0.33\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1292s 7.8015ms 128.1808 Ops/s 128.1943 Ops/s $\color{#d91a1a}-0.01\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.6177ms 16.2903ms 61.3864 Ops/s 61.5289 Ops/s $\color{#d91a1a}-0.23\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.4164ms 1.5103ms 662.1185 Ops/s 673.5932 Ops/s $\color{#d91a1a}-1.70\%$

@vmoens
Update
61e2eb6 
[ghstack-poisoned]
This was referenced Aug 8, 2024
Merged
Open
@vmoens vmoens added the enhancement New feature or request label Aug 8, 2024
@vmoens vmoens linked an issue Aug 8, 2024 that may be closed by this pull request
[Feature Request] Partial steps in env #2356
Open
@vmoens
Copy link
Contributor Author

vmoens commented Aug 8, 2024

closing #1868

@vmoens
Update
ff82fd5 
[ghstack-poisoned]
@vmoens
Update
3f73153 
[ghstack-poisoned]
@vmoens
Update
e783d97 
[ghstack-poisoned]
@vmoens
Update
be1c691 
[ghstack-poisoned]
@vmoens
Update
79d0bb1 
[ghstack-poisoned]
@vmoens
Update
7e73fd0 
[ghstack-poisoned]
@vmoens
Update
6c01e9c 
[ghstack-poisoned]
@vmoens
Update
8a0308b 
[ghstack-poisoned]
@vmoens
Update
5409a2a 
[ghstack-poisoned]
@vmoens
Update
3b7c5a7 
[ghstack-poisoned]
@vmoens
Update
65ca9b8 
[ghstack-poisoned]
@vmoens
Update
f56b8e0 
[ghstack-poisoned]
@vmoens vmoens merged commit f56b8e0 into gh/vmoens/10/base Aug 12, 2024
56 of 68 checks passed
vmoens added a commit that referenced this pull request Aug 12, 2024
@vmoens
[Feature] Partial steps in batched envs
430f1bd 
ghstack-source-id: a1a69e55cddf10290cb59dc1a3c6136bd257368a
Pull Request resolved: #2377
@vmoens vmoens deleted the gh/vmoens/10/head branch August 12, 2024 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Feature Request] Partial steps in env
2 participants
@vmoens @facebook-github-bot