Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refac: TreeIndexToSectorMap optimize #3697

Conversation

Strepto
Copy link
Contributor

@Strepto Strepto commented Sep 14, 2023

Type of change

Refactor

Jira ticket 📘

https://cognitedata.atlassian.net/browse/

Description 📝

Builds upon the changes in #3695 , consider reviewing that first

Allocating a new Set for each treeindex in the model has some memory overhead when profiled.

Stats for a custom sample model scene in the reveal "example" viewer:

See JS Heap Size
image

Initial load of custom 3d model for profiling.

More stats for the camera viewpoint I took all profilings from:
Unique treeIndexes: 347012 Average sectors per treeIndex 1.0567 (The scene.json has ~2.3 million treeindexes so this is just a subset)

Distribution of sectors per TreeIndex in the loaded sectors

1: 327558  (Read: 327558 (>90%) treeindexes are in just 1 sector)
2: 19232 
3: 197 
4: 24 
5: 1 

When we have so few items the set.has vs array.contains becomes equal or better for array. And the overhead of an Set vs array is considerable when we have millions of them.

How has this been tested? 🔍

Unit test coverage should cover the use.

Tested in browser, and found no performance regression (possible perf increase) and found significant memory saving.

Test instructions ℹ️

Checklist ☑️

  • I am proud of this feature.
  • I have performed a self-review of my own code.
  • I have added PR type (Feat, Bug, Chore, Test, Docs, Style, Refactor) to the PR title.
  • I have manually tested this for different scenarios (different models, formats, devices, browsers).
  • I have commented my code in hard-to-understand areas.
  • I have made corresponding changes to the public documentation.
  • I have added unit and visuals tests to prove that my feature works to the best of my ability.
  • I have refactored the code for readability to the best of my ability.
  • I have checked that my changes do not introduce regressions in the public documentation.
  • I have outlined any known defects / lacking capabilities in the contents of this PR.
  • I have listed any remaining work that should follow this PR in the description or jira/miro/etc.
  • I have added TSDoc to any public facing changes.
  • I have added "developer documentation" in any package README.md that is related / applicable to this PR.
  • I have noted down and am currently tracking any technical debt introduced in this PR.
  • I have thoroughly thought about the architecture of this implementation.
  • I have asked peers to test this feature.

@Strepto Strepto requested a review from a team as a code owner September 14, 2023 13:18
@codecov
Copy link

codecov bot commented Sep 14, 2023

Codecov Report

Merging #3697 (3d19a0d) into master (887b847) will increase coverage by 0.00%.
Report is 41 commits behind head on master.
The diff coverage is 87.50%.

@@           Coverage Diff           @@
##           master    #3697   +/-   ##
=======================================
  Coverage   71.70%   71.71%           
=======================================
  Files         346      346           
  Lines       33762    33766    +4     
  Branches     2650     2650           
=======================================
+ Hits        24210    24214    +4     
  Misses       9445     9445           
  Partials      107      107           
Files Coverage Δ
...s/cad-model/src/utilities/TreeIndexToSectorsMap.ts 100.00% <100.00%> (ø)
viewer/packages/cad-model/src/wrappers/CadNode.ts 86.40% <100.00%> (ø)
...packages/cad-model/src/wrappers/CogniteCadModel.ts 78.65% <0.00%> (ø)

Reduces allocation by using array instead of set for values.
Uses indices instead of a set for lookup
@Strepto Strepto force-pushed the nih/feat/treeindexmap_optimize branch from 4dfb552 to 3d19a0d Compare September 19, 2023 07:18
Copy link
Contributor

@haakonflatval-cognite haakonflatval-cognite left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

I ran some tests - they showed a slightly higher average FPS, while the memory use is much smaller. Nice contribution!

@haakonflatval-cognite haakonflatval-cognite added the auto-update Makes bulldozer automatically update this PR when there are changes to the target branch label Sep 28, 2023
@haakonflatval-cognite haakonflatval-cognite merged commit 51f3f23 into cognitedata:master Sep 28, 2023
14 checks passed
@Strepto Strepto deleted the nih/feat/treeindexmap_optimize branch September 28, 2023 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-update Makes bulldozer automatically update this PR when there are changes to the target branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants