feat(Build): use bench get-app cache

[18alantom]

Use bench get-app cache when running bench get-app during build. Related PRs:

• feat: cache get-app artifacts by commit_hash bench#1519
• feat(Bench): add using-cached flag frappe#24412
• fix: delete node_modules if frappe supports app_cache bench#1525
• fix: increase tar file limit bench#1526
• fix: coerce git_url to app_name bench#1527

Since this is bench dependent it will be restricted to bench versions 5.21.3 and up only.

TODO:

• Ability to check contents of mounted cache
• Update Dockerfile with new bench get-app command
• Prevent git clone if get-app cache is present
• Update build steps not required since clone will be marked as cached
• Feature flag on Release Group(?)
• Delete docker image after app cache list check
• Utils to check and delete cache
• If previous deploy present as layer cache, skip app cache
• Add Compress artifacts (hard coded default, else massive tarfiles most above 200MB)
• Layer cache miss cause bind mount has changed (handling of repos in bind mount)
• Does it work?
• Fix CI

Notes

Further Improvements

• Caching bench init artifacts mainly frappe and its dependencies.
• Caching app Python dependencies
• Reducible Inefficiencies:
  • Due to bench not handling clone:
    • ~15s to check app cache for every build
    • few seconds to clone repo even if app cache present (else potential layer cache miss)
  • Due to partial build code being in frappe repo
    • node_modules have to be included in cache
    • tar creation and extraction times increase cause of node_modules
      • eg hrms: 305ms to 11,429ms
    • cache space requirement increases cause of node_modules
      • eg hrms: 31MB in 1744 entries to 71MB in 32,198 entries
    • with node_modules compression may be required cause size diff without compression

Current Caveats

• Due to dependence on frappe>15.12.0 for build node_modules have to be included in the cache.
• Compressing and decompressing the cache takes a lot of time due to inclusion of node_modules.
• Python dependencies are not cached and so pip install has to be run, this too is very time consuming.

Get App Cache in Action

First Run (cache empty): time to build 9 apps 1780s (~30 minutes)

• Apps are cached by bench after being built
• Build times include caching times.
• The cache times are high due to inclusion of node_modules and compressing cached assets.

Second Run (cache used): time to build 9 apps 697s (~12 minutes)

• Wiki added to app list to cause docker layer cache miss.
• Bench get-app cache is used for apps that were present in the first run.
• Install times include pip installation which cannot be cached and decompressing the cache.

Note: without compression, build times are:

• Build 1 (cache empty): 1443s (~24 minutes)
• Build 2 (cache used): 606s (~10 minutes)

Bench Get App Cache Virtual Doctype

Added to allow viewing cached artifacts, and checking report view. Delete also has been added but since each delete requires building a docker image (~15s) it's preferable to just use docker prune to free space.

Additional functionality can be added later on using run_command_in_docker_cache

Images

[codecov]

Codecov Report

Attention: 34 lines in your changes are missing coverage. Please review.

Comparison is base (aaa7dd6) 44.78% compared to head (ee675f7) 45.16%.

Files	Patch %	Lines
...ress/press/doctype/deploy_candidate/cache_utils.py	81.70%	15 Missing ⚠️
...doctype/bench_get_app_cache/bench_get_app_cache.py	80.59%	13 Missing ⚠️
press/press/doctype/release_group/release_group.py	75.00%	3 Missing ⚠️
press/press/doctype/app/app.py	66.66%	1 Missing ⚠️
...press/doctype/deploy_candidate/deploy_candidate.py	92.30%	1 Missing ⚠️
press/utils/__init__.py	95.83%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1396      +/-   ##
==========================================
+ Coverage   44.78%   45.16%   +0.38%     
==========================================
  Files         310      312       +2     
  Lines       20742    20935     +193     
==========================================
+ Hits         9289     9456     +167     
- Misses      11453    11479      +26     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[18alantom]

⚠️ Roadblock ⚠️

1. Install fails if app fetched from cache due to absent node_modules.
2. Bench needs to not delete node_modules if app depends on frappe<15.12.0.
3. Apps need to be updated with required frappe version.
4. Bench needs to be updated to account for points 2. and 3.
5. Need to know version documentation method for 4.

Will be putting it on hold until we've decided a way for apps to document minimum required frappe version.

@ankush

---

Edit: Roadblock removed. Format mentioned in frappe/bench#1524. Bench updated in frappe/bench#1525.