Test Verdaccio plugin for build improvements

[DawoudSheraz]

Announced on https://discuss.openedx.org/t/local-npm-registry-for-mfe-builds/13526, Maksim from Raccoon Gang shared a PoC on using Verdaccio as a npm registry. Tutor-mfe build causes npm cache and other issues consistently. The PoC plugin is an attempt to pave to path for having a dedicated registry meant for npm operations that can be expanded to Tutor officially.
In Tutor Users meet on July 29, it was mentioned that Tutor Maintainers will test the plugin out and share their feedback https://openedx.atlassian.net/wiki/spaces/COMM/pages/3583016961/Tutor+Users+Group#2024-07-29. This ticket aims to cover that testing. Once tested, the stats will be shared on the discuss link.

[DawoudSheraz]

I have max-parallelism=2 set for the builds. Here are the stats so far:

Verdaccio enabled, max2cpu buildx config:

• Mfe build (no options, some existing cache?) — 866.7s
• tutor images build mfe --no-cache --no-registry-cache — 1789.9s
• tutor tutor images build mfe --no-cache — 880.1s
• tutor images build mfe --no-registry-cache — 649.4s
• Tutor images build mfe (cache filled) — 766.2s

Verdaccio disabled, max2cpu config

• MFE build (no options, some existing cache?) — 828.2s
• tutor images build mfe --no-cache --no-registry-cache — 1481.8s

I have yet to do more builds with verdaccio disabled.

The plugin did not work out of the box. I had to change tutor version constraint and remove network:host from docker-compose to make it work on Mac.

[Danyal-Faheem]

Here are my stats for this testing:

With Max-parallelism = 2:

• tutor images build mfe --no-cache (Verdaccio empty): 1438.9 secs
• tutor images build mfe --no-cache (Verdaccio filled): 1271.7 secs
• tutor images build mfe --no-cache (Verdaccio disabled): Crashed with network timeout

With Max-parallelism = 1:

• tutor images build mfe --no-cache (Verdaccio disabled): 2107.8s

Note: I had to remove network mode: host from the docker-compose configuration for Verdaccio to get it working on MacOS.

[hinakhadim]

I also had to remove "network_mode:host" from docker-compose and I was also working on MacOS. Testing on my system results in:

Verdaccio	No of CPUs (parallelism)	Command	No. of MFEs	Verdaccio Cache	Time (in seconds)	Notes
Enabled	2	build mfe --no-cache	3	Empty	816
Enabled	2	build mfe --no-cache	3	Filled	841
Enabled	2	build mfe --no-cache	7	Empty	1728.9
Enabled	2	build mfe --no-cache	7	Filled	1839.2
Enabled	2	build mfe --no-cache --no-registry-cache	7	Filled	1906.5
Enabled	2	build mfe	7	Filled	1843.6	[docker cache is empty for mfe image here. Registry cache is filled here]
Enabled	2	build mfe	7	Filled	10	[docker cache is filled for mfe image here. Registry cache is also filled here]
Disabled	2	build mfe --no-cache	7		1850
Disabled	2	build mfe --no-cache --no-registry-cache	7		1933
Disabled	2	build mfe	7		9.1	[docker cache is filled for mfe image here. Registry cache is also filled here]

My conclusion:

• I did not test with different levels of parallelism because I believe parallelism is a separate factor that affects build time independently. The impact of parallelism is not related to the Verdaccio cache, as the results consistently show for a constant level of parallelism.
• The differences in build times are minimal, with only about a 100-second variation between the different configurations. This suggests that the impact of Verdaccio on build performance is negligible, indicating that its benefits may be minimal in this context.

[DawoudSheraz]

Next steps:

• Use verdaccio but do not set max-parallelism. Using verdaccio should not cause any connection errors that we have seen with mfe builds in the past.

[regisb]

Are you guys quite sure that you were hitting verdaccio in your tests, and not the remote NPM registry? The fact that Verdaccio is slower when its cache is filled is perplexing...

Also, the Verdaccio plugin is definitely not working for me on Linux. I get:

 > [learner-dashboard-common 3/5] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-fund --registry=http://host.docker.internal:4873:
82.54 npm ERR! code ENOTFOUND
82.54 npm ERR! syscall getaddrinfo
82.54 npm ERR! errno ENOTFOUND
82.55 npm ERR! network request to http://host.docker.internal:4873/npm/-/npm-8.19.4.tgz failed, reason: getaddrinfo ENOTFOUND host.docker.internal
82.55 npm ERR! network This is a problem related to network connectivity.
82.55 npm ERR! network In most cases you are behind a proxy or have bad network settings.
82.55 npm ERR! network 
82.55 npm ERR! network If you are behind a proxy, please make sure that the
82.55 npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
82.55 
82.55 npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-08-14T11_17_44_546Z-debug-0.log
------
Dockerfile:397

Turns out that host.docker.internal is not supported on Linux: https://stackoverflow.com/questions/48546124/what-is-the-linux-equivalent-of-host-docker-internal

EDIT: I think that the proper command to test optimizations with/without Verdaccio is:

tutor images build --docker-arg=--no-cache-filter=authn-common,authn-prod,account-common,account-prod,communications-common,communications-prod,course-authoring-common,course-authoring-prod,discussions-common,discussions-prod,gradebook-common,gradebook-prod,learner-dashboard-common,learner-dashboard-prod,learning-common,learning-prod,ora-grading-common,ora-grading-prod,profile-common,profile-prod mfe

[DawoudSheraz]

Are you guys quite sure that you were hitting verdaccio in your tests, and not the remote NPM registry? The fact that Verdaccio is slower when its cache is filled is perplexing...

Also, the Verdaccio plugin is definitely not working for me on Linux. I get:

 > [learner-dashboard-common 3/5] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-fund --registry=http://host.docker.internal:4873:
82.54 npm ERR! code ENOTFOUND
82.54 npm ERR! syscall getaddrinfo
82.54 npm ERR! errno ENOTFOUND
82.55 npm ERR! network request to http://host.docker.internal:4873/npm/-/npm-8.19.4.tgz failed, reason: getaddrinfo ENOTFOUND host.docker.internal
82.55 npm ERR! network This is a problem related to network connectivity.
82.55 npm ERR! network In most cases you are behind a proxy or have bad network settings.
82.55 npm ERR! network 
82.55 npm ERR! network If you are behind a proxy, please make sure that the
82.55 npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
82.55 
82.55 npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-08-14T11_17_44_546Z-debug-0.log
------
Dockerfile:397

Turns out that host.docker.internal is not supported on Linux: https://stackoverflow.com/questions/48546124/what-is-the-linux-equivalent-of-host-docker-internal

EDIT: I think that the proper command to test optimizations with/without Verdaccio is:

tutor images build --docker-arg=--no-cache-filter=authn-common,authn-prod,account-common,account-prod,communications-common,communications-prod,course-authoring-common,course-authoring-prod,discussions-common,discussions-prod,gradebook-common,gradebook-prod,learner-dashboard-common,learner-dashboard-prod,learning-common,learning-prod,ora-grading-common,ora-grading-prod,profile-common,profile-prod mfe

It did not work out of the box. During the build, the MFE container could not connect to Verdaccio container. Once network:host was removed from docker-compose, the connection established successfully. Verdaccio container logs showed whenever the registry was hit.