[WIP] Add builds of Ceph and Rook binaries and OCIs #498
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cryslith
reviewed
Mar 3, 2020
| # remove symlinks that create cycles (which break glob) | ||
| "find -name '.qa' -delete", | ||
| # remove filenames with ":" in them (which are disallowed in filegroups) | ||
| "rm src/test/common/test_blkdev_sys_block/sys/dev/block/8:0 src/test/common/test_blkdev_sys_block/sys/dev/block/9:0", |
There was a problem hiding this comment.
any reason not to use another find command for this? or at least to verify that there aren't any unexpected filenames with that character?
There was a problem hiding this comment.
There definitely might be some unexpected names that happen to include colons, but verifying that there aren't any further names like this is a great idea.
cryslith
reviewed
Mar 8, 2020
platform/ceph/BUILD.bazel
Outdated
| # OOM killer gets fed up with Bazel, but I can't come up with a good | ||
| # way around that, and hopefully this code is correct enough that it | ||
| # never happens. | ||
| "make -j$(expr $(expr 1 + $(cat /proc/cpuinfo | cut -f 1 | grep -E '^processor$' | wc -l)) / 2)", |
There was a problem hiding this comment.
-
I don't think you need the nested
$(expr ...)here,exprunderstands parentheses just fine. -
use
</proc/cpuinfo ..., notcat /proc/cpuinfo | ... -
There are a couple ways to detect the inner command failing in bash, such as setting a variable and using an explicit test. But at this level of complexity it might be appropriate to refactor this into a separate script.
There was a problem hiding this comment.
- Will fix
- Will fix
- The make_commands parameter is badly implemented, and there appears to be no way to embed a reference to an external file in it, hence inlining the logic that really deserves a separate script. I will try to update the logic to be smarter, just inline.
celskeggs
changed the title
cryslith
reviewed
Apr 11, 2020
platform/ceph/BUILD.bazel
Outdated
| # which requires working pip in a virtualenv, which is not working for | ||
| # some reason relating to DNS resolution. | ||
| # (and it's too much hell to make it functional for me to want to deal | ||
| # with this mess right now.) |
There was a problem hiding this comment.
I think it's a fine justification without this acerbic parenthetical :)
It takes so long to build Ceph that Jenkins times out. Increase the amount of time we have to run our build so that this doesn't happen.
Ceph wants LVM on the server hosts. Make sure we include it, then.
The previous workspace format name corresponds to the import style used by gazelle, but this particular etcd dependency isn't appropriate for use in imports. Since rook is going to require an appropriate etcd dependency, change this dependency to use a different name that won't conflict with the importable one.
celskeggs
force-pushed
the
29-setup-ceph
branch
3 times, most recently
from
cd6b40a
to
9415874
Compare
In order to meet our supply chain protection standards, we have to build our own version of Ceph to use with Rook. This commit brings in rules_foreign_cc, a set of bazel rules to more easily run cmake builds, and then uses those rules to build Ceph using cmake. Of course, we end up needing to patch both Ceph and rules_foreign_cc to make them work correctly together, and we end up needing to install many more packages into the build chroot.
In order to manage our disk cluster, we need a build of rook. Our software supply chain standards mean that we have to build it ourselves. We also need to pull in numerous Rook dependencies, update the version of at least one dependency, build tini, and package up Rook in a usable OCI.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In order to meet our supply chain protection standards, we have to build our own versions of Rook and Ceph.
I am not yet planning on merging this PR, because I want to make more progress on Ceph before concluding that I did this part correctly, but I would like a code review as things stand.
The testing carried out for Ceph to determine that this worked correctly was manually deploying a container running the
cephimage, execing into it with kubectl, and confirming that every one of the ceph commands could be launched. (This took a lot of iteration, to make sure all of the dependencies -- both built as part of Ceph and not built as part of Ceph -- worked out.) The same testing process was performed for the Rook binaries.See #29.
Checklist: