CI workflow with Github Actions

[mhratson]

Summary

Why

1. GIthub Actions workflow are native GH workflows
2. Github Actions do not require additional non-github accounts unlike CircleCI
3. plenty of compute resources¹ available for OSS projects
4. unlike CircleCI resource limits (don't have details)

What

1. creates CI workflow ci.yaml
2. creates Artifactory workflow: artifactory.yaml

Workflow structure is documented in the spec²

Expected Behavior

CI is expected to

1. execute unit tests
2. execute integration tests
3. execute hw platform unit tests
4. publish artifacts to the artifactory when a tag is published
5. provide ability to re-run tests on failures
6. report results to corresponding PR/branch

which is to be used as quality gates for PR merging.

Actual Behavior

1. current Circle CI integration provides 1 2 3 4 from the expected behavior
2. but re-run-ing checks requires additional efforts like logging in into the Circle CI
3. which slows PR feedback loop as users may not have CircleCI credentials and knowledge of the system

Steps to reproduce

1. see failed PR checks, ie Add completed / remaining bytes rebalance metrics #2133

Known Workarounds

1. asking PR authors to trigger build

Migration Plan

1. add GH Actions integration along with CircleCI
2. confirm GH Actions provide equivalent or better functionality
3. remove CircleCI integration
4. ensure publishing via GH actions works

Categorization

• refactor

Example

This PR has both GH Action and CircleCI checks running:

Note how CircleCI doesn't report timing, and is actually slower in current configuration, making GH action complete workflows in 32m vs 40m

Footnotes

1. https://docs.github.com/en/actions/administering-github-actions/usage-limits-billing-and-administration#availability ↩

2. https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions ↩

[CCisGG]

Hi @mhratson , thanks for the change! Let's have more details included in the PR description. E.g. the issue link, what will happen after this merged, what will be the follow up steps, etc. This is a change that impacts every developer, so it would be better to be more clear. Thanks!

[CCisGG]

This looks amazing. Thanks Maryan! We can ship this and improve things along the way when we find anything are missing. We probably need a few other PR check runs to validate this actually works fine.

[CCisGG]

One more question: have we tested new tag publishing, or is that the plan when we merge this PR? I think we need to ensure the release publishing pipeline also works after this migration.

[mhratson]

One more question: have we tested new tag publishing, or is that the plan when we merge this PR? I think we need to ensure the release publishing pipeline also works after this migration.

This is tricky, as to test tag pushing we have to disable the circleCI. So won't be validated until cirlceCI workflow is removed.
But then enabling it back should as easy as reverting the removal commit.

[CCisGG]

This is tricky, as to test tag pushing we have to disable the circleCI. So won't be validated until cirlceCI workflow is removed.
But then enabling it back should as easy as reverting the removal commit.

Got it. So it means that merging this PR itself won't trigger the switch from circleCI to github actions. We probably need to have another PR to disable it?

[mhratson]

Got it. So it means that merging this PR itself won't trigger the switch from circleCI to github actions. We probably need to have another PR to disable it?

Yup. Yeah, for now both pipelines run in parallel. While running tests in parallel is harmless running publishing in parallel is a race. So maybe i'll disable automatic publishing via GH actions until CircleCI is disabled.

[CCisGG]

Yup. Yeah, for now both pipelines run in parallel. While running tests in parallel is harmless running publishing in parallel is a race. So maybe i'll disable automatic publishing via GH actions until CircleCI is disabled.

This makes sense to me. Let's do it.

[mhratson]

Ran publishing¹ for the latest tag, and looks like

1. CircleCI publishing works² as expected
2. looks like we'll have to configure artifactory secrets in GH for it to work:

> Task :artifactoryDeploy FAILED

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':artifactoryDeploy'.
> java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: JFrog service failed. Received 401: {
    "errors" : [ {
      "status" : 401,
      "message" : "Unauthorized"
    } ]
  }

CircleCI secrets[^3]

Using environment variables from project settings and/or contexts:
  ARTIFACTORY_KEY=**REDACTED**
  ARTIFACTORY_USER=**REDACTED**

Footnotes

1. https://github.com/linkedin/cruise-control/actions/runs/10748084760/job/29811352729 ↩

2. https://app.circleci.com/pipelines/github/linkedin/cruise-control/2449/workflows/9f0f0a0c-cdf7-4db4-9b81-753ab197004c/jobs/6708 ↩