Skip to content

Commit

Permalink
doc: synthetic Integrations
Browse files Browse the repository at this point in the history
  • Loading branch information
squakez committed Jan 9, 2024
1 parent 5cedfed commit a1601b7
Show file tree
Hide file tree
Showing 9 changed files with 156 additions and 123 deletions.
3 changes: 3 additions & 0 deletions config/manager/operator-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,9 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
# Change to true to be able to create synthetic Integrations
- name: CAMEL_K_SYNTHETIC_INTEGRATIONS
value: "false"
livenessProbe:
httpGet:
path: /healthz
Expand Down
3 changes: 2 additions & 1 deletion docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,9 @@
** xref:running/dev-mode.adoc[Developer mode]
** xref:running/dry-run.adoc[Dry run]
** xref:running/runtime-version.adoc[Camel version]
** xref:running/camel-runtimes.adoc[Camel runtimes]
** xref:running/quarkus-native.adoc[Quarkus Native]
** xref:running/camel-runtimes.adoc[Camel runtimes]
** xref:running/import.adoc[Import existing Camel apps]
** xref:running/run-from-github.adoc[Run from GitHub]
** xref:running/promoting.adoc[Promote an Integration]
** xref:running/knative-sink.adoc[Knative Sinks]
44 changes: 44 additions & 0 deletions docs/modules/ROOT/pages/running/import.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
= Importing existing Camel applications

You may have already a Camel application running on your cluster. You may have created it via a manual deployment, a CICD or any other deployment mechanism you have in place. Since the Camel K operator is meant to operate any Camel application out there, then, you will be able to import it and monitor in a similar fashion of any other Camel K **managed Integration**.

This feature is disabled by default. In order to enable it, you need to run the operator deployment with an environment variable, `CAMEL_K_SYNTHETIC_INTEGRATIONS`, set to `true`.

NOTE: you will be only able to monitor the synthetic Integrations. Camel K won't be able to alter the lifecycle of non managed Integrations (ie, rebuild the original application).

It's important to notice that the operator won't be altering any field of the original application in order to avoid breaking any deployment procedure which is already in place. As it cannot make any assumption on the way the application is built and deployed, it will only be able to **watch** for any changes happening around it.

[[deploy-and-monitor]]
== Deploy externally, monitor via Camel K Operator

An imported Integration is known as **synthetic Integration**. You can import any Camel application deployed as a **Deployment**, **CronJob** or **Knative Service**. We control this behavior via a label (`camel.apache.org/integration`) that the user need to apply on the Camel application (either manually or introducing in the deployment process, ie, via CICD).

NOTE: the example here will work in a similar way using CronJob and Knative Service.

As an example, we show how to import a Camel application which was deployed with the Deployment kind. Let's assume it is called `my-deploy`.
```
$ kubectl label deploy my-camel-sb-svc camel.apache.org/integration=my-it
```
The operator immediately creates a synthetic Integration:
```
$ kubectl get it
NAMESPACE NAME PHASE RUNTIME PROVIDER RUNTIME VERSION KIT REPLICAS
test-79c385c3-d58e-4c28-826d-b14b6245f908 my-it Cannot Monitor Pods
```
You can see it will be in `Cannot Monitor Pods` status phase. This is expected because the way Camel K operator monitor Pods. It requires that the same label applied to the Deployment is inherited by the generated Pods. For this reason, beside labelling the Deployment, we need to add a label in the Deployment template.
```
$ kubectl patch deployment my-camel-sb-svc --patch '{"spec": {"template": {"metadata": {"labels": {"camel.apache.org/integration": "my-it"}}}}}'
```
Also this operator can be performed manually or automated in the deployment procedure. We can see now that the operator will be able to monitor accordingly the status of the Pods:
```
$ kubectl get it
NAMESPACE NAME PHASE RUNTIME PROVIDER RUNTIME VERSION KIT REPLICAS
test-79c385c3-d58e-4c28-826d-b14b6245f908 my-it Running 1
```
From now on, you will be able to monitor the status of the synthetic Integration in a similar fashion of what you do with managed Integrations. If, for example, your Deployment will scale up or down, then, you will see this information reflecting accordingly:
```
$ kubectl scale deployment my-camel-sb-svc --replicas 2
$ kubectl get it
NAMESPACE NAME PHASE RUNTIME PROVIDER RUNTIME VERSION KIT REPLICAS
test-79c385c3-d58e-4c28-826d-b14b6245f908 my-it Running 2
```
26 changes: 0 additions & 26 deletions e2e/common/synthetic/default.go

This file was deleted.

93 changes: 0 additions & 93 deletions e2e/common/synthetic/synthetic_test.go

This file was deleted.

File renamed without changes.
100 changes: 100 additions & 0 deletions e2e/commonwithcustominstall/synthetic_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
//go:build integration
// +build integration

// To enable compilation of this file in Goland, go to "Settings -> Go -> Vendoring & Build Tags -> Custom Tags" and add "integration"

/*
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package commonwithcustominstall

import (
"testing"

. "github.com/onsi/gomega"

. "github.com/apache/camel-k/v2/e2e/support"
v1 "github.com/apache/camel-k/v2/pkg/apis/camel/v1"
. "github.com/onsi/gomega/gstruct"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
)

func TestSyntheticIntegrationFromDeployment(t *testing.T) {
RegisterTestingT(t)
WithNewTestNamespace(t, func(ns string) {
// Install Camel K with the synthetic Integration feature variable
operatorID := "camel-k-synthetic-env"
Expect(KamelInstallWithID(operatorID, ns,
"--operator-env-vars", "CAMEL_K_SYNTHETIC_INTEGRATIONS=true",
).Execute()).To(Succeed())

// Run the external deployment
ExpectExecSucceed(t, Kubectl("apply", "-f", "files/deploy.yaml", "-n", ns))
Eventually(DeploymentCondition(ns, "my-camel-sb-svc", appsv1.DeploymentProgressing), TestTimeoutShort).
Should(MatchFields(IgnoreExtras, Fields{
"Status": Equal(corev1.ConditionTrue),
"Reason": Equal("NewReplicaSetAvailable"),
}))

// Label the deployment --> Verify the Integration is created (cannot still monitor)
ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration=my-it", "-n", ns))
Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseCannotMonitor))
Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))

// Label the deployment template --> Verify the Integration is monitored
ExpectExecSucceed(t, Kubectl("patch", "deployment", "my-camel-sb-svc", "--patch", `{"spec": {"template": {"metadata": {"labels": {"camel.apache.org/integration": "my-it"}}}}}`, "-n", ns))
Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseRunning))
Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
one := int32(1)
Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&one))

// Delete the deployment --> Verify the Integration is in missing status
ExpectExecSucceed(t, Kubectl("delete", "deploy", "my-camel-sb-svc", "-n", ns))
Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseImportMissing))
Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionFalse))
zero := int32(0)
Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&zero))

// Recreate the deployment and label --> Verify the Integration is monitored
ExpectExecSucceed(t, Kubectl("apply", "-f", "files/deploy.yaml", "-n", ns))
ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration=my-it", "-n", ns))
ExpectExecSucceed(t, Kubectl("patch", "deployment", "my-camel-sb-svc", "--patch", `{"spec": {"template": {"metadata": {"labels": {"camel.apache.org/integration": "my-it"}}}}}`, "-n", ns))
Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseRunning))
Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&one))

// Remove label from the deployment --> Verify the Integration is deleted
ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration-", "-n", ns))
Eventually(Integration(ns, "my-it"), TestTimeoutShort).Should(BeNil())

// Add label back to the deployment --> Verify the Integration is created
ExpectExecSucceed(t, Kubectl("label", "deploy", "my-camel-sb-svc", "camel.apache.org/integration=my-it", "-n", ns))
Eventually(IntegrationPhase(ns, "my-it"), TestTimeoutShort).Should(Equal(v1.IntegrationPhaseRunning))
Eventually(IntegrationConditionStatus(ns, "my-it", v1.IntegrationConditionReady), TestTimeoutShort).Should(Equal(corev1.ConditionTrue))
Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&one))
// Scale the deployment --> verify replicas are correctly set
ExpectExecSucceed(t, Kubectl("scale", "deploy", "my-camel-sb-svc", "--replicas", "2", "-n", ns))
two := int32(2)
Eventually(IntegrationStatusReplicas(ns, "my-it"), TestTimeoutShort).Should(Equal(&two))

// Delete Integration and deployments --> verify no Integration exists any longer
Expect(Kamel("delete", "--all", "-n", ns).Execute()).To(Succeed())
ExpectExecSucceed(t, Kubectl("delete", "deploy", "my-camel-sb-svc", "-n", ns))
Eventually(Integration(ns, "my-it"), TestTimeoutShort).Should(BeNil())
})
}
9 changes: 7 additions & 2 deletions pkg/cmd/operator/operator.go
Original file line number Diff line number Diff line change
Expand Up @@ -232,8 +232,13 @@ func Run(healthPort, monitoringPort int32, leaderElection bool, leaderElectionID
install.OperatorStartupOptionalTools(installCtx, bootstrapClient, watchNamespace, operatorNamespace, log)
exitOnError(findOrCreateIntegrationPlatform(installCtx, bootstrapClient, operatorNamespace), "failed to create integration platform")

log.Info("Starting the synthetic Integration manager")
exitOnError(synthetic.ManageSyntheticIntegrations(ctx, ctrlClient, mgr.GetCache(), mgr.GetAPIReader()), "synthetic Integration manager error")
synthEnvVal, synth := os.LookupEnv("CAMEL_K_SYNTHETIC_INTEGRATIONS")
if synth && synthEnvVal == "true" {
log.Info("Starting the synthetic Integration manager")
exitOnError(synthetic.ManageSyntheticIntegrations(ctx, ctrlClient, mgr.GetCache(), mgr.GetAPIReader()), "synthetic Integration manager error")
} else {
log.Info("Synthetic Integration manager not configured, skipping")
}
log.Info("Starting the manager")
exitOnError(mgr.Start(ctx), "manager exited non-zero")
}
Expand Down
1 change: 0 additions & 1 deletion script/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,6 @@ test-common: do-build
go test -timeout 30m -v ./e2e/common/misc -tags=integration $(TEST_INTEGRATION_COMMON_LANG_RUN) $(GOTESTFMT) || FAILED=1; \
go test -timeout 60m -v ./e2e/common/traits -tags=integration $(TEST_INTEGRATION_COMMON_LANG_RUN) $(GOTESTFMT) || FAILED=1; \
go test -timeout 20m -v ./e2e/common/runtimes -tags=integration $(TEST_INTEGRATION_COMMON_LANG_RUN) $(GOTESTFMT) || FAILED=1; \
go test -timeout 10m -v ./e2e/common/synthetic -tags=integration $(TEST_INTEGRATION_COMMON_LANG_RUN) $(GOTESTFMT) || FAILED=1; \
go test -timeout 10m -v ./e2e/common/support/teardown_test.go -tags=integration $(TEST_INTEGRATION_COMMON_LANG_RUN) $(GOTESTFMT) || FAILED=1; \
exit $${FAILED}

Expand Down

0 comments on commit a1601b7

Please sign in to comment.