add Use WorkloadSpread with customized workload manual (#209)

Signed-off-by: AiRanthem <zhongtianyun.zty@alibaba-inc.com> Co-authored-by: AiRanthem <zhongtianyun.zty@alibaba-inc.com>
openkruise · Sep 12, 2024 · cc4ea61 · cc4ea61
1 parent 2fe54d1
commit cc4ea61
Show file tree

Hide file tree

Showing 8 changed files with 1,018 additions and 814 deletions.
diff --git a/.github/workflows/pre_dict.json b/.github/workflows/pre_dict.json
@@ -864,5 +864,18 @@
     "th",
     "updateReadyPods",
     "hostpath",
-    "featuregate"
+    "featuregate",
+    "replicasPath",
+    "subResources",
+    "ApiGroup",
+    "SubResources",
+    "rbac",
+    "ClusterRole",
+    "kruise-rollouts-access",
+    "apiGroups",
+    "ClusterRoleBinding",
+    "kruise-rollouts-access-binding",
+    "ServiceAccount",
+    "roleRef",
+    "apiGroup"
 ]
diff --git a/docs/user-manuals/workloadspread.md b/docs/user-manuals/workloadspread.md
@@ -27,7 +27,7 @@ Since Kruise `1.3.0`, WorkloadSpread supports `StatefulSet`.
 
 In particular, for `StatefulSet`, WorkloadSpread supports manage its subsets only when `scale up`. The order of `scale down` is still controlled by StatefulSet controller. The subset management of StatefulSet is based on ordinals of Pods, and more details can be found [here](https://github.com/openkruise/kruise/blob/f46097db1fa5a4ed9c002eba050b888344884e11/pkg/util/workloadspread/workloadspread.go#L305).
 
-Since Kruise `1.5.0`, WorkloadSpread supports `customized workloads that have [scale sub-resource](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#scale-subresource)`.
+Since Kruise `1.5.0`, WorkloadSpread supports `customized workloads` that have [scale sub-resource](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#scale-subresource).
 
 ## Demo
 
@@ -204,6 +204,94 @@ The workload managed by WorkloadSpread will scale according to the defined order
 # deletion order: b -> a -> c
 ```
 
+## Use WorkloadSpread with customized workload
+
+If you want to use WorkloadSpread with custom workloads, which is disabled by default, some
+additional configuration is required. This section uses
+the [Rollout Workload from the Argo community](https://argoproj.github.io/argo-rollouts/) as an example to
+demonstrate how to integrate it with WorkloadSpread.
+
+### Configure the custom workload watch whitelist
+
+First, you need to add the custom workload to the `WorkloadSpread_Watch_Custom_Workload_WhiteList` to ensure it can be
+read and understood by WorkloadSpread.
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kruise-configuration
+  namespace: kruise-system
+data:
+  "WorkloadSpread_Watch_Custom_Workload_WhiteList": |
+    {
+      "workloads": [
+        {
+          "Group": "argoproj.io",
+          "Kind": "Rollout",
+          "replicasPath": "spec.replicas",
+        }
+      ]
+    }
+```
+
+The specific configuration items are explained as follows:
+
+- **Group:** ApiGroup of the customized workload.
+- **Kind:** Kind of the customized workload.
+- **subResources:** SubResources of the customized workload, including Group and Kind. For example: Deployment's
+  ReplicaSet. This field is optional, and can be left as empty slice if no sub-workload is used for the customized workload.
+- **replicasPath:** Resource path to the replicas in the resource. For example: spec.replicas.
+
+### Authorize kruise-manager
+
+To use WorkloadSpread with custom workloads, you need to grant the kruise-manager service account read permissions for
+the respective resources.
+
+**Caution**: The WorkloadSpread Webhook does not set a deletion cost for Pods created by custom workloads, so it cannot ensure the scaling-down order of custom workloads.
+
+```yaml
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: kruise-rollouts-access
+rules:
+  - apiGroups: [ "argoproj.io" ]
+    resources: [ "rollouts" ]
+    verbs: [ "get" ]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: kruise-rollouts-access-binding
+subjects:
+  - kind: ServiceAccount
+    name: kruise-manager
+    namespace: kruise-system
+roleRef:
+  kind: ClusterRole
+  name: kruise-rollouts-access
+  apiGroup: rbac.authorization.k8s.io
+```
+
+### Reference the custom workload in WorkloadSpread
+
+Once the configuration is complete, the custom workload can be referenced in the `targetRef` field of WorkloadSpread.
+
+```yaml
+apiVersion: apps.kruise.io/v1alpha1
+kind: WorkloadSpread
+metadata:
+  name: workloadspread-demo
+spec:
+  targetRef:
+    apiVersion: argoproj.io/v1alpha1
+    kind: Rollout
+    name: rollouts-demo
+  subsets:
+    ...
+```
+
 ## feature-gates
 WorkloadSpread feature is turned off by default, if you want to turn it on set feature-gates *WorkloadSpread*.
 

diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/user-manuals/workloadspread.md b/i18n/zh/docusaurus-plugin-content-docs/current/user-manuals/workloadspread.md
@@ -203,6 +203,89 @@ WorkloadSpread所管理的workload会按照`subsets`中定义的顺序扩缩容
 # deletion order: b -> a -> c
 ```
 
+## 在自定义工作负载上使用 WorkloadSpread
+
+WorkloadSpread 默认不会监听自定义工作负载。如果想要在自定义工作负载上使用 WorkloadSpread，需要进行额外的配置。本节以 [Argo
+社区的 Rollout Workload](https://argoproj.github.io/argo-rollouts/) 为例，介绍如何将其与 WorkloadSpread 配合使用。
+
+**注意**：WorkloadSpread Webhook 不会为自定义工作负载所创建的 Pod 设置 deletion cost，因而无法保证自定义工作负载的缩容顺序。
+
+### 配置自定义工作负载监听白名单
+
+首先，需要将自定义工作负载加入`WorkloadSpread_Watch_Custom_Workload_WhiteList` 白名单，以使其能够被 WorkloadSpread 读取并理解。
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kruise-configuration
+  namespace: kruise-system
+data:
+  "WorkloadSpread_Watch_Custom_Workload_WhiteList": |
+    {
+      "workloads": [
+        {
+          "Group": "argoproj.io",
+          "Kind": "Rollout",
+          "replicasPath": "spec.replicas",
+        }
+      ]
+    }
+```
+
+具体的配置项说明如下：
+
+- **Group:** 自定义工作负载的 ApiGroup。
+- **Kind:** 自定义工作负载的 Kind。
+- **subResources:** 自定义工作负载的子资源，字段包括 Group 与 Kind。例如：Deployment 的 ReplicaSet。该字段为可选字段，如果自定义资源不包含子资源，那么可以留空。
+- **replicasPath:** 自定义工作负载中用于指定副本数的资源字段路径，例如：spec.replicas。
+
+### 向 kruise-manager 授予权限
+
+要在自定义工作负载上使用 WorkloadSpread，需要给 kruise-manager 服务账号授予相应资源的读取权限。
+
+```yaml
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: kruise-rollouts-access
+rules:
+  - apiGroups: [ "argoproj.io" ]
+    resources: [ "rollouts" ]
+    verbs: [ "get" ]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: kruise-rollouts-access-binding
+subjects:
+  - kind: ServiceAccount
+    name: kruise-manager
+    namespace: kruise-system
+roleRef:
+  kind: ClusterRole
+  name: kruise-rollouts-access
+  apiGroup: rbac.authorization.k8s.io
+```
+
+### 指定自定义工作负载
+
+当配置完成后，即可在 WorkloadSpread 的 `targetRef` 字段中指定自定义工作负载。
+
+```yaml
+apiVersion: apps.kruise.io/v1alpha1
+kind: WorkloadSpread
+metadata:
+  name: workloadspread-demo
+spec:
+  targetRef:
+    apiVersion: argoproj.io/v1alpha1
+    kind: Rollout
+    name: rollouts-demo
+  subsets:
+    ...
+```
+
 ## feature-gates
 WorkloadSpread 默认是关闭的，如果要开启请通过设置 feature-gates *WorkloadSpread*.
 

diff --git a/i18n/zh/docusaurus-plugin-content-docs/version-v1.6/user-manuals/workloadspread.md b/i18n/zh/docusaurus-plugin-content-docs/version-v1.6/user-manuals/workloadspread.md
@@ -203,6 +203,89 @@ WorkloadSpread所管理的workload会按照`subsets`中定义的顺序扩缩容
 # deletion order: b -> a -> c
 ```
 
+## 在自定义工作负载上使用 WorkloadSpread
+
+WorkloadSpread 默认不会监听自定义工作负载。如果想要在自定义工作负载上使用 WorkloadSpread，需要进行额外的配置。本节以 [Argo
+社区的 Rollout Workload](https://argoproj.github.io/argo-rollouts/) 为例，介绍如何将其与 WorkloadSpread 配合使用。
+
+**注意**：WorkloadSpread Webhook 不会为自定义工作负载所创建的 Pod 设置 deletion cost，因而无法保证自定义工作负载的缩容顺序。
+
+### 配置自定义工作负载监听白名单
+
+首先，需要将自定义工作负载加入`WorkloadSpread_Watch_Custom_Workload_WhiteList` 白名单，以使其能够被 WorkloadSpread 读取并理解。
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kruise-configuration
+  namespace: kruise-system
+data:
+  "WorkloadSpread_Watch_Custom_Workload_WhiteList": |
+    {
+      "workloads": [
+        {
+          "Group": "argoproj.io",
+          "Kind": "Rollout",
+          "replicasPath": "spec.replicas",
+        }
+      ]
+    }
+```
+
+具体的配置项说明如下：
+
+- **Group:** 自定义工作负载的 ApiGroup。
+- **Kind:** 自定义工作负载的 Kind。
+- **subResources:** 自定义工作负载的子资源，字段包括 Group 与 Kind。例如：Deployment 的 ReplicaSet。该字段为可选字段，如果自定义资源不包含子资源，那么可以留空。
+- **replicasPath:** 自定义工作负载中用于指定副本数的资源字段路径，例如：spec.replicas。
+
+### 向 kruise-manager 授予权限
+
+要在自定义工作负载上使用 WorkloadSpread，需要给 kruise-manager 服务账号授予相应资源的读取权限。
+
+```yaml
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: kruise-rollouts-access
+rules:
+  - apiGroups: [ "argoproj.io" ]
+    resources: [ "rollouts" ]
+    verbs: [ "get" ]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: kruise-rollouts-access-binding
+subjects:
+  - kind: ServiceAccount
+    name: kruise-manager
+    namespace: kruise-system
+roleRef:
+  kind: ClusterRole
+  name: kruise-rollouts-access
+  apiGroup: rbac.authorization.k8s.io
+```
+
+### 指定自定义工作负载
+
+当配置完成后，即可在 WorkloadSpread 的 `targetRef` 字段中指定自定义工作负载。
+
+```yaml
+apiVersion: apps.kruise.io/v1alpha1
+kind: WorkloadSpread
+metadata:
+  name: workloadspread-demo
+spec:
+  targetRef:
+    apiVersion: argoproj.io/v1alpha1
+    kind: Rollout
+    name: rollouts-demo
+  subsets:
+    ...
+```
+
 ## feature-gates
 WorkloadSpread 默认是关闭的，如果要开启请通过设置 feature-gates *WorkloadSpread*.
 

diff --git a/i18n/zh/docusaurus-plugin-content-docs/version-v1.7/user-manuals/workloadspread.md b/i18n/zh/docusaurus-plugin-content-docs/version-v1.7/user-manuals/workloadspread.md
@@ -203,6 +203,89 @@ WorkloadSpread所管理的workload会按照`subsets`中定义的顺序扩缩容
 # deletion order: b -> a -> c
 ```
 
+## 在自定义工作负载上使用 WorkloadSpread
+
+WorkloadSpread 默认不会监听自定义工作负载。如果想要在自定义工作负载上使用 WorkloadSpread，需要进行额外的配置。本节以 [Argo
+社区的 Rollout Workload](https://argoproj.github.io/argo-rollouts/) 为例，介绍如何将其与 WorkloadSpread 配合使用。
+
+**注意**：WorkloadSpread Webhook 不会为自定义工作负载所创建的 Pod 设置 deletion cost，因而无法保证自定义工作负载的缩容顺序。
+
+### 配置自定义工作负载监听白名单
+
+首先，需要将自定义工作负载加入`WorkloadSpread_Watch_Custom_Workload_WhiteList` 白名单，以使其能够被 WorkloadSpread 读取并理解。
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kruise-configuration
+  namespace: kruise-system
+data:
+  "WorkloadSpread_Watch_Custom_Workload_WhiteList": |
+    {
+      "workloads": [
+        {
+          "Group": "argoproj.io",
+          "Kind": "Rollout",
+          "replicasPath": "spec.replicas",
+        }
+      ]
+    }
+```
+
+具体的配置项说明如下：
+
+- **Group:** 自定义工作负载的 ApiGroup。
+- **Kind:** 自定义工作负载的 Kind。
+- **subResources:** 自定义工作负载的子资源，字段包括 Group 与 Kind。例如：Deployment 的 ReplicaSet。该字段为可选字段，如果自定义资源不包含子资源，那么可以留空。
+- **replicasPath:** 自定义工作负载中用于指定副本数的资源字段路径，例如：spec.replicas。
+
+### 向 kruise-manager 授予权限
+
+要在自定义工作负载上使用 WorkloadSpread，需要给 kruise-manager 服务账号授予相应资源的读取权限。
+
+```yaml
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: kruise-rollouts-access
+rules:
+  - apiGroups: [ "argoproj.io" ]
+    resources: [ "rollouts" ]
+    verbs: [ "get" ]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: kruise-rollouts-access-binding
+subjects:
+  - kind: ServiceAccount
+    name: kruise-manager
+    namespace: kruise-system
+roleRef:
+  kind: ClusterRole
+  name: kruise-rollouts-access
+  apiGroup: rbac.authorization.k8s.io
+```
+
+### 指定自定义工作负载
+
+当配置完成后，即可在 WorkloadSpread 的 `targetRef` 字段中指定自定义工作负载。
+
+```yaml
+apiVersion: apps.kruise.io/v1alpha1
+kind: WorkloadSpread
+metadata:
+  name: workloadspread-demo
+spec:
+  targetRef:
+    apiVersion: argoproj.io/v1alpha1
+    kind: Rollout
+    name: rollouts-demo
+  subsets:
+    ...
+```
+
 ## feature-gates
 WorkloadSpread 默认是关闭的，如果要开启请通过设置 feature-gates *WorkloadSpread*.