datashim-io · srikumar003 · Jan 23, 2024 · Jun 14, 2022 · Jun 25, 2023 · Jun 26, 2023
diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
@@ -43,7 +43,7 @@ jobs:
       - name: Create sample Dataset
         run: |
           cat <<EOF | kubectl apply -f -
-          apiVersion: com.ie.ibm.hpsys/v1alpha1
+          apiVersion: datashim.io/v1alpha1
           kind: Dataset
           metadata:
             name: example-dataset

diff --git a/.gitignore b/.gitignore
@@ -7,3 +7,9 @@ _tmp
 *s3-secret.yaml
 .vscode/*
 bin/*
+chart/Chart.lock
+chart/values.yaml.orig
+example-dummy.yaml
+example-pod.yaml
+src/dataset-operator/testing/csi_test_suite.go
+src/dataset-operator/testing/csi_test_utils.go
diff --git a/README.md b/README.md
@@ -13,6 +13,12 @@
 A Kubernetes Framework to provide easy access to S3 and NFS **Datasets** within pods. Orchestrates the provisioning of
 **Persistent Volume Claims** and **ConfigMaps** needed for each **Dataset**. Find more details in our [FAQ](https://datashim-io.github.io/datashim/FAQ/)
 
+## Alert (23 Jan 2024) - Group Name Change
+
+__If you have an existing installation of Datashim, please DO NOT follow the instructions below to upgrade it to version `0.4.0` or latest__. The group name of the Dataset and DatasetInternal CRDs (objects) is changing from `com.ie.ibm.hpsys` to `datashim.io`. An upgrade in place will invalidate your Dataset definitions and will cause problems in your installation. You can upgrade up to version `0.3.2` without any problems. 
+
+To upgrade to `0.4.0` and beyond, please a) delete all datasets safely; b) uninstall Datashim; and c) reinstall Datashim either through Helm or using the manifest file as follows.
+
 ## Quickstart
 
 First, create the namespace for installing Datashim, if not present
@@ -56,7 +62,7 @@ and populate it with data._
 We will create now a Dataset named `example-dataset` pointing to your S3 bucket.
 ```yaml
 cat <<EOF | kubectl apply -f -
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset
@@ -112,18 +118,26 @@ To install, search for the latest stable release
 ```bash
 helm search repo datashim
 ```
+which will result in:
+```
+NAME                    	CHART VERSION	APP VERSION	DESCRIPTION
+datashim/datashim-charts	0.3.2        	0.3.2      	Datashim chart
+```
+
+__Note:__Version `0.3.2` still has `com.ie.ibm.hpsys` as the apiGroup name. So, please proceed with caution. It is fine for upgrading from an existing Datashim installation but going forward the apiGroup will be `datashim.io` 
+
 
 Pass the option to create namespace, if you are installing Datashim for the first time:
 ```bash
-helm install --namespace=dlf --create-namespace datashim datashim/datashim-charts
+helm install --namespace=dlf --create-namespace datashim datashim/datashim-charts --version <version_string>
 ```
 Do not forget to label the target namespace to support pod labels, as shown in the previous section
 
 ### Uninstalling through Helm
 
 To uninstall, use `helm uninstall` like so:
 ```bash
-helm uninstall -n datashim datashim
+helm uninstall -n dlf datashim
 ```
 
 ### Installing intermediate releases
@@ -136,7 +150,7 @@ helm search repo datashim --devel
 
 To install an intermediate version, 
 ```bash
-helm install --namespace=dlf --create-namespace datashim datashim/datashim-charts --devel
+helm install --namespace=dlf --create-namespace datashim datashim/datashim-charts --devel --version <version_name>
 ```
 
 ## Questions

diff --git a/docs/Archive-based-Datasets.md b/docs/Archive-based-Datasets.md
@@ -49,7 +49,7 @@ minio-7979c89d5c-khncd              0/1     Running     0          3m
 Now we can create a Dataset based on a remote archive as follows:
 ```yaml
 cat <<EOF | kubectl apply -f -
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset

diff --git a/docs/Ceph-Caching.md b/docs/Ceph-Caching.md
@@ -86,7 +86,7 @@ dataset-operator-7b8f65f7d4-hg8n5   1/1     Running   0          6s
 ```
 Create an s3 dataset by replacing the values and invoking `kubectl create -f my-dataset.yaml`
 ``` yaml
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset

diff --git a/docs/index.md b/docs/index.md
@@ -50,7 +50,7 @@ and populate it with data._
 We will create now a Dataset named `example-dataset` pointing to your S3 bucket.
 ```yaml
 cat <<EOF | kubectl apply -f -
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset

diff --git a/docs/kubeflow/Data-Volumes-for-Notebook-Servers.md b/docs/kubeflow/Data-Volumes-for-Notebook-Servers.md
@@ -12,7 +12,7 @@ In this guide, we assume that your data are already stored in a remote s3 bucket
 Let's assume that you will launch your notebook server on the namespace `{my-namespace}`
 
 ``` yaml
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: your-dataset

diff --git a/docs/kubeflow/Model-Storing-and-Serving-with-DLF.md b/docs/kubeflow/Model-Storing-and-Serving-with-DLF.md
@@ -31,7 +31,7 @@ Otherwise follow the instructions in [Configure IBM COS Storage](https://github.
 
 Now we need to create a dataset to point to the newly created bucket. Create a file that looks like this:
 ``` yaml
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: your-dataset

diff --git a/docs/kubeflow/PVCs-for-Pipelines-SDK.md b/docs/kubeflow/PVCs-for-Pipelines-SDK.md
@@ -13,7 +13,7 @@ We will just how you can adopt the examples located in [contrib/volume_ops](http
 
 First you need to create a Dataset to point to the bucket you want to use. Create a file that looks like this:
 ``` yaml
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: your-dataset
@@ -138,7 +138,7 @@ def volume_op_dag():
 def get_dataset_yaml(name,accessKey,secretAccessKey,endpoint,bucket,region):
     print(region)
     dataset_spec = f"""
-    apiVersion: com.ie.ibm.hpsys/v1alpha1
+    apiVersion: datashim.io/v1alpha1
     kind: Dataset
     metadata:
       name: {name}

diff --git a/examples/hive/sampleapp/bookdataset.yaml b/examples/hive/sampleapp/bookdataset.yaml
@@ -1,5 +1,5 @@
 ---
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: bookds

diff --git a/examples/kubeflow/volumeop.py b/examples/kubeflow/volumeop.py
@@ -62,7 +62,7 @@ def volume_op_dag():
 def get_dataset_yaml(name,accessKey,secretAccessKey,endpoint,bucket,region):
     print(region)
     dataset_spec = f"""
-    apiVersion: com.ie.ibm.hpsys/v1alpha1
+    apiVersion: datashim.io/v1alpha1
     kind: Dataset
     metadata:
       name: {name}

diff --git a/examples/scheduling/README.md b/examples/scheduling/README.md
@@ -113,7 +113,7 @@ First lets create a dataset
 ```
 $ cat <<EOF | kubectl apply -f -
 ---
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset

diff --git a/examples/templates/example-archive.yaml b/examples/templates/example-archive.yaml
@@ -1,4 +1,4 @@
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset

diff --git a/examples/templates/example-dataset-host.yaml b/examples/templates/example-dataset-host.yaml
@@ -1,4 +1,4 @@
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset

diff --git a/examples/templates/example-dataset-nfs.yaml b/examples/templates/example-dataset-nfs.yaml
@@ -1,4 +1,4 @@
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset

diff --git a/examples/templates/example-dataset-s3-provision.yaml b/examples/templates/example-dataset-s3-provision.yaml
@@ -1,4 +1,4 @@
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset

diff --git a/examples/templates/example-dataset-s3-secrets.yaml b/examples/templates/example-dataset-s3-secrets.yaml
@@ -1,4 +1,4 @@
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset

diff --git a/examples/templates/example-dataset-s3.yaml b/examples/templates/example-dataset-s3.yaml
@@ -1,4 +1,4 @@
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datasets.datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset

diff --git a/plugins/ceph-cache-plugin/deploy/role.yaml b/plugins/ceph-cache-plugin/deploy/role.yaml
@@ -67,7 +67,7 @@ rules:
   verbs:
   - get
 - apiGroups:
-  - com.ie.ibm.hpsys
+  - datashim.io
   resources:
   - '*'
   verbs:

diff --git a/release-tools/manifests/dlf-ibm-k8s.yaml b/release-tools/manifests/dlf-ibm-k8s.yaml
@@ -160,9 +160,9 @@ metadata:
   annotations:
     controller-gen.kubebuilder.io/version: v0.8.0
   creationTimestamp: null
-  name: datasetsinternal.com.ie.ibm.hpsys
+  name: datasetsinternal.datashim.io
 spec:
-  group: com.ie.ibm.hpsys
+  group: datashim.io
   names:
     kind: DatasetInternal
     listKind: DatasetInternalList
@@ -257,9 +257,9 @@ metadata:
   annotations:
     controller-gen.kubebuilder.io/version: v0.8.0
   creationTimestamp: null
-  name: datasets.com.ie.ibm.hpsys
+  name: datasets.datashim.io
 spec:
-  group: com.ie.ibm.hpsys
+  group: datashim.io
   names:
     kind: Dataset
     listKind: DatasetList
@@ -565,7 +565,7 @@ rules:
   verbs:
   - get
 - apiGroups:
-  - com.ie.ibm.hpsys
+  - datashim.io
   resources:
   - '*'
   - datasetsinternal

diff --git a/release-tools/manifests/dlf-ibm-oc.yaml b/release-tools/manifests/dlf-ibm-oc.yaml
@@ -160,9 +160,9 @@ metadata:
   annotations:
     controller-gen.kubebuilder.io/version: v0.8.0
   creationTimestamp: null
-  name: datasetsinternal.com.ie.ibm.hpsys
+  name: datasetsinternal.datashim.io
 spec:
-  group: com.ie.ibm.hpsys
+  group: datashim.io
   names:
     kind: DatasetInternal
     listKind: DatasetInternalList
@@ -257,9 +257,9 @@ metadata:
   annotations:
     controller-gen.kubebuilder.io/version: v0.8.0
   creationTimestamp: null
-  name: datasets.com.ie.ibm.hpsys
+  name: datasets.datashim.io
 spec:
-  group: com.ie.ibm.hpsys
+  group: datashim.io
   names:
     kind: Dataset
     listKind: DatasetList
@@ -565,7 +565,7 @@ rules:
   verbs:
   - get
 - apiGroups:
-  - com.ie.ibm.hpsys
+  - datashim.io
   resources:
   - '*'
   - datasetsinternal

diff --git a/release-tools/manifests/dlf-oc.yaml b/release-tools/manifests/dlf-oc.yaml
@@ -160,9 +160,9 @@ metadata:
   annotations:
     controller-gen.kubebuilder.io/version: v0.8.0
   creationTimestamp: null
-  name: datasetsinternal.com.ie.ibm.hpsys
+  name: datasetsinternal.datashim.io
 spec:
-  group: com.ie.ibm.hpsys
+  group: datashim.io
   names:
     kind: DatasetInternal
     listKind: DatasetInternalList
@@ -257,9 +257,9 @@ metadata:
   annotations:
     controller-gen.kubebuilder.io/version: v0.8.0
   creationTimestamp: null
-  name: datasets.com.ie.ibm.hpsys
+  name: datasets.datashim.io
 spec:
-  group: com.ie.ibm.hpsys
+  group: datashim.io
   names:
     kind: Dataset
     listKind: DatasetList
@@ -565,7 +565,7 @@ rules:
   verbs:
   - get
 - apiGroups:
-  - com.ie.ibm.hpsys
+  - datashim.io
   resources:
   - '*'
   - datasetsinternal

diff --git a/release-tools/manifests/dlf.yaml b/release-tools/manifests/dlf.yaml
@@ -160,9 +160,9 @@ metadata:
   annotations:
     controller-gen.kubebuilder.io/version: v0.8.0
   creationTimestamp: null
-  name: datasetsinternal.com.ie.ibm.hpsys
+  name: datasetsinternal.datashim.io
 spec:
-  group: com.ie.ibm.hpsys
+  group: datashim.io
   names:
     kind: DatasetInternal
     listKind: DatasetInternalList
@@ -257,9 +257,9 @@ metadata:
   annotations:
     controller-gen.kubebuilder.io/version: v0.8.0
   creationTimestamp: null
-  name: datasets.com.ie.ibm.hpsys
+  name: datasets.datashim.io
 spec:
-  group: com.ie.ibm.hpsys
+  group: datashim.io
   names:
     kind: Dataset
     listKind: DatasetList
@@ -565,7 +565,7 @@ rules:
   verbs:
   - get
 - apiGroups:
-  - com.ie.ibm.hpsys
+  - datashim.io
   resources:
   - '*'
   - datasetsinternal

diff --git a/src/apiclient/pkg/clientset/versioned/typed/com/v1alpha1/com_client.go b/src/apiclient/pkg/clientset/versioned/typed/com/v1alpha1/com_client.go
diff --git a/src/apiclient/pkg/clientset/versioned/typed/com/v1alpha1/fake/fake_dataset.go b/src/apiclient/pkg/clientset/versioned/typed/com/v1alpha1/fake/fake_dataset.go
diff --git a/src/apiclient/pkg/clientset/versioned/typed/com/v1alpha1/fake/fake_datasetinternal.go b/src/apiclient/pkg/clientset/versioned/typed/com/v1alpha1/fake/fake_datasetinternal.go
diff --git a/src/dataset-operator/README.md b/src/dataset-operator/README.md
@@ -7,7 +7,7 @@ to a k8s cluster and allows you to create objects like the example
 in [dataset_cr](deploy/crds/com_v1alpha1_dataset_cr.yaml)
 
 ```
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset
@@ -47,7 +47,7 @@ dataset-operator-644f8d854-dct95   1/1     Running   0          15s
 
 Now you can do `kubectl create -f dataset.yaml` with a yaml which looks like this:
 ```
-apiVersion: com.ie.ibm.hpsys/v1alpha1
+apiVersion: datashim.io/v1alpha1
 kind: Dataset
 metadata:
   name: example-dataset