Kubeflow

There are few ways you can leverage DLF to assist you while working with kubeflow.

Requirements

You have permissions to install DLF and a namespace where you can use for kubeflow (to create TFJobs, launch Jupyter notebooks etc) Lets assume the namespace you can use is {my-namespace}. Feel free to change accordingly.

Installation using kubectl

git clone https://github.com/IBM/dataset-lifecycle-framework.git
cd dataset-lifecycle-framework
git checkout fixed-caching #TODO remove when branch merged
make DATASET_OPERATOR_NAMESPACE={my-namespace} NAMESPACES_TO_MONITOR={my-namespace} deployment

If everything worked well you should see this:

NAME                               READY   STATUS              RESTARTS   AGE
csi-attacher-nfsplugin-0           2/2     Running             0          78s
csi-attacher-s3-0                  1/1     Running             0          78s
csi-nodeplugin-nfsplugin-j4ljv     2/2     Running             0          78s
csi-provisioner-s3-0               1/1     Running             0          78s
csi-s3-2gwcs                       2/2     Running             0          79s
dataset-operator-76f795587-cljfm   1/1     Running             0          77s
generate-keys-q8s99                0/1     Completed           0          67s

We will loosely follow the example posted in mnist_vanilla_k8s.ipynb