Provide different OpenShift and non-OpenShift watches yaml files

[jmazzitelli]

fixes: kiali/kiali#6790

[jmazzitelli]

To test on OpenShift, you need this PR and the helm PR

Set up

1. Get an OpenShift cluster and install Istio
   hack/crc-openshift.sh start && hack/istio/install-istio-via-istioctl.sh -c oc
2. Build and push the OSSMC plugin from your local plugin repo (your fork of https://github.com/kiali/openshift-servicemesh-plugin)
   cd <plugin repo> && make cluster-push
3. Build and push the Kiali server and operator from your local kiali dev setup
   cd <kiali repo> && make build build-ui cluster-push

Test kiali-ossm OLM bundle

4. Test the "kial-ossm" OLM bundle (this is what the Red Hat Catalog will have) - install the Operator, Kiali, and OSSMC to ensure all CRDs are installed and the operator has the correct watches yaml
   a. Install Kiali: make -e OC=$(which oc) -e OLM_BUNDLE_PACKAGE=kiali-ossm olm-operator-create kiali-create
   b. Wait for Kiali to start: oc wait pod --for condition=Ready -n istio-system -l app=kiali
   c. Install OSSMC: make ossmconsole-create
5. Confirm OSSMC is running via oc wait pod --for condition=Ready -n ossmconsole -l app=ossmconsole
6. You can log into the OpenShift Console and see that you get the OSSMC plugin functionality (it will take a few minutes for the OpenShift Console to load it in, so be patient).
7. This will return two CRDs - this is just sanity checking you have the Kiali CRD and OSSMConsole CRD:
   oc get crds | grep kiali
8. Confirm the operator is using the new watches-os.yaml file - it is an argument that is passed to the container:

$ oc get pods --all-namespaces -l app=kiali-operator -o jsonpath={..spec.containers..args} | grep watches-os.yaml
["--zap-log-level=info","--leader-election-id=kiali-operator","--watches-file=./watches-os.yaml"]

9. Uninstall everything: make olm-operator-delete

Test kiali-community OLM bundle

10. Test the "kial-community" OLM bundle (this is what the Community Catalog will have) - install the Operator, Kiali, and OSSMC to ensure all CRDs are installed and the operator has the correct watches yaml
    a. Install Kiali: make -e OC=$(which oc) -e OLM_BUNDLE_PACKAGE=kiali olm-operator-create kiali-create
    b. Wait for Kiali to start: oc wait pod --for condition=Ready -n istio-system -l app=kiali
    c. Install OSSMC: make ossmconsole-create
11. Confirm OSSMC is running via oc wait pod --for condition=Ready -n ossmconsole -l app=ossmconsole
12. You can log into the OpenShift Console and see that you get the OSSMC plugin functionality (it will take a few minutes for the OpenShift Console to load it in, so be patient).
13. This will return two CRDs - this is just sanity checking you have the Kiali CRD and OSSMConsole CRD:
    oc get crds | grep kiali
14. Confirm the operator is using the new watches-os.yaml file - it is an argument that is passed to the container:

$ oc get pods --all-namespaces -l app=kiali-operator -o jsonpath={..spec.containers..args} | grep watches-os.yaml
["--zap-log-level=info","--leader-election-id=kiali-operator","--watches-file=./watches-os.yaml"]

15. Uninstall everything: make olm-operator-delete

Test operator helm chart

16. Build the helm charts from your local helm repo
    cd <kiali helm-charts repo> && make clean build-helm-charts
17. Install the operator via helm (the make target uses helm under the covers)
    cd <kiali repo> && make operator-create
18. Confirm the operator is using the new watches-os.yaml file - it is an argument that is passed to the container:

$ oc get pods --all-namespaces -l app=kiali-operator -o jsonpath={..spec.containers..args} | grep watches-os.yaml
["--zap-log-level=info","--leader-election-id=kiali-operator","--watches-file=./watches-os.yaml"]

19. Create Kiali and then OSSMC
    a. make kiali-create
    c. Wait for Kiali to start: oc wait pod --for condition=Ready -n istio-system -l app=kiali
    d. Install OSSMC: make ossmconsole-create
20. Confirm OSSMC is running via oc wait pod --for condition=Ready -n ossmconsole -l app=ossmconsole
21. You can log into the OpenShift Console and see that you get the OSSMC plugin functionality (it will take a few minutes for the OpenShift Console to load it in, so be patient).
22. This will return two CRDs - this is just sanity checking you have the Kiali CRD and OSSMConsole CRD:
    oc get crds | grep kiali
23. Uninstall everything: make operator-delete

[jmazzitelli]

To test on non-OpenShift Kubernetes, you need this PR and the helm PR

Set up

1. Get a minikube cluster and install Istio
   hack/k8s-minikube.sh start && hack/istio/install-istio-via-istioctl.sh -c kubectl
2. Build and push the Kiali server and operator from your local kiali dev setup
   cd <kiali repo> && make -e CLUSTER_TYPE=minikube build build-ui cluster-push

Test kiali-upstream OLM bundle

3. Test the "kial-upstream" OLM bundle (this is what the OperatorHub.io Catalog will have) - install the Operator, Kiali, and ensure only the Kiali CRD is installed and the operator has the correct watches yaml
   a. Install OLM, the operator, and Kiali (you must use kubectl client here!): make -e CLUSTER_TYPE=minikube -e OC=$(which kubectl) -e OLM_BUNDLE_PACKAGE=kiali olm-install olm-operator-create kiali-create
   b. Wait for Kiali to start: kubectl wait pod --for condition=Ready -n istio-system -l app=kiali
4. This will return one CRD - you should only have the Kiali CRD and NOT have the OSSMConsole CRD:
   kubectl get crds | grep kiali
5. Confirm the operator is using the new watches-k8s.yaml file - it is an argument that is passed to the container:

$ kubectl get pods --all-namespaces -l app=kiali-operator -o jsonpath={..spec.containers..args} | grep watches-k8s.yaml
["--zap-log-level=info","--leader-election-id=kiali-operator","--watches-file=./watches-k8s.yaml"]

6. Uninstall everything: make -e CLUSTER_TYPE=minikube olm-operator-delete

Test operator helm chart

7. Build the helm charts from your local helm repo
   cd <kiali helm-charts repo> && make clean build-helm-charts
8. Install the operator via helm (the make target uses helm under the covers)
   cd <kiali repo> && make -e CLUSTER_TYPE=minikube operator-create
9. Confirm the operator is using the new watches-k8s.yaml file - it is an argument that is passed to the container:

$ kubectl get pods --all-namespaces -l app=kiali-operator -o jsonpath={..spec.containers..args} | grep watches-k8s.yaml
["--zap-log-level=info","--leader-election-id=kiali-operator","--watches-file=./watches-k8s.yaml"]

10. Create Kiali
    a. make -e CLUSTER_TYPE=minikube kiali-create
    c. Wait for Kiali to start: kubectl wait pod --for condition=Ready -n istio-system -l app=kiali
11. This will return one CRD - you should only have the Kiali CRD and NOT have the OSSMConsole CRD:
    kubectl get crds | grep kiali
12. Uninstall everything: make -e CLUSTER_TYPE=minikube operator-delete

[leandroberetta]

Looks good, tested all the configurations.