Installation Process

Roles in the installation process

Red Hat OpenShift cluster administrator

The cluster administrator is responsible for:

• Creating projects (namespaces)

• Creating roles, role bindings and service-accounts

• Installing Custom Resource (CR) definitions

• Configuring the OpenShift image-registry

Project administrator

The project administrator is responsible for:

• Installing Solution Hub and Designer

• Installing Envoys on prepared namespaces

• Providing necessary configuration data

• Pushing images to the image-registry

• Executing Helm after installation

Before you begin

In order to install IBM Financial Services Workbench the following requirements should be met on the machine from where the installer is executed:

• You are logged-in into the OpenShift cluster as a user with sufficient rights for the task at hand

$ oc login

• A local docker daemon is installed and running

$ systemctl start docker

• tar utility and bash are available and the installer package was downloaded and unpacked

$ tar xvzf ssob-2.3.0-ibm-openshift-4.3-cpd-installations.tgz

• Your current working directory is set to the directory ssob-install of the unpacked installer package

$ cd ssob-install 

Step 1: Setup cluster-wide resources

Introduction

Cluster-wide resources are setup via executing the script ssob_admin_setup.sh , that is part of the installer. It must be executed as a user with cluster-admin permissions in the OpenShift cluster (Role: Red Hat OpenShift cluster administrator). It can be configured via command line arguments, that are described later on.

Description

The ssob_admin_setup.sh will create and configure the following:

• Create Custom Resource definition

  • solutions.sol.rt.cp.knowis.de

  • envoys.env.rt.cp.knowis.de

  • k5clients.k5.project.operator

  • k5dashboards.k5.project.operator

  • k5pipelinemanagers.k5.project.operator

  • k5projects.k5.project.operator

  • k5realms.k5.project.operator

  • k5topics.k5.project.operator

• Create instances of Custom Resource ConsoleYAMLSample

• Optionally: creates a route for the OpenShift docker registry

• Create Roles

  • k5-aggregate-edit (provides edit access to the created Custom Resource Definitions above and aggregates it to the cluster roles admin and edit)

  • k5-aggregate-view (provides view access to the created Custom Resource Definitions above and aggregates it to the cluster role view)

  • cpd-admin-additional-role (provides admin access to routes and pods/portforward)

• Create ServiceAccounts

  • k5-operator-sa

• Creates role bindings (associates permissions to service-accounts in the namespace)

  • Service Account k5-operator-sa: cpd-admin-role, cpd-viewer-role, edit

  • Service Account cpd-viewer-sa: view

  • Service Account cpd-admin-sa-rolebinding: cpd-admin-additional-role

Parameters

The ssob_admin_setup.sh script has the following parameters:

$ oc -n openshift-image-registry get routes

Example

$ cd deployments
$ chmod +x ssob_admin_setup.sh
$ ./ssob_admin_setup.sh  \
  --accept_license  \
  --cpd_namespace=zen \
  --external_address_image_registry=image-registry.apps.{your.cluster.domain}

Step 2: Installation of IBM Financial Services Workbench

Introduction

The installation of IBM Financial Services Workbench is done by executing the script ssob_install.sh . It required to be executed as a user with OpenShift project admin permissions in the chosen OpenShift namespace, i.e. zen . The installation script requires a set of command line arguments.

Description

The installer will configure and install the following components via Helm charts into the already existing namespace of CPD:

• solution-designer

• solution-hub

Without any further configuration the components will be reusing the existing cpd service-accounts respectively the created ones.

The installer will create all default secrets, that are necessary for an envoy project. It creates all secrets containing sensitive data (needed for the deployments and jobs) not via Helm chart in order to avoid breach of secrecy.

Each package contains all used docker images and Helm charts. During the installation all provided and used docker images are pushed into the configured registry. For convenience it should be the internal OpenShift registry.

Also after the installation of each Helm chart the Helm tests will be executed. The installation will fail, if one or more Helm tests are failing. Per default the Helm test is enabled and the Helm tests pods will be removed automatically. These steps can be skipped. In case of failing Helm tests, the pod description and if available the pod logs will be printed.

The installation script ( ssob_install.sh ) creates the following required k8s secrets (containing all needed sensitive data) using the oc cli and the install_init_data.yaml file:

• default mongodb binding for solution-envoy

• default kafka binding for solution-envoy

• default iam binding for solution-envoy

• mongodb binding for the solution-designer

• token master key for encryption of git tokens stored in the solution-designer

• optional access credentials for the s3 storage binding of the solution-designer (local marketplace)

• secret containing the admin credentials for the identity server

Parametrization

The installer is parametrized and adapted to your particular environment via CLI parameters and 3 parameter or value files.

There are 3 different types of parameters:

• Those that you will always have to adapt to your environment. They are given as CLI parameters to the ssob_install.sh script.

• Those that contain sensitive data and need to be handled with care. These are configured via the values file install_init_data.yaml

• Those that are likely to be used in their default form. These are configured via the yaml files solution-hub-values.yaml and solution-designer-values.yaml

You can find these files in the ssob-install/deployments/configs directory of the unpacked installation package.

Customize the configuration files

$ scp user@remote_host:remote_file ~/remote_directory 
$ scp root@11.11.11.11:/root/install/ssob_2.4.0/ssob-install/deployments/configs/install_init_data.yaml ~/install

$ scp local_file user@remote_host:remote_file
$ scp install_init_data.yaml root@11.11.11.11:/root/install/ssob_2.4.0/ssob-install/deployments/configs/install_init_data.yaml

$ cd ~/install/ssob_2.4.0/ssob-install/deployments
$ mv configs configs.bak
$ mkdir configs
$ cp ./configs.bak/* configs

Parameters

The ssob_install.sh script has the following parameters:

$ oc get services -n openshift-image-registry

$ oc get secret helm-secret -n zen -o jsonpath='{.data.ca\.cert\.pem}' | base64 -d > /root/install/ssob_2.4.0/zenhelm/ca.cert.pem
$ oc get secret helm-secret -n zen -o jsonpath='{.data.helm\.cert\.pem}' | base64 -d > /root/install/ssob_2.4.0/zenhelm/helm.cert.pem
$ oc get secret helm-secret -n zen -o jsonpath='{.data.helm\.key\.pem}' | base64 -d > /root/install/ssob_2.4.0/zenhelm/helm.key.pem    

$ oc -n "openshift-image-registry" get route image-registry    

$ oc -n "openshift-image-registry" get services    

Configuration

YAML file install_init_data_file.yaml

The following configuration yaml file can be found in the directory ../ssob-install/deployments/configs and needs to be adjusted:

$ oc -n "foundation" get services   

YAML file solution-hub-values.yaml

The following configuration yaml file can be found in the directory ../ssob-install/deployments/resources and needs to be adjusted:

trustMap:
  identity: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
  mycert: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----

$ echo | openssl s_client -showcerts -connect #KEYCLOAK#DOMAIN#:443 2>/dev/null

YAML file solution-designer-values.yaml

The following configuration yaml file can be found in the directory ../ssob-install/deployments/configs and needs to be adjusted:

apiVersion: v1
kind: Secret
type: Opaque
data:
  key: BASE64_ENCODED_ACCESSKEY

apiVersion: v1
kind: Secret
type: Opaque
data:
  accesskey: BASE64_ENCODED_ACCESSKEY
  secretkey: BASE64_ENCODED_SECRETKEY

apiVersion: v1
kind: Secret
type: Opaque
data:
  accesskey: BASE64_ENCODED_ACCESSKEY
  secretkey: BASE64_ENCODED_SECRETKEY

apiVersion: v1
kind: Secret
type: Opaque
data:
  accesskey: BASE64_ENCODED_ACCESSKEY
  secretkey: BASE64_ENCODED_SECRETKEY

cd deployments
chmod +x ssob_install.sh
./ssob_install.sh \
  --accept_license \
  --external_address_image_registry=image-registry.apps.{your.cluster.domain} \
  --internal_address_image_registry=image-registry.openshift-image-registry.svc:5000 \
  --cpd_namespace=zen \
  --identity_provider_host=https://keycloak.apps.{your.cluster.domain} \
  --host_domain=apps.{your.cluster.domain} \
  --helm-tls-ca-cert=/root/.helm/ca.pem \
  --helm-tls-cert=/root/.helm/cert.pem \
  --helm-tls-key=/root/.helm/key.pem

Validate the installation

To validate the results of the previous installation step do the following:

1. Check the deployment versions by running this command and make sure that the expected versions of Solution Designer and Solution Hub are deployed as listed in section Deployment Versions:

   $ helm ls --tls --tiller-namespace zen \
   --tls-cert=/root/.helm/cert.pem \
   --tls-key=/root/.helm/key.pem \
   | grep release-ssob-solution
   NAME                            STATUS    CHART                     APP VERSION
   release-ssob-solution-designer  DEPLOYED  k5-designer-bundle-2.9.5  2.9.5      
   release-ssob-solution-hub       DEPLOYED  ssob-solution-hub-2.3.3   2.3.3      

2. Check the deployments by running this command and compare your output to the output provided here

   $ oc get deployments -n zen
   NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
   auto-devops-mvn-dependencies   2/2     2            2           1d
   configuration-management       2/2     2            2           1d
   k5-cli-provider                3/3     3            3           1d
   k5-code-generation-provider    3/3     3            3           1d
   k5-designer-backend            3/3     3            3           1d
   k5-designer-frontend           3/3     3            3           1d
   k5-git-integration-controller  3/3     3            3           1d
   k5-hub-backend                 3/3     3            3           1d
   k5-hub-frontend                3/3     3            3           1d
   k5-plantuml-server             2/2     2            2           1d
   k5-query                       2/2     2            2           1d
   k5-s3-storage                  1/1     1            1           1d
   k5-solution-controller         2/2     2            2           1d
   k5projectoperator              1/1     1            1           1d
   tiller-deploy                  1/1     1            1           1d

3. Open the Solution Designer Web application in your browser
   1. Retrieve the URL to point your browser to:

      $ oc -n zen get routes | grep k5-designer-frontend-service
      k5-designer-frontend-route ssob-sc.apps.{your.cluster.domain} / k5-designer-frontend-service https reencrypt/Redirect None

   2. Point your browser to the retrieved URL, e.g https://ssob-sc.apps.{your.cluster.domain}/. As a result you should see a login form.

Troubleshooting

Docker daemon

For above command to work you need a running docker daemon on the host you are running the command on (your bastion host). If you get an error message like the following

$ Warning: failed to get default registry endpoint from daemon (Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?). Using system default: https://index.docker.io/v1/
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Start docker like this (system dependent):

$ systemctl start docker
$ systemctl enable docker

Docker and self-signed certificates

If your container registry uses self signed certificates you might see something like the following:

Extracting modules
checking structure
structure check skip for now
docker login https://image-registry.apps.{your.cluster.domain}
Error response from daemon: Get https://image-registry.apps.{your.cluster.domain}/v1/users/: x509: certificate signed by unknown authority

To fix the issues with self signed certificates, please follow the instructions on: https://docs.docker.com/registry/insecure/ and perform the following steps:

• retrieve the CA Cert from the image registry

• add it to the local trust store

• reload the trusted certificates

$ oc -n openshift-ingress get secret router-certs-default -o jsonpath='{.data.tls\.crt}' | base64 -d  > /etc/pki/ca-trust/source/anchors/image-registry.apps.{your.cluster.domain}.crt
update-ca-trust
$ systemctl restart docker  
$ docker login -u admin -p {password} https://image-registry.{your.cluster.domain}

In case of failing k5-s3-storage with that log

ERROR Unable to initialize backend: Insufficient permissions to access path
> Please ensure the specified path can be accessed

And if the storage class is nfs

> oc get pvc k5-s3-storage -o yaml | grep "storageClassName"
  storageClassName: nfs

the permissions must be repaired.

To do so, please follow these stops

1. Get the user id of the k5 s3 storage

> oc get pod  | grep k5-s3-storage
k5-s3-storage-775db6d5fc-sjlqn                                  1/1     Running     0          6h6m
> oc get pod k5-s3-storage-775db6d5fc-sjlqn -o yaml | grep runAsUser
      runAsUser: 1000320900

2. Login to your nfs mount

3. Check the current permissions

> ls -la *
insgesamt 336
drwxr-xr-x. 2 1001 root   4096 17. Jun 19:27 .
drwxrwxrwx. 4 root root     53 23. Jun 13:46 ..
-rw-r--r--. 1 1001 root    462 17. Jun 19:22 c48a5afe-3d1b-4c18-ad61-2dd9c5d078eb-meta.yml
-rw-r--r--. 1 1001 root   1269 17. Jun 19:22 c48a5afe-3d1b-4c18-ad61-2dd9c5d078eb-preview.json
-rw-r--r--. 1 1001 root 144790 17. Jun 19:22 c48a5afe-3d1b-4c18-ad61-2dd9c5d078eb.zip
-rw-r--r--. 1 1001 root    960 17. Jun 19:27 index.yml

4. Change owner

> chown -R 1000320900:1000320900 .

5. Check result

> ls -la *
insgesamt 336
drwxr-xr-x. 2 1000320900 1000320900   4096 17. Jun 19:27 .
drwxrwxrwx. 4 1000320900 1000320900     53 23. Jun 13:48 ..
-rw-r--r--. 1 1000320900 1000320900    462 17. Jun 19:22 c48a5afe-3d1b-4c18-ad61-2dd9c5d078eb-meta.yml
-rw-r--r--. 1 1000320900 1000320900   1269 17. Jun 19:22 c48a5afe-3d1b-4c18-ad61-2dd9c5d078eb-preview.json
-rw-r--r--. 1 1000320900 1000320900 144790 17. Jun 19:22 c48a5afe-3d1b-4c18-ad61-2dd9c5d078eb.zip
-rw-r--r--. 1 1000320900 1000320900    960 17. Jun 19:27 index.yml

6. Change permission

chmod g+rwx . -R

7. Check result

> ls -la *
insgesamt 336
drwxrwxr-x. 2 1000320900 1000320900   4096 17. Jun 19:27 .
drwxrwxrwx. 4 1000320900 1000320900     53 23. Jun 13:48 ..
-rw-rwxr--. 1 1000320900 1000320900    462 17. Jun 19:22 c48a5afe-3d1b-4c18-ad61-2dd9c5d078eb-meta.yml
-rw-rwxr--. 1 1000320900 1000320900   1269 17. Jun 19:22 c48a5afe-3d1b-4c18-ad61-2dd9c5d078eb-preview.json
-rw-rwxr--. 1 1000320900 1000320900 144790 17. Jun 19:22 c48a5afe-3d1b-4c18-ad61-2dd9c5d078eb.zip
-rw-rwxr--. 1 1000320900 1000320900    960 17. Jun 19:27 index.yml