Installation Process

Detailed information how to install IBM Financial Services Workbench

Preparation

Ensure that all system requirements are met.

For executing the installation itself following requirements should be met on the machine from where the installer is executed:

  • Logged in into the OpenShift cluster as cluster-admin

  • A local docker daemon is installed and running

  • Bash and tar utility are available

  • The installer package (e.g. ssob-*-ibm-openshift-4.1-cpd-installations.tgz) was downloaded and unpacked

For the installation of FSW you will need a Keycloak admin with the privileges to create realms and to configure them. In general those included jobs can also be omitted by configuration. So no realm or clients for the components would be created while the installation.

Furthermore, the user for the messagehub (kafka) needs the privileges to create topics and to create events and listen to them.

Step 1: Setup cluster-wide resources

Introduction

Cluster-wide resources are setup via executing the script ssob_admin_setup.sh, that is part of the installer. It must be executed as a user with cluster-admin permissions in the OpenShift cluster. It can be configured via commandline arguments, that are described later on.

Description

The ssob_admin_setup.sh will create and configure the following:

  • An extra OpenShift project (referred to as envoy namespace) for the installation of envoy components
  • Add permission to view CustomResourceDefinitions in the envoy namespace
  • Create a CustomResourceDefinition for built solutions (solutions.sol.rt.cp.knowis.de)
  • Add permission to push images into the envoy project to the builder service account of the cpd namespace
  • Optionally: creates a route for the OpenShift docker registry
  • Add permissions to access the crd for built solutions (solutions.sol.rt.cp.knowis.de) to the service accounts in the cpd namespace and envoy project
  • Configure the envoy project as tethered namespace to cpd, meaning it associates permissions of the envoy project to the serviceaccounts available in the cpd namespace

Parameters

The ssob_admin_setup.sh script has the following parameters:

Variable Description Example value
--accept_license Set flag to confirm that the license aggreement has been read and accepted (license files can be found in the installation package folder ./ssob-install/deployments/LICENSES)
--cpdlite_namespace Name of the cpd-lite project in the OpenShift cluster cpd
--tethered_namespace Name of the envoy project in the OpenShift cluster cpd-runtime-default
--external_address_image_registry External address of the internal docker registry for creating a route (without namespace), if not set the route will not be created image-registry.apps.openshift-cluster.mydomain.cloud
--image_registry_namespace Namespace of internal docker registry openshift-image-registry (default value)
--image_registry_service Service of internal docker registry image-registry (default value)

Additional Notes

The route for the OpenShift image registry will only be created if the parameter --external_image_registry is set.

If you want to create an additional envoy, the admin script must also be executed for the additional envoy (see --tethered_namespace)

Example

./ssob_admin_setup.sh --accept_license --cpdlite_namespace=cpd --tethered_namespace=cpd-runtime-default

Step 2: Manually create secrets

The secret containing the access data to the s3 storage service is currently not created by the installer.

The secret containing data for the personal access token of a root user in GitLab (gitlab-admin-token) is currently not created by the installer.

Therefore those two secrets must be provided manually.

  1. Log in to the OpenShift cluster and choose the appropriate namespace, where cpd is installed

    Example for login to the cluster and chossing the namespace cpd:

    oc login https://openshift-cluster.mydomain.cloud:6443 --token=ABCDEFGHIJKLMN
oc project cpd

  2. Create a secret named s3storage-secret and fill it with the appropriate values for accessing the s3 storage, that is used for the marketplace. Typically a minio installation is used for this, which is included in the GitLab installation.

    The secret must contain the following information:

    Key Description
    accesskey The accesskey for accessing the s3 storage endpoint
    secretkey The secretkey for accessing the s3 storage endpoint

    For a minio service provided by GitLab the values can be found in a yet exisiting secret, that is located in the namespace where GitLab was installed. By default the secret ends with -minio-secret.

    Example for creating a secret containing the s3 storage access data

    oc create secret generic "s3storage-secret" --type="Opaque" \
--from-literal="accesskey"="ICnh47lt3YmwSmy801Dfobi9Wf82oK6gHxJzA9LbxgiFcAW5pSPPcJPjxiy8YylN" \
--from-literal="secretkey"="BGh6SkSi35egd7mkMxKj#QFB6UnpQyyXw5remhMLwwrwiiQVVmIgKzchc3CuEY5xL"

  3. Create a secret named gitlab-admin-token and fill it with the appropriate values for accessing GitLab with admin permissions. Typically a personal access token for the user root is used for this.

    The secret must contain the following information:

    Key Description
    name The name of the token
    token The value of the personal access token

    A personal access token in GitLab can be created via the web interface. Please login as administrative user and open the personal settings of the user (right upper corner). In the menu on the left click on Access Tokens. Create a token by filling out the form and granting the scopes api and sudo. After submitting the form, the token will be deisplayed.

    Example for creating a secret containing the GitLab admin token

    oc create secret generic "gitlab-admin-token" --type="Opaque" \
--from-literal="name"="admin-token" \
--from-literal="token"="31ajxZJzwhxtoqRGThyV"

Step 3: Installation of IBM Financial Services Workbench

Introduction

The installer provides three packages for installing three components:

  • solution-designer-*.tar.gz
  • solution-hub-*.tar.gz
  • solution-envoy-*.tar.gz

The installation itself is done via executing the script ssob_install.sh. It can be executed as a user with admin permissions in the touched OpenShift namespaces. It can be configured via commandline arguments, that are described later on.

Description

The installer will configure and install the following componentes via helm charts into the already existing namespace of cpd:

  • solution-designer
  • solution-hub

Without any further configuration the components will be reusing the existing cpd service-accounts.

The installer will create and configure the envoy project and install a new solution-envoy via helm chart. It creates all secrets containing sensitive data (needed for the deployments and jobs).

Each package contains all used docker images and helm charts. During the installation all provided and used docker images are pushed into the configured registry. For convenience it should be the internal OpenShift registry.

The installation script (ssob_install.sh) creates the following needed k8s secrets (containing all needed sensitive data) using the oc cli and the install_init_data.yaml file:

  • mongodb binding for the solution-envoy
  • kafka binding for the solution-envoy
  • mongodb binding for the solution-designer
  • tls secrets for all components
  • secret containing the admin credentials for the identity server

The installer contains also an additional package (config-solution-pipeline-gitlab-*.tar.gz). This package should be used for configuring the GitLab pipeline. Its an example and should be enabled via parameter (see --include_config_gitlab_pipeline). It contains suitable default configuration values and can be adjusted to the needs. This package creates:

  • Keycloak client for the solution pipeline to call the configuration management
  • Needed GitLab ci variables for executing the solution pipeline
  • Pipeline definition of the solution pipeline
  • Adjust gitaly timeouts

Parameters

The ssob_install.sh script has the following parameters:

Variable Description Example
--accept_license Set flag to confirm that the license aggreement has been read and accepted (license files can be found in the installation package folder ./ssob-install/deployments/LICENSES)
--external_address_image_registry Docker registry for pushing the images into the used registry (normally the external available address) (without namespace) image-registry.apps.openshift-cluster.mydomain.cloud
--internal_address_image_registry Docker registry for pulling images in OpenShift (usually the internal docker registry) image-registry.openshift-image-registry.svc:5000
--cpdlite_namespace Name of the cpd-lite project in the OpenShift cluster cpd
--tethered_namespace Name of the envoy project in the OpenShift cluster cpd-runtime-default
--identity_provider_host URL of the identity provider https://identity.apps.openshift-cluster.mydomain.cloud
--host_domain Hostname of the OpenShift cluster apps.openshift-cluster.mydomain.cloud
--only_solution_envoy Optional: install only a new solution-envoy (default is to install all 3 components at once)
--only_solution_designer Optional: install only a new solution-designer (default is to install all 3 components at once)
--only_solution_hub Optional: install only a new solution-hub (default is to install all 3 components at once)
--include_config_gitlab_pipeline Recommended: include and install the configuration package for the solution pipeline in GitLab
--skip_docker_login Optional: skip the docker login (can be used if you are already logged in)
--skip_helm_tls Optional: disable tls for helm requests
--helm-tls-ca-cert Optional: path to the TLS CA certificate file (default "$HELM_HOME/ca.pem") for helm commands /path/to/my/ca.cert.pem
--helm-tls-cert Optional: path to TLS certificate file (default "$HELM_HOME/cert.pem") for helm commands /path/to/my/cert.pem
--helm-tls-key Optional: path to TLS key file (default "$HELM_HOME/key.pem") for helm commands /path/to/my/helm.key.pem
--start_docker_daemon Optional: start the docker daemon via the script

Configurations

Note: The installer replaces values that are surrounded by # with appropriate values, e.g. registry: "#DOCKER_REGISTRY#/#NAMESPACE#" will be resolved to registry: "image-registry.openshift-image-registry.svc:5000/cpd".

install_init_data_file

Note: This file contains settings for creating secrets with sensitive data (credentials, bindings, tls certificates and keys)

The following configuration yaml file can be found in the directory ../ssob-install/deployments/ing-poc and needs to be adjusted:

Variable Description Example
global.identity.adminUser Username of a Keycloak admin admin
global.identity.adminPassword Password of a Keycloak admin
global.ingress.tls.crt Ingress tls certificate, typically that one that is used as the default router certificate in OpenShift. This is most commonly a wildcard certificate. 
-----BEGIN CERTIFICATE-----
MIIC+jCCAmOgAwIBAgIJAParOnPwEkKjMA0GCSqGSIb3DQEBBQUAMIGKMQswCQYD
...
tSHcLfefKeidq6NDBJ8DhWHi/zvC9YbT0KkCToEgvCTBpRZgdSFxTJcUksqoFA==
-----END CERTIFICATE-----
global.ingress.tls.key Ingress tls key, typically that one that is used as key for the default router certificate in OpenShift 
-----BEGIN PRIVATE KEY-----
MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAJgo2C8xtpSjesms
...
nA23T7y+dfnJNg==
-----END PRIVATE KEY-----
global.mongodb.dt.connectionString Mongo database connection string, that will be used for the Solution Designer mongodb://admin:password@mongodb.foundation.svc.cluster.local:27017/admin?ssl=false
global.mongodb.rt.connectionString Mongo database connection string, that will be used for the Solution Envoy mongodb://admin:password@mongodb.foundation.svc.cluster.local:27017/admin?ssl=false
global.messagehub.brokersSasl MessageHub bootstrap servers adress array [\"kafka-cluster-kafka-bootstrap.foundation.svc.cluster.local:9093\"]
global.messagehub.user Username for the MessageHub service user kafka-user
global.messagehub.password Password of the user for the MessageHub service sercret123

solution-hub-values.yaml

Note: This file contains settings for helm deployment of the Solution Hub. It contains settings for the configuration-management, solution-controller and the GitLab integration.

The following configuration yaml file can be found in the directory ../ssob-install/deployments/ing-poc and needs to be adjusted:

Variable Description Example
global.identity.realm Used realm in the identity provider ssob (default value)
global.hosts.codeGenerationProvider.name Hostname for the coden generation provider (without protocol) ssob-code-gen.apps.openshift-cluster.mydomain.cloud (default value)
global.hosts.cliProvider.name Hostname for the cli provider (without protocol) ssob-cli-provider.apps.openshift-cluster.mydomain.cloud (default value)
global.hosts.configurationManagement.name Hostname for the configuration management (without protocol) ssob-config.apps.openshift-cluster.mydomain.cloud (default value)
global.hosts.envoyController.name Hostname for the envoy controller (without protocol) ssob-envoy-controller.apps.openshift-cluster.mydomain.cloud (default value)
global.hosts.solutionController.name Hostname for the solution controller (without protocol) ssob-sdo.apps.openshift-cluster.mydomain.cloud (default value)
global.truststore.trustMap List of certificates that is used for the proper communication over SSL. The list should include all certificates in PEM ASCII format for the communication with
  • the identity server
  • the storage service of the marketplace
  • the git repository
 
trustMap:
  identity: |
    -----BEGIN CERTIFICATE-----
    MIIC+jCCAmOgAwIBAgIJAParOnPwEkKjMA0GCSqGSIb3DQEBBQUAMIGKMQswCQYD
    ...
    tJBcLfefKeidq6NDBJ8DhWHi/zvC9YbT0KkCToEgvCTBpRZgdSFxTJcUksqoFA==
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    MIIE9TCCA92gAwIBAgIETA6MOTANBgkqhkiG9w0BAQUFADCBtDEUMBIGA1UEChML
    ...
    wfsm5p9GJKaxB825DOgNghYAHZaS/KYIoA==
    -----END CERTIFICATE-----
  minio: |
    -----BEGIN CERTIFICATE-----
    MIIC+jCCAmOgAwIBAgIJAParOnPwEkKjMA0GCSqGSIb3DQEBBQUAMIGKMQswCQYD
    ...
    tSHcLfefKeidq6NDBJ8DhWHi/zvC9YbT0KkCToEgvCTBpRZgdSFxTJcUksqoFA==
    -----END CERTIFICATE-----
solution-controller.gitLabApi.baseUrl Url of the GitLab service (the internal service address could be used as well) https://gitlab.apps.openshift-cluster.mydomain.cloud
solution-controller.marketplace.storage.endpoint Endpoint to a s3 storage service (minio). Minio is typically provided by GitLab https://minio.apps.openshift-cluster.mydomain.cloud
solution-controller.marketplace.storage.secretName Name of a secret containing the credetials to access the s3 storage service. It must contain the keys accesskey and secretkey. It must be located in the same namespace as the Solution Hub. The secret was created manually in step 2. s3storage-secret
solution-controller.gitlabApi.adminTokenSecret Name of a secret containing the credetials to access the GitLab service as an admin. It must contain the keys name and token for a root user in gitlab. It must be located in the same namespace as the Solution Hub. The secret was created manually in step 2. gitlab-admin-token

solution-designer-values.yaml

Note: This file contains settings for helm deployment of the Solution Designer

The following configuration yaml file can be found in the directory ../ssob-install/deployments/ing-poc and needs to be adjusted:

Variable Description Example
global.environment.host Hostname for the solution designer (without protocol) ssob-sc.#DOMAIN# (default value)
global.identity.realm Used realm in the identity provider. It must correspond to the URL that was specified at global.identity.realm in the file solution-hub-values.yaml. ssob (default value)
cp-dt-backend.mongoDb.dbName Name of the Mongo database ssob-sc (default value)
cp-dt-backend.solutionController.url Url to the solution controller. It must correspond to the URL that was specified at global.hosts.solutionController.name in the file solution-hub-values.yaml. https://ssob-sdo.#DOMAIN# (default value)
cp-dt-backend.solutionController.internalUrl Cluster internal URL to the solution controller service https://solution-controller.#NAMESPACE#.svc.cluster.local (default value)
cp-dt-backend.codeGenerationProvider.url URL to the code generation provider service. It must correspond to the URL that was specified at global.hosts.codeGenerationProvider.name in the file solution-hub-values.yaml. https://ssob-code-gen.#DOMAIN# (default value)
cp-dt-backend.cliProvider.url URL to the code generation provider service. It must correspond to the URL that was specified at global.hosts.cliProvider.name in the file solution-hub-values.yaml. https://ssob-cli-provider.#DOMAIN# (default value)
global.truststore.trustMap.identity List of certificates that is used for the proper communication over SSL. The list is represented as a single string containing multiple certificates in PEM ASCII format and covers communication with
  • the identity server
  • the git repository
  • the mongo database
  • the Solution Hub (if not called via the internal URL)
 
trustMap:
  caCerts: |
    -----BEGIN CERTIFICATE-----
    MIIC+jCCAmOgAwIBAgIJAParOnPwEkKjMA0GCSqGSIb3DQEBBQUAMIGKMQswCQYD
    ...
    tJBcLfefKeidq6NDBJ8DhWHi/zvC9YbT0KkCToEgvCTBpRZgdSFxTJcUksqoFA==
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    MIIE9TCCA92gAwIBAgIETA6MOTANBgkqhkiG9w0BAQUFADCBtDEUMBIGA1UEChML
    ...
    wfsm5p9GJKaxB825DOgNghYAHZaS/KYIoA==
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    MIIEKjCCAxKgAwIBAgIEOGPe+DANBgkqhkiG9w0BAQUFADCBtDEUMBIGA1UEChML
    ...
    rSHcLfefKeidq6NDBJ8DhWHi/zvC9YbT0LOCToEgvCTBpRZgdSFxTJcUksqoFB==
    -----END CERTIFICATE-----

solution-envoy-values.yaml

Note: This file contains settings for helm deployment of the Solution Envoy. It contains general settings for running built solutions.

The following configuration yaml file can be found in the directory ../ssob-install/deployments/ing-poc and needs to be adjusted:

Variable Description Example
global.edition Edition of the solution envoy either dev or prod in regards to the licensing dev (default value)
global.environment.host Hostname for the solution envoy (without protocol). It will form the url of the envoy and will also be used as base url for all deployed solutions. dev-default.#DOMAIN# (default value), would lead to a url like https://dev-default.apps.openshift-cluster.mydomain.cloud
global.identity.realm Used realm in the identity provider ssob (default value)
global.messagehub.secretName Name of a secret containing the binding (access credentials) for the messagehub (kafka) binding-#NAMESPACE#-message-service (default value)
global.messagehub.saslMechanism Authentication mechanism for the usage with the messagehub (kafka) SCRAM-SHA-512 (default value) or PLAIN
global.messagehub.saslJaasConfigLoginModule Login module for the authentication mechanism for the usage with the messagehub (kafka). Must be in sync with the authentication mechanism given at global.messagehub.saslMechanism org.apache.kafka.common.security.scram.ScramLoginModule (default value) or org.apache.kafka.common.security.plain.PlainLoginModule
dashboard.feature.debugServer Enable or disable the debug server api in the dashboard (possible values are true or false) false (default value)
dashboard.feature.mvnDeps Enable the maven dependencies service for downloading needed sdk for local development for java custom solutions (possible values are true or false) false (default value)
debugserver.install Enable installation of the debug-server (recommended only for dev and test environments) (possible values are true or false) false (default value)
runtime-infrastructure.envoy.type Type of the envoy (possible values are DEV, TEST or PROD) DEV (default value)
runtime-infrastructure.envoy.stage Stage of the envoy where its deployed (can be any String) Envoy Dev (default value)
global.truststore.trustMap List of certificates that is used for the proper communication over SSL. The list should include all certificates in PEM ASCII format for the communication with
  • the identity server
  • the mongo database
  • the messagehub service (kafka)
  • any other systems that are called from solution implementations
 
trustMap:
  identity: |
    -----BEGIN CERTIFICATE-----
    MIIC+jCCAmOgAwIBAgIJAParOnPwEkKjMA0GCSqGSIb3DQEBBQUAMIGKMQswCQYD
    ...
    tJBcLfefKeidq6NDBJ8DhWHi/zvC9YbT0KkCToEgvCTBpRZgdSFxTJcUksqoFA==
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    MIIE9TCCA92gAwIBAgIETA6MOTANBgkqhkiG9w0BAQUFADCBtDEUMBIGA1UEChML
    ...
    wfsm5p9GJKaxB825DOgNghYAHZaS/KYIoA==
    -----END CERTIFICATE-----
  messagehub: |
    -----BEGIN CERTIFICATE-----
    MIIC+gJwCDOgAwIBAgIJAParOnPwEkKjMA0GCSqGSIb3DQEBBQUAMIGKMQswCQYD
    ...
    mSHcLfefKeidq6NDBJ8DhWHi/zvC9YbT0KkCToEgvCTBpRZgdSFxTJcUasdzaG==
    -----END CERTIFICATE-----

cfg-gitlab-values.yaml

Note: This file contains settings for the configuration of the solution pipeline in GitLab. GitLab needs to be installed already. Adjustment of this file is only needed if the the parameter --include_config_gitlab_pipeline was set.

The following configuration yaml file can be found in the directory ../ssob-install/deployments/ing-poc and needs to be adjusted:

Variable Description Example
global.identity.realm Used realm in the identity provider. It should correspond to the realm that was specified at global.identity.realm in the file solution-hub-values.yaml. ssob (default value)
global.hosts.gitlab.name Hostname of GitLab server (without protocol) gitlab.{{ .Values.global.hosts.domain }} (default value)
global.hosts.configurationManagement.name Hostname of the configuration management (without protocol). It must correspond to the URL that was specified at global.hosts.solutionController.name in the file solution-hub-values.yaml. ssob-config.{{ .Values.global.hosts.domain }} (default value)
Note: The previous listings are describing only a minimum set of variables. There are a lot of more configuration possibilities available in the helm charts directly. Please find the relevant documentation directly at this place, but also keep in mind, that these possibilities might change in future.

Additional Notes

The installer can also be used for only adding an additional solution-envoy.

GitLab should also be integrated with Keycloak to use the same user base. The preferred realm is the realm (by default ssob), that was created during the installation. In general the user mapping is done by comparing the claim username of the user's JWT with the username of the GitLab user.

Certificates for services that are called with a cluster based url (e.g. myservice.namespace.svc.cluster.local) may be omitted in truststores, if the services are using certificates issued by the internal cluster ca.

The certificates in the truststore should always contain the complete certificate chain.

To fully comply with the license terms it is required that the CPD Control Plane is installed in each tethered project as well. This is not required in POC situations.

Example

./ssob_install.sh \
--external_address_image_registry=image-registry.apps.openshift-cluster.mydomain.cloud \
--internal_address_image_registry=image-registry.openshift-image-registry.svc:5000 \
--cpdlite_namespace=cpd \
--tethered_namespace=cpd-runtime-default \
--identity_provider_host=https://identity.apps.openshift-cluster.mydomain.cloud  \
--host_domain=apps.openshift-cluster.mydomain.cloud \
--include_config_gitlab_pipeline
Note: Please note that the addresses of the internal docker-registry differ from OpenShift 3.11 to OpenShift 4.1 or higher. In OpenShift 3.11 the address is docker-registry.default.svc:5000 whereas in OpenShift 4.1 or higher it points to image-registry.openshift-image-registry.svc:5000.

Step 4: Post-Install Configuration

After the installation was successfull and all components are up and running. The following configurations have to be done before the product is ready to use:

FAQs

  1. Q. Where is ssob_install.sh (release package is ssob-cpd-installations.tgz)?

    Ans. Once you have extracted release package, it can be found under ssob-cpd-installations/ssob-install/deployments/ssob_install.sh

  2. Q. Pushing images to docker registry failed, where to see error logs?

    Ans. Error logs of installutil can be accessed under: /tmp/wdp.install.log

  3. Q. How can I add additional envoys (Stages)?

    Ans. The installation of a new additional envoy happens again with the two scripts:

    • Setup cluster-wide resources (ssob_admin_setup.sh)

    • Install IBM Financial Services Workbench (ssob_install.sh)

    First adjust the following values:

    • the script parameters for ssob_admin_setup.sh (e.g. --tethered_namespace)

    • in the yaml files (install_init_data.yaml and solution-envoy-values.yaml)

    • the script parameters for ssob_install.sh (e.g. --tethered_namespace)

    • add the parameter --envoy_only for executing the ssob_install.sh.