JFrog-Cloud-Installers/Openshift4/helm/openshift-pipelines/README.md

JFrog Pipelines on Kubernetes Helm Chart

JFrog Pipelines

Prerequisites Details

• Kubernetes 1.12+

Chart Details

This chart will do the following:

• Deploy PostgreSQL (optionally with an external PostgreSQL instance)
• Deploy RabbitMQ (optionally as an HA cluster)
• Deploy Redis (optionally as an HA cluster)
• Deploy Vault (optionally as an HA cluster)
• Deploy JFrog Pipelines

Requirements

• A running Kubernetes cluster
  • Dynamic storage provisioning enabled
  • Default StorageClass set to allow services using the default StorageClass for persistent storage
• A running Artifactory 7.7.x with Enterprise+ License
  • Precreated repository jfrogpipelines in Artifactory type Generic with layout maven-2-default
• Kubectl installed and setup to use the cluster
• Helm v2 or v3 installed

Install JFrog Pipelines

Add ChartCenter Helm repository

Before installing JFrog helm charts, you need to add the ChartCenter helm repository to your helm client

helm repo add center https://repo.chartcenter.io
helm repo update

Artifactory Connection Details

In order to connect Pipelines to your Artifactory installation, you have to use a Join Key, hence it is MANDATORY to provide a Join Key and Jfrog Url to your Pipelines installation. Here's how you do that:

Retrieve the connection details of your Artifactory installation, from the UI - https://www.jfrog.com/confluence/display/JFROG/General+Security+Settings#GeneralSecuritySettings-ViewingtheJoinKey.

Install Pipelines Chart with Ingress

Pre-requisites

Before deploying Pipelines you need to have the following

• A running Kubernetes cluster
• An Artifactory or Artifactory HA with Enterprise+ License
  • Precreated repository jfrogpipelines in Artifactiry type Generic with layout maven-2-default
• Deployed Nginx-ingress controller
• [Optional] Deployed Cert-manager for automatic management of TLS certificates with Lets Encrypt
• [Optional] TLS secret needed for https access

Prepare configurations

Fetch the JFrog Pipelines helm chart to get the needed configuration files

helm fetch center/jfrog/pipelines --untar

Edit local copies of values-ingress.yaml, values-ingress-passwords.yaml and values-ingress-external-secret.yaml with the needed configuration values

• URLs in values-ingress.yaml
  • Artifactory URL
  • Ingress hosts
  • Ingress tls secrets
• Passwords uiUserPassword, postgresqlPassword and rabbitmq.password must be set, and same for masterKey and joinKey in values-ingress-passwords.yaml

Install JFrog Pipelines

Install JFrog Pipelines

kubectl create ns pipelines
helm upgrade --install pipelines --namespace pipelines center/jfrog/pipelines -f pipelines/values-ingress.yaml -f pipelines/values-ingress-passwords.yaml

Use external secret

Note: Best practice is to use external secrets instead of storing passwords in values.yaml files.

Don't forget to update URLs in values-ingress-external-secret.yaml file.

Fill in all required passwords, masterKey and joinKey in values-ingress-passwords.yaml and then create and install the external secret.

Note: Helm release name for secrets generation and helm install must be set the same, in this case it is pipelines.

With Helm v2:

## Generate pipelines-system-yaml secret
helm template --name-template pipelines pipelines/ -x templates/pipelines-system-yaml.yaml \
    -f pipelines/values-ingress-external-secret.yaml -f pipelines/values-ingress-passwords.yaml | kubectl apply --namespace pipelines -f -

## Generate pipelines-database secret
helm template --name-template pipelines pipelines/ -x templates/database-secret.yaml \
    -f pipelines/values-ingress-passwords.yaml | kubectl apply --namespace pipelines -f -

## Generate pipelines-rabbitmq-secret secret
helm template --name-template pipelines pipelines/ -x templates/rabbitmq-secret.yaml \
    -f pipelines/values-ingress-passwords.yaml | kubectl apply --namespace pipelines -f -

With Helm v3:

## Generate pipelines-system-yaml secret
helm template --name-template pipelines pipelines/ -s templates/pipelines-system-yaml.yaml \
    -f pipelines/values-ingress-external-secret.yaml -f pipelines/values-ingress-passwords.yaml | kubectl apply --namespace pipelines -f -

## Generate pipelines-database secret
helm template --name-template pipelines pipelines/ -s templates/database-secret.yaml \
    -f pipelines/values-ingress-passwords.yaml | kubectl apply --namespace pipelines -f -

## Generate pipelines-rabbitmq-secret secret
helm template --name-template pipelines pipelines/ -s templates/rabbitmq-secret.yaml \
    -f pipelines/values-ingress-passwords.yaml | kubectl apply --namespace pipelines -f -

Install JFrog Pipelines:

helm upgrade --install pipelines --namespace pipelines center/jfrog/pipelines -f values-ingress-external-secret.yaml

Using external Rabbitmq

If you want to use external Rabbitmq, set rabbitmq.enabled=false and create values-external-rabbitmq.yaml with below yaml configuration

rabbitmq:
  enabled: false
  internal_ip: "{{ .Release.Name }}-rabbitmq"
  msg_hostname: "{{ .Release.Name }}-rabbitmq"
  port: 5672
  manager_port: 15672
  ms_username: admin
  ms_password: password
  cp_username: admin
  cp_password: password
  build_username: admin
  build_password: password    
  root_vhost_exchange_name: rootvhost
  erlang_cookie: secretcookie
  build_vhost_name: pipelines
  root_vhost_name: pipelinesRoot
  protocol: amqp

helm upgrade --install pipelines --namespace pipelines center/jfrog/pipelines -f values-external-rabbitmq.yaml

Using external Vault

If you want to use external Vault, set vault.enabled=false and create values-external-vault.yaml with below yaml configuration

vault:
  enabled: false

global:
  vault:
    host: vault_url
    port: vault_port
    token: vault_token
    ## Set Vault token using existing secret
    # existingSecret: vault-secret

If you store external Vault token in a pre-existing Kubernetes Secret, you can specify it via existingSecret.

To create a secret containing the Vault token:

kubectl create secret generic vault-secret --from-literal=token=${VAULT_TOKEN}

helm upgrade --install pipelines --namespace pipelines center/jfrog/pipelines -f values-external-vault.yaml

Status

See the status of deployed helm release:

With Helm v2:

helm status pipelines

With Helm v3:

helm status pipelines --namespace pipelines

Pipelines Version

• By default, the pipelines images will use the value appVersion in the Chart.yml. This can be over-ridden by adding version to the pipelines section of the values.yml

Build Plane

Build Plane with static and dynamic node-pool VMs

To start using Pipelines you need to setup a Build Plane:

• For Static VMs Node-pool setup, please read Managing Node Pools.

• For Dynamic VMs Node-pool setup, please read Managing Dynamic Node Pools.

• For Kubernetes Node-pool setup, please read Managing Dynamic Node Pools.

Useful links

• https://www.jfrog.com/confluence/display/JFROG/Pipelines+Quickstart
• https://www.jfrog.com/confluence/display/JFROG/Using+Pipelines
• https://www.jfrog.com/confluence/display/JFROG/Managing+Runtimes