How to create custom Helm charts
Here we create a custom helm chart for OpenProject with data persistence and an external database.
TL;DR
Make sure you have Helm installed
helm create [chart name]- Edit files to change to a Docker image that you want to deploy
- Customize with any K8s features within the
templatesfolder - Deploy with
helm install [release name] [chart folder]
Grab the custom chart created in this article for the community version of the project management software OpenProject, on Github.
Helm 3 overview
Helm 3 is a complete departure from the previous versions of helm. There is no server side component required. Whereas before you would need to add an application called Tiller to your cluster. Dependencies are handled directly within the chart.yaml file. No more requirements.yaml file needed. Helm 3 now also supports RBAC.
Even with all these changes under the hood, and some changes to the CLI as well, Helm 2 charts are still largely compatible with Helm 3. There are API versions that must be changed and there may be other chages required, depening on the complexity of the chart, but everything on Helm Hub should be compatible.
Helm create
First we will use the helm create command to create our custom chart. I will be using OpenProject as my application of choice for this article. You can use whichever application you like, the catch is that it needs to have a Docker image hosted on a registry.
Be warned, every application has it's own nuances and requirements. If you choose a different application please make sure to read their documentation on deployment requirements.
We will be creating a helm chart for OpenProject a project management software. Let's start by using the helm create command to create the baseline of our project.
helm create openproject-helm-chart
We will see the following files are generated by the helm CLI.
├── charts
├── Chart.yaml
├── templates
│ ├── deployment.yaml
│ ├── _helpers.tpl
│ ├── hpa.yaml
│ ├── ingress.yaml
│ ├── NOTES.txt
│ ├── serviceaccount.yaml
│ ├── service.yaml
│ └── tests
│ └── test-connection.yaml
└── values.yaml
All kubernetes spec files are contained within the templates folder. They are all templated using the Go Template Engine. In this article we will mainly focus on chart.yaml, deployment.yaml, and values.yaml files.
First we will need to modify the values.yaml file to update the Docker image being used for this chart. Here we will also change the service.type to LoadBalancer so we can have external access to our OpenProject instance.
Side node: It is recommended that you keep the
service.typeasClusterIPand useingressto direct traffic to your internal instances. But that is outside that scope of this article. I will do an article in the future to deploy this properly behind an ingress.
Your values.yaml file should look like this:
# Default values for openproject-helm-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
replicaCount: 1
image:
repository: openproject/community
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: "10.6.5"
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
podAnnotations: {}
podSecurityContext: {}
# fsGroup: 2000
securityContext: {}
# capabilities:
# drop:
# - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
service:
type: LoadBalancer
port: 8080
ingress:
enabled: false
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
hosts:
- host: chart-example.local
paths: []
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 100
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity: {}
And your chart.yaml file should look like this:
apiVersion: v2
name: openproject-helm-chart
description: A Helm chart for Kubernetes
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
appVersion: 10.6.5
Shameless plug
If you like content on this blog you can support us by using the DO referral link to sign up and get free credit. And you'll support us as well.
Referral link: https://m.do.co/c/590c0c82c1fc
Also you can subscribe directly signing up for free, or support us with a small monthly or yearly fee. Simply click the button at the bottom right corner.
Adding persistent storage
Now let's add some persistent storage to our chart so we don't lose our data when we update. To do this we will need to update two and add a new one. So first we will add pvc.yaml file in the templates folder. This will define the template for a Persistent Volume Claim. It should look like this:
{{- if and .Values.persistence.enabled (not .Values.persistence.existingClaim) }}
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: {{ template "openproject-helm-chart.fullname" . }}
labels: {{- include "openproject-helm-chart.labels" . | nindent 4 }}
spec:
accessModes:
- {{ .Values.persistence.accessMode | quote }}
resources:
requests:
storage: {{ .Values.persistence.size | quote }}
{{- end }}
Now we will update deployment.yaml and values.yaml files to make use of this new template.
deployment.yaml: Notice the volumes and volumeMounts sections
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "openproject-helm-chart.fullname" . }}
labels:
{{- include "openproject-helm-chart.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
{{- include "openproject-helm-chart.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "openproject-helm-chart.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "openproject-helm-chart.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: 8080
protocol: TCP
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
resources:
{{- toYaml .Values.resources | nindent 12 }}
volumeMounts:
- mountPath: /var/openproject
name: openproject-data
volumes:
- name: openproject-data
{{- if .Values.persistence.enabled }}
persistentVolumeClaim:
claimName: {{ .Values.persistence.existingClaim | default (include "openproject-helm-chart.fullname" .) }}
{{- else }}
emptyDir: {}
{{ end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
values.yaml: Added the persistence section
# Default values for openproject-helm-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
replicaCount: 1
image:
repository: openproject/community
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: "10.6.5"
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
podAnnotations: {}
podSecurityContext: {}
# fsGroup: 2000
securityContext: {}
# capabilities:
# drop:
# - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
service:
type: LoadBalancer
port: 8080
ingress:
enabled: false
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
hosts:
- host: chart-example.local
paths: []
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 100
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity: {}
persistence:
enabled: true
accessMode: ReadWriteOnce
size: 48Gi
This will add a 48 GB volume to our OpenProject instance. It will be mounted on /var/openproject where all data is stored by OpenProject.
Adding external database
To add an external database to our OpenProject instance we will need to add a dependency to our chart. Plus update the _helpers.tpl with a helper function, update the deployment.yaml and values.yaml files to make use of the external database. First we will update chart.yaml file to add PostgreSQL as a dependency. We will be using the excellent Bitnami chart for PostgreSQL. Your chart.yaml file should look like this:
apiVersion: v2
name: openproject-helm-chart
description: A Helm chart for Kubernetes
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
appVersion: 10.6.5
dependencies:
- name: postgresql
version: "9.1.1"
repository: "https://charts.bitnami.com/bitnami"
Now we have to update the _helpers.tpl file to add the following section to the file.
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
*/}}
{{- define "openproject-helm-chart.postgresql.fullname" -}}
{{- printf "%s-%s" .Release.Name "postgresql" | trunc 63 | trimSuffix "-" -}}
{{- end -}}
Now we will update the values.yaml file to customize the some of the values provided by the PostgreSQL chart file.
# Default values for openproject-helm-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
replicaCount: 1
image:
repository: openproject/community
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: "10.6.5"
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
podAnnotations: {}
podSecurityContext: {}
# fsGroup: 2000
securityContext: {}
# capabilities:
# drop:
# - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
service:
type: LoadBalancer
port: 8080
ingress:
enabled: false
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
hosts:
- host: chart-example.local
paths: []
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 100
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity: {}
persistence:
enabled: true
accessMode: ReadWriteOnce
size: 48Gi
postgresql:
persistence:
enabled: true
size: 15G
postgresqlPassword: superSecretPassword
postgresqlDatabase: openproject
Finally it's time to update the deployment.yaml file to to make use of these values in an environment variable. We will also take this opportunity to add initialDelaySeconds and periodSeconds to our livenessProbe and readinessProbe. This will allow the application to startup before kubernetes thinks it's failed and restarts it.
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "openproject-helm-chart.fullname" . }}
labels:
{{- include "openproject-helm-chart.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
{{- include "openproject-helm-chart.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "openproject-helm-chart.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "openproject-helm-chart.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
env:
- name: DATABASE_URL
value: "postgresql://postgres:{{ .Values.postgresql.postgresqlPassword }}@{{ include "openproject-helm-chart.postgresql.fullname" . }}:5432/{{ .Values.postgresql.postgresqlDatabase }}"
ports:
- name: http
containerPort: 8080
protocol: TCP
livenessProbe:
initialDelaySeconds: 120
periodSeconds: 120
httpGet:
path: /
port: http
readinessProbe:
initialDelaySeconds: 120
periodSeconds: 120
httpGet:
path: /
port: http
resources:
{{- toYaml .Values.resources | nindent 12 }}
volumeMounts:
- mountPath: /var/openproject
name: openproject-data
volumes:
- name: openproject-data
{{- if .Values.persistence.enabled }}
persistentVolumeClaim:
claimName: {{ .Values.persistence.existingClaim | default (include "openproject-helm-chart.fullname" .) }}
{{- else }}
emptyDir: {}
{{ end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
With the above changes we now a database that is external to our application.
Deploy
With the above changes, we have a working chart for OpenProject. We can deploy the chart with a few simple steps, assuming you have your kubernetes cluster all setup:
helm dependency updatehelm install openproject ./openproject-helm-chart
Give it a few minutes and you should see the final product on the IP of the your load balancer. Make sure specify port 8080
