Production ready, custom Kubernetes (k8s) cluster in minutes
A production ready, highly available, gossip based kubernetes cluster.
Using tools such as kops, kubctl, cloud provider's CLI, a production ready Kubernetes cluster is created on the cloud. It is then enhanced with some basic tools like a dashboard, and an autoscaler. All of this is done through YAML configuration files.
Some notes about the environment this tutorial was created on. Fedora Linux version 32 was used to write and create this tutorial. The tools mentioned here are available for most platforms out there; although the exact commands and steps may vary between platforms.
It is beneficial to have some prior knowledge of the basic architecture of Kubernetes. However, it is not a hard requirement for this article. Describing Kubernetes in detail is outside the scope of this article; for those that want to know more can head over to Kubernetes docs for details.
This cluster was created with three masters and fours nodes. All of these were placed in different availability zones to ensure high availability and redundancy. The diagram below details the cluster that was created here.
kops: is a tool for creating, and managing k8s clusters. It provides all the necessary packages and configuration to make an operational cluster.
kubctl: is main tool used for interacting with any k8s cluster. It is an easy tool to manage all applications, services, and resources that are deployed on a k8s cluster.
awscli/glcoud/doctl: these tools provide access to their respective cloud providers. They are required for the initial set-up of authentication and authorization. This article will focus on awscli
- Install tools
- kubectl: On Fedora 32, the kubectl is a simple command to install
sudo dnf install kubernetes-client. For other platforms please follow the instructions at https://kubernetes.io/docs/tasks/tools/install-kubectl/
- kops: Will need to downloaded from the project's Github release page https://github.com/kubernetes/kops/releases. On Fedora32, download the latest stable release and place the binary in the
~/.local/binto automatically make it available in the shell.
- awscli: On Fedora 32, awscli can be installed again with a simple command
sudo dnf install awscli. For other platforms, the project's Github release page can be used to download the tool https://github.com/aws/aws-cli/releases. CLI tools for platforms can be downloaded from their respective project pages
2. setup cloud cli
- Follow this guide to setup
awsclilocally. Provide credentials as promted
- TL;DR: create admin user in AWS IAM, generate key and secret, run
aws configure, enter all required information, and make sure
~/.aws/configfiles are created
- optionaly provide environemnt variables documented here
3. create cluster
1. Create an S3 bucket to store the state of the cluster.
aws s3api create-bucket --bucket cluster-com-state-store
2. Create the cluster
kops create cluter cluster.k8s.local \ --node-count=3 \ --zones="us-east-1a,us-east-1b,us-east-1c,us-east-1d" \ --node-size="t3a.large" \ --master-zones="us-east-1a,us-east-1b,us-east-1c" \ --master-size="t3a-small" \ --yes \ --state=s3://cluster-com-state-store
The above command will create a highly available, gossip based cluster.
The above clusters should create a production ready, highly available cluster. Verify the cluster the configuration by running
kops validate cluster --state=s3://cluster-com-state-store
The resulting cluster should look like the following
Next we will be deploying applications to this cluster using Helm