docker

Production ready, custom Kubernetes (k8s) cluster in minutes

A production ready, highly available, gossip based kubernetes cluster.

Mohamed Baig

22 Jun 2020 — 3 min read

Summary

Using tools such as kops, kubctl, cloud provider's CLI, a production ready Kubernetes cluster is created on the cloud. It is then enhanced with some basic tools like a dashboard, and an autoscaler. All of this is done through YAML configuration files.

Details

Some notes about the environment this tutorial was created on. Fedora Linux version 32 was used to write and create this tutorial. The tools mentioned here are available for most platforms out there; although the exact commands and steps may vary between platforms.

It is beneficial to have some prior knowledge of the basic architecture of Kubernetes. However, it is not a hard requirement for this article. Describing Kubernetes in detail is outside the scope of this article; for those that want to know more can head over to Kubernetes docs for details.

This cluster was created with three masters and fours nodes. All of these were placed in different availability zones to ensure high availability and redundancy. The diagram below details the cluster that was created here.

Tools

kops: is a tool for creating, and managing k8s clusters. It provides all the necessary packages and configuration to make an operational cluster.

kubctl: is main tool used for interacting with any k8s cluster. It is an easy tool to manage all applications, services, and resources that are deployed on a k8s cluster.

awscli/glcoud/doctl: these tools provide access to their respective cloud providers. They are required for the initial set-up of authentication and authorization. This article will focus on awscli

Process

Install tools

kubectl: On Fedora 32, the kubectl is a simple command to install sudo dnf install kubernetes-client. For other platforms please follow the instructions at https://kubernetes.io/docs/tasks/tools/install-kubectl/
kops: Will need to downloaded from the project's Github release page https://github.com/kubernetes/kops/releases. On Fedora32, download the latest stable release and place the binary in the ~/.local/bin to automatically make it available in the shell.
awscli: On Fedora 32, awscli can be installed again with a simple command sudo dnf install awscli. For other platforms, the project's Github release page can be used to download the tool https://github.com/aws/aws-cli/releases. CLI tools for platforms can be downloaded from their respective project pages

2. setup cloud cli

Follow this guide to setup awscli locally. Provide credentials as promted
TL;DR: create admin user in AWS IAM, generate key and secret, run aws configure, enter all required information, and make sure ~/.aws/credentials and ~/.aws/config files are created
optionaly provide environemnt variables documented here

3. create cluster

1. Create an S3 bucket to store the state of the cluster. aws s3api create-bucket --bucket cluster-com-state-store

2. Create the cluster

kops create cluter cluster.k8s.local \
--node-count=3 \
--zones="us-east-1a,us-east-1b,us-east-1c,us-east-1d" \
--node-size="t3a.large" \
--master-zones="us-east-1a,us-east-1b,us-east-1c" \
--master-size="t3a-small" \
--yes \
--state=s3://cluster-com-state-store

The above command will create a highly available, gossip based cluster.