Kubernetes has been a bit of a buzzword lately. I haven't had much exposure to it, focusing my energy on VMware products over the last few years. Today, we'll cover the Kubernetes install process on my go to Linux distribution of choice, CentOS 7.
Step -1: Why?
I like to practice exercises to get a better understanding. A lot of the documentation surrounding Kubernetes, like most technologies, covers the "what" but not the "why". To add some color to "why containers", we'll solve a problem that I would have for personal use: A Plex media server.
It used to be that Plex had to be installed on Windows or Linux bare metal. Virtualization helped by allowing a VM to be right sized for this purpose, and freed up other resources for other machines. Containers simplified this even more; instead of having to install a VM from scratch or from a template, update the OS, and then install the application, I can deploy Plex with a single command. Kubernetes takes this one step further, allowing for management and orchestration, similar to a command-line only version of vCenter managing ESXi hosts.
Step 0: Prerequisites
For this exercise, I'm going to be using an ESXi 7.0 host with an NFS datastore mounted. Any type of datastore will do, we just need a place to put VMDKs.
We'll be creating 3 virtual machines for this, one master node and two worker nodes. The minimum requirements for each VM is as follows:
- 2 vCPUs for the master node, 1 for each worker node (official requirements are 1.5 and 0.7)
- 2GB of memory for the master, 1GB for workers
- I'm using 100GB of storage each but could probably get away with less
When in the installation environment, be sure to select "Virtualization Host" option. This should install docker automagically.
I'll include the step when the time comes in the guide, but a big callout that should be made is that installing Kubernetes requires disabling SELinux. I initially wasn't comfortable with this, if I have an issue caused by SELinux, journalctl usually gives me the reason why and the steps to fix it; however, kubeadm notes indicate that SELinux must be disabled so that containers can access the filesystem. I don't plan on putting anything else on these VMs, and neither should you.
While this isn't an official requirement, I would strongly recommend configuring NTP. Kubernetes requires the nodes to maintain the same time. NTP is a great way to maintain consistent time between VMs without much manual intervention.
After installing CentOS 7, run the following commands on all three VMs:
Install ntp and ntpdate (ntp sync)
yum install ntp ntpdate
Start the service
systemctl start ntpd
Enable the service
systemctl enable ntpd
Configure NTP servers
ntpdate -u -s 0.centos.pool.ntp.org 1.centos.pool.ntp.org 2.centos.pool.ntp.org
Restart the service
systemctl restart ntpd
Check time, ensure that it's the same on all three VMs
timedatectl
Set the hardware clock to match current system time
Step 1: Configure Kubernetes Repository
Kubernetes packages are not on the official CentOS 7 repo, so we'll add the repo. Skipping this will cause step 2 to fail. Feel free to use the text editor of your choice to add the repository. In this tutorial, I'll be using vi. Be sure to carry out the following on all of the nodes.
vi /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Save and quit.
Step 2: Install required packages
Install the following packages on each node as a super user:
yum install -y kubelet kubeadm kubectl
Enable kubelet on each node:
systemctl enable kubelet
And finally, start kubelet on each node:
systemctl start kubelet
Step 3: Set hostname
Record the IP addresses of each VM (we'll need this later to update DNS records)
ip addr
Choose one of the VMs to be the master node, and run the following command:
hostnamectl set-hostname master-node
Choose which VMs will be worker-node1 and worker-node2. Run this on the first:
hostnamectl set-hostname worker-node1
And this on the second:
hostnamectl set-hostname worker-node2
Once complete, be sure to edit /etc/hosts to reflect these changes (using the IPs recorded previously):
vi /etc/hosts
192.168.0.10 master-node
192.168.0.11 worker-node1
192.168.0.12 worker-node2
Save and quit.
Step 4: Configure firewall-cmd
We'll need to open up some ports to allow the nodes to communicate with one another.
Run the following on the master node:
firewall-cmd --permanent --add-port=6443/tcp
firewall-cmd --permanent --add-port=2379-2380/tcp
firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10251/tcp
firewall-cmd --permanent --add-port=10252/tcp
firewall-cmd --permanent --add-port=10255/tcp
firewall-cmd --reload
On each of the worker nodes:
firewall-cmd --permanent --add-port=6783/tcp
firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10255/tcp
firewall-cmd --permanent --add-port=30000-32767/tcp
firewall-cmd --reload
Step 5: Update Kubernetes firewall configuration
Kubernetes uses iptables by default. To ensure that our firewalld rules work, we'll need to modify the sysctl.conf file. Run this command on each node:
echo 'net.bridge.bridge-nf-call-iptables=1' | sudo tee -a /etc/sysctl.conf
This should persist through reboots. If for some reason you only wish to have it valid for this session, run:
echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
Step 6: Disable SELinux
As mentioned in Step 0 above, we need to disable SELinux (in other words, set it to permissive mode) so containers can access the host filesystem. Run the following on each node:
setenforce 0
sed -i ‘s/^SELINUX=enforcing$/SELINUX=permissive/’ /etc/selinux/config
Step 7: Disable swap
We'll need to disable swap on each node:
swapoff -a
Step 8: Deploy the cluster
Use the following command to initialize the cluster:
kubeadm init
This will take a few minutes, and should generate a token that we'll need to copy for future use, similar to:
--token i5tx56.q5q6tx37m9j2acea --discovery-token-ca-cert-hash sha256:2a0c84ce92c6185009941730aac1955e7a445d5115b768c5955dab356e645fd5
Before joining the worker nodes, we need to install a pod network, which will allow nodes to communicate. There are several options to choose from, for this we'll use weave.
Create a regular expression that copies your current kubectl version:
export kubever=$(kubectl version | base64 | tr -d '\n')
Then run this command to apply the weave network:
kubectl apply -f “https://cloud.weave.works/k8s/net?k8s-version=$kubever"
Once complete, join the worker nodes by using the following command on each (using the token documented above as an example:
kubeadm join 192.168.0.10:6443 --token i5tx56.q5q6tx37m9j2acea --discovery-token-ca-cert-hash sha256:2a0c84ce92c6185009941730aac1955e7a445d5115b768c5955dab356e645fd5
Once complete, you should be able to check the status of the cluster. Run this on the master node:
kubectl get nodes
Finally, we need to enable a user to start the cluster. I ran all of the previous commands as root, and used:
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
If you're using a sudo enabled user, run:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
And that should do it! In the next blog post, I'll be checking out a project that goes back to the example used previously... can we get a Plex transcode running on a Kubernetes cluster?
