kubeadm

kubeadm Quick Notes

What kubeadm owns

kubeadm generates certificates, static-Pod manifests, and kubeconfig files, then keeps their version in a ConfigMap (kubeadm-config in kube-system). It does not install a CNI plugin, a StorageClass, or an Ingress controller. Those are addons you apply after init.

The kubelet is a host service (systemd), not a static Pod. kubeadm installs a drop-in so the kubelet loads /var/lib/kubelet/config.yaml.

Before init

On every node:

• Container runtime installed and enabled (containerd). SystemdCgroup = true in /etc/containerd/config.toml matches the kubelet’s cgroup driver. Then systemctl restart containerd.
• Swap off: swapoff -a and remove the swap line from /etc/fstab. The kubelet refuses to start while swap is on unless you explicitly relax that.
• Kernel: overlay and br_netfilter loaded, and these sysctls set to 1:

net.ipv4.ip_forward
net.bridge.bridge-nf-call-iptables
net.bridge.bridge-nf-call-ip6tables

• Unique node identity: hostname, MAC address, and /sys/class/dmi/id/product_uuid must differ per VM. Cloned images that share a product UUID confuse the cloud provider and node registration.
• Ports: control plane needs 6443 (API), 2379–2380 (etcd), 10250 (kubelet), 10259 (scheduler), 10257 (controller-manager). Workers need 10250 and the NodePort range if you use it. CNI may add its own (VXLAN UDP 4789 is common).

Install matching packages. The kubelet version should be the version you intend to run:

kubeadm version
kubelet --version
kubectl version --client

init

On the first control plane:

kubeadm init \
  --apiserver-advertise-address=10.0.0.10 \
  --pod-network-cidr=192.168.0.0/16 \
  --control-plane-endpoint=cp.lab:6443 \
  --kubernetes-version=v1.35.0

init prints the next two commands. Keep them.

Admin kubeconfig for the user you will work as:

mkdir -p $HOME/.kube
sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get nodes

Recent kubeadm also writes /etc/kubernetes/super-admin.conf. That file’s client certificate is in system:masters and bypasses RBAC. admin.conf is the one to copy for normal administration. Leave super-admin.conf on the control plane for break-glass.

Install a CNI manifest whose Pod CIDR matches --pod-network-cidr before you expect Ready. Then:

kubectl get nodes

A single-node lab that must run workloads on the control plane removes the taint:

kubectl taint nodes --all node-role.kubernetes.io/control-plane-

Do that only when there are no workers. An HA control plane should keep the taint.

Files init leaves behind

/etc/kubernetes/manifests/*.yaml static Pods: apiserver, etcd, scheduler, controller-manager
/etc/kubernetes/pki/ cluster CA and friends
/etc/kubernetes/admin.conf admin kubeconfig
/etc/kubernetes/super-admin.conf system:masters kubeconfig
/etc/kubernetes/kubelet.conf kubelet’s own kubeconfig
/var/lib/kubelet/config.yaml kubelet configuration

The apiserver manifest’s command: list is where advertise address, service CIDR, and feature gates live. Editing the file restarts the static Pod. If the API is down afterward, on that node run crictl ps -a and crictl logs <container-id>. A YAML error or an unknown flag is in that log and in journalctl -u kubelet. Move the manifest out of /etc/kubernetes/manifests and back to force another restart.

Join a worker

From kubeadm init output, or regenerate:

kubeadm token create --print-join-command

On the worker:

kubeadm join cp.lab:6443 \
  --token <token> \
  --discovery-token-ca-cert-hash sha256:<hash>

• Token is short-lived (24 hours). kubeadm token list shows expiry. Create a new one rather than editing bootstrap files by hand.
• The hash pins the CA the worker expects. A wrong hash means you are joining a different cluster or the CA was rotated. Copy it from the kubeadm init join command or from kubeadm token create --print-join-command. You can recompute it from the CA:

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt \
  | openssl rsa -pubin -outform der 2>/dev/null \
  | openssl dgst -sha256 -hex | sed 's/^.* //'

Prefix the hex with sha256: in the join command.

• Discovery via a file is the alternative: --discovery-file=kubeconfig that already contains the cluster CA.

After join, kubectl get nodes on the control plane shows the worker NotReady until the CNI DaemonSet lands on it, then Ready.

Join another control plane

The init flag --upload-certs (or a later kubeadm init phase upload-certs --upload-certs) prints a certificate key. It lets the new control plane download the shared PKI. The key expires in two hours.

kubeadm join cp.lab:6443 \
  --token <token> \
  --discovery-token-ca-cert-hash sha256:<hash> \
  --control-plane \
  --certificate-key <key>

--control-plane is the difference between a worker and a second etcd member plus apiserver. Confirm with kubectl get nodes (role control-plane) and kubectl get pods -n kube-system (a local etcd-* and kube-apiserver-*).

Upgrade

One minor at a time. Snapshot etcd first:

sudo ETCDCTL_API=3 etcdctl snapshot save /opt/etcd-backup.db \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key

snapshot restore writes a new directory (--data-dir=/var/lib/etcd-restore). Point the etcd static Pod manifest at that path. Do not restore over a live data directory.

On the first control plane:

apt-get update && apt-get install -y kubeadm=<new-version>
kubeadm upgrade plan
kubeadm upgrade apply v1.35.0
apt-get install -y kubelet=<new-version> kubectl=<new-version>
systemctl restart kubelet

On each extra control plane, then on each worker:

kubectl drain <node> --ignore-daemonsets --delete-emptydir-data
apt-get install -y kubeadm=<new-version>
kubeadm upgrade node
apt-get install -y kubelet=<new-version> kubectl=<new-version>
systemctl restart kubelet
kubectl uncordon <node>

upgrade apply runs only on the first control plane (it updates the cluster-wide ConfigMap). Every other node uses upgrade node. Package names look like 1.35.0-1.1 on Debian-based images. apt-cache madison kubelet shows the exact string.

Certificates

kubeadm certs check-expiration
kubeadm certs renew all

Renewal rewrites PEMs under /etc/kubernetes/pki and regenerates the embedded admin kubeconfigs. Restart the control-plane static Pods so they load the new server certificates (edit-free approach: kubectl delete pod -n kube-system -l component=kube-apiserver, or move the manifest out of the directory and back). Copy a fresh admin.conf to ~/.kube/config after renew if kubectl starts failing TLS.

Client certificates you issued through the Certificates API are not renewed by kubeadm certs renew. Re-issue those CSRs.

A cluster that will not come up

Official docs

• Installing kubeadm
• Creating a cluster with kubeadm
• Upgrading kubeadm clusters
• Certificate management