使用kubeadm部署一套高可用k8s集群

时间:2022-01-24 23:59:55

使用kubeadm部署一套高可用k8s集群

有疑问的地方可以看官方文档

准备环境

我的机器如下, 系统为ubuntu20.04, kubernetes版本1.21.0

hostname IP 硬件配置(最低)
ha 10.0.0.10 1核1G
k8s-master0 10.0.0.20 4核4G
k8s-master1 10.0.0.21 4核4G
k8s-master2 10.0.0.22 4核4G
k8s-work0 10.0.0.30 4核4G

基础环境配置

基础环境是不管master还是work都需要的环境

  1. 禁用swap
  2. 确保每个节点上 MAC 地址和 product_uuid 的唯一性

product_uuid可以这样查看

sudo cat /sys/class/dmi/id/product_uuid
  1. 修改hostname
  2. 允许 iptables 检查桥接流量
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system
  1. 安装runtime

    这里安装docker
curl -fsSL get.docker.com | bash

修改docker配置

cat > /etc/docker/daemon.json << EOF
{
"registry-mirrors": ["https://go38p9zi.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
}
}
EOF
  1. 安装kubeadm、kubelet 和 kubectl

kubectl 并不是每台机器都需要安装

更新 apt 包索引并安装使用 Kubernetes apt 仓库所需要的包:

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl

下载 Google Cloud 公开签名秘钥:

sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

添加 Kubernetes apt 仓库:

echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

更新 apt 包索引,安装 kubelet、kubeadm 和 kubectl,并锁定其版本:

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

准备负载均衡

在ha机器上执行

安装haproxy

apt install -y haproxy

配置haproxy

--- 前面保持默认配置 ---

frontend k8s_api_fe
bind :6443
default_backend k8s_api_be
mode tcp
option tcplog
backend k8s_api_be
balance source
mode tcp
server k8s-master0 10.0.0.20:6443 check
server k8s-master1 10.0.0.21:6443 check
server k8s-master2 10.0.0.22:6443 check frontend http_ingress_traffic_fe
bind :80
default_backend http_ingress_traffic_be
mode tcp
option tcplog
backend http_ingress_traffic_be
balance source
mode tcp
server k8s-work0 10.0.0.30:80 check frontend https_ingress_traffic_fe
bind *:443
default_backend https_ingress_traffic_be
mode tcp
option tcplog
backend https_ingress_traffic_be
balance source
mode tcp
server k8s-work0 10.0.0.30:443 check

创建集群

kubeadm init

在k8s-master0上执行

kubeadm init \
--apiserver-advertise-address=10.0.0.10 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16

其他参数:

--image-repository registry.aliyuncs.com/k8sxio

--kubernetes-version v1.18.0

安装flannel插件

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

获取join命令, 增加新的节点

node

kubeadm init 后会输出在终端上, 有效期2小时, 超时后可以重新生成

生成添加命令:

kubeadm token create --print-join-command

master

生成证书, 记录输出结果

kubeadm init phase upload-certs --upload-certs

获取加入命令

kubeadm token create --print-join-command

上面两步可以简化成

echo "`kubeadm token create --print-join-command` --control-plane --certificate-key `kubeadm init phase upload-certs --upload-certs | tail -1`"

移除节点

移除节点

kubectl drain k8s-node1 --ignore-daemonsets
kubectl delete node k8s-node1

如果是master节点还需要移除etcd member

kubectl exec -it -n kube-system etcd-k8s-master0 /bin/sh

etcdctl --endpoints 127.0.0.1:2379 --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/server.crt --key /etc/kubernetes/pki/etcd/server.key member list
可以看到有多少个节点 etcdctl --endpoints 127.0.0.1:2379 --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/server.crt --key /etc/kubernetes/pki/etcd/server.key member remove 12637f5ec2bd02b8
通过ID来删除