K8S从入门到放弃系列-(10)kubernetes集群之kube-proxy部署

时间：2021-04-22 20:34:25
摘要：
　　kube-proxy的作用主要是负责service的实现，具体来说，就是实现了内部从pod到service和外部的从node port向service的访问

新版本目前 kube-proxy 组件全部采用 ipvs 方式负载，所以为了 kube-proxy 能正常工作需要预先处理一下 ipvs 配置以及相关依赖(每台 node 都要处理)
## 开启ipvs

[root@k8s-master01 ~]# ansible k8s-node -m shell -a "yum install -y ipvsadm ipset conntrack"

[root@k8s-master01 ~]# ansible k8s-node -m shell -a 'modprobe -- ip_vs'

[root@k8s-master01 ~]# ansible k8s-node -m shell -a 'modprobe -- ip_vs_rr'

[root@k8s-master01 ~]# ansible k8s-node -m shell -a 'modprobe -- ip_vs_wrr'

[root@k8s-master01 ~]# ansible k8s-node -m shell -a 'modprobe -- ip_vs_sh'

[root@k8s-master01 ~]# ansible k8s-node -m shell -a 'modprobe -- nf_conntrack_ipv4'
1）创建kube-proxy证书请求

[root@k8s-master01 ~]# vim /opt/k8s/certs/kube-proxy-csr.json

{

    "CN": "system:kube-proxy",

    "hosts": [],

    "key": {

        "algo": "rsa",

        "size":

    },

    "names": [

        {

            "C": "CN",

            "ST": "ShangHai",

            "L": "ShangHai",

            "O": "system:kube-proxy",

            "OU": "System"

        }

    ]

}
2）生成kube-proxy证书与私钥

[root@k8s-master01 ~]# cd /opt/k8s/certs/

[root@k8s-master01 certs]# cfssl gencert -ca=/etc/kubernetes/ssl/ca.pem \

 -ca-key=/etc/kubernetes/ssl/ca-key.pem \

 -config=/opt/k8s/certs/ca-config.json \

 -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy

// :: [INFO] generate received request

// :: [INFO] received CSR

// :: [INFO] generating key: rsa-

// :: [INFO] encoded CSR

// :: [INFO] signed certificate with serial number

// :: [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for

websites. For more information see the Baseline Requirements for the Issuance and Management

of Publicly-Trusted Certificates, v.1.1., from the CA/Browser Forum (https://cabforum.org);

specifically, section 10.2. ("Information Requirements").
3）查看证书生成

[root@k8s-master01 certs]# ll kube-proxy*

-rw-r--r--  root root  Apr  : kube-proxy.csr

-rw-r--r--  root root   Apr  : kube-proxy-csr.json

-rw-------  root root  Apr  : kube-proxy-key.pem

-rw-r--r--  root root  Apr  : kube-proxy.pem
4）证书分发

[root@k8s-master01 certs]# ansible k8s-node -m copy -a 'src=/opt/k8s/certs/kube-proxy-key.pem dest=/etc/kubernetes/ssl/'

[root@k8s-master01 certs]# ansible k8s-node -m copy -a 'src=/opt/k8s/certs/kube-proxy.pem dest=/etc/kubernetes/ssl/'
5）创建kube-proxy kubeconfig文件
kube-proxy组件连接 apiserver 所需配置文件
## 配置集群参数

[root@k8s-master01 ~]# kubectl config set-cluster kubernetes \

 --certificate-authority=/etc/kubernetes/ssl/ca.pem \

 --embed-certs=true \

 --server=https://127.0.0.1:6443 \

 --kubeconfig=kube-proxy.kubeconfig

Cluster "kubernetes" set.

## 配置客户端认证参数

[root@k8s-master01 ~]# kubectl config set-credentials system:kube-proxy \

 --client-certificate=/opt/k8s/certs/kube-proxy.pem \

 --embed-certs=true \

 --client-key=/opt/k8s/certs/kube-proxy-key.pem \

 --kubeconfig=kube-proxy.kubeconfig

User "system:kube-proxy" set.

## 配置集群上下文

[root@k8s-master01 ~]# kubectl config set-context system:kube-proxy@kubernetes \

 --cluster=kubernetes \

 --user=system:kube-proxy \

 --kubeconfig=kube-proxy.kubeconfig

Context "system:kube-proxy@kubernetes" created.

## 配置集群默认上下文

[root@k8s-master01 ~]# kubectl config use-context system:kube-proxy@kubernetes --kubeconfig=kube-proxy.kubeconfig

Switched to context "system:kube-proxy@kubernetes".

##配置文件分发至node节点

[root@k8s-master01 ~]# ansible k8s-node -m copy -a 'src=/root/kube-proxy.kubeconfig dest=/etc/kubernetes/config/'
6）配置kube-proxy参数

[root@k8s-master01 ~]# vim /opt/k8s/cfg/kube-proxy.conf

###

# kubernetes proxy config

# default config should be adequate

# Add your own!

KUBE_PROXY_ARGS="   --bind-address=0.0.0.0 \

                    --cleanup-ipvs=true \

                    --cluster-cidr=10.254.0.0/ \

                    --hostname-override=k8s-node01 \

                    --healthz-bind-address=0.0.0.0 \

                    --healthz-port= \

                    --masquerade-all=true \

                    --proxy-mode=ipvs \

                    --ipvs-min-sync-period=5s \

                    --ipvs-sync-period=5s \

                    --ipvs-scheduler=wrr \

                    --kubeconfig=/etc/kubernetes/config/kube-proxy.kubeconfig \

                    --logtostderr=true \

                    --v="

## 分发参数配置文件

### 修改hostname-override字段所属主机名
[root@k8s-master01 ~]# ansible k8s-node -m copy -a 'src=/opt/k8s/cfg/kube-proxy.conf dest=/etc/kubernetes/config/'
7）kube-proxy系统服务脚本

[root@k8s-master01 ~]# vim /opt/k8s/unit/kube-proxy.service
[Unit]

Description=Kubernetes Kube-Proxy Server

Documentation=https://github.com/GoogleCloudPlatform/kubernetes

After=network.target

[Service]

EnvironmentFile=-/etc/kubernetes/config/kube-proxy.conf

ExecStart=/usr/local/bin/kube-proxy $KUBE_PROXY_ARGS

Restart=on-failure

LimitNOFILE=

[Install]

WantedBy=multi-user.target

## 分发至node节点

[root@k8s-master01 ~]# ansible k8s-node -m copy -a 'src=/opt/k8s/unit/kube-proxy.service dest=/usr/lib/systemd/system/'

## 启动服务

[root@k8s-master01 ~]# ansible k8s-node -m shell -a 'systemctl daemon-reload'

[root@k8s-master01 ~]# ansible k8s-node -m shell -a 'systemctl enable kube-proxy'

[root@k8s-master01 ~]# ansible k8s-node -m shell -a 'systemctl start kube-proxy'
8）查看ipvs路由规则

检查LVS状态，可以看到已经创建了一个LVS集群，将来自10.254.0.1:443的请求转到三台master的6443端口，而6443就是api-server的端口
[root@k8s-node01 ~]# ipvsadm -L -n

IP Virtual Server version 1.2. (size=)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.254.0.1: wrr

  -> 10.10.0.18:              Masq

  -> 10.10.0.19:              Masq

  -> 10.10.0.20:              Masq

相关文章