Docker+Nginx+Keepalived实现架构高可用

时间:2022-02-28 18:01:13

一、背景

通过keepalived实现nginx高可用,由于在家不想弄多台主机来搞,所以将运行环境用docker封装来模拟跨主机

docker基础镜像:centos

说之前,简单介绍一下:

Keepalived是基于vrrp协议的一款高可用软件。Keepailived有一台主服务器和多台备份服务器,在主服务器和备份服务器上面部署相同的服务配置,使用一个虚拟IP地址对外提供服务,当主服务器出现故障时,虚拟IP地址会自动漂移到备份服务器。

双机高可用方法目前分为两种:

  • 双机主从(也叫双机热备)
  • 双机主主 (也叫双机互备)

下述介绍,高可用中的双机主从模式,双机主主模式,主要是keepalived.conf配置会有所不同,方法都是一样。

二、具体操作

1、安装centos 镜像

docker pull centos

说明:通过用centos镜像来安装高可用所需要的所有环境,再启两个容器,再真实模拟跨主机的场景

2、在centos上安装所需环境(nginx和其它工具)

运行centos容器

docker run -it centos /bash/bin

安装依赖和所需要的包  

#使用yum安装nginx需要包括Nginx的库,安装Nginx的库

rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm

# 使用下面命令安装nginx
#yum install nginx #安装网络包(需要使用ifconfig和ping命令)
yum install net-tools #安装vim
yum install vim

3、在centos安装keepalvied

#安装keepalived环境依赖

yum install -y gcc openssl-devel popt-devel

#安装keepalived

通过yum install keepalived

#或者通过源码安装

wget http://124.205.69.132/files/90630000053A2BB4/www.keepalived.org/software/keepalived-1.3.4.tar.gz

tar zxvf keepalived-1.3.4.tar.gz 
cd keepalived-1.3.4
./configure --prefix=/usr/local/keepalived
make && make install 拷贝几个文件到CentOS7环境中: cp keepalived-1.3.4/keepalived/etc/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp keepalived-1.3.4/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

4、修改/etc/keepalived/keepalived.conf文件

! Configuration File for keepalived
global_defs {
notification_email {
762357658@qq.com
}
notification_email_from itsection@example.com
smtp_server mail.example.com
smtp_connect_timeout 30
router_id LVS_DEVEL
} vrrp_script chk_nginx {
script "/etc/keepalived/nginx_check.sh"
interval 2
weight -5
fall 3
rise 2
} vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 2
priority 101
advert_int 2
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.17.0.210
}
track_script {
chk_nginx
} }

  

/etc/keepalived/check_nginx.sh文件

A=`ps -ef | grep nginx | grep -v grep | wc -l`
if [ $A -eq 0 ];then
nginx
sleep 2
if [ `ps -ef | grep nginx | grep -v grep | wc -l` -eq 0 ];then
#killall keepalived
ps -ef|grep keepalived|grep -v grep|awk '{print $2}'|xargs kill -9
fi fi  

再对check_nginx.sh赋于执行权限:

chmod +x check_nginx.sh

注:keepalived是通过检测keepalived进程是否存在判断服务器是否宕机,如果keepalived进程在但是nginx进程不在了那么keepalived是不会做主备切换,所以我们需要写个脚本来监控nginx进程是否存在,如果nginx不存在就将keepalived进程杀掉。

在主nginx上需要编写nginx进程检测脚本(check_nginx.sh),判断nginx进程是否存在,如果nginx不存在就将keepalived进程杀掉,并将vip漂移到备份机器上

5、设置开机启动

chkconfig keepalived on

或者

systemctl enable keepalived.service  设置开机自动启动

启动keepalived服务:

systemctl start keepalived.service 启动

6、安装所有需要的依赖和环境后,将容器新增的内容重新提交

docker commit 5d112 centos_keepalived_nginx:v1

注:5d112为,上述安装软件所对应的容器id

6、启动含有(keepalived+nginx)的容器

docker run --privileged  -tid --name  keepalived_master centos_keepalived_nginx:v1 /usr/sbin/init

Docker+Nginx+Keepalived实现架构高可用

进入keepalived_master容器:

docker exec -it keepalived_master bash

进入/usr/share/nginx/html,修改index.html文件

Docker+Nginx+Keepalived实现架构高可用

修改标题为:

Welcome to nginx Master!

7、启动keepalived_salve容器

#启动一个容器

docker run --privileged  -tid --name  keepalived_slave centos_keepalived_nginx:v1 /usr/sbin/init

#进入容器

docker exec -it keepalived_slave bash

8、修改keepalived_salve容器中nginx index.html文件

vim /usr/share/nginx/html/index.html

Docker+Nginx+Keepalived实现架构高可用

修改标题为:

Welcome to nginx Slave!

9、修改keepalived_salve容器中keepalived.conf文件 (master容器中,保持和镜像中设置一样即可,不需要更改)

! Configuration File for keepalived
global_defs {
notification_email {
@qq.com
}
notification_email_from itsection@example.com
smtp_server mail.example.com
smtp_connect_timeout
router_id LVS_DEVEL
} vrrp_script chk_nginx {
script "/etc/keepalived/nginx_check.sh"
interval
weight -
fall
rise
} vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id
priority 100
advert_int
authentication {
auth_type PASS
auth_pass
}
virtual_ipaddress {
172.17.0.210
}
track_script {
chk_nginx
} }

其实,由配置中可以看出,主要是state和priority两个参数的调整,其中master节点的priority值一定要比backup大才行!

原理说明:
1、 通过vrrp协议广播,每个keepalived vrrp都去争取master
2、 以virtual_router_id为组队标识。  同为一个vip服务的keepalived的virtual_router_id要保持相同

3、 以priority 为权值,同一个virtual_router_id下那个priority大那个就是master,其它为backup

改完之后,重新加载

systemctl daemon-reload 
systemctl restart keepalived.service

10、验证

查看两个容器中keepalived服务状态

systemctl status keepalived.service 

keepalived_master服务状态效果:

Docker+Nginx+Keepalived实现架构高可用

keepalived_slave服务状态效果图:

Docker+Nginx+Keepalived实现架构高可用

可以看到,keepalived服务运行正常

启动nginx: nginx

在master容器中 curl 172.17.0.210

Docker+Nginx+Keepalived实现架构高可用

在slave容器中 curl 172.17.0.210:

Docker+Nginx+Keepalived实现架构高可用

可以看现,此时master和slave容器两边通过虚拟vip : 172.17.0.210 访问nginx数据,请求返回的数据都是master容器中nginx配置的数据: welcome to nginx master

继续验证,关掉master容器的keepalived服务:

Docker+Nginx+Keepalived实现架构高可用

验证得到的结果是当master容器中的keepalived服务关掉后,curl 172.17.0.210请求返回的数据来自slave,welcome to nginx slave

再继续验证,把关掉master容器的keepalived服务再开启:

Docker+Nginx+Keepalived实现架构高可用

可以看到,当master容器中的keepalived服务开启后,请求返回的数据会再次转到master中。

到此,所有的验证和预期的一致,也达到我们借助docker为基础来实现了整套基于Nginx+Keepalived高可用的方案了。

三、Keepalived服务命令

  • systemctl daemon-reload  重新加载
  • systemctl enable keepalived.service  设置开机自动启动
  • systemctl disable keepalived.service 取消开机自动启动
  • systemctl start keepalived.service 启动
  • systemctl stop keepalived.service停止
  • systemctl status keepalived.service  查看服务状态