利用LVS+Keepalived DR 实现高性能高可用负载均衡

时间:2022-01-20 22:08:10

最近公司用做负责均衡,研究了一下lvs,做为我的第一篇博客吧。

一.            LVS+Keepalived 介绍

1.      LVS

LVSLinux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立,是中国国内最早出现的*软件项目之一。目前有三种IP负载均衡技术(VS/NAT、VS/TUN和VS/DR);

八种调度算法(rr,wrr,lc,wlc,lblc,lblcr,dh,sh)。

 

2.      Keepalvied

       Keepalived在这里主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现

 

二. 网站负载均衡拓朴图

利用LVS+Keepalived DR 实现高性能高可用负载均衡

三. 安装LVS和Keepalvied软件包

 

    

#lsmod | rep ip_vs
#uname -r
显示2.6.18-53.el5PAE
#ln -s /usr/src/kernels/2.6.18-53.el5PAE-i686/ /usr/src/linux

#tar zxvf ipvsadm-1.24.tar.gz
#cd ipvsadm-1.24
#make all && make install(安装相关包:gcc*)
#find / -name ipvsadm # 查看ipvsadm的位置

#tar zxvf keepalived-1.1.15.tar.gz
#cd keepalived-1.1.15
#./configure (提示安装openssl*)
# make && make install
#find / -name keepalived # 查看keepalived位置

#cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
#cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
#mkdir /etc/keepalived
#cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
#cp /usr/local/sbin/keepalived /usr/sbin/
#chkconfig —add keepalived
#chkconfig keepalived on


 

四. 配置LVS

在LVS机上配置(两台都要配置)

  

#vim /usr/local/sbin/lvsdr.sh
#!/bin/bash
# description: start LVS of DirectorServer
#Written by :NetSeek
GW=192.168.5.2

# website director vip.
WEB_VIP=192.168.6.200
WEB_RIP1=192.168.6.152
WEB_RIP2=192.168.6.159
. /etc/rc.d/init.d/functions

logger $0 called with $1

case "$1" in
start)
# Clear all iptables rules.
#/sbin/iptables -F
# Reset iptables counters.
#/sbin/iptables -Z
# Clear all ipvsadm rules/services.
/sbin/ipvsadm -C
#set lvs vip for dr
/sbin/ipvsadm --set 30 5 60
#/sbin/ifconfig eth0:0 $WEB_VIP broadcast $WEB_VIP netmask 255.255.255.255 up
/sbin/route add -host $WEB_VIP dev eth0
/sbin/ipvsadm -A -t $WEB_VIP:1758 -s wrr -p 3
/sbin/ipvsadm -a -t $WEB_VIP:1758 -r $WEB_RIP1:1758 -g -w 1
/sbin/ipvsadm -a -t $WEB_VIP:1758 -r $WEB_RIP2:1758 -g -w 1
touch /var/lock/subsys/ipvsadm >/dev/null 2>&1
# set Arp
/sbin/arping -I eth0 -c 5 -s $WEB_VIP $GW >/dev/null 2>&1 #备份节点上把这行注释掉
;;
stop)
/sbin/ipvsadm -C
/sbin/ipvsadm -Z
#ifconfig eth0:0 down
route del $WEB_VIP >/dev/null 2>&1
rm -rf /var/lock/subsys/ipvsadm >/dev/null 2>&1
/sbin/arping -I eth0 -c 5 -s $WEB_VIP $GW #备份节点上把这行注释掉
echo "ipvsadm stoped"
;;
status)
if [ ! -e /var/lock/subsys/ipvsadm ];then
echo "ipvsadm is stoped"
exit 1
else
ipvsadm -ln
echo "..........ipvsadm is OK."
fi
;;
*)
echo "Usage: $0 {start|stop|status}"
exit 1
esac
exit 0


五. 利用Keepalvied实现负载均衡和和高可用性

1.主LVS上:修改/etc/keepalived/keepalived.conf

global_defs {
router_id LVS_DEVEL
}
vrrp_sync_group lvs_1 {
group {
VI_1
}
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.6.200
}
}

virtual_server 192.168.6.200 1758 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.6.152 1758 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 1758
}
}
real_server 192.168.6.159 1758 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 1758
}
}
}

2.从LVS:

global_defs {
router_id LVS_DEVEL
}
vrrp_sync_group lvs_1 {
group {
VI_1
}
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 50
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.6.200
}

virtual_server 192.168.6.200 1758 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.6.152 1758 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 1758
}
}
real_server 192.168.6.159 1758 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 1758
}
}
}


 

3.开启IP转发,可以不用,如果是nat方式就要配置了,用sysctl命令配置

net.ipv4.ip_forward=1


六、配置xcloud(我们项目的名字)端(192.168.6.152和192.168.6.159)

1.      配置IP转发功能

#vim /etc/rc.d/init.d/realserver.sh

#!/bin/bash
# description: Config realserver lo and apply noarp

SNS_VIP=192.168.6.200
/etc/rc.d/init.d/functions

case "$1" in
start)
ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
/sbin/route add -host $SNS_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
route del $SNS_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0


 

七、启动服务

#/usr/local/sbin/lvsdr.sh start (把这条语句写到/etc/rc.local中,开机启动)

#/etc/init.d/keepalivedstart  启动keepalived 服务,keepalived就能利用keepalived.conf 配置文件,实现负载均衡和高可用.

真是服务器上:运行 /etc/rc.d/init.d/realserver.shstart 也可以加入/etc/rc.local里面

八、测试

在主lvs上:注意红色的ip

1.      # ip addr show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu16436 qdisc noqueue state UNKNOWN

   link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

   inet 127.0.0.1/8 scope host lo

   inet6 ::1/128 scope host

      valid_lft forever preferred_lft forever

2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000

   link/ether e4:1f:13:ec:d2:aa brd ff:ff:ff:ff:ff:ff

   inet 192.168.6.175/16 brd 192.168.255.255 scope global eth0

   inet 192.168.6.200/32 scope global eth0

   inet6 fe80::e61f:13ff:feec:d2aa/64 scope link

      valid_lft forever preferred_lft forever

3: eth1:<NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen1000

link/ethere4:1f:13:ec:d2:ab brd ff:ff:ff:ff:ff:ff

    ………

最后把主机宕机(通过在主机上用/etc/init.d/keepalived stop也可以),到从机上使用ip addr show 查看,同上则OK。

2.用客户端连接一下xcloudd(192.168.6.200),如果连接成功,通过下面命令可以看到,真实连接到了哪台服务器。

  #watch ipvsadm –ln

IP Virtual Server version 1.2.1 (size=4096)

ProtLocalAddress:Port Scheduler Flags

 -> RemoteAddress:Port          Forward Weight ActiveConn InActConn

TCP  192.168.6.200:1758 wrr persistent 3

 -> 192.168.6.152:1758          Route   1      0          0

 -> 192.168.6.159:1758           Route  1      1          0

 

 #tail –f /var/log/message  监听日志,查看状态。

 

3.将一台xcloudd服务器关闭,然后在LVS 上用ipvsadm 命令查看,关闭的服务器应该从

lvs集群中剔除了,再将关闭的服务器启动起来,用ipvsadm查看,又回来了。