Keepalived

Nginx

Tomcat

1、规划

虚IP：192.168.56.101

真实IP |主机名 |Nginx端口|Tomcat端口|主从

192.168.56.103|leo.dis2| 88 | 8080 |主

192.168.56.104|leo.dis3| 88 | 8080 |从

CentOS6.8

Nginx 1.13.1

Keepalived 1.3.5

Tomcat7.0.78

JDK1.8.0_131

参看我以前的文章，打开防火墙端口。

2、安装Nginx

使用root用户，安装依赖包：

yum install gcc gcc-c++ make automake autoconf libtool pcre pcre-devel zlib zlib-devel openssl openssl-devel

把Nginx包放到/usr/local/src内，解压缩并安装：

tar -zxvf nginx-1.13.1.tar.gz

cd nginx-1.13.1

./configure --prefix=/usr/local/nginx

make && make install

3、配置Nginx

vi /usr/local/nginx/conf/nginx.conf

在开头增加：

user root;

监听端口80改为88。

vi /usr/local/nginx/html/index.html

分别给两台服务器的Nginx页面增加103/104的标识。

启动Nginx

/usr/local/nginx/sbin/nginx

# 重新载入配置文件

/usr/local/nginx/sbin/nginx -s reload

# 重启 Nginx

/usr/local/nginx/sbin/nginx -s reopen

# 停止 Nginx

/usr/local/nginx/sbin/nginx -s stop

4、Nginx开机启动

vi /etc/rc.local

添加 /usr/local/nginx/sbin/nginx

5、Keepalived安装

安装依赖包：

yum install openssl-devel libnl3-devel ipset-devel iptables-devel libnfnetlink-devel popt popt-static popt-devel gcc kernel-headers kernel-devel net-snmp-devel -y

把Keepalived包放到/usr/local/src内，解压缩并安装：

tar -zxvf keepalived-1.3.5.tar.gz

cd keepalived-1.3.5

./configure --prefix=/usr/local/nginx

make && make install

6、设置为开机启动

接下来是跟很多教程不一样的地方。

在很多教程里，安装完毕后下一步应该是复制、链接一些文件，以便把keepalived设置成系统服务，但是请注意/usr/local/keepalived/etc/rc.d/init.d/keepalived这个文件，教程上都说有这个文件，复制到/etc/init.d/即可，可是在我安装过程中，没有这个目录和文件，所以要按照以下的方法来处理：

mkdir -p /etc/keepalived

cd /usr/local/keepalived/

ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/

ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin/

ln -s /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

touch /etc/rc.d/init.d/keepalived

chmod +x /etc/rc.d/init.d/keepalived

vi /etc/rc.d/init.d/keepalived

keepalived脚本内容：

#!/bin/sh

#

# keepalived High Availability monitor built upon LVS and VRRP

#

# chkconfig: - 86 14

# description: Robust keepalive facility to the Linux Virtual Server project \

# with multilayer TCP/IP stack checks.

### BEGIN INIT INFO

# Provides: keepalived

# Required-Start: $local_fs $network $named $syslog

# Required-Stop: $local_fs $network $named $syslog

# Should-Start: smtpdaemon httpd

# Should-Stop: smtpdaemon httpd

# Default-Start:

# Default-Stop: 0 1 2 3 4 5 6

# Short-Description: High Availability monitor built upon LVS and VRRP

# Description: Robust keepalive facility to the Linux Virtual Server

# project with multilayer TCP/IP stack checks.

### END INIT INFO

# Source function library.

. /etc/rc.d/init.d/functions

exec="/usr/sbin/keepalived"

prog="keepalived"

config="/etc/keepalived/keepalived.conf"

[ -e /etc/sysconfig/$prog ] && . /etc/sysconfig/$prog

lockfile=/var/lock/subsys/keepalived

start() {

[ -x $exec ] || exit 5

[ -e $config ] || exit 6

echo -n $"Starting $prog: "

daemon $exec $KEEPALIVED_OPTIONS

retval=$?

echo

[ $retval -eq 0 ] && touch $lockfile

return $retval

}

stop() {

echo -n $"Stopping $prog: "

killproc $prog

retval=$?

echo

[ $retval -eq 0 ] && rm -f $lockfile

return $retval

}

restart() {

stop

start

}

reload() {

echo -n $"Reloading $prog: "

killproc $prog -1

retval=$?

echo

return $retval

}

force_reload() {

restart

}

rh_status() {

status $prog

}

rh_status_q() {

rh_status &>/dev/null

}

case "$1" in

start)

rh_status_q && exit 0

$1

;;

stop)

rh_status_q || exit 0

$1

;;

restart)

$1

;;

reload)

rh_status_q || exit 7

$1

;;

force-reload)

force_reload

;;

status)

rh_status

;;

condrestart|try-restart)

rh_status_q || exit 0

restart

;;

*)

echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload}"

exit 2

esac

exit $?

注意脚本里面的目录，要跟上面cp、ln的目录一致。

设置服务开机启动：

chkconfig --add keepalived

chkconfig --level 35 keepalived on

7、设置Keepalived

主节点（192.168.56.103）：

vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

router_id leo.dis2 #主机名

}

vrrp_script chk_nginx {

script "/etc/keepalived/nginx_check.sh"

interval 2

weight -20

}

vrrp_instance VI_1 {

state MASTER#主节点 建议设置为BACKUP，具体解释见后

interface eth1#指定监测的网卡

virtual_router_id 101#虚拟路由ID，节点保持一致

mcast_src_ip 192.168.56.103 #本机IP

priority 100#权重

nopreempt#设置为不抢占资源

advert_int 1

authentication {

auth_type PASS

auth_pass 1357924680

}

track_script {

chk_nginx

}

virtual_ipaddress {

192.168.56.101#虚拟IP

}

}

从节点（192.168.56.104）：

vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

router_id leo.dis3

}

vrrp_script chk_nginx {

script "/etc/keepalived/nginx_check.sh"

interval 2

weight -20

}

vrrp_instance VI_1 {

state BACKUP

interface eth1

virtual_router_id 101

mcast_src_ip 192.168.56.104

priority 90

nopreempt

advert_int 1

authentication {

auth_type PASS

auth_pass 1357924680

}

track_script {

chk_nginx

}

virtual_ipaddress {

192.168.56.101

}

}

注意：nopreempt这个选项要解释解释。

这个设置项，允许一个priority比较低的节点作为master，即使有priority更高的节点启动。

通常如果master服务死掉后backup会变成master，但是当master服务又好了的时候 master此时会抢占VIP，这样就会发生两次切换对业务繁忙的网站来说是不好的。

所以我们要在配置文件加入 nopreempt 非抢占，但是这个参数只能用于state 为backup，故我们在用的时候最好master 和backup的state都设置成backup 让其通过priority来竞争。

但是我在测试的时候，不管怎么配置， VIP资源总是会被抢占，后来查资料说要在防火墙上开启组播：

vi /etc/sysconfig/iptables

#组播

-A INPUT -d 224.0.0.18 -j ACCEPT

重启防火墙。

监测Nginx脚本：

#!/bin/bash

A=`ps -C nginx –no-header |wc -l`

if [ $A -eq 0 ];then

/usr/local/nginx/sbin/nginx

sleep 2

if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then

killall keepalived

fi

fi

脚本赋权：

chmod +x /etc/keepalived/nginx_check.sh

启动Keepalived：

service keepalived start

8、验证

启动Nginx、Keepalived。

分别访问：http://192.168.56.103:88/ http://192.168.56.104:88/ 会看到之前做的标识。

访问http://192.168.56.101:88/ 会看到103的标识。

关闭103的KA，刷新页面则会看到104的标识。

9、Nginx+Tomcat负载均衡

Nginx+Tomcat负载均衡的教程特别多，不再多介绍，简单提几个点。

1）两个nginx.conf修改以下几处：

#增加

upstream tomcat.com{

#Tomcat1

192.168.56.103:8080 weight=1;

#Tomcat2

192.168.56.104:8080 weight=1;

}

#修改

location / {

#root html;

#index index.html index.htm;

proxy_pass http://tomcat.com;

proxy_redirect default;

}

2）JDK要安装，防火墙要打开端口。

3）修改两个Tomcat/webapps/ROOT/index.jsp，分别做上标识。

访问http://192.168.56.101:88，即可看到不同标识的Tomcat主页。