Nginx manage multiple https website with keepalived

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Nginx manage multiple https website with keepalived

linbo liao

I already submit an issue in keepalived github issue page and stackoverflow. Paste again for more people's help.

I want to use Nginx to manager multiple https website, refer to nginx document (Name-based HTTPS servers section), one method is to assign a separate IP for every HTTPS servers. And in our environment, this is the only method.

Due to single-point issue, I want to use keepalived to manage master-backup Nginx node. The logic is:

1. Setup master/backup nginx node

2. Master nginx will assign multiple vip via keepalived

3. Master nginx will be up, backup nginx is down. (due to backup nginx has no vip, start will fail)

4. If master nginx is down, vip transfer to backup node, backup nginx start.

I test in Centos 7 with keepalived v1.3.5, but meet some issue.

Configuration

master node

global_defs {
   router_id LVS_DEVEL
}

vrrp_script chk_nginx {
    script "/usr/sbin/pidof nginx"
    interval 3
    !weight -5
    rise 1
    fall 2
}

vrrp_instance VI_1 {
    state MASTER
    interface eth1
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.2.16
        192.168.2.17
    }
    track_script {
        chk_nginx
    }

    notify /etc/keepalived/notify_keepalived.sh
    notify_stop "systemctl stop nginx"
}

backup node

global_defs {
   router_id LVS_DEVEL
}

vrrp_script chk_nginx {
    script "/usr/sbin/pidof  nginx"
    interval 3
    !weight -5
    rise 1
    fall 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth1
    virtual_router_id 51
    priority 96
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.2.16
        192.168.2.17
    }
    track_script {
        chk_nginx
    }

    notify /etc/keepalived/notify_keepalived.sh
    notify_stop "systemctl stop nginx"
}

check script:

$ cat /etc/keepalived/notify_keepalived.sh
#!/bin/bash
TYPE=$1
NAME=$2
STATE=$3

echo $STATE > /tmp/k.log
:
case $STATE in
        "MASTER") systemctl start nginx
                  exit 0
                  ;;
        "BACKUP") systemctl stop nginx
                  exit 0
                  ;;
        "FAULT")  systemctl stop nginx
                  exit 0
                  ;;
        *)        echo "ipsec unknown state"
                  exit 1
                  ;;
esac

method 1

If unset weight, keepalived startup will check nginx pid immediately, even I set interval and fall parameter. Master nginx won't enter master state, all node will enter fault state. No master will elect and no active nginx will come up.

Dec 30 04:59:00 localhost systemd: Starting LVS and VRRP High Availability Monitor... Dec 30 04:59:00 localhost Keepalived[20039]: Starting Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2 Dec 30 04:59:00 localhost Keepalived[20039]: Unable to resolve default script username 'keepalived_script' - ignoring Dec 30 04:59:00 localhost Keepalived[20039]: Opening file '/etc/keepalived/keepalived.conf'. Dec 30 04:59:00 localhost systemd: PID file /var/run/keepalived.pid not readable (yet?) after start. Dec 30 04:59:00 localhost Keepalived[20040]: Starting Healthcheck child process, pid=20041 Dec 30 04:59:00 localhost Keepalived[20040]: Starting VRRP child process, pid=20042 Dec 30 04:59:00 localhost systemd: Started LVS and VRRP High Availability Monitor. Dec 30 04:59:00 localhost Keepalived_healthcheckers[20041]: Opening file '/etc/keepalived/keepalived.conf'. Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: Registering Kernel netlink reflector Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: Registering Kernel netlink command channel Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: Registering gratuitous ARP shared channel Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: Opening file '/etc/keepalived/keepalived.conf'. Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: VRRP_Instance(VI_1) removing protocol VIPs. Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: WARNING - script `systemctl` resolved by path search to `/usr/bin/systemctl`. Please specify full path. Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: SECURITY VIOLATION - scripts are being executed but script_security not enabled. Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: Using LinkWatch kernel netlink reflector... Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: VRRP sockpool: [ifindex(3), proto(112), unicast(0), fd(10,11)] Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: /usr/sbin/pidof nginx exited with status 1 Dec 30 04:59:01 localhost Keepalived_vrrp[20042]: VRRP_Instance(VI_1) Now in FAULT state Dec 30 04:59:03 localhost Keepalived_vrrp[20042]: /usr/sbin/pidof nginx exited with status 1 Dec 30 04:59:06 localhost Keepalived_vrrp[20042]: /usr/sbin/pidof nginx exited with status 1

method 2

If uncomment weigth, startup works fine. Master node assign vip and master nginx startup. Backup nginx is down.

However, when I shutdown master nginx, master node priority (100-5) > backup node (96-5). Although master nginx is down, but vip still be in master node.

method 3

set master weight -5, set backup weigth 2.

  1. Startup keepalived, master node get vip, master nginx start. Backup nginx is down.
  2. Shutdown master nginx, master node priority 95 < backup node 96, backup node get vip, backup nginx start.
  3. Shutdown backup nginx, master node priority 95 < backup node 96(98-2), backup still hold vip, no active nginx come up.

For this scenario, program startup depends on vip, how to manage HA?

Thanks.


_______________________________________________
nginx mailing list
[hidden email]
http://mailman.nginx.org/mailman/listinfo/nginx