enira.net

We are all living in a technological wasteland.

RSS
people

Slitaz project – Part 6 – Load balancer

This last part will focus on the loadbalancers. These loadbalancers will balance the load over our four webnodes. Because I couldn’t get lvs working for heartbeat, I’ve created a simple python script which takes over the virtual IP. Basically the same functionality, only no pain in the ass with missing dependencies for compiling an open-source package.

Let’s start by prepping ‘loadbalancer1’, I will use IP 192.168.1.200 to manage incoming virtual connections. And IP 192.168.1.201 for loadbalancer1 and IP 192.168.1.202 for loadbalancer 2.

/home/base/ip.sh loadbalancer1 192.168.1 201 1
reboot

Now before we can use HAProxy, we must disable port 80, which is default still open on the LighTTPD servers. Just edit ‘lighttpd.conf’ and change the default port to 81 (our admin instance).

nano /etc/lighttpd/lighttpd.conf
# Port, default for HTTP traffic is 80.
#
server.port = 81

Now we can start with HAProxy, grab the toolchain. As we need to compile.

tazpkg get-install slitaz-toolchain
tazpkg get-install python
tazpkg get-install fcron

It is possible that you recieve following error:

Installation of : fcron
================================================================================
Copying fcron...                                                     [ OK ]
Extracting fcron...                                                  [ OK ]
Extracting the pseudo fs... (lzma)                                   [ OK ]
Installing fcron... cp: can't create '/etc/init.d': File exists
                                                                     [ Failed ]
Removing all tmp files...                                            [ OK ]
================================================================================
fcron (3.0.4) is installed.

This is because the service file is called ‘init.d’ in the package, and not ‘fcron’.Which tries to install the startup folder. I provided a renamed copy of this on this site. If you wish to do it manually, unextract the fcron tazpkg. It contains a small lzma filesystem in which you will see the ‘init.d’ file.
Grabbing it from this site:

wget http://enira.net/wp-content/uploads/2012/07/fcron.txt -O /etc/init.d/fcron

Allow execute:

chmod +x /etc/init.d/fcron

Now let’s cleanup again:

tazpkg clean-cache

And let’s also grab our sources and make the application. I found that pcre is already installed so I can use the ‘USE_PCRE=1’ flag.

cd /home/base
wget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.21.tar.gz
tar xzvf /home/base/haproxy-1.4.21.tar.gz
cd /home/base/haproxy-1.4.21
 
make TARGET=linux26 USE_PCRE=1 ARCH=i386
make install

And a little bit of cleaning.

cd /
rm -rf /home/base/haproxy-1.4.210
rm /home/base/haproxy-1.4.21.tar.gz

Now it’s time to make our configuration file. I will place this in ‘/etc/haproxy/haproxy.conf’.

mkdir /etc/haproxy
nano /etc/haproxy/haproxy.conf

So now this is the configuration file for haproxy. I defined my gluster nodes and mysql servers too as a backend machine. This is easy so I can get a quick view of the ‘slitaz farm’ health on ‘http://192.168.1.201/stats’ or ‘http://192.168.1.202/stats’.

global
	daemon
        maxconn 4096
	pidfile /var/run/haproxy.pid
 
defaults
        mode http
        timeout connect 5000ms
        timeout client 50000ms
        timeout server 50000ms
 
frontend http
        bind *:80
	balance leastconn 
        default_backend webnodes
 
backend loadbalancers
        stats enable
        server loadbalancer1 192.168.1.201:81 check
        server loadbalancer2 192.168.1.202:81 check
 
backend webnodes
	stats enable
	stats auth slitaz:slitaz
        stats uri /stats
 
      	option httpclose
      	option forwardfor
 
	server webnode1 192.168.1.211:80 weight 1 maxconn 512 check
	server webnode2 192.168.1.212:80 weight 1 maxconn 512 check
	server webnode3 192.168.1.213:80 weight 1 maxconn 512 check 
	server webnode3 192.168.1.214:80 weight 1 maxconn 512 check 
 
backend storage
        stats enable
	server glusternode1 192.168.1.221:81 check
	server glusternode2 192.168.1.222:81 check
	server glusternode3 192.168.1.223:81 check
	server glusternode4 192.168.1.224:81 check
	server glusterclient 192.168.1.229:81 check
 
backend mysql
        stats enable
	server mysqlmaster 192.168.1.231:81 check
	server mysqlslave 192.168.1.232:81 check

Anyway now this is over, it’s time to make a service script.

nano /etc/init.d/haproxy
#!/bin/sh
# /etc/init.d/haproxy: Start, stop and restart haproxy loadbalancer on SliTaz,
# at boot time or with the command line. Daemons options are configured
# with /etc/daemons.conf
#
. /etc/init.d/rc.functions
. /etc/daemons.conf
 
NAME=HAproxy
DESC="load balancer"
DAEMON=/usr/local/sbin/haproxy
OPTIONS=$HAPROXY_OPTIONS
PIDFILE=/var/run/haproxy.pid
 
case "$1" in
  start)
    if active_pidfile $PIDFILE haproxy ; then
      echo "$NAME already running."
      exit 1
    fi
    echo -n "Starting $DESC: $NAME... "
    $DAEMON $OPTIONS
    status
    ;;
  stop)
    if ! active_pidfile $PIDFILE haproxy ; then
      echo "$NAME is not running."
      exit 1
    fi
    echo -n "Stopping $DESC: $NAME... "
    kill `cat $PIDFILE`
    rm $PIDFILE
    status
    ;;
  restart)
    if ! active_pidfile $PIDFILE haproxy ; then
      echo "$NAME is not running."
      exit 1
    fi
    echo -n "Restarting $DESC: $NAME... "
    kill `cat $PIDFILE`
    rm $PIDFILE
    sleep 2
    $DAEMON $OPTIONS
    status
    ;;
  *)
    echo ""
    echo -e "\033[1mUsage:\033[0m /etc/init.d/`basename $0` [start|stop|restart]"
    echo ""
    exit 1
    ;;
esac
 
exit 0

Add rights to it:

chmod +x /etc/init.d/haproxy

Now let’s edit the deamon config file:

nano /etc/daemons.conf

And add the line:

# HAproxy options.
HAPROXY_OPTIONS="-f /etc/haproxy/haproxy.conf"

And edit our start up rcS file.

nano /etc/rcS.conf

Also add the haproxy and crontabs to the services:

RUN_DAEMONS="dbus hald slim firewall dropbear lighttpd haproxy fcron"

Now let’s add the heartbeat service. For this I created a small script with python (thats why you need to install python at the beginning of this post).

nano /home/base/heartbeat.py
#! /usr/bin/python
 
import socket
import fcntl
import struct
import array
import os
import time
import datetime
 
#Variables
########################
VIRTUAL = "192.168.1.200"
MASK = "255.255.255.0"
VIRTUAL_IFACE = "eth0:0"
LOOP_PING = 1
LOOP_SLEEP = 5
LOOP_EXECUTE = 9
########################
 
def all_interfaces():
    max_possible = 128  # arbitrary. raise if needed.
    bytes = max_possible * 32
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    names = array.array('B', '\0' * bytes)
    outbytes = struct.unpack('iL', fcntl.ioctl(
        s.fileno(),
        0x8912,  # SIOCGIFCONF
        struct.pack('iL', bytes, names.buffer_info()[0])
    ))[0]
    namestr = names.tostring()
    return [namestr[i:i+32].split('\0', 1)[0] for i in range(0, outbytes, 32)]
 
def get_ip_address(ifname):
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    return socket.inet_ntoa(fcntl.ioctl(
        s.fileno(),
        0x8915,  # SIOCGIFADDR
        struct.pack('256s', ifname[:15])
    )[20:24])
 
t0 = datetime.datetime.now()
 
for num in range(LOOP_EXECUTE):
        interfaces = all_interfaces()
        virtual = False
 
        for interface in interfaces:
                ip = get_ip_address(interface)
 
                if interface == VIRTUAL_IFACE:
                        # we found a virtual interface
                        virtual = True
 
        # Ping and see if the virtual works
        response = os.system("ping -c 1 -W " + str(LOOP_PING) + " " + VIRTUAL)
        if response == 0:
                print "Virtual responding... done"
        else:
                print "Virtual not responding .. binding"
                os.system("ifconfig " + VIRTUAL_IFACE + " " + VIRTUAL + " netmask " + MASK + " up")
 
        time.sleep(LOOP_SLEEP)
 
        print "Execution: " + str(datetime.datetime.now() - t0) +"s"

Add the script to the crontab:

fcrontab -u root -e

press ‘i’ and add:

@ 1 /usr/bin/python /home/base/heartbeat.py >> /dev/null

Quit by pressing: ‘:’, ‘w’, ‘q’

To view your cron jobs type:

fcrontab -u root -l

Now shutdown the server and copy the loadbalancer1. This will function as loadbalancer2:

/home/base/ip.sh loadbalancer2 192.168.1 202 1
reboot

Start loadbalancer1 again and you should now be balancing connections.

You should see HAProxy stats at http://192.168.1.201/stats loadbalancer1 (username: slitaz, password:slitaz) or http://192.168.1.202/stats loadbalancer2 (username: slitaz, password:slitaz)

lb2stats

(As you can see from this stats page example glusterclient and loadbalancer1 is down. Easy no?)

Now you can connect to the virtual IP too, it will redirect to one of our four nodes: http://192.168.1.200/
webnode3

Now it’s time to test the loadbalancer. Find the loadbalancer which has ‘eth0:0’ (our virtual interface).
Tip:

ifconfig
root@loadbalancer1:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:0E:AC:A0
          inet addr:192.168.1.201  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2144 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2937 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:150148 (146.6 KiB)  TX bytes:201556 (196.8 KiB)
          Interrupt:19 Base address:0x2000
 
eth0:0    Link encap:Ethernet  HWaddr 00:0C:29:0E:AC:A0
          inet addr:192.168.1.200  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:19 Base address:0x2000
 
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:383 errors:0 dropped:0 overruns:0 frame:0
          TX packets:383 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:21900 (21.3 KiB)  TX bytes:21900 (21.3 KiB)

Once you found it, shut it down or reboot it. (In my case loadbalancer1)

C:\Users\Enira>ping 192.168.1.200 -t
 
Pinging 192.168.1.200 with 32 bytes of data:
Reply from 192.168.1.200: bytes=32 time=492ms TTL=64
Reply from 192.168.1.200: bytes=32 time

As you can see the timeouts occur when I rebooted the machine. And an ifconfig on loadbalancer2 clearly shows that the has taken over the virtual eth0:0 interface:

root@loadbalancer2:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:DB:4B:CF
          inet addr:192.168.1.202  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6551 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9197 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:457143 (446.4 KiB)  TX bytes:641346 (626.3 KiB)
          Interrupt:19 Base address:0x2000
 
eth0:0    Link encap:Ethernet  HWaddr 00:0C:29:DB:4B:CF
          inet addr:192.168.1.200  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:19 Base address:0x2000
 
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1119 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1119 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:61932 (60.4 KiB)  TX bytes:61932 (60.4 KiB)

That’s it, for now. And the VMWare images: loadbalancers.7z (67.1 MB)

Leave a Reply

You must be logged in to post a comment.