XtendWeb cluster setup¶
Your business is critical - Never Lose an email . Never Show a “WebSite Down!” message to a client .
Never worry how your server will handle the next high profile product launch
DuET cPanel provides a horizontally scalable high-performance web stack on a cPanel server that makes 3 critical infrastructure high available
- High Available web stack using Round Robin DNS
- High Available MySQL/MariaDB using master-master replication
- PostFix backup MX that will accept and queue up emails in case the master goes down and deliver it to master once master is online
DuET cPanel needs at least 2 servers ( thus the name! ) .It is best that they be identical , but except for the drives, it is not mandatory.
- cPanel Licenced server (master)
- cPanel DNS only server (slave)
You can add more cPanel DNS only installed slave servers at any time and scale your infrastructure horizontally
The recommended method to switch to DuET cPanel is set up the DuET cpanel in a cloud ( Why Cloud?) and migrate your domains over. You can Also migrate your existing cPanel server setup to DuET cPanel , but this is not discussed here.
Basic 2 server setup¶
1. Setup cPanel
master.sysally.net ==> CentOS7 ,cPanel ,MariaDB 10.1
Install cPanel - cd /home && curl -o latest -L https://securedownloads.cpanel.net/latest && sh latest
slave.sysally.net ==> CentOS7 ,cPanel DNS only
Install DNS Only - cd /home && curl -o latest-dnsonly -L https://securedownloads.cpanel.net/latest-dnsonly && sh latest-dnsonly
- Install nDeploy on the master . Follow http://ndeploy.readthedocs.io/en/latest/#installation
3. Install ansible
yum -y install python-pip libffi-devel python-paramiko python-jinja2
pip install ansible
- Ensure master and slave can communicate via SSH without password (SSH key based login must work from both ends)
Login at least once from both ends and ensure the SSH key signatures are accepted and SSH login works without any prompt
On Slave:
ssh root@masters-FQDN -p port
On Master:
ssh root@slaves-FQDN -p port
- Setup the cluster
On Master
cd /opt/nDeploy/conf/nDeploy-cluster
cp -p hosts.sample hosts
#Modify the hosts file created above as required .Use FQDN's for hostnames
ansible-playbook -i ./hosts cluster.yml
#If any of the tasks fail; you will need to check it and rerun the playbook
#Copy rpm repos on master to slave for php and mysql
cd /etc/yum.repos.d
rsync -av EA4.repo MariaDB101.repo root@slave:/etc/yum.repos.d/
mkdir -p /etc/cpanel/ea4
touch /etc/cpanel/ea4/is_ea4
Configuring the rest of the system to work as a cluster¶
6. Setup the BackupMX
WHM >> Home »DNS Functions »Edit Zone Templates
Modify the DNS zone template on master and set slave as a second MX entry with lower priority
%domain%. IN A %ip%
%domain%. IN AAAA %ipv6%
%domain%. IN MX 0 %domain%.
%domain%. IN MX 5 slave.gnusys.net.
WHM >> Home »Service Configuration »Exim Configuration Manager
Access Lists
Backup MX hosts => Edit
and add the hostname of the slave to this list
DNS
- WHM >> Home »Clusters »DNS Cluster
Setup DNS clustering between master and slave with DNS role as “Synchronize Changes”
MYSQL
- There are 3 ways to setup MySQL to be used with nDeploy cluster.
Dedicated MySQL server(recommended):
This setup is recommeded for its simplicity .The downside is that the MySQL server is a single point of failure and Sites will show database error on the database server outage.But since the dedicated mysql server runs only mysql it can be fine tuned and downtime can be limited to cases where there is a planned downtime by the host itself (which are very few in modern cloud)
In each server the /var/lib/mysql/mysql.sock and 127.0.0.1:3306 sockets are bound by nginx stream proxy.
MySQL master-master replication with nginx as proxy:
yum install MariaDB-server MariaDB-client MariaDB-shared MariaDB-common MariaDB-devel
On master and slave setup the my.cnf file ( sample my.cnf file is provided as /opt/nDeploy/conf/cluster_mysql.ini.sample)
server-id and auto_increment_offset must be different on both servers .Dump and restore all data on master on the slave.
On Master:
MariaDB [(none)]> grant replication slave on *.* to replicator@'slaves-ip' identified by 'thesecretpass';
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> flush privileges;
On Slave:
MariaDB [(none)]> grant replication slave on *.* to replicator@'masters-ip' identified by 'thesecretpass';
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> flush privileges;
Verify the master log file and position on both servers and configure each other to be a slave
On Master:
MariaDB [(none)]> show master status;
+-------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+------------------+
| master-bin.000002 | 641 | | |
+-------------------+----------+--------------+------------------+
On Slave:
MariaDB [(none)]> CHANGE MASTER TO master_host='masters-ip', master_port=13306, master_user='replicator', master_password='thesecretpass', master_log_file='master-bin.000002', master_log_pos=641;
Query OK, 0 rows affected (0.03 sec)
MariaDB [(none)]> start slave;
Do the same by reversing the server roles thus making each server master to and slave of the other.
Setup nginX as a proxy to the MySQL service and serve the port and socket where PHP expects it to be
Sample config files /etc/nginx/conf.d/mysql_stream_master.conf and /etc/nginx/conf.d/mysql_stream_slave.conf is already supplied by the RPM Copy and adjust it according to your setup and include it in nginx.conf from the file /etc/nginx/conf.d/main_custom_include.conf
On Master:
cp -p /etc/nginx/conf.d/mysql_stream_master.conf /etc/nginx/conf.d/mysql_stream_master.conf.local
[root@master ~]# cat /etc/nginx/conf.d/main_custom_include.conf
include /etc/nginx/conf.d/mysql_stream_master.conf.local;
[root@master ~]# cat /etc/nginx/conf.d/mysql_stream_master.conf.local
stream {
upstream mysql_backend {
server unix:/var/lib/mysql/mysql_original.sock;
server slaves-ip:13306 backup; # Use Slaves IP here
}
server {
listen 127.0.0.1:3306;
listen unix:/var/lib/mysql/mysql.sock;
proxy_pass mysql_backend;
}
}
On Slave:
cp -p /etc/nginx/conf.d/mysql_stream_slave.conf /etc/nginx/conf.d/mysql_stream_slave.conf.local
root@slave [~]# cat /etc/nginx/conf.d/main_custom_include.conf
include /etc/nginx/conf.d/mysql_stream_slave.conf.local;
root@slave [~]# cat /etc/nginx/conf.d/mysql_stream_slave.conf.local
stream {
upstream mysql_backend {
server masters-ip:13306; # Use Masters IP here
server unix:/var/lib/mysql/mysql_original.sock backup;
}
server {
listen 127.0.0.1:3306;
listen unix:/var/lib/mysql/mysql.sock;
proxy_pass mysql_backend;
}
}
restart Nginx on both master and slave
- Ensure database access work from both servers.
WHM >> Home »SQL Services »Additional MySQL Access Hosts
Add both master and slave servers IP address here
APPLICATION SERVERS
10. Setup PHP-FPM backends (for PHP) and Phusion Passenger ( RUBY , PYTHON , NODEJS) on all servers in the cluster
For PHP support
================
On Master:
/opt/nDeploy/scripts/easy_php_setup.sh
On Slave:
/opt/nDeploy/scripts/easy_php_setup.sh
For RUBY, PYTHON and NODEJS support
======================
On Master:
/usr/nginx/scripts/nginx-passenger-setup.sh
On Slave:
/usr/nginx/scripts/nginx-passenger-setup.sh
Start the cluster¶
As a last step you must provide nDeploy with a ipmap file that maps IP address on master to IP address on slave server for configuration generation
- Make the ipmap file that maps each IP on the master to an ip on the slave
/opt/nDeploy/scripts/update_cluster_ipmap.py
usage: update_cluster_ipmap.py [-h] slave_hostname ip_here remote_ip
update_cluster_ipmap.py: error: too few arguments
/opt/nDeploy/scripts/update_cluster_ipmap.py slavehostname ip-on-master corresponding-ip-on-slave
The above step creates the /opt/nDeploy/conf/ndeploy_cluster.yaml
Example:
/opt/nDeploy/scripts/update_cluster_ipmap.py slave.sysally.net 162.243.56.192 162.243.54.157
[root@master ~]# cat /opt/nDeploy/conf/ndeploy_cluster.yaml
slave.sysally.net:
ipmap:
162.243.56.192: 162.243.54.157