Corosync+Pacemaker

-
A: Availability
MTBF: Mean Time Between Failure
MTTR: Mean Time to Repair

A=MTBF/(MTBF+MTTR)

0<A<1
99%: 365*24*3600*1% = 315360 seconds = 87.6hours
99.9%: 365*24*3600*0.1% = 31536 seconds = 8.76hours 
99.99%  365*24*3600*0.01% = 3153.6 seconds
99.999% 62*24*3600*0.001% = 315.36 seconds

Heart Beat Check --> Corosync
Resource Tranfser --> Pacemaker
ping / port / curl
Example script to HA check script
#!/bin/bash
function heart {
#check the serve status
}

function ipvs_del {
#del the stop serve information in ipvs
}

function ipvs_add {
#add the active server information in ipvs
}

main {
    #endless check program
    while true; do
        for i in {}
            if  ; then
                if (heart $i); then
                    ipvsdel $i;
                fi
            else
                if ! heart $i; then
                    ipvsadd $i;
                fi        
            fi
        done
        sleep 2
    done
}

Installation Prerequestment
1. time sync;
2. hosts file;
3. root ssh visit;
4. judge machine;
5. resource can not be start automatically;
6. disable selinux / iptables

Three Corosync + Pacemaker Test mysql HA (data file save in NFS)
NFS: 192.168.199.4
Node 1: 192.168.199.2
Node 2: 192.168.199.3
VIP(load balance): 192.168.199.11

1. Disable selinux, iptables

2. Time sync
config vm to connect internet
crontab -e
*/3 * * * * /usr/sbin/ntpdata <ntp server> &> /dev/null

3. Hosts Config
A. Node 1: node1.lianshu.com
hostname node1.lianshu.com
uname -n
cat /proc/sys/kernel/hostname
vim /etc/hosts
192.168.199.2 node1.lianshu.com node1
192.168.199.3 node2.lianshu.com node2

B. Node 2: node2.lianshu.com
hostname node2.lianshu.com
uname -n
cat /proc/sys/kernel/hostname
vim /etc/hosts
192.168.199.2 node1.lianshu.com node1
192.168.199.3 node2.lianshu.com node2

4. root communicate with ssh
A. Node 1
ssh-keygen -P ''
ssh-copy-id -i .ssh/id_rsa.pub root@node1
ssh-copy-id -i .ssh/id_rsa.pub root@node2

B. Node 2
ssh-keygen -P ''
ssh-copy-id -i .ssh/id_rsa.pub root@node1
ssh-copy-id -i .ssh/id_rsa.pub root@node2

5. Install package
for i in node1 node2;do ssh $i 'yum install corosync pacemakder -y'; done

6. Change config
cd /etc/corosync/
cp corosync.conf.example corosync.conf
vim corosync.conf
A. change from secauth: off
to secauth: on
B. Change from bindnetaddr: 192.168.1.0
to bindnetaddr: 192.168.199.0
C. Change from mcastaddr: 239.255.1.1
to mcastaddr: 228.24.25.1
D. Change from to_syslog: yes
to to_syslog: no
E. Add following context
service {
    ver: 0
    name: pacemaker
}

7. Generate Auth
cd /etc/corosync/
corosync-keygen 
during it, upload files to generate key instead of keyboard input
scp corosync.conf authkey node2:/etc/corosync

8. Start service
for i in node1 node2; do ssh $i 'service corosync start'; done

9. Check corosync status
crm_mon
ss -nult

10. Client terminal to write rules
visit the url and copy paste into yum repo
node1: yum install crmsh
After install, run crm to command line
A. status: check status

11. NFS MySQL configuration
A. NFS Server
#groupadd -r -g 400 mysql
#useradd -r -g 400 -u 400 mysql
#mkdir /mydata
#chown -R mysql:mysql mydata
#vim /etc/exports
/mydata 192.168.199.0/24(rw,no_root_squash)
#service rpcbind start
#service nfs start
#showmount -e 192.168.199.4
# service iptables stop

B. Node1 server
#showmount -e 192.168.199.4
#mkdir /mydata
#mount -t nfs 192.168.199.4:/mydata /mydata
#groupadd -r -g 400 mysql
#useradd -r -g 400 -u 400 mysql
#chown -R mysql:mysql /mydata
Install mariadb
#tar xvf mariadb-5.5.36-linux-x86_64.tar.gz -C /usr/local
#cd /usr/local
#ln -sv mariadb-5.5.36-linux-x86_64 mysql
#chown -R mysql:mysql mysql
#mkdir /etc/mysql
#cd /usr/local/mysql/support-files
#cp my-large.cnf /etc/mysql/my.cnf
#vim /etc/mysql/my.cnf
Add "datadir = /mydata"
#cd /usr/local/mysql/scripts
#./mysql_install_db --user=mysql --datadir=/mydata
#cp /usr/local/mysql/support-files/mysql.server /etc/rc.d/init.d/mysqld
#chkconfig --add mysqld
#chkconfig mysqld off
Check mysql status #service mysqld start
#mysql
#grant all *.* to 'root@192.168.199.%' identified by '123456';
#flush privileges;
#service mysqld stop
#umount /mydata

C. Node2 server
#showmount -e 192.168.199.4
#mkdir /mydata
#mount -t nfs 192168.199.4:/mydata /mydata
#groupadd -r -g 400 mysql
#useradd -r -g 400 -u 400 mysql
#chown -R mysql:mysql /mydata
Install mariadb
#tar xvf mariadb-5.5.36-linux-x86_64.tar.gz -C /usr/local
#cd /usr/local
#ln -sv mariadb-5.5.36-linux-x86_64 mysql
#chown -R mysql:mysql mysql
#mkdir /etc/mysql
#cp /usr/local/mysql/support-files/my-large.cnf /etc/mysql/my.cnf
#vim /etc/mysql/my.cnf
Add "datadir=/mydata"
#cp /usr/local/mysql/support-files/mysql.server /etc/rc.d/init.d/mysqld
#chkconfig --add mysqld
#chkconfig mysqld off
Check mysql status: #service mysqld start
#mysql
#grant all *.* to 'root@192.168.199.%' identified by '123456';
#flush privileges;
#service mysqld stop
#umount /mydata

D. Node1 server 
Configure corosync rules
#crm
Check status: status
Go to configure setting: configure
Configure no ping node server: property stonith-enabled=false
Configure no vote: property no-quorum-policy=ignore
Configure the setting: property default-resource-stickiness=100
Configure the public ip address and set rule: primitive mysqlip ocf:heartbeat:IPaddr  params  ip='192.168.199.11' nic='eth0' cidr_netmask='24' broadcast='192.168.199.0' op monitor interval=30s(启动延迟时间) timeout=20s（监控超时时间）
Configure the nfs rousrce: primitive mysqlnfs ocf:heartbeat:Filesystem params device='192.168.199.4:/mydata' directory='/mydata' fstype='nfs' op monitor interval=20s timeout=40s
Check grammer: verify
Configure the mysql service: primitive myserver lsb:mysqld op monitor interval=30s timeout=15s
Configure the group of service: colocation  mysqlserver inf: mysqlip mysqlnfs myserver
Configure the service order: order mysqlnfs_before_myserver mandatory: mysqlnfs myserver
Exit
Check status: status

E. Client machine to test connect the mysql 
i. Install mysql client: yum install mysql
mysql-u root -h 192.168.199.11 -p
ii. Stop the node1: 
crm
node
standby
iii. Recover the node1:
crm 
node 
online

Two Server Corosync + Pacemaker Test mysql HA (data file save in NFS)
NFS: 192.168.20.61
Node 1: 192.168.20.61
Node 2: 192.168.20.77
VIP(load balance): 192.168.20.79
Client:192.168.20.41

1. Disable selinux, iptables

2. Time sync
config vm to connect internet
crontab -e
*/3 * * * * /usr/sbin/ntpdata <ntp server> &> /dev/null

3. Hosts Config
A. Node 1: node1.test.com
hostname node1.test.com
uname -n
cat /proc/sys/kernel/hostname
vim /etc/hosts
192.168.20.61 node1.test.com node1
192.168.20.77 node2.test.com node2

B. Node 2: node2.test.com
hostname node2.test.com
uname -n
cat /proc/sys/kernel/hostname
vim /etc/hosts
192.168.20.61 node1.test.com node1
192.168.20.77 node2.test.com node2

4. root communicate with ssh
A. Node 1
ssh-keygen -P ''
ssh-copy-id -i .ssh/id_rsa.pub root@node1
ssh-copy-id -i .ssh/id_rsa.pub root@node2

B. Node 2
ssh-keygen -P ''
ssh-copy-id -i .ssh/id_rsa.pub root@node1
ssh-copy-id -i .ssh/id_rsa.pub root@node2

5. Install package
for i in node1 node2;do ssh $i 'yum install corosync pacemakder -y'; done

6. Change config
cd /etc/corosync/
cp corosync.conf.example corosync.conf
vim corosync.conf
A. change from secauth: off
to secauth: on
B. Change from bindnetaddr: 192.168.1.0
to bindnetaddr: 192.168.20.0
C. Change from mcastaddr: 239.255.1.1
to mcastaddr: 228.24.25.1
D. Change from to_syslog: yes
to to_syslog: no
E. Add following context
service {
    ver: 0
    name: pacemaker
}

7. Generate Auth
cd /etc/corosync/
corosync-keygen 
during it, upload files to generate key instead of keyboard input
scp corosync.conf authkey node2:/etc/corosync

8. Start service
for i in node1 node2; do ssh $i 'service corosync start'; done

9. Check corosync status
crm_mon
ss -nult

10. Client terminal to write rules
copy following context into yum repo /etc/yum.repos.d/CentOS-Base.repo
[network_ha-clustering_Stable]
name=Stable High Availability/Clustering packages (CentOS_CentOS-6)
type=rpm-md
baseurl=http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/
gpgcheck=1
gpgkey=http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/repodata/repomd.xml.key
enabled=1 
node1: yum install crmsh
After install, run crm to command line
A. status: check status

11. NFS MySQL configuration
A. NFS Server
#groupadd -r -g 400 mysql
#useradd -r -g 400 -u 400 mysql
#mkdir /mydata
#chown -R mysql:mysql mydata
#vim /etc/exports
/mydata 192.168.20.0/24(rw,no_root_squash)
#service rpcbind start
#service nfs start
#showmount -e 192.168.20.61
# service iptables stop

B. Node1 server
#mount -t nfs 192.168.20.61:/mydata /mydata
Install mariadb
#tar xvf mariadb-5.5.36-linux-x86_64.tar.gz -C /usr/local
#cd /usr/local
#ln -sv mariadb-5.5.36-linux-x86_64 mysql
#chown -R mysql:mysql mysql
#mkdir /etc/mysql
#cd /usr/local/mysql/support-files
#cp my-large.cnf /etc/mysql/my.cnf
#vim /etc/mysql/my.cnf
Add "datadir = /mydata"
#cd /usr/local/mysql
#./scripts/mysql_install_db --user=mysql --datadir=/mydata
#cp /usr/local/mysql/support-files/mysql.server /etc/rc.d/init.d/mysqld
#chkconfig --add mysqld
#chkconfig mysqld off
Check mysql status #service mysqld start
#./usr/local/mysql/bin/mysql
#grant all on *.* to 'root@192.168.199.%' identified by '123456';
#flush privileges;
#service mysqld stop
#umount /mydata

C. Node2 server
#showmount -e 192.168.20.61
#mkdir /mydata
#mount -t nfs 192.168.20.61:/mydata /mydata
#groupadd -r -g 400 mysql
#useradd -r -g 400 -u 400 mysql
#chown -R mysql:mysql /mydata
Install mariadb
#tar xvf mariadb-5.5.36-linux-x86_64.tar.gz -C /usr/local
#cd /usr/local
#ln -sv mariadb-5.5.36-linux-x86_64 mysql
#chown -R mysql:mysql mysql
#mkdir /etc/mysql
#cp /usr/local/mysql/support-files/my-large.cnf /etc/mysql/my.cnf
#vim /etc/mysql/my.cnf
Add "datadir=/mydata"
#cp /usr/local/mysql/support-files/mysql.server /etc/rc.d/init.d/mysqld
#chkconfig --add mysqld
#chkconfig mysqld off
Check mysql status: #service mysqld start
#mysql
#grant all *.* to 'root'@'192.168.199.%' identified by '123456';
#flush privileges;
#service mysqld stop
#umount /mydata

D. Node1 server 
Configure corosync rules
#crm
Check status: status
Go to configure setting: configure
Configure no ping node server: property stonith-enabled=false
Configure no vote: property no-quorum-policy=ignore
Configure the setting: property default-resource-stickiness=100
Configure the public ip address and set rule: primitive mysqlip ocf:heartbeat:IPaddr  params  ip='192.168.20.79' nic='eth0' cidr_netmask='24' broadcast='192.168.20.0' op monitor interval=30s(启动延迟时间) timeout=20s（监控超时时间）
Configure the nfs rousrce: primitive mysqlnfs ocf:heartbeat:Filesystem params device='192.168.20.61:/mydata' directory='/mydata' fstype='nfs' op monitor interval=20s timeout=40s
Check grammer: verify
Configure the mysql service: primitive myserver lsb:mysqld op monitor interval=30s timeout=15s
Configure the group of service: colocation  mysqlserver inf: mysqlip mysqlnfs myserver
Configure the service order: order mysqlnfs_before_myserver mandatory: mysqlnfs myserver
Exit
Check status: status

E. Client machine to test connect the mysql 
i. Install mysql client: yum install mysql
mysql-u root -h 192.168.20.79 -p
ii. Stop the node1: 
crm
node
standby node1.test.com
iii. Recover the node1:
crm 
node 
online node1.test.com

Two Server Corosync + Pacemaker Test Apache HA (data file save in NFS)
NFS: 192.168.20.61
Node 1: 192.168.20.61
Node 2: 192.168.20.77
VIP(load balance): 192.168.20.79
Client:192.168.20.44

1. Disable selinux, iptables

2. Time sync
config vm to connect internet
crontab -e
*/3 * * * * /usr/sbin/ntpdata <ntp server> &> /dev/null

3. Hosts Config
A. Node 1: node1.test.com
hostname node1.test.com
uname -n
cat /proc/sys/kernel/hostname
vim /etc/hosts
192.168.20.61 node1.test.com node1
192.168.20.77 node2.test.com node2

B. Node 2: node2.test.com
hostname node2.test.com
uname -n
cat /proc/sys/kernel/hostname
vim /etc/hosts
192.168.20.61 node1.test.com node1
192.168.20.77 node2.test.com node2

4. root communicate with ssh
A. Node 1
ssh-keygen -P ''
ssh-copy-id -i .ssh/id_rsa.pub root@node1
ssh-copy-id -i .ssh/id_rsa.pub root@node2

B. Node 2
ssh-keygen -P ''
ssh-copy-id -i .ssh/id_rsa.pub root@node1
ssh-copy-id -i .ssh/id_rsa.pub root@node2

5. Install package
for i in node1 node2;do ssh $i 'yum install corosync pacemakder -y'; done

6. Change config
cd /etc/corosync/
cp corosync.conf.example corosync.conf
vim corosync.conf
A. change from secauth: off
to secauth: on
B. Change from bindnetaddr: 192.168.1.0
to bindnetaddr: 192.168.20.0
C. Change from mcastaddr: 239.255.1.1
to mcastaddr: 228.24.25.1
D. Change from to_syslog: yes
to to_syslog: no
E. Add following context
service {
    ver: 0
    name: pacemaker
}

7. Generate Auth
cd /etc/corosync/
corosync-keygen 
during it, upload files to generate key instead of keyboard input
scp corosync.conf authkey node2:/etc/corosync

8. Start service
for i in node1 node2; do ssh $i 'service corosync start'; done

9. Check corosync status
crm_mon
ss -nult

10. Client terminal to write rules
copy following context into yum repo /etc/yum.repos.d/CentOS-Base.repo
[network_ha-clustering_Stable]
name=Stable High Availability/Clustering packages (CentOS_CentOS-6)
type=rpm-md
baseurl=http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/
gpgcheck=1
gpgkey=http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/repodata/repomd.xml.key
enabled=1 
node1: yum install crmsh
After install, run crm to command line
A. status: check status

11. NFS Apache configuration
A. NFS Server
#mkdir /www
#vim /etc/exports
/www 192.168.20.0/24(rw)
#service rpcbind start
#service nfs start
#showmount -e 192.168.20.61
# service iptables stop

B. Node1 server
#mount -t nfs 192.168.20.61:/www /www
Install apache
#yum install httpd
#vim /etc/httpd/conf/httpd.conf

#service httpd stop
#umount /www

C. Node2 server
#mount -t nfs 192.168.20.61:/www /www
Install apache
#yum install httpd
#vim /etc/httpd/conf/httpd.conf

#service httpd stop
#umount /www

D. Node1 server 
Configure corosync rules
#crm
Check status: status
Go to configure setting: configure
Configure no ping node server: property stonith-enabled=false
Configure no vote: property no-quorum-policy=ignore
Configure the setting: property default-resource-stickiness=100
Configure the public ip address and set rule: primitive apacheip ocf:heartbeat:IPaddr  params  ip='192.168.20.79' nic='eth0' cidr_netmask='24' broadcast='192.168.20.0' op monitor interval=30s(启动延迟时间) timeout=20s（监控超时时间）
Configure the nfs rousrce: primitive apachenfs ocf:heartbeat:Filesystem params device='192.168.20.61:/www' directory='/www' fstype='nfs' op monitor interval=20s timeout=40s
Check grammer: verify
Configure the service: primitive apacheservice lsb:httpd op monitor interval=30s timeout=15s
Configure the group of service: colocation apacheserver inf: apacheip apachenfs apacheservice
Configure the service order: order apachenfs_before_apacheservice mandatory: apachenfs apacheservice
Exit
Check status: status

E. Client machine to test httpd
i. Chrome http://192.168.20.79
ii. Stop the node1: 
crm
node
standby node1.test.com
iii. Recover the node1:
crm 
node 
online node1.test.com

Comments

Post a Comment

Popular posts from this blog

Nginx Proxy & Load Balance & LNMP

-
Nginx Proxy Service E-commerce website Server 1: provide image request response (  apache / nginx ) Server 2: provide dynamic request response ( php-fpm / apache-php ) Server 3: provide static request response ( apache / nginx ) Server 4: provide data request response ( mysql ) It need one additional nginx server 1. Install nginx 2. proxy to specific server Example location / {      proxy_pass http://192.168.199.187:80/; } location /bbs/ {      proxy_pass http://192.168.199.187:80/bbs/; } location ~* \.(jpg|png|gif)$  {      proxy_pass http://172.16.100.7; } Enhance the log information and specific the hostname location / {      proxy_pass  http://192.168.199.187:80/;      proxy_set_header <parameter name>  $host;  (client requst the hostname )      proxy_set_header <parameter name> $remote_addr; ...
Read more

Snort+barnyard2+Snorby CentOS 6.5_64 Installation

-
#download packages: daq-2.0.2.tar.gz ImageMagicK-6.8.9-8.tar.gz oinkmaster-2.0.tar.gz ruby-1.9.3-p547.tar.gz snort-2.9.6.2.tar.gz snortrules-snapshot-2962.tar.gz MySQL-server-5.5.40-1.rhel5.x86_64.rpm MySQL-client-5.6.21-1.rhel5.x86_64.rpm MySQL-devel-5.6.21-1.rhel5.x86_64.rpm MySQL-shared-5.6.21-1.rhel5.x86_64.rpm epel-release-6-8.noarch.rpm barnyard2-2.1.13.tar.gz #install yum package rpm -ivh epel-release-6-8.noarch.rpm yum install gcc-c++ patch readline readline-devel zlib zlib-devel flex git yum install libyaml-devel libffi-devel openssl-devel make  yum install bzip2 autoconf automake libtool bison iconv-devel libxml2-devel libxslt  yum install gcc g++ build-essential libssl-devel libreadline5-devel zlib1g-devel linux-headers-generic libsqlite3-devel libxslt-devel curl-devel sqlite-devel yum --enablerepo=epel install jasper jasper-libs jasper-devel #Mysql install yum remove mysql mysql-libs rpm -ivh MySQL-server-5.5.40-1.rh...
Read more

ORACLE Error

-
Image
oracle official error explanation  http://docs.oracle.com/cd/E11882_01/server.112/e17766/toc.htm 从服务器下载ALERT日志文件，传输方式需要选择"ASCII" ORA-01078: failure in processing system parameters 发生场景：新建数据库第一次启动时 SQL> startup ORA-01078: failure in processing system parameters LRM-00109: could not open parameter file ‘/home/oracle/oracle/product/10.2.0/db_1/dbs/initorcl.ora’ 原因：oracle_sid没有配置正确 解决方案： 1. 修改oracle用户.bash_profile文件中ORACLE_SID后的内容； 2. 重新已oracle用户登录； ORA-01157: cannot identify/lock data file string - see DBWR trace file  Cause: The background process was either unable to find one of the data files or failed to lock it because the file was already in use. The database will prohibit access to this file but other files will be unaffected. However the first instance to open the database will need to access all online data files. Accompanying error from the operating system describes why the file could not be identified. Action: Have operating system make file ...
Read more