[MySQL - 4node MHA 구축] part 1.

그동안 master-slave 구조에서의 mha는 2node가 보통입니다. 이번 포스팅에선 총 4node를 구성할 예정이며 master와 slave 3대를 구축하여 LB를 통해 부하분산이 최종 목표입니다.

 

 

 

테스트 환경
Hostname
Public IP
Private IP
DB version
Role
VIP
jh-mgr
115.85.183.80
192.168.100.35
 
MHA manager
 
jh-master
101.101.210.29
192.168.100.36
Mysql 8.0.28
Master
192.168.100.40
jh-slave1
49.50.161.52
192.168.100.37
Mysql 8.0.28
Slave
 
jh-slave2
118.67.131.204
192.168.100.38
Mysql 8.0.28
Slave
 
jh-slave3
27.96.131.176
192.168.100.39
Mysql 8.0.28
Slave
 

 

※ 사전 MySQL DB 설치 진행

 

■ my.cnf Config

<Master>
[root@jh-master ~]# vi /etc/my.cnf

[mysqld]
server-id=1

log-bin=/data/mariadb-bin
log-bin-index=/data/bin.index
expire-logs-days=7

relay-log=/data/relay-log
relay_log_purge=off
expire_logs_days=7

datadir=/data
socket=/tmp/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd
log-error=/log/mariadb.log
pid-file=/log/mariadb.pid

[mysqld_safe]
log-error=/log/mariadb.log
pid-file=/log/mariadb.pid

#
# include all files from the config directory
#
!includedir /etc/my.cnf.d



<Slave1,2,3>
[mysqld]
server-id=2

log-bin=/data/mariadb-bin
log-bin-index=/data/bin.index
expire-logs-days=7

relay-log=/data/relay-log
relay_log_purge=off
expire_logs_days=7

read_only=1

datadir=/data
socket=/tmp/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd
log-error=/log/mariadb.log
pid-file=/log/mariadb.pid

[mysqld_safe]
log-error=/log/mariadb.log
pid-file=/log/mariadb.pid

#
# include all files from the config directory
#
!includedir /etc/my.cnf.d



[mysqld]
server-id=3

log-bin=/data/mariadb-bin
log-bin-index=/data/bin.index
expire-logs-days=7

relay-log=/data/relay-log
relay_log_purge=off
expire_logs_days=7

read_only=1

datadir=/data
socket=/tmp/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd
log-error=/log/mariadb.log
pid-file=/log/mariadb.pid

[mysqld_safe]
log-error=/log/mariadb.log
pid-file=/log/mariadb.pid

#
# include all files from the config directory
#
!includedir /etc/my.cnf.d




[mysqld]
server-id=4

log-bin=/data/mariadb-bin
log-bin-index=/data/bin.index
expire-logs-days=7

relay-log=/data/relay-log
relay_log_purge=off
expire_logs_days=7

read_only=1

datadir=/data
socket=/tmp/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
# Settings user and group are ignored when systemd is used.
# If you need to run mysqld under a different user or group,
# customize your systemd unit file for mariadb according to the
# instructions in http://fedoraproject.org/wiki/Systemd
log-error=/log/mariadb.log
pid-file=/log/mariadb.pid

[mysqld_safe]
log-error=/log/mariadb.log
pid-file=/log/mariadb.pid

#
# include all files from the config directory
#
!includedir /etc/my.cnf.d

 

■ replication 유저 생성(모든 노드)

mysql> create user repl@'%' identified WITH mysql_native_password by 'repl';
Query OK, 0 rows affected (0.01 sec)

mysql> grant replication slave,replication client on *.* to repl@'%';
Query OK, 0 rows affected (0.01 sec)
 
-master status(Master)
mysql> show master status;
+--------------------+----------+--------------+------------------+-------------------+
| File               | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+--------------------+----------+--------------+------------------+-------------------+
| mariadb-bin.000002 |     1030 |              |                  |                   |
+--------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

 

-Slave 등록 & 실행(Slave 1,2,3)

mysql> CHANGE MASTER TO MASTER_HOST='192.168.100.36', MASTER_USER='repl',MASTER_PASSWORD='repl',MASTER_PORT=3306,MASTER_LOG_FILE='mariadb-bin.000002', MASTER_LOG_POS=1030,MASTER_CONNECT_RETRY=10;

mysql> start slave;
 
-Slave 상태 확인(Slave 1, 2, 3)
mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for source to send event
                  Master_Host: 192.168.100.36
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 10
              Master_Log_File: mariadb-bin.000002
          Read_Master_Log_Pos: 1731
               Relay_Log_File: relay-log.000002
                Relay_Log_Pos: 1029
        Relay_Master_Log_File: mariadb-bin.000002
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1731
              Relay_Log_Space: 1233
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 9dd71463-07d3-11ed-a6c4-f220cde1e464
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
       Master_public_key_path:
        Get_master_public_key: 0
            Network_Namespace:
1 row in set, 1 warning (0.01 sec)



mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for source to send event
                  Master_Host: 192.168.100.36
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 10
              Master_Log_File: mariadb-bin.000002
          Read_Master_Log_Pos: 1731
               Relay_Log_File: relay-log.000002
                Relay_Log_Pos: 1029
        Relay_Master_Log_File: mariadb-bin.000002
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1731
              Relay_Log_Space: 1233
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 9dd71463-07d3-11ed-a6c4-f220cde1e464
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
       Master_public_key_path:
        Get_master_public_key: 0
            Network_Namespace:
1 row in set, 1 warning (0.00 sec)



mysql> show slave status\G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for source to send event
                  Master_Host: 192.168.100.36
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 10
              Master_Log_File: mariadb-bin.000002
          Read_Master_Log_Pos: 1731
               Relay_Log_File: relay-log.000002
                Relay_Log_Pos: 1029
        Relay_Master_Log_File: mariadb-bin.000002
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1731
              Relay_Log_Space: 1233
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 9dd71463-07d3-11ed-a6c4-f220cde1e464
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0
         Replicate_Rewrite_DB:
                 Channel_Name:
           Master_TLS_Version:
       Master_public_key_path:
        Get_master_public_key: 0
            Network_Namespace:
1 row in set, 1 warning (0.00 sec)

 

-Master에서 Slave 조회

mysql> show slave hosts\G;
*************************** 1. row ***************************
Server_id: 2
      Host:
      Port: 3306
Master_id: 1
Slave_UUID: aa511e17-07d3-11ed-aea9-f220cda626a6
*************************** 2. row ***************************
Server_id: 3
      Host:
      Port: 3306
Master_id: 1
Slave_UUID: 87cf362c-07e1-11ed-b7f8-f220cd646c8a
*************************** 3. row ***************************
Server_id: 4
      Host:
      Port: 3306
Master_id: 1
Slave_UUID: 9dd5ca74-07d3-11ed-a50d-f220cd8a1dd8


3 rows in set, 1 warning (0.00 sec)

 

■ VIP 생성(Master, Slave 1,2,3) 
띄어 놓는건 Master에만 띄어놓습니다.
[root@jh-master ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth1:1

[root@jh-master ~]# /etc/sysconfig/network-scripts/ifup ifcfg-eth1:1

[root@jh-master ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.41.216.208  netmask 255.255.254.0  broadcast 10.41.217.255
        ether f2:20:cd:e1:e4:64  txqueuelen 1000  (Ethernet)
        RX packets 282998  bytes 1286099185 (1.1 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 188056  bytes 21221577 (20.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.100.36  netmask 255.255.255.0  broadcast 192.168.100.255
        ether f2:d7:cf:cc:bb:f0  txqueuelen 1000  (Ethernet)
        RX packets 28340  bytes 1433805 (1.3 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2539  bytes 277764 (271.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.100.40  netmask 255.255.255.0  broadcast 192.168.100.255
        ether f2:d7:cf:cc:bb:f0  txqueuelen 1000  (Ethernet)

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1  (Local Loopback)
        RX packets 12  bytes 1164 (1.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 12  bytes 1164 (1.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

 

 

■ MHA 필수 패키지 설치 (모든 노드)

[root@jh-master ~]# yum -y install epel*

[root@jh-master ~]# yum install -y epel perl-devel perl-CPAN perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Parallel-ForkManager perl-Module-Install

 

 

■ SSh 설정(모든 노드)

[root@jh-master ~]# vi /etc/ssh/sshd_config
PermitRootLogin yes


[root@jh-master ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
d3:56:44:0e:4d:52:17:93:64:31:25:3c:6f:d9:72:52 root@jh-master
The key's randomart image is:
+--[ RSA 2048]----+
|          o==oX=.|
|           =.o++E|
|            o  +o|
|         . .  o.=|
|        S o    = |
|         o       |
|                 |
|                 |
|                 |
+-----------------+



[root@jh-master ~]# chmod 700 ~/.ssh
[root@jh-master ~]# chmod 600 ~/.ssh/id_rsa
[root@jh-master ~]# chmod 644 ~/.ssh/id_rsa.pub
[root@jh-master ~]# cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys
[root@jh-master ~]# chmod 644 ~/.ssh/authorized_keys


## 각 노드별 Public key 공유
[root@jh-mgr .ssh]# vi authorized_keys

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6/zs03al+j0Bf/i3IvtaqbA2or/4Rsb1VwB1LIHRSxDyVrYS+bQsiJBOA/qlOYiFMCSPUSQx3PWJNgeT/HqsEznUutpEqgnM/i7D3946w4X87laExWJ3veNd5No8WchX3ol8HxHaF5WH8mgXB7Sj+zoB5Xw2pDnjlCaCdZGc73OUYeW2shSm7eCgrCltLM0tEhUo72sbcJhvGEN/PvEpFsicHwLBy/UFbx4ZY3Ow5WYRi3VF4CI3pLOwcHmsR6eewh3a4IA4CLiy6G744ziyJvkZMVFs2zWOwxjlKeVudcM0k6axkzix4iAy+vOMy7ZzgtrVIcCrSqR5GFgfkzX9h root@jh-mgr

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgv9r7c/inJus0WqGs/5u+EKZyPNSO+76Me2D4gTct+8in7ml6SaMQWGbS+JbquFnoTKXWRuZ703LNWC1koY2ATW+7IEWPh6VcA+FxCEB7oYDeUPm1zSVm9IypAKgq2NcUuzZJWMnpRuLwKm7F03iwC7EBwTdr05LXF+QRY1xaIoGFtH5GhWreCc2AcsCCUfuDtZp+25f4RTfv9CWjEHdU2Qd0hD57OOfU2OQ866LQgySn7XymU5zwl3wyQUVV6MSuGkaKcGpVhxR1KQfnvNr3FTjDh2XQMG4czEOaQ+6Sp9DeJTXA0kQp4O04X3NfIlgjPW0MZbyOxSgbzS6t1uL5 root@jh-master

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDifiOMYKLP787SFjSlCc8DLNkA34nZ+QrWsRXGpNgNhztxaCGtK4ru/8YuwgqjkODH8Gmwz0VVSsrNAhtw1ARv4Sd+oSKB6vX/fxkYPeI6fDqsP4E1gUNMtrjfj5sArQS1Qkmrp7ZipqGC2Ir3M4MGEzXBfLrEZku3hCxtXz8m7pXGVpTzOvfZU+Zkizad21eqnUUpkmsklqNxZrRGcikvqJhs8g/qTowFK7/m+JsYt8U1llojtUnJLgfO6UVbUaOllvcEpBwZk5CqK/7XiVgxTNJc4HmCb1w9BRndUIouC+qkDrclTYxYh6t9UOO7z2W3dOXDkAwTYKYHfyKRj1wR root@jh-slave1

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyWG0YbJ/QQVnsuxyV4PHxodINTrJodem9MNbhNyJQYiYdAavnWBMO1D60+eeIi0fUX9UA+Ptsg6b1mlRy9wIaY1K1KqZieqAMR3HH4qg9fc+JQJzuAbXgeWGBnbqzfrMgD1PrIND3jhBxgrV4h+vMXSBdwBZ6QKCOOKqSygJk03PmB6vXG+HunQ27BE1xUMft7AP6fdw5kX4LgM9AmYuAJ+Wc5bN+twaV8fWWjR4PLLGpgqCnNidPyohIoiccd3tdftuZfm2nshpNZAcwvX/1m0zJ+vYSh1F5w2dSiXSFyUpNExIfqzkJ7msIEC3TdVkO6YvnPb1ETvarOtRtdhkv root@jh-slave2

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8zRiaVvzWiLKMqlaTokDUbPZDN5IUWjIYSpnvPqtLFmluXnl9A6qVMaLS82BQHnAJmiNKBNGFkdZFPcEvrPl1vm4H8SoPs8xvDVSXMQp90DZaIAu+vwiVJlwClaO5QKj88CbFiRn33mJxLJ3chjxGhDsflOQybGd2xA1+LG6sp5GCZVm09IRfmveSrtGFF6oOAwARyP0Tus0Z+JvKXtwH/GYBwnEUcke406JO+2RY0Nv571bAcBssyS7T5c/tzlzluX6c2YL7Ie7PmGpoZrOaCTXcu+m/y8E50aI0VGTRS9dxJLMWI4u4uRKSloViFMbQpgZG65OwZ1UVqy5CUuB3 root@jh-slave3


[root@jh-mgr .ssh]# systemctl restart sshd


## 패스워드 없이 접속 가능 여부 확인
[root@jh-mgr ~]# ssh 192.168.100.36
[root@jh-mgr ~]# ssh 192.168.100.37
[root@jh-mgr ~]# ssh 192.168.100.38
[root@jh-mgr ~]# ssh 192.168.100.39
 
■ mha4mysql-node 컴파일 & 인스톨 (모든서버)
[root@jh-mgr ~]# tar -zxvf mha4mysql-node-0.57.tar.gz
[root@jh-mgr ~]# cd mha4mysql-node-0.57/

[root@jh-mgr mha4mysql-node-0.57]# perl Makefile.PL
[root@jh-mgr mha4mysql-node-0.57]# make && make install

 

■ mha4mysql-manager 컴파일 & 인스톨 (Manager)

[root@jh-mgr ~]# tar -zxvf mha4mysql-manager-0.57.tar.gz


[root@jh-mgr ~]# cd mha4mysql-manager-0.57/


[root@jh-mgr mha4mysql-manager-0.57]# cpan YAML

[root@jh-mgr mha4mysql-manager-0.57]# perl -MCPAN -e "install File::Remove"

[root@jh-mgr mha4mysql-manager-0.57]# perl -MCPAN -e "install Build"

[root@jh-mgr mha4mysql-manager-0.57]# perl -MCPAN -e "install Module::Install"

[root@jh-mgr mha4mysql-manager-0.57]# perl -MCPAN -e "install Net::Telnet"

[root@jh-mgr mha4mysql-manager-0.57]# perl -MCPAN -e "install Log::Dispatch"

[root@jh-mgr mha4mysql-manager-0.57]# perl Makefile.PL

[root@jh-mgr mha4mysql-manager-0.57]# make && make install

 

■ MHA Configure (Manager)

[root@jh-mgr ~]#  vi ~/.bash_profile

# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH

alias sshcheck='/usr/local/bin/masterha_check_ssh --conf=/etc/mha.cnf'

alias replcheck='/usr/local/bin/masterha_check_repl --conf=/etc/mha.cnf'

alias start='/usr/local/bin/masterha_manager --conf=/etc/mha.cnf &'

alias stop='/usr/local/bin/masterha_stop --conf=/etc/mha.cnf'

alias status='/usr/local/bin/masterha_check_status --conf=/etc/mha.cnf'

alias log='tail -f /var/log/masterha/app1/app1.log'




[root@jh-mgr ~]# cp /root/mha4mysql-manager-0.57/samples/conf/app1.cnf  /etc/mha.cnf

[root@jh-mgr ~]# mkdir /mha


[root@jh-mgr ~]# vi /etc/mha.cnf

[server default]
user=repl
password=repl
ssh_user=root


# working directory on the manager
manager_workdir=/var/log/masterha/app1


# manager log file
manager_log=/var/log/masterha/app1/app1.log


# working directory on MySQL servers
remote_workdir=/var/log/masterha/app5


repl_user=repl
repl_password=repl


master_binlog_dir=/data

#ping interval, 3 times trial
ping_interval=5


#custom scripts for master ip failover
master_ip_failover_script=/var/log/masterha/app1/custom_scripts/master_ip_failover


#custom scripts for shutdown using fencing network custom script
#shutdown_script=/var/log/masterha/app1/custom_scripts/power_manager


#custom scripts for manual master switch
master_ip_online_change_script=/var/log/masterha/app1/custom_scripts/master_ip_online_change


[server1]
hostname=192.168.100.36
port=3306
ignore_fail=1


[server2]
hostname=192.168.100.37
port=3306
ignore_fail=1


[server3]
hostname=192.168.100.38
port=3306
ignore_fail=1


[server4]
hostname=192.168.100.39
port=3306
ignore_fail=1



[root@jh-mgr ~]# mkdir -p /var/log/masterha/app1/custom_scripts
[root@jh-mgr ~]# mkdir /var/log/masterha/app5

 

 

■ sshcheck

[root@jh-mgr ~]# sshcheck
Wed Jul 20 14:25:52 2022 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Jul 20 14:25:52 2022 - [info] Reading application default configuration from /etc/mha.cnf..
Wed Jul 20 14:25:52 2022 - [info] Reading server configuration from /etc/mha.cnf..
Wed Jul 20 14:25:52 2022 - [info] Starting SSH connection tests..
Wed Jul 20 14:25:53 2022 - [debug]
Wed Jul 20 14:25:52 2022 - [debug]  Connecting via SSH from root@192.168.100.36(192.168.100.36:22) to root@192.168.100.37(192.168.100.37:22)..
Warning: Permanently added '192.168.100.37' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:53 2022 - [debug]   ok.
Wed Jul 20 14:25:53 2022 - [debug]  Connecting via SSH from root@192.168.100.36(192.168.100.36:22) to root@192.168.100.38(192.168.100.38:22)..
Warning: Permanently added '192.168.100.38' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:53 2022 - [debug]   ok.
Wed Jul 20 14:25:53 2022 - [debug]  Connecting via SSH from root@192.168.100.36(192.168.100.36:22) to root@192.168.100.39(192.168.100.39:22)..
Warning: Permanently added '192.168.100.39' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:53 2022 - [debug]   ok.
Wed Jul 20 14:25:54 2022 - [debug]
Wed Jul 20 14:25:53 2022 - [debug]  Connecting via SSH from root@192.168.100.37(192.168.100.37:22) to root@192.168.100.36(192.168.100.36:22)..
Warning: Permanently added '192.168.100.36' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:53 2022 - [debug]   ok.
Wed Jul 20 14:25:53 2022 - [debug]  Connecting via SSH from root@192.168.100.37(192.168.100.37:22) to root@192.168.100.38(192.168.100.38:22)..
Warning: Permanently added '192.168.100.38' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:53 2022 - [debug]   ok.
Wed Jul 20 14:25:53 2022 - [debug]  Connecting via SSH from root@192.168.100.37(192.168.100.37:22) to root@192.168.100.39(192.168.100.39:22)..
Warning: Permanently added '192.168.100.39' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:54 2022 - [debug]   ok.
Wed Jul 20 14:25:54 2022 - [debug]
Wed Jul 20 14:25:53 2022 - [debug]  Connecting via SSH from root@192.168.100.38(192.168.100.38:22) to root@192.168.100.36(192.168.100.36:22)..
Warning: Permanently added '192.168.100.38' (ECDSA) to the list of known hosts.
Warning: Permanently added '192.168.100.36' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:54 2022 - [debug]   ok.
Wed Jul 20 14:25:54 2022 - [debug]  Connecting via SSH from root@192.168.100.38(192.168.100.38:22) to root@192.168.100.37(192.168.100.37:22)..
Warning: Permanently added '192.168.100.37' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:54 2022 - [debug]   ok.
Wed Jul 20 14:25:54 2022 - [debug]  Connecting via SSH from root@192.168.100.38(192.168.100.38:22) to root@192.168.100.39(192.168.100.39:22)..
Warning: Permanently added '192.168.100.39' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:54 2022 - [debug]   ok.
Wed Jul 20 14:25:55 2022 - [debug]
Wed Jul 20 14:25:54 2022 - [debug]  Connecting via SSH from root@192.168.100.39(192.168.100.39:22) to root@192.168.100.36(192.168.100.36:22)..
Warning: Permanently added '192.168.100.39' (ECDSA) to the list of known hosts.
Warning: Permanently added '192.168.100.36' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:54 2022 - [debug]   ok.
Wed Jul 20 14:25:54 2022 - [debug]  Connecting via SSH from root@192.168.100.39(192.168.100.39:22) to root@192.168.100.37(192.168.100.37:22)..
Warning: Permanently added '192.168.100.37' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:54 2022 - [debug]   ok.
Wed Jul 20 14:25:54 2022 - [debug]  Connecting via SSH from root@192.168.100.39(192.168.100.39:22) to root@192.168.100.38(192.168.100.38:22)..
Warning: Permanently added '192.168.100.38' (ECDSA) to the list of known hosts.
Wed Jul 20 14:25:54 2022 - [debug]   ok.
Wed Jul 20 14:25:55 2022 - [info] All SSH connection tests passed successfully.
Use of uninitialized value in exit at /usr/local/bin/masterha_check_ssh line 44.

 

■ 심볼릭 링크 생성(Master, Slave 1,2,3)

[root@jh-master ~]# ln -s /mysql/bin/mysqlbinlog  /usr/bin/mysqlbinlog
[root@jh-master ~]# ln -s /mysql/bin/mysql /usr/bin/mysql

 

■ replcheck
[root@jh-mgr ~]# replcheck
Wed Jul 20 14:39:28 2022 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Jul 20 14:39:28 2022 - [info] Reading application default configuration from /etc/mha.cnf..
Wed Jul 20 14:39:28 2022 - [info] Reading server configuration from /etc/mha.cnf..
Wed Jul 20 14:39:28 2022 - [info] MHA::MasterMonitor version 0.57.
Wed Jul 20 14:39:29 2022 - [info] GTID failover mode = 0
Wed Jul 20 14:39:29 2022 - [info] Dead Servers:
Wed Jul 20 14:39:29 2022 - [info] Alive Servers:
Wed Jul 20 14:39:29 2022 - [info]   192.168.100.36(192.168.100.36:3306)
Wed Jul 20 14:39:29 2022 - [info]   192.168.100.37(192.168.100.37:3306)
Wed Jul 20 14:39:29 2022 - [info]   192.168.100.38(192.168.100.38:3306)
Wed Jul 20 14:39:29 2022 - [info]   192.168.100.39(192.168.100.39:3306)
Wed Jul 20 14:39:29 2022 - [info] Alive Slaves:
Wed Jul 20 14:39:29 2022 - [info]   192.168.100.37(192.168.100.37:3306)  Version=8.0.28 (oldest major version between slaves) log-bin:enabled
Wed Jul 20 14:39:29 2022 - [info]     Replicating from 192.168.100.36(192.168.100.36:3306)
Wed Jul 20 14:39:29 2022 - [info]   192.168.100.38(192.168.100.38:3306)  Version=8.0.28 (oldest major version between slaves) log-bin:enabled
Wed Jul 20 14:39:29 2022 - [info]     Replicating from 192.168.100.36(192.168.100.36:3306)
Wed Jul 20 14:39:29 2022 - [info]   192.168.100.39(192.168.100.39:3306)  Version=8.0.28 (oldest major version between slaves) log-bin:enabled
Wed Jul 20 14:39:29 2022 - [info]     Replicating from 192.168.100.36(192.168.100.36:3306)
Wed Jul 20 14:39:29 2022 - [info] Current Alive Master: 192.168.100.36(192.168.100.36:3306)
Wed Jul 20 14:39:29 2022 - [info] Checking slave configurations..
Wed Jul 20 14:39:29 2022 - [info]  read_only=1 is not set on slave 192.168.100.37(192.168.100.37:3306).
Wed Jul 20 14:39:29 2022 - [info]  read_only=1 is not set on slave 192.168.100.38(192.168.100.38:3306).
Wed Jul 20 14:39:29 2022 - [info]  read_only=1 is not set on slave 192.168.100.39(192.168.100.39:3306).
Wed Jul 20 14:39:29 2022 - [info] Checking replication filtering settings..
Wed Jul 20 14:39:29 2022 - [info]  binlog_do_db= , binlog_ignore_db=
Wed Jul 20 14:39:29 2022 - [info]  Replication filtering check ok.
Wed Jul 20 14:39:29 2022 - [info] GTID (with auto-pos) is not supported
Wed Jul 20 14:39:29 2022 - [info] Starting SSH connection tests..
Wed Jul 20 14:39:32 2022 - [info] All SSH connection tests passed successfully.
Wed Jul 20 14:39:32 2022 - [info] Checking MHA Node version..
Wed Jul 20 14:39:32 2022 - [info]  Version check ok.
Wed Jul 20 14:39:32 2022 - [info] Checking SSH publickey authentication settings on the current master..
Wed Jul 20 14:39:32 2022 - [info] HealthCheck: SSH to 192.168.100.36 is reachable.
Wed Jul 20 14:39:32 2022 - [info] Master MHA Node version is 0.57.
Wed Jul 20 14:39:32 2022 - [info] Checking recovery script configurations on 192.168.100.36(192.168.100.36:3306)..
Wed Jul 20 14:39:32 2022 - [info]   Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/data --output_file=/var/log/masterha/app5/save_binary_logs_test --manager_version=0.57 --start_file=mariadb-bin.000002
Wed Jul 20 14:39:32 2022 - [info]   Connecting to root@192.168.100.36(192.168.100.36:22)..
  Creating /var/log/masterha/app5 if not exists..    ok.
  Checking output directory is accessible or not..
   ok.
  Binlog found at /data, up to mariadb-bin.000002
Wed Jul 20 14:39:33 2022 - [info] Binlog setting check done.
Wed Jul 20 14:39:33 2022 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Wed Jul 20 14:39:33 2022 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='repl' --slave_host=192.168.100.37 --slave_ip=192.168.100.37 --slave_port=3306 --workdir=/var/log/masterha/app5 --target_version=8.0.28 --manager_version=0.57 --relay_dir=/data --current_relay_log=relay-log.000002  --slave_pass=xxx
Wed Jul 20 14:39:33 2022 - [info]   Connecting to root@192.168.100.37(192.168.100.37:22)..
  Checking slave recovery environment settings..
    Relay log found at /data, up to relay-log.000002
    Temporary relay log file is /data/relay-log.000002
    Testing mysql connection and privileges..mysql: [Warning] Using a password on the command line interface can be insecure.
done.
    Testing mysqlbinlog output.. done.
    Cleaning up test file(s).. done.
Wed Jul 20 14:39:33 2022 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='repl' --slave_host=192.168.100.38 --slave_ip=192.168.100.38 --slave_port=3306 --workdir=/var/log/masterha/app5 --target_version=8.0.28 --manager_version=0.57 --relay_dir=/data --current_relay_log=relay-log.000002  --slave_pass=xxx
Wed Jul 20 14:39:33 2022 - [info]   Connecting to root@192.168.100.38(192.168.100.38:22)..
  Checking slave recovery environment settings..
    Relay log found at /data, up to relay-log.000002
    Temporary relay log file is /data/relay-log.000002
    Testing mysql connection and privileges..mysql: [Warning] Using a password on the command line interface can be insecure.
done.
    Testing mysqlbinlog output.. done.
    Cleaning up test file(s).. done.
Wed Jul 20 14:39:33 2022 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='repl' --slave_host=192.168.100.39 --slave_ip=192.168.100.39 --slave_port=3306 --workdir=/var/log/masterha/app5 --target_version=8.0.28 --manager_version=0.57 --relay_dir=/data --current_relay_log=relay-log.000002  --slave_pass=xxx
Wed Jul 20 14:39:33 2022 - [info]   Connecting to root@192.168.100.39(192.168.100.39:22)..
  Checking slave recovery environment settings..
    Relay log found at /data, up to relay-log.000002
    Temporary relay log file is /data/relay-log.000002
    Testing mysql connection and privileges..mysql: [Warning] Using a password on the command line interface can be insecure.
done.
    Testing mysqlbinlog output.. done.
    Cleaning up test file(s).. done.
Wed Jul 20 14:39:33 2022 - [info] Slaves settings check done.
Wed Jul 20 14:39:33 2022 - [info]
192.168.100.36(192.168.100.36:3306) (current master)
+--192.168.100.37(192.168.100.37:3306)
+--192.168.100.38(192.168.100.38:3306)
+--192.168.100.39(192.168.100.39:3306)

Wed Jul 20 14:39:33 2022 - [info] Checking replication health on 192.168.100.37..
Wed Jul 20 14:39:33 2022 - [info]  ok.
Wed Jul 20 14:39:33 2022 - [info] Checking replication health on 192.168.100.38..
Wed Jul 20 14:39:33 2022 - [info]  ok.
Wed Jul 20 14:39:33 2022 - [info] Checking replication health on 192.168.100.39..
Wed Jul 20 14:39:33 2022 - [info]  ok.
Wed Jul 20 14:39:33 2022 - [warning] master_ip_failover_script is not defined.
Wed Jul 20 14:39:33 2022 - [warning] shutdown_script is not defined.
Wed Jul 20 14:39:33 2022 - [info] Got exit code 0 (Not master dead).

MySQL Replication Health is OK.

 

part 2에서 이어서 스크립트 수정 후 테스트 진행하겠습니다.

 

 

 

참고 & 사진 출처

 

https://hoing.io/archives/9175

 

MySQL MHA(Master High Availability) 1 - MHA 기능 설명 및 아키텍처 설명

 

hoing.io