使用pgpool搭建高可用PostgreSQL HA的步骤

pgpool，功能不较多，具有连接池，复制，负载均衡，并行查询，和HA等。今天博主测试下HA的配置。

环境信息：

两台虚拟机，ip分别由192.168.56.101和192.168.56.102 。

PostgreSQL版本：9.3.5

PGHOME：/opt/PostgreSQL/93

PGDATA：/data/pgsql

PGPORT：5432

实验架构：

只有一个pgpool，只测试PostgreSQL热备自动切换功能。

实验步骤

1、准备PostgreSQL环境

在操作之前，需要安装好PostgreSQL，然后配置流复制环境。

这个步骤不明白的朋友可以参考之前的文章。

编译安装PostgreSQL：http://blog.nbqykj.cn/?p=1535.html

PostgreSQL流复制搭建：http://blog.nbqykj.cn/?p=1764.html

2、安装pgpool

pgpool的下载地址：http://www.pgpool.net/download.php?f=pgpool-II-3.3.4.tar.gz 。

tar zxvf pgpool-II-3.3.4.tar.gz

cd pgpool-II-3.3.4

mkdir -p /opt/pgpool

./configure –prefix=/opt/pgpool -with-pgsql=path -with-pgsql=/opt/PostgreSQL/93

make

make install

3、安装pgpool相关函数

 pgpool 函数不是必需安装，但建议安装 pgpool_regclass, pgpool_recovery 函数。

cd /root/pgpool-II-3.3.4/sql

make

make install

安装完成后可以在/opt/PostgreSQL/93/share/postgresql/extension/看到pgpool相关文件。

root@ubuntu:~/pgpool-II-3.3.4/sql# ls -l /opt/PostgreSQL/93/share/postgresql/extension/
total 36
-rw-r–r– 1 root root 791 Nov 11 11:44 pgpool_recovery–1.0.sql
-rw-r–r– 1 root root 160 Nov 11 11:44 pgpool_recovery.control
-rw-r–r– 1 root root 551 Nov 11 11:44 pgpool-recovery.sql
-rw-r–r– 1 root root 283 Nov 11 11:44 pgpool_regclass–1.0.sql
-rw-r–r– 1 root root 152 Nov 11 11:44 pgpool_regclass.control
-rw-r–r– 1 root root 142 Nov 11 11:44 pgpool-regclass.sql
-rw-r–r– 1 root root 332 Aug 22 11:52 plpgsql–1.0.sql
-rw-r–r– 1 root root 179 Aug 22 11:52 plpgsql.control
-rw-r–r– 1 root root 381 Aug 22 11:52 plpgsql–unpackaged–1.0.sql

登陆需要安装的库中，和安装插件一样执行以下两条命令。

create extension pgpool_regclass;
create extension pgpool_recovery;

4、配置两台数据库之间的信任关系

下面这些在两台机器上分别执行，互相登陆不需要密码。

postgres@ubuntu:~$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/postgres/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/postgres/.ssh/id_rsa.
Your public key has been saved in /home/postgres/.ssh/id_rsa.pub.
The key fingerprint is:
3d:47:a9:3e:9e:36:5b:ef:2e:0e:21:66:0b:12:76:cd postgres@ubuntu
The key’s randomart image is:
+–[ RSA 2048]—-+
|                 |
|       o     .   |
|    o . E   o    |
|   . o   . o     |
|    . . S = .    |
|     . + + +     |
|        . + .    |
|         .o=..   |
|         .++.++  |
+—————–+
postgres@ubuntu:~$ ssh-copy-id postgres@192.168.56.102
The authenticity of host ‘192.168.56.102 (192.168.56.102)’ can’t be established.
ECDSA key fingerprint is 8c:f2:97:55:b4:1f:e4:81:1d:55:fd:42:77:3e:c7:c2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘192.168.56.102’ (ECDSA) to the list of known hosts.
postgres@192.168.56.102’s password:
Now try logging into the machine, with “ssh ‘postgres@192.168.56.102′”, and check in:

~/.ssh/authorized_keys

to make sure we haven’t added extra keys that you weren’t expecting.

5、配置pgpool

root@ubuntu:/opt/pgpool# chown -R postgres:postgres etc/

root@ubuntu:/opt/pgpool# su – postgres

postgres@ubuntu:~$ echo “export PATH=\”\$PATH:/opt/pgpool/bin\”” >>/home/postgres/.bashrc

postgres@ubuntu:~$ source /home/postgres/.bashrc

postgres@ubuntu:~$ cd /opt/pgpool/etc/

a.配置  pcp.conf

pgpool 提供 pcp 接口，可以查看、管理 pgpool 的状态，并且可以远程操作 pgpool 。pcp.conf是用来对 pcp 相关命令认证的文件，格式为  USERID:MD5PASSWD。

postgres@ubuntu:/opt/pgpool/etc$ cp pcp.conf.sample pcp.conf

postgres@ubuntu:/opt/pgpool/etc$ pg_md5 -u pgpool -p
password:
e6a52c828d56b46129fbf85c4cd164b3

在pcp.conf文件中末尾添加pgpool:e6a52c828d56b46129fbf85c4cd164b3 。

b.配置 pgpool.conf

listen_addresses = ‘*’
port = 9999
socket_dir = ‘/tmp’
pcp_port = 9898
pcp_socket_dir = ‘/tmp’
backend_hostname0 = ‘192.168.56.101’        //配置数据节点192.168.56.101
backend_port0 = 5432
backend_weight0 = 1        //负载均衡中的权重值
backend_flag0 = ‘ALLOW_TO_FAILOVER’        //允许自动切换
backend_hostname1 = ‘192.168.56.102’        //配置数据节点192.168.56.102
backend_port1 = 5432
backend_weight1 = 1
backend_flag1 = ‘ALLOW_TO_FAILOVER’
enable_pool_hba = on        //打开pool_passwd
pool_passwd = ‘pool_passwd’
authentication_timeout = 60
ssl = off
num_init_children = 32
max_pool = 4
child_life_time = 300
child_max_connections = 0
connection_life_time = 0
client_idle_limit = 0
log_destination = ‘stderr’
print_timestamp = on
log_connections = on
log_hostname = on
log_statement = on
log_per_node_statement = off
log_standby_delay = ‘none’
syslog_facility = ‘LOCAL0’
syslog_ident = ‘pgpool’
debug_level = 0
pid_file_name = ‘/var/run/pgpool/pgpool.pid’
logdir = ‘/tmp’
connection_cache = on
reset_query_list = ‘ABORT; DISCARD ALL’
replication_mode = off
replicate_select = off
insert_lock = on
lobj_lock_table = ”
replication_stop_on_mismatch = off
failover_if_affected_tuples_mismatch = off
load_balance_mode = on        //负载均衡开关
ignore_leading_white_space = on
white_function_list = ”
black_function_list = ‘nextval,setval’
master_slave_mode = on        //主备模式开关
master_slave_sub_mode = ‘stream’        //设置流复制模式
sr_check_period = 5
sr_check_user = ‘repuser’
sr_check_password = ‘123456’
delay_threshold = 0
follow_master_command = ”
parallel_mode = off
pgpool2_hostname = ”
system_db_hostname  = ‘localhost’
system_db_port = 5432
system_db_dbname = ‘pgpool’
system_db_schema = ‘pgpool_catalog’
system_db_user = ‘pgpool’
system_db_password = ”
health_check_period = 0
health_check_timeout = 20
health_check_user = ‘nobody’
health_check_password = ”
health_check_max_retries = 0
health_check_retry_delay = 1
failover_command = ‘/home/postgres/scripts/failover_stream.sh %d %H /data/pgsql/postgresql.trigger.5432’
failback_command = ”
fail_over_on_backend_error = on
search_primary_node_timeout = 10
recovery_user = ‘nobody’
recovery_password = ”
recovery_1st_stage_command = ”
recovery_2nd_stage_command = ”
recovery_timeout = 90
client_idle_limit_in_recovery = 0
use_watchdog = off
trusted_servers = ”
ping_path = ‘/bin’
wd_hostname = ”
wd_port = 9000
wd_authkey = ”
delegate_IP = ”
ifconfig_path = ‘/sbin’
if_up_cmd = ‘ifconfig eth0:0 inet $_IP_$ netmask 255.255.255.0’
if_down_cmd = ‘ifconfig eth0:0 down’
arping_path = ‘/usr/sbin’           # arping command path
arping_cmd = ‘arping -U $_IP_$ -w 1’
clear_memqcache_on_escalation = on
wd_escalation_command = ”
wd_lifecheck_method = ‘heartbeat’
wd_interval = 10
wd_heartbeat_port = 9694
wd_heartbeat_keepalive = 2
wd_heartbeat_deadtime = 30
heartbeat_destination0 = ‘host0_ip1’
heartbeat_destination_port0 = 9694
heartbeat_device0 = ”
wd_life_point = 3
wd_lifecheck_query = ‘SELECT 1’
wd_lifecheck_dbname = ‘template1’
wd_lifecheck_user = ‘nobody’
wd_lifecheck_password = ”
relcache_expire = 0
relcache_size = 256
check_temp_table = on
memory_cache_enabled = off
memqcache_method = ‘shmem’
memqcache_memcached_host = ‘localhost’
memqcache_memcached_port = 11211
memqcache_total_size = 67108864
memqcache_max_num_cache = 1000000
memqcache_expire = 0
memqcache_auto_cache_invalidation = on
memqcache_maxcache = 409600
memqcache_cache_block_size = 1048576
memqcache_oiddir = ‘/var/log/pgpool/oiddir’
white_memqcache_table_list = ”
black_memqcache_table_list = ”

c.failover脚本

/home/postgres/scripts/failover_stream.sh

#! /bin/sh
# Failover command for streaming replication.
# This script assumes that DB node 0 is primary, and 1 is standby.
#
# If standby goes down, do nothing. If primary goes down, create a
# trigger file so that standby takes over primary node.
#
# Arguments: $1: failed node id. $2: new master hostname. $3: path to
# trigger file.

failed_node=$1
new_master=$2
trigger_file=$3

# Do nothing if standby goes down.
if [ $failed_node = 1 ]; then
exit 0;
fi

# Create the trigger file.
/usr/bin/ssh -T $new_master /bin/touch $trigger_file

exit 0;

d.配置pool_hba.conf

postgres@ubuntu:/opt/pgpool/etc$ cp pgpool.conf.sample pgpool.conf

这里我把认证权限都设置成md5，和PostgreSQL中的pg_hba.conf一样。

e.配置pool_passwd

配置哪些用户可以通过pgpool登陆数据库，格式user:md5 。

例如：postgres:md5cdf7c8e298f988acbc8d7b6b16cdd5f8 。

6、启动pgpool

启动之前创建两个目录。

mkdir /var/log/pgpool
chown postgres.postgres /var/log/pgpool
mkdir /var/run/pgpool
chown postgres.postgres /var/run/pgpool

启动：

postgres@ubuntu:/opt/pgpool/etc$ pgpool -n >/var/log/pgpool/pgpool.log 2>&1 &

7、登陆pgpool、测试HA

postgres@ubuntu:/opt/pgpool/etc$ psql -p 9999 -U postgres -d postgres
psql (9.3.5)
Type “help” for help.

postgres=# show pool_nodes ;
node_id |    hostname    | port | status | lb_weight |  role
———+—————-+——+——–+———–+———
0       | 192.168.56.101 | 5432 | 2      | 0.500000  | primary
1       | 192.168.56.102 | 5432 | 2      | 0.500000  | standby
(2 rows)

热备和负载均衡这样就配置好了，接下来手动关掉主库。

再次登陆pgpool，查看node状态，可以发现192.168.56.102已经自动切换成primary。

#192.168.56.101

postgres@ubuntu:/opt/pgpool/etc$ psql -p 9999 -U postgres -d postgres
psql (9.3.5)
Type “help” for help.

postgres=# show pool_nodes ;
node_id |    hostname    | port | status | lb_weight |  role
———+—————-+——+——–+———–+———
0       | 192.168.56.101 | 5432 | 3      | 0.500000  | standby
1       | 192.168.56.102 | 5432 | 2      | 0.500000  | primary
(2 rows)

#192.168.56.102

postgres@ubuntu:/data/pgsql$ ls -l recovery.done
-rw-r–r– 1 postgres postgres 4938 Nov 11 11:07 recovery.done

热备自动切换成功。

8、其他问题

如果192.168.56.101重新做了备库后再启动，输入以下命令，status状态变成2 。

/opt/pgpool/bin/pcp_attach_node -d 5 localhost 9898 postgres 123456 0

postgres和密码123456由之前pcp.conf中设置所得。

常见连接异常问题

pg_hba.conf	pool_hba.conf	pool_passwd	result
md5	md5	yes	md5 auth
md5	md5	no	“MD5” authentication with pgpool failed for user “XX”
md5	trust	yes/no	MD5 authentication is unsupported in replication, master-slave and parallel mode
trust	md5	yes	no auth
trust	md5	no	“MD5” authentication with pgpool failed for user “XX”
trust	trust	yes/no	no auth

备注： 这个表格很好地描述了各种连接异常问题，对照表格很容易找到原因。

实验总结

使用pgpool可以实现主备之间的自动切换，而且也可以有效实现负载均衡功能，个人觉得可以在生产中进行试用。

参考链接：

http://francs3.blog.163.com/blog/static/4057672720149285445881/

http://www.pgpool.net/docs/latest/pgpool-en.html