Amoeba(变形虫)项目,该开源框架于2008年开始发布一款 Amoeba for Mysql软件。这个软件致力于MySQL的分布式数据库前端代理层,它主要在应用 层访问MySQL的 时候充当SQL路由功能,专注于分布式数据库代理层(Database Proxy)开发。座落与 Client、DB Server(s)之间,对客户端透明。具有负载均衡 、高可用性、SQL 过滤、读写分离、可路由相关的到目标数据库、可并发请求多台数据库合并结果。 通过Amoeba你能够完成多数据源的高可用、负载均衡、数据 切片的功能,目前Amoeba已在很多企业的生产线上面使用

目录

Mysql读写分离-Amoeba

Amoeba的优缺点

优点:
  1.降低费用,简单易用
  2.提高系统整体可用性
  3.易于扩展处理能力与系统规模
  4.可以直接实现读写分离及负载均衡效果,而不用修改代码

缺点:
  1.不支持事务与存储过程
  2.暂不支持分库分表,amoeba目前只做到分数据库实例
  3.不适合从amoeba导数据的场景或者对大数据量查询的query并不合适(比如一次请求返回10w以上甚至更多数据的场合)

角色 系统 服务 IP地址
master centos7 mysql5.7 192.168.100.10
slave centos7 mysql5.7 192.168.100.20
slave centos7 mysql5.7 192.168.100.30
proxy centos7 Amoeba 192.168.100.40

搭建主从

1.安装mysql5.7并配置主从
请参考Mysql主从复制

2.主从创建proxy用户
在主库操作即可,从库自动同步

1
2
> grant all on *.* to 'proxy'@'%' identified by '123456.Com';
> flush privilegse;

安装Amoeba

3.安装jdk

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
$ yum -y remove java* tzdata-java.noarch
$ yum -y install java
$ java -version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)

$ vim /etc/profile
...
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-1.el7_7.x86_64
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$JAVA_HOME/lib:$JAVA_HOME/jre/bin/:$PATH:$HOME/bin
export AMOEBA_HOME=/usr/local/amoeba
export PATH=$PATH:$AMOEBA_HOME/bin 

$ source /etc/profile

4.下载amoeba

1
2
3
$ wget https://sourceforge.net/projects/amoeba/files/Amoeba%20for%20mysql/3.x/amoeba-mysql-3.0.5-RC-distribution.zip
$ unzip amoeba-mysql-3.0.5-RC-distribution.zip
$ mv amoeba-mysql-3.0.5-RC /usr/local/amoeba

5.配置amoeba
Amoeba的配置文件在本环境下位于/usr/local/amoeba/conf目录下。配置文件比较多,但是仅仅使用读写分离功能,只需配置两个文件即可,分别是dbServers.xml和amoeba.xml,如果需要配置ip访问控制,还需要修改access_list.conf文件

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
$ vim dbServers.xml 
<?xml version="1.0" encoding="gbk"?>

<!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd">
<amoeba:dbServers xmlns:amoeba="https://amoeba.meidusa.com/">

		<!-- 
			Each dbServer needs to be configured into a Pool,
			If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration:
			 add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig
			 such as 'multiPool' dbServer   
		-->
		
	<dbServer name="abstractServer" abstractive="true">
		<factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory">
			<property name="connectionManager">${defaultManager}</property>
			<property name="sendBufferSize">64</property>
			<property name="receiveBufferSize">128</property>
				
			<!-- mysql port -->
			<property name="port">3306</property>    #设置Amoeba要连接的mysql数据库的端口,默认是3306
			
			<!-- mysql schema -->
			<property name="schema">test</property>    #设置缺省的数据库,当连接amoeba时,操作表必须显式的指定数据库名,即采用dbname.tablename的方式,不支持 use dbname指定缺省库,因为操作会调度到各个后端dbserver
			
			<!-- mysql user -->
			<property name="user">proxy</property>   #设置amoeba连接后端数据库服务器的账号和密码,因此需要在所有后端数据库上创建该用户,并授权amoeba服务器可连接
			
			<property name="password">123456.Com</property>
		</factoryConfig>

		<poolConfig class="com.meidusa.toolkit.common.poolable.PoolableObjectPool">
			<property name="maxActive">500</property>    #最大连接数,默认500
			<property name="maxIdle">500</property>    #最大空闲连接数
			<property name="minIdle">1</property>    #最新空闲连接数
			<property name="minEvictableIdleTimeMillis">600000</property>
			<property name="timeBetweenEvictionRunsMillis">600000</property>
			<property name="testOnBorrow">true</property>	
			<property name="testOnReturn">true</property>
			<property name="testWhileIdle">true</property>
		</poolConfig>
	</dbServer>

	<dbServer name="master"  parent="abstractServer">    #设置一个后端可写的dbServer,这里定义为master,这个名字可以任意命名,后面还会用到
		<factoryConfig>
			<!-- mysql ip -->
			<property name="ipAddress">192.168.100.10</property>
		</factoryConfig>
	</dbServer>
	
	<dbServer name="slave1"  parent="abstractServer">    #设置后端可写slave1
		<factoryConfig>
			<!-- mysql ip -->
			<property name="ipAddress">192.168.100.20</property>
		</factoryConfig>
	</dbServer>

        <dbServer name="slave2"  parent="abstractServer">    #设置后端可写slave2
                <factoryConfig>
                        <!-- mysql ip -->
                        <property name="ipAddress">192.168.100.30</property>
                </factoryConfig>
        </dbServer>

	
	<dbServer name="slave" virtual="true">    #设置定义一个虚拟的dbserver,实际上相当于一个dbserver组,这里将可读的数据库ip统一放到一个组中,将这个组的名字命名为slave
		<poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">
			<!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA-->
			<property name="loadbalance">1</property>    #选择调度算法,1表示复制均衡,2表示权重,3表示HA, 这里选择1
			
			<!-- Separated by commas,such as: server1,server2,server1 -->
			<property name="poolNames">slave1,slave2</property>    #slave组成员
		</poolConfig>
	</dbServer>
		
</amoeba:dbServers>
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
$ vim amoeba.xml

<?xml version="1.0" encoding="gbk"?>

<!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd">
<amoeba:configuration xmlns:amoeba="https://amoeba.meidusa.com/">

	<proxy>
	
		<!-- service class must implements com.meidusa.amoeba.service.Service -->
		<service name="Amoeba for Mysql" class="com.meidusa.amoeba.mysql.server.MySQLService">
			<!-- port -->
			<property name="port">8066</property>    #设置amoeba监听的端口,默认是8066
			
			<!-- bind ipAddress -->
			<!-- 
			<property name="ipAddress">127.0.0.1</property>    #配置监听的接口,如果不设置,默认监听所以的IP
			 -->
			
			<property name="connectionFactory">
				<bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory">
					<property name="sendBufferSize">128</property>
					<property name="receiveBufferSize">64</property>
				</bean>
			</property>
			
			<property name="authenticateProvider">
				<bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator">
					
					<property name="user">root</property>    #配置客户端连接amoeba时需要使用这里设定的账号 (这里的账号密码和amoeba连接后端数据库服务器的密码无关)
					
					<property name="password">123456</property>
					
					<property name="filter">
						<bean class="com.meidusa.toolkit.net.authenticate.server.IPAccessController">
							<property name="ipFile">${amoeba.home}/conf/access_list.conf</property>
						</bean>
					</property>
				</bean>
			</property>
			
		</service>
		
		<runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext">
			
			<!-- proxy server client process thread size -->
			<property name="executeThreadSize">128</property>
			
			<!-- per connection cache prepared statement size  -->
			<property name="statementCacheSize">500</property>
			
			<!-- default charset -->
			<property name="serverCharset">utf8</property>
			
			<!-- query timeout( default: 60 second , TimeUnit:second) -->
			<property name="queryTimeout">60</property>
		</runtime>
		
	</proxy>
	
	<!-- 
		Each ConnectionManager will start as thread
		manager responsible for the Connection IO read , Death Detection
	-->
	<connectionManagerList>
		<connectionManager name="defaultManager" class="com.meidusa.toolkit.net.MultiConnectionManagerWrapper">
			<property name="subManagerClassName">com.meidusa.toolkit.net.AuthingableConnectionManager</property>
		</connectionManager>
	</connectionManagerList>
	
		<!-- default using file loader -->
	<dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader">
		<property name="configFile">${amoeba.home}/conf/dbServers.xml</property>
	</dbServerLoader>
	
	<queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter">
		<property name="ruleLoader">
			<bean class="com.meidusa.amoeba.route.TableRuleFileLoader">
				<property name="ruleFile">${amoeba.home}/conf/rule.xml</property>
				<property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property>
			</bean>
		</property>
		<property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property>
		<property name="LRUMapSize">1500</property>
		<property name="defaultPool">master</property>    #设置amoeba默认的池,这里设置为master
		
		<property name="writePool">master</property>    #这两个选项默认是注销掉的,需要取消注释,这里用来指定前面定义好的俩个读写池
		<property name="readPool">slave</property>
		<property name="needParse">true</property>
	</queryRouter>
</amoeba:configuration>

6.启动Amoeba
设置jvm内存大小

1
2
3
4
5
6
7
8
9
$ vim  /usr/local/amoeba/jvm.properties 

修改58行的Xss参数:

JVM_OPTIONS="-server -Xms256m -Xmx1024m -Xss196k -XX:PermSize=16m -XX:MaxPermSize=96m"

修改为:

JVM_OPTIONS="-server -Xms1024m -Xmx1024m -Xss256k -XX:PermSize=16m -XX:MaxPermSize=96m"

启动

1
$ /usr/local/amoeba/bin/launcher

关闭

1
$ /usr/local/amoeba/bin/shutdown

登录

1
2
3
4
5
6
7
8
$ mysql -h192.168.100.40 -P8066 -uroot -p123456
> select * from test.student;
+----+--------+-----+-----+
| id | name   | age | sex |
+----+--------+-----+-----+
|  1 | 张三   |  20 ||
|  2 | 李四   |  19 ||
+----+--------+-----+-----+

验证读写分离

7.在master,salve机器上,用下面命令查询访问量

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
192.168.100.10:

> show  global  status like 'Question%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 100   |
+---------------+-------+

192.168.100.20:

> show  global  status like 'Question%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 100   |
+---------------+-------+

192.168.100.30:

> show  global  status like 'Question%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 100   |
+---------------+-------+

每执行一次访问量增加1,确保两个从库访问量相同

8.登录Amoeba接口
查询两次student表,然后查看访问量

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
192.168.100.10:

> show  global  status like 'Question%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 100   |
+---------------+-------+

192.168.100.20:

> show  global  status like 'Question%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 102   |
+---------------+-------+

192.168.100.30:

> show  global  status like 'Question%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 102   |
+---------------+-------+

从库值访问量各增加1,读取负载成功

student表添加两条数据,然后查看访量

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
192.168.100.10:

> show  global  status like 'Question%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 103   |
+---------------+-------+

192.168.100.20:

> show  global  status like 'Question%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 102   |
+---------------+-------+

192.168.100.30:

> show  global  status like 'Question%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Questions     | 102   |
+---------------+-------+

主库访问量增加2,读写分离成功