Sharding-JDBC教程:Spring Boot整合Sharding-JDBC实现分库分表+读写分离
在上一篇文章介绍了如何 使用Sharding-jdbc进行分库+读写分离,这篇文章将讲述如何使用Sharding-jdbc进行分库分表+读写分离。
架构回顾
在数据量不是很多的情况下,我们可以将数据库进行读写分离,以应对高并发的需求,通过水平扩展从库,来缓解查询的压力。如下:
在数据量达到500万的时候,这时数据量预估千万级别,我们可以将数据进行分表存储。
在数据量继续扩大,这时可以考虑分库分表,将数据存储在不同数据库的不同表中,如下:
案例详解
本案例有6个数据库,两个主库,四个从库,信息如下:
数据库类型 | 数据库 | ip |
---|---|---|
主 | cool | 10.0.0.3 |
从 | cool | 10.0.0.13 |
从 | cool | 10.0.0.17 |
主 | cool2 | 10.0.0.3 |
从 | cool2 | 10.0.0.13 |
从 | cool2 | 10.0.0.17 |
在主库主机的Mysql执行以下脚本,分别为数据库cool和cool2创建5个表,这5个表分别为user_0、user_1、user_2、user_3、user_4。
执行的脚本如下:
USE `cool`;
/*Table structure for table `user_0` */
DROP TABLE IF EXISTS `user_0`;
CREATE TABLE `user_0` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`username` varchar(12) NOT NULL,
`password` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx-username` (`username`)
) ENGINE=InnoDB AUTO_INCREMENT=149 DEFAULT CHARSET=utf8;
/*Table structure for table `user_1` */
DROP TABLE IF EXISTS `user_1`;
CREATE TABLE `user_1` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`username` varchar(12) NOT NULL,
`password` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx-username` (`username`)
) ENGINE=InnoDB AUTO_INCREMENT=150 DEFAULT CHARSET=utf8;
/*Table structure for table `user_2` */
DROP TABLE IF EXISTS `user_2`;
CREATE TABLE `user_2` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`username` varchar(12) NOT NULL,
`password` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx-username` (`username`)
) ENGINE=InnoDB AUTO_INCREMENT=147 DEFAULT CHARSET=utf8;
/*Table structure for table `user_3` */
DROP TABLE IF EXISTS `user_3`;
CREATE TABLE `user_3` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`username` varchar(12) NOT NULL,
`password` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx-username` (`username`)
) ENGINE=InnoDB AUTO_INCREMENT=148 DEFAULT CHARSET=utf8;
CREATE TABLE `user_4` (
`id` INT(12) NOT NULL AUTO_INCREMENT,
`username` VARCHAR(12) NOT NULL,
`password` VARCHAR(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx-username` (`username`)
) ENGINE=INNODB AUTO_INCREMENT=148 DEFAULT CHARSET=utf8;
USE `cool2`;
/*Table structure for table `user_0` */
DROP TABLE IF EXISTS `user_0`;
CREATE TABLE `user_0` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`username` varchar(12) NOT NULL,
`password` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx-username` (`username`)
) ENGINE=InnoDB AUTO_INCREMENT=149 DEFAULT CHARSET=utf8;
/*Table structure for table `user_1` */
DROP TABLE IF EXISTS `user_1`;
CREATE TABLE `user_1` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`username` varchar(12) NOT NULL,
`password` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx-username` (`username`)
) ENGINE=InnoDB AUTO_INCREMENT=150 DEFAULT CHARSET=utf8;
/*Table structure for table `user_2` */
DROP TABLE IF EXISTS `user_2`;
CREATE TABLE `user_2` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`username` varchar(12) NOT NULL,
`password` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx-username` (`username`)
) ENGINE=InnoDB AUTO_INCREMENT=147 DEFAULT CHARSET=utf8;
/*Table structure for table `user_3` */
DROP TABLE IF EXISTS `user_3`;
CREATE TABLE `user_3` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`username` varchar(12) NOT NULL,
`password` varchar(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx-username` (`username`)
) ENGINE=InnoDB AUTO_INCREMENT=148 DEFAULT CHARSET=utf8;
CREATE TABLE `user_4` (
`id` INT(12) NOT NULL AUTO_INCREMENT,
`username` VARCHAR(12) NOT NULL,
`password` VARCHAR(30) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx-username` (`username`)
) ENGINE=INNODB AUTO_INCREMENT=148 DEFAULT CHARSET=utf8;
案例的工程是在上一篇文章的工程基础上进行改造,其中pom文件的依赖包不变,详情可见源码。
在工程的application中做sharding-jdbc的分库分表配置,代码如下:
sharding.jdbc.datasource.names=ds-master-0,ds-master-1,ds-master-0-slave-0,ds-master-0-slave-1,ds-master-1-slave-0,ds-master-1-slave-1
sharding.jdbc.datasource.ds-master-0.type=com.alibaba.druid.pool.DruidDataSource
sharding.jdbc.datasource.ds-master-0.driver-class-name=com.mysql.jdbc.Driver
sharding.jdbc.datasource.ds-master-0.url=jdbc:mysql://10.0.0.3:3306/cool?useUnicode=true&characterEncoding=utf8&tinyInt1isBit=false&useSSL=false&serverTimezone=GMT
sharding.jdbc.datasource.ds-master-0.username=root
sharding.jdbc.datasource.ds-master-0.password=
sharding.jdbc.datasource.ds-master-0-slave-0.type=com.alibaba.druid.pool.DruidDataSource
sharding.jdbc.datasource.ds-master-0-slave-0.driver-class-name=com.mysql.jdbc.Driver
sharding.jdbc.datasource.ds-master-0-slave-0.url=jdbc:mysql://10.0.0.13:3306/cool?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT
sharding.jdbc.datasource.ds-master-0-slave-0.username=root
sharding.jdbc.datasource.ds-master-0-slave-0.password=
sharding.jdbc.datasource.ds-master-0-slave-1.type=com.alibaba.druid.pool.DruidDataSource
sharding.jdbc.datasource.ds-master-0-slave-1.driver-class-name=com.mysql.jdbc.Driver
sharding.jdbc.datasource.ds-master-0-slave-1.url=jdbc:mysql://10.0.0.17:3306/cool?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT
sharding.jdbc.datasource.ds-master-0-slave-1.username=root
sharding.jdbc.datasource.ds-master-0-slave-1.password=
sharding.jdbc.datasource.ds-master-1.type=com.alibaba.druid.pool.DruidDataSource
sharding.jdbc.datasource.ds-master-1.driver-class-name=com.mysql.jdbc.Driver
sharding.jdbc.datasource.ds-master-1.url=jdbc:mysql://10.0.0.3:3306/cool2?useUnicode=true&characterEncoding=utf8&tinyInt1isBit=false&useSSL=false&serverTimezone=GMT
sharding.jdbc.datasource.ds-master-1.username=root
sharding.jdbc.datasource.ds-master-1.password=
sharding.jdbc.datasource.ds-master-1-slave-0.type=com.alibaba.druid.pool.DruidDataSource
sharding.jdbc.datasource.ds-master-1-slave-0.driver-class-name=com.mysql.jdbc.Driver
sharding.jdbc.datasource.ds-master-1-slave-0.url=jdbc:mysql://10.0.0.13:3306/cool2?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT
sharding.jdbc.datasource.ds-master-1-slave-0.username=root
sharding.jdbc.datasource.ds-master-1-slave-0.password=
sharding.jdbc.datasource.ds-master-1-slave-1.type=com.alibaba.druid.pool.DruidDataSource
sharding.jdbc.datasource.ds-master-1-slave-1.driver-class-name=com.mysql.jdbc.Driver
sharding.jdbc.datasource.ds-master-1-slave-1.url=jdbc:mysql://10.0.0.17:3306/cool2?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT
sharding.jdbc.datasource.ds-master-1-slave-1.username=root
sharding.jdbc.datasource.ds-master-1-slave-1.password=
sharding.jdbc.config.sharding.default-database-strategy.inline.sharding-column=id
sharding.jdbc.config.sharding.default-database-strategy.inline.algorithm-expression=ds_$->{id % 2}
sharding.jdbc.config.sharding.tables.user.actual-data-nodes=ds_$->{0..1}.user_$->{0..4}
sharding.jdbc.config.sharding.tables.user.table-strategy.inline.sharding-column=id
sharding.jdbc.config.sharding.tables.user.table-strategy.inline.algorithm-expression=user_$->{id % 5}
sharding.jdbc.config.sharding.tables.user.key-generator-column-name=id
sharding.jdbc.config.sharding.master-slave-rules.ds_0.master-data-source-name=ds-master-0
sharding.jdbc.config.sharding.master-slave-rules.ds_0.slave-data-source-names=ds-master-0-slave-0, ds-master-0-slave-1
sharding.jdbc.config.sharding.master-slave-rules.ds_1.master-data-source-name=ds-master-1
sharding.jdbc.config.sharding.master-slave-rules.ds_1.slave-data-source-names=ds-master-1-slave-0, ds-master-1-slave-1
- 在上面的配置中,其中sharding.jdbc.datasource部分是配置数据库的信息,配置了6个数据库。
- sharding.jdbc.config.sharding.master-slave-rules.ds_0.master-data-source-name配置的是ds_0区的的主库名称,同理ds_1。
- sharding.jdbc.config.sharding.master-slave-rules.ds_0.slave-data-source-names配置的是ds_0区的的从库名称,同理ds_1。
- sharding.jdbc.config.sharding.default-database-strategy.inline.sharding-column配置的分库的字段,本案例是根据id进行分。
- sharding.jdbc.config.sharding.default-database-strategy.inline.algorithm-expression配置的分库的逻辑,根据id%2进行分。
- sharding.jdbc.config.sharding.tables.user.actual-data-nodes配置的是user表在真实数据库中的位置,ds_->{0..1}.user_->{0..4}表示
数据在ds_0和ds_1中的user_0、user_1、user_2、user_3、user_4中。 - sharding.jdbc.config.sharding.tables.user.table-strategy.inline.sharding-column,配置user表数据切分的字段
- sharding.jdbc.config.sharding.tables.user.table-strategy.inline.algorithm-expression=user_$->{id % 5},配置user表数据切分的策略。
- sharding.jdbc.config.sharding.tables.user.key-generator-column-name=id 自动生成id。
然后在Spring Boot启动类的注解@SpringBootApplication,加上exclude={DataSourceAutoConfiguration.class},代码如下:
@SpringBootApplication(exclude={DataSourceAutoConfiguration.class})
@EnableConfigurationProperties
public class ShardingJdbcDbMsTblApplication {
public static void main(String[] args) {
SpringApplication.run(ShardingJdbcDbMsTblApplication.class, args);
}
}
测试
测试同上一篇文章,这里不在重复。
源码下载
https://github.com/forezp/SpringBootLearning/tree/master/sharding-jdbc-example/sharding-jdbc-db-ms-tbl
资料参考
https://github.com/apache/incubator-shardingsphere-example/releases/tag/3.1.0.M1
https://shardingsphere.apache.org/document/current/cn/overview/
https://github.com/apache/incubator-shardingsphere
看完两件小事
如果你觉得这篇文章对你挺有启发,我想请你帮我两个小忙:
- 把这篇文章分享给你的朋友 / 交流群,让更多的人看到,一起进步,一起成长!
- 关注公众号 「方志朋」,公众号后台回复「666」 免费领取我精心整理的进阶资源教程
本文著作权归作者所有,如若转载,请注明出处
转载请注明:文章转载自「 Java极客技术学习 」https://www.javajike.com