我只做了分表,其实分表和分库是一模一样的道理,连配置也是一样的。还有sharding jdbc改名字了,sharding 3.0版本是叫Sharding Sphere。
1.sharding jdbc简述
sharding jdbc是国人的项目,当当的项目,那时候就更新很频繁到1.x的版本,2.x就从当当内部抽出来了,用的人多了嘛,3.就改名Sharding Sphere。
http://shardingsphere.io
去看看官方文档就知道,他有很多策略,只能使用一种策略。我这里就用的标准策略。
还有你要注意既然分了表,那sharding也不是神仙,有很多sql是不支持了的。官方也有说自己也能想到。去重distinct就不支持
2.mysql建表
CREATE TABLE `test` ( `id` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL , `datadate` timestamp NULL DEFAULT NULL , PRIMARY KEY (`id`) ) CREATE TABLE `test_2017` ( `id` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL , `datadate` timestamp NULL DEFAULT NULL , PRIMARY KEY (`id`) ) CREATE TABLE `test_2018` ( `id` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL , `datadate` timestamp NULL DEFAULT NULL , PRIMARY KEY (`id`) )
3.pom.xml(既然是用了spring boot那就很简单了)
<dependency> <groupId>io.shardingsphere</groupId> <artifactId>sharding-jdbc-spring-boot-starter</artifactId> <version>3.0.0.M1</version> </dependency>
4.application.properties配置。sharding配置的数据源自动注入到mybatis,所以mybatis就不用配置数据源了。当然也可以配置用多数据源,毕竟分表的数据源sql有限制
#数据源 sharding.jdbc.datasource.names=ds sharding.jdbc.datasource.ds.type=com.alibaba.druid.pool.DruidDataSource sharding.jdbc.datasource.ds.driverClassName=com.mysql.jdbc.Driver sharding.jdbc.datasource.ds.url=jdbc:mysql://rm-wz99823k9gaz076cn0o.mysql.rds.aliyuncs.com/zee_bi?useUnicode=true&characterEncoding=UTF-8&useSSL=false sharding.jdbc.datasource.ds.username=root sharding.jdbc.datasource.ds.password=q1w2e3r4Q! sharding.jdbc.config.sharding.props.sql.show=true #用于单分片键的标准分片场景 sharding.jdbc.config.sharding.default-table-strategy.standard.sharding-column=datadate sharding.jdbc.config.sharding.default-table-strategy.standard.precise-algorithm-class-name=com.zee.sharding.ShardingAlgorithmYear sharding.jdbc.config.sharding.default-table-strategy.standard.range-algorithm-class-name=com.zee.sharding.ShardingAlgorithmYear #分表 sharding.jdbc.config.sharding.tables.test.actual-data-nodes=ds.test_2017,ds.test_2018
后面的四个配置,每个要分表的都需要配置,只不过如果策略一样(我这里都是根据同一个日期字段根据年份分)。那就直接使用默认的,要单独配置就要把default改成表名
5.java代码 实现PreciseShardingAlgorithm和RangeShardingAlgorithm接口,前者是基础策略,后面是实现between范围查询
public class ShardingAlgorithmYear implements PreciseShardingAlgorithm,RangeShardingAlgorithm { private final Logger logger = LoggerFactory.getLogger(this.getClass()); public ShardingAlgorithmYear(){ } @Override public String doSharding(Collection collection, PreciseShardingValue preciseShardingValue) { String tableNode=null; try { SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); Calendar calendar=Calendar.getInstance(); String dateStr=preciseShardingValue.getValue().toString(); Date date=format.parse(dateStr); calendar.setTime(date); String year=calendar.get(Calendar.YEAR)+""; for(Object obj:collection){ String oneNode=obj+""; if(oneNode.endsWith(year)){ tableNode=oneNode; break; } } } catch (ParseException e) { logger.error(e.getMessage(),e); } return tableNode; } @Override public Collection<String> doSharding(Collection collection, RangeShardingValue rangeShardingValue) { Collection<String> collect = new ArrayList<String>(); try { SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); Calendar calendar=Calendar.getInstance(); String dateUpperStr=rangeShardingValue.getValueRange().upperEndpoint().toString(); String dateLowerStr=rangeShardingValue.getValueRange().lowerEndpoint().toString(); Date dateUpper=format.parse(dateUpperStr); Date dateLower=format.parse(dateLowerStr); calendar.setTime(dateUpper); int yearUpper=calendar.get(Calendar.YEAR); calendar.setTime(dateLower); int yearLower=calendar.get(Calendar.YEAR); for(Object obj:collection){ String tableNoe=obj+""; for(int i=yearLower;i<=yearUpper;i++){ if(tableNoe.endsWith(i+"")){ collect.add(tableNoe); break; } } } } catch (ParseException e) { logger.error(e.getMessage(),e); } return collect; } }
道理很简单 就是根据你的规则找到要查的表,然后把表返回回去让他自动查询。
6.测试代码
Calendar calendar=Calendar.getInstance(); List<TestModel> testModelList=new ArrayList<>(); TestModel testModel1 = new TestModel(); testModel1.setDatadate(calendar.getTime()); testModel1.setShopId(11120); testModelList.add(testModel1); calendar.add(Calendar.YEAR, -1); TestModel testModel2 = new TestModel(); testModel2.setDatadate(calendar.getTime()); testModel2.setShopId(111190); testModelList.add(testModel2); testDao.batchInsert(testModelList); calendar.setTime(new Date()); calendar.add(Calendar.YEAR, 1); Date dateEnd=calendar.getTime(); calendar.add(Calendar.YEAR, -3); Date dateStart=calendar.getTime(); Wrapper wrapper=new EntityWrapper<TestModel>(); wrapper.between("datadate", dateStart, dateEnd); int a=testDao.selectCount(wrapper); List<TestModel> list1=testDao.selectList(wrapper);
遇到很多坑,还是要仔细看官方的文档,另外分库也分表一个道理。配置一样。