likes
comments
collection
share

小白学习封装多数据源starter: dynamic-datasource-spring-boot-starter

作者站长头像
站长
· 阅读数 25

前言

springboot业务量太大的时候，在一个数据源同时进行读写操作的时候，会增加数据库的压力。为减轻单个数据库的压力，使用主从数据源，读写分离。

方案

方案一

就像配置多个数据源那样（见博文spring boot学习6之mybatis+PageHelper分页插件+jta多数据源事务整合），将dao都分别放到不通的包下，指明哪个包下dao接口或配置文件走哪个数据库，service层程序员决定走主库还是从库。

缺点：相同的dao接口和配置文件要复制多份到不同包路径下，不易维护和扩展。

方案二

使用AbstractRoutingDataSource+aop+annotation在dao层决定数据源。

缺点：不支持事务。因为事务在service层开启时，就必须拿到数据源了。

方案三

使用AbstractRoutingDataSource+aop+annotation在service层决定数据源，可以支持事务.

缺点：类内部方法通过this.xx()方式相互调用时，aop不会进行拦截，需进行特殊处理。

搭建

一、新建一个空的springboot项目

引入依赖，pom.xml内容如下：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-parent</artifactId>
       <version>2.6.12</version>
       <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <groupId>io.dynamic</groupId>
    <artifactId>dynamic-datasource-spring-boot-starter</artifactId>
    <version>2.0.0</version>

    <name>dynamic-datasource-spring-boot-starter</name>
    <description>Demo project for Spring Boot</description>

    <dependencies>
       <dependency>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter</artifactId>
       </dependency>

       <dependency>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-autoconfigure</artifactId>
       </dependency>

       <dependency>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-configuration-processor</artifactId>
          <optional>true</optional>
       </dependency>

       <dependency>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter-aop</artifactId>
       </dependency>

       <dependency>
          <groupId>org.mybatis.spring.boot</groupId>
          <artifactId>mybatis-spring-boot-starter</artifactId>
          <version>2.2.2</version>
       </dependency>

       <dependency>
          <groupId>mysql</groupId>
          <artifactId>mysql-connector-java</artifactId>
          <version>5.1.6</version>
          <scope>runtime</scope>
       </dependency>

       <!-- 添加多个连接池实现的依赖 -->
       <dependency>
          <groupId>com.zaxxer</groupId>
          <artifactId>HikariCP</artifactId>
       </dependency>
       <dependency>
          <groupId>org.apache.commons</groupId>
          <artifactId>commons-dbcp2</artifactId>
       </dependency>
       <dependency>
          <groupId>org.apache.tomcat</groupId>
          <artifactId>tomcat-jdbc</artifactId>
       </dependency>
       <dependency>
          <groupId>com.alibaba</groupId>
          <artifactId>druid</artifactId>
          <version>1.2.8</version>
       </dependency>

       <dependency>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter-test</artifactId>
          <scope>test</scope>
       </dependency>

       <dependency>
          <groupId>org.projectlombok</groupId>
          <artifactId>lombok</artifactId>
          <scope>provided</scope>
       </dependency>
    </dependencies>

    <distributionManagement>
       <!-- 这里可以配置多个比如snapshots快照仓库 这里只配置一个 -->
       <repository>
          <id>maven-releases</id>
          <name>maven的发布仓库id与setting.xml中在server节点id保持一致</name>
          <url>http://192.168.3.5:18088/repository/maven-releases/</url>
       </repository>
       <!-- 这里的 id 要和上面的 server 的 id 保持一致,name 随意写-->
       <snapshotRepository>
          <id>maven-snapshots</id>
          <name>Snapshot Repository</name>
          <url>http://192.168.3.5:18088/repository/maven-snapshots/</url>
       </snapshotRepository>
    </distributionManagement>
</project>

二. 新建io.dynamic.config和io.dynamic.core包

  1. 在config包下新建DataSourceProperties、DataSourceConfigure、MybatisMultiDatasourceAutoConfigure三个java文件
  • DataSourceProperties内容如下：
package io.dynamic.config;

import io.dynamic.core.constant.DatasourceConstant;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.ConstructorBinding;

import java.util.HashMap;
import java.util.Map;

@ConfigurationProperties(prefix = "spring.datasource")
public class DataSourceProperties {

    private Map<String, DataSourceConfig> dynamic = new HashMap<>();

    private String master = DatasourceConstant.default_master_datasource;

    public Map<String, DataSourceConfig> getDynamic( ) {
        return dynamic;
    }

    public void setDynamic(Map<String, DataSourceConfig> dynamic) {
        this.dynamic = dynamic;
    }

    public String getMaster( ) {
        return master;
    }

    public void setMaster(String master) {
        this.master = master;
    }

    public static class DataSourceConfig {

        private String type = "com.zaxxer.hikari.HikariDataSource";

        private String url;
        private String username;
        private String password;
        private String driverClassName;

        public String getType( ) {
            return type;
        }

        public void setType(String type) {
            this.type = type;
        }

        public String getUrl() {
            return url;
        }

        public void setUrl(String url) {
            this.url = url;
        }

        public String getUsername() {
            return username;
        }

        public void setUsername(String username) {
            this.username = username;
        }

        public String getPassword() {
            return password;
        }

        public void setPassword(String password) {
            this.password = password;
        }

        public String getDriverClassName() {
            return driverClassName;
        }

        public void setDriverClassName(String driverClassName) {
            this.driverClassName = driverClassName;
        }
    }
}
  • DataSourceConfigure内容如下：

MapperScanner注解的basePackages属性支持占位符动态配置: "${spring.datasource.mapper}"

package io.dynamic.config;

import com.alibaba.druid.pool.DruidDataSource;
import com.zaxxer.hikari.HikariDataSource;
import io.dynamic.core.annotation.MapperScanner;
import io.dynamic.core.datasource.DynamicRoutingDataSource;
import org.apache.commons.dbcp2.BasicDataSource;
import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.SqlSessionFactoryBean;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;
import org.springframework.util.StringUtils;

import javax.sql.DataSource;
import java.util.HashMap;
import java.util.Map;

@Configuration
@EnableConfigurationProperties(DataSourceProperties.class)
@MapperScanner(basePackages = { "${spring.datasource.mapper}" })
public class DataSourceConfigure {

    private final DataSourceProperties properties;

    public DataSourceConfigure(DataSourceProperties dataSourceProperties) {
        this.properties = dataSourceProperties;
    }

    @Bean
    public DataSource dataSource() {
        Map<Object, Object> dataSourceMap = new HashMap<>();
        properties.getDynamic().forEach((key, value) -> {
            DataSource dataSource = createDataSource(value);
            dataSourceMap.put(key, dataSource);
        });

        AbstractRoutingDataSource routingDataSource = new DynamicRoutingDataSource();
        routingDataSource.setDefaultTargetDataSource(dataSourceMap.get(properties.getMaster()));
        routingDataSource.setTargetDataSources(dataSourceMap);

        routingDataSource.afterPropertiesSet();

        return routingDataSource;
    }

    @Bean
    public SqlSessionFactory sqlSessionFactory(DataSource dynamicDataSource) throws Exception {
        SqlSessionFactoryBean sqlSessionFactoryBean = new SqlSessionFactoryBean();
        sqlSessionFactoryBean.setDataSource(dynamicDataSource);
        sqlSessionFactoryBean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath*:mapper/*.xml"));
        return sqlSessionFactoryBean.getObject();
    }

    @Bean
    public DataSourceTransactionManager transactionManager(DataSource dynamicDataSource) {
        return new DataSourceTransactionManager(dynamicDataSource);
    }

    private DataSource createDataSource(DataSourceProperties.DataSourceConfig dataSourceConfig) {
        if (!StringUtils.hasText(dataSourceConfig.getType())) {
            throw new IllegalArgumentException("DataSource type must be specified");
        }

        switch (dataSourceConfig.getType()) {
            case "com.zaxxer.hikari.HikariDataSource":
                HikariDataSource hikariDataSource = new HikariDataSource();
                hikariDataSource.setDriverClassName(dataSourceConfig.getDriverClassName());
                hikariDataSource.setJdbcUrl(dataSourceConfig.getUrl());
                hikariDataSource.setUsername(dataSourceConfig.getUsername());
                hikariDataSource.setPassword(dataSourceConfig.getPassword());

                return hikariDataSource;

            case "org.apache.commons.dbcp2.BasicDataSource":
                BasicDataSource dbcpDataSource = new BasicDataSource();
                dbcpDataSource.setDriverClassName(dataSourceConfig.getDriverClassName());
                dbcpDataSource.setUrl(dataSourceConfig.getUrl());
                dbcpDataSource.setUsername(dataSourceConfig.getUsername());
                dbcpDataSource.setPassword(dataSourceConfig.getPassword());
                return dbcpDataSource;

            case "org.apache.tomcat.jdbc.pool.DataSource":
                org.apache.tomcat.jdbc.pool.DataSource tomcatDataSource = new org.apache.tomcat.jdbc.pool.DataSource();
                tomcatDataSource.setDriverClassName(dataSourceConfig.getDriverClassName());
                tomcatDataSource.setUrl(dataSourceConfig.getUrl());
                tomcatDataSource.setUsername(dataSourceConfig.getUsername());
                tomcatDataSource.setPassword(dataSourceConfig.getPassword());
                return tomcatDataSource;

            case "com.alibaba.druid.pool.DruidDataSource":
                DruidDataSource druidDataSource = new DruidDataSource();
                druidDataSource.setDriverClassName(dataSourceConfig.getDriverClassName());
                druidDataSource.setUrl(dataSourceConfig.getUrl());
                druidDataSource.setUsername(dataSourceConfig.getUsername());
                druidDataSource.setPassword(dataSourceConfig.getPassword());
                return druidDataSource;

            default:
                throw new IllegalArgumentException("Unsupported DataSource type: " + dataSourceConfig.getType());
        }
    }
}

不能使用MapperScannerConfigurer代替@MappeScan注解。代码如下：

@Bean
public MapperScannerConfigurer mapperScannerConfigurer() {
   MapperScannerConfigurer mapperScannerConfigurer = new MapperScannerConfigurer();
   mapperScannerConfigurer.setBasePackage("io.stall.mapper");
   mapperScannerConfigurer.setSqlSessionFactoryBeanName("sqlSessionFactory");
   
   return mapperScannerConfigurer;
}

也不支持BeanDefinitionRegistryPostProcessor动态配置，代码如下：

@Bean
public static BeanDefinitionRegistryPostProcessor mapperScannerConfigurer(DynamicDataSourceProperties properties) {
       return new BeanDefinitionRegistryPostProcessor() {
           @Override
           public void postProcessBeanDefinitionRegistry(BeanDefinitionRegistry registry) {
               properties.getDynamic().forEach((key, value) -> {
                   String beanName = key + "MapperScannerConfigurer";
                   BeanDefinitionBuilder builder = >BeanDefinitionBuilder.genericBeanDefinition(MapperScannerConfigurer.class);
                   builder.addPropertyValue("basePackage", value.getMapperPackage());
                   builder.addPropertyValue("sqlSessionFactoryBeanName", key + "SqlSessionFactory");
                   registry.registerBeanDefinition(beanName, builder.getBeanDefinition());
               });
           }

           @Override
           public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) {
               // No implementation needed
           }
       };
}
  • 这个Bean和BeanDefinitionRegistryPostProcessor会第一个执行，导致DataSourceProperties注入为null。
  • 我的解决办法是重写了一个注解代替 @MappeScan。新注解支持动态配置($占位符)，从yml文件中读取配置。
  • MybatisMultiDatasourceAutoConfigure内容如下：
package io.dynamic.config;

import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;

@Configuration
@Import({DataSourceConfigure.class})
public class MybatisMultiDatasourceAutoConfigure {
}
  1. 在core包下新建io.dynamic.core.registrar.MapperScannerRegistrar、io.dynamic.core.annotation.MapperScanner
  • MapperScannerRegistrar内容如下：
package io.dynamic.core.registrar;

import io.dynamic.core.annotation.MapperScanner;
import org.mybatis.spring.mapper.ClassPathMapperScanner;
import org.mybatis.spring.mapper.MapperFactoryBean;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.BeanUtils;
import org.springframework.beans.factory.support.BeanDefinitionRegistry;
import org.springframework.beans.factory.support.BeanNameGenerator;
import org.springframework.context.EnvironmentAware;
import org.springframework.context.ResourceLoaderAware;
import org.springframework.context.annotation.ImportBeanDefinitionRegistrar;
import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;
import org.springframework.core.annotation.AnnotationAttributes;
import org.springframework.core.env.Environment;
import org.springframework.core.io.ResourceLoader;
import org.springframework.core.type.AnnotationMetadata;
import org.springframework.util.ClassUtils;
import org.springframework.util.StringUtils;

import java.lang.annotation.Annotation;
import java.util.*;
import java.util.stream.Collectors;

public class MapperScannerRegistrar implements ImportBeanDefinitionRegistrar, ResourceLoaderAware,EnvironmentAware {

    private Environment environment;

    private ResourceLoader resourceLoader;

    private  static  final Logger logger = LoggerFactory.getLogger(MapperScannerRegistrar. class);

    @Override
    public void setEnvironment(Environment environment) {
        this.environment = environment;
    }

    @Override
    public void setResourceLoader(ResourceLoader resourceLoader) {
        this.resourceLoader = resourceLoader;
    }

    @Override
    public void registerBeanDefinitions(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry) {

        AnnotationAttributes mapperScanAttrs = AnnotationAttributes.fromMap(importingClassMetadata.getAnnotationAttributes(MapperScanner.class.getName()));
        if (mapperScanAttrs != null) {
            this.registerBeanDefinitions(mapperScanAttrs, registry);
        }
    }

    void registerBeanDefinitions(AnnotationAttributes annoAttrs, BeanDefinitionRegistry registry) {
        ClassPathMapperScanner scanner = new ClassPathMapperScanner(registry);
        Optional var10000 = Optional.ofNullable(this.resourceLoader);
        Objects.requireNonNull(scanner);
        Class<? extends Annotation> annotationClass = annoAttrs.getClass("annotationClass");
        if (!Annotation.class.equals(annotationClass)) {
            scanner.setAnnotationClass(annotationClass);
        }

        Class<?> markerInterface = annoAttrs.getClass("markerInterface");
        if (!Class.class.equals(markerInterface)) {
            scanner.setMarkerInterface(markerInterface);
        }

        Class<? extends BeanNameGenerator> generatorClass = annoAttrs.getClass("nameGenerator");
        if (!BeanNameGenerator.class.equals(generatorClass)) {
            scanner.setBeanNameGenerator((BeanNameGenerator) BeanUtils.instantiateClass(generatorClass));
        }

        Class<? extends MapperFactoryBean> mapperFactoryBeanClass = annoAttrs.getClass("factoryBean");
        if (!MapperFactoryBean.class.equals(mapperFactoryBeanClass)) {
            scanner.setMapperFactoryBeanClass(mapperFactoryBeanClass);
        }

        scanner.setSqlSessionTemplateBeanName(annoAttrs.getString("sqlSessionTemplateRef"));
        scanner.setSqlSessionFactoryBeanName(annoAttrs.getString("sqlSessionFactoryRef"));
        List<String> basePackages = new ArrayList<String>();
        basePackages.addAll(Arrays.stream(annoAttrs.getStringArray("value")).filter(StringUtils::hasText).collect(Collectors.toList()));
        for (String pkg : annoAttrs.getStringArray("basePackages")) {
            if (StringUtils.hasText(pkg)) {
                String value = parsePlaceHolder(pkg);
                if(StringUtils.hasText(value)){
                    List<String> valueList = Arrays.asList(value.split(","));
                    for(String base : valueList){
                        basePackages.add(base);
                    }
                }
            }
        }
        basePackages.addAll(Arrays.stream(annoAttrs.getClassArray("basePackageClasses")).map(ClassUtils::getPackageName).collect(Collectors.toList()));
        scanner.registerFilters();
        scanner.doScan(StringUtils.toStringArray(basePackages));
    }

    private String parsePlaceHolder(String pro) {
        if (StringUtils.hasText(pro) && pro.contains(PropertySourcesPlaceholderConfigurer.DEFAULT_PLACEHOLDER_PREFIX)) {
            String value = environment.getProperty(pro.substring(2, pro.length() - 1));

            if (logger.isDebugEnabled()) {
                logger.debug("find placeholder value " + value + " for key " + pro);
            }

            if (null == value) {
                throw new IllegalArgumentException("property " + pro + " can not find!!!");
            }
            return value;
        }
        return pro;
    }
}
  • MapperScanner内容如下：
package io.dynamic.core.annotation;

import io.dynamic.core.registrar.MapperScannerRegistrar;
import org.mybatis.spring.mapper.MapperFactoryBean;
import org.springframework.beans.factory.support.BeanNameGenerator;
import org.springframework.context.annotation.Import;

import java.lang.annotation.*;

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
@Documented
@Import({MapperScannerRegistrar.class})
public @interface MapperScanner {
    String[] value() default {};

    String[] basePackages() default {};

    Class<?>[] basePackageClasses() default {};

    Class<? extends BeanNameGenerator> nameGenerator() default BeanNameGenerator.class;

    Class<? extends Annotation> annotationClass() default Annotation.class;

    Class<?> markerInterface() default Class.class;

    String sqlSessionTemplateRef() default "";

    String sqlSessionFactoryRef() default "";

    Class<? extends MapperFactoryBean> factoryBean() default MapperFactoryBean.class;
}
  1. 新建io.dynamic.core.datasource.DynamicRoutingDataSource

该类用于动态切换数据源

package io.dynamic.core.datasource;

import org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource;

public class DynamicRoutingDataSource extends AbstractRoutingDataSource {
    private static final ThreadLocal<String> CONTEXT_HOLDER = new ThreadLocal<>();

    public static void setDataSourceKey(String key) {
        CONTEXT_HOLDER.set(key);
    }

    public static void clearDataSourceKey() {
        CONTEXT_HOLDER.remove();
    }

    @Override
    protected Object determineCurrentLookupKey() {
        return CONTEXT_HOLDER.get();
    }
}

yml中配置多数据源：

spring:
  datasource:
    dynamic:
      primary:
        type: com.zaxxer.hikari.HikariDataSource
        driver-class-name: com.mysql.cj.jdbc.Driver
        url: jdbc:mysql://localhost:3306/primary_db
        username: root
        password: root
        mapperPackage: com.example.primary.mapper
        typeAliasesPackage: com.example.primary.domain
      secondary:
        type: org.apache.commons.dbcp2.BasicDataSource
        driver-class-name: com.mysql.cj.jdbc.Driver
        url: jdbc:mysql://localhost:3306/secondary_db
        username: root
        password: root
        mapperPackage: com.example.secondary.mapper
        typeAliasesPackage: com.example.secondary.domain
      tertiary:
        type: com.alibaba.druid.pool.DruidDataSource
        driver-class-name: com.mysql.cj.jdbc.Driver
        url: jdbc:mysql://localhost:3306/tertiary_db
        username: root
        password: root
        mapperPackage: com.example.tertiary.mapper
        typeAliasesPackage: com.example.tertiary.domain

调用示例如下：

package com.example.service;

import com.example.dynamicdatasource.DynamicRoutingDataSource;
import com.example.primary.mapper.UserMapperPrimary;
import com.example.secondary.mapper.UserMapperSecondary;
import com.example.tertiary.mapper.UserMapperTertiary;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class UserService {

    @Autowired
    private UserMapperPrimary userMapperPrimary;

    @Autowired
    private UserMapperSecondary userMapperSecondary;

    @Autowired
    private UserMapperTertiary userMapperTertiary;

    @Transactional
    public void queryFromDefaultDataSource() {
        // 使用默认数据源（未设置数据源，会使用 defaultDataSource）
        userMapperPrimary.findAll();
    }

    @Transactional
    public void queryFromPrimaryDataSource() {
        DynamicRoutingDataSource.setDataSourceKey("primary");
        // 查询主数据源
        userMapperPrimary.findAll();
        DynamicRoutingDataSource.clearDataSourceKey();
    }

    @Transactional
    public void queryFromSecondaryDataSource() {
        DynamicRoutingDataSource.setDataSourceKey("secondary");
        // 查询第二数据源
        userMapperSecondary.findAll();
        DynamicRoutingDataSource.clearDataSourceKey();
    }

    @Transactional
    public void queryFromTertiaryDataSource() {
        DynamicRoutingDataSource.setDataSourceKey("tertiary");
        // 查询第三数据源
        userMapperTertiary.findAll();
        DynamicRoutingDataSource.clearDataSourceKey();
    }
}

primary、secondary是配置的key。

但这样太麻烦，可以使用注解。 4. 新建io.dynamic.core.annotation.TargetDataSource、io.dynamic.core.aop.DataSourceRoutingAspect

  • TargetDataSource内容如下：
package io.dynamic.core.annotation;

import java.lang.annotation.*;

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface TargetDataSource {
    String value();
}
  • DataSourceRoutingAspect内容如下：
package io.dynamic.core.aop;

import io.dynamic.core.annotation.TargetDataSource;
import io.dynamic.core.datasource.DynamicRoutingDataSource;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.springframework.stereotype.Component;

@Aspect
@Component
public class DataSourceRoutingAspect {

    @Around("@annotation(targetDataSource)")
    public Object routingWithDataSource(ProceedingJoinPoint joinPoint, TargetDataSource targetDataSource) throws Throwable {
        String key = targetDataSource.value();

        DynamicRoutingDataSource.setDataSourceKey(key);

        return joinPoint.proceed();
    }
}

三、在resources/META-INF新建spring.factories

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
  io.dynamic.config.MybatisMultiDatasourceAutoConfigure

发布

进入终端控制台,执行如下命令：

mvn deploy

引入依赖

<dependency>
    <groupId>io.dynamic</groupId>
    <artifactId>dynamic-datasource-spring-boot-starter</artifactId>
    <version>2.0.0</version>
</dependency>

参考

转载自:https://juejin.cn/post/7387662864122331174
评论
请登录