上下文
我正在尝试使用JPA Repository开发使用Spring Boot的批处理服务。使用两个不同的数据源，我想要在内存数据库中创建与批处理相关的表，这样它就不会污染我的业务数据库。
关于网络上的多个主题，我想出了我的两个数据源的配置：
@Configuration
public class DataSourceConfiguration {
@Bean(name = "mainDataSource")
@Primary
@ConfigurationProperties(prefix="spring.datasource")
public DataSource mainDataSource(){
return DataSourceBuilder.create().build();
}
@Bean(name = "batchDataSource")
public DataSource batchDataSource( @Value("${batch.datasource.url}") String url ){
return DataSourceBuilder.create().url( url ).build();
}
}
第一个是mainDataSource，它使用默认的Spring数据库配置。 batchDataSource定义了一个嵌入式HSQL数据库，我希望在其中创建批处理和步骤表。
# DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.url=jdbc:mariadb://localhost:3306/batch_poc
spring.datasource.username=root
spring.datasource.password=
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
spring.datasource.max-age=10000
spring.datasource.initialize=false
# JPA (JpaBaseConfiguration, HibernateJpaAutoConfiguration)
spring.jpa.generate-ddl=false
spring.jpa.show-sql=true
spring.jpa.database=MYSQL
# SPRING BATCH (BatchDatabaseInitializer)
spring.batch.initializer.enabled=false
# ----------------------------------------
# PROJECT SPECIFIC PROPERTIES
# ----------------------------------------
# BATCH DATASOURCE
batch.datasource.url=jdbc:hsqldb:file:C:/tmp/hsqldb/batchdb
这是我的批量配置：
@Configuration
@EnableBatchProcessing
public class BatchConfiguration {
private static final Logger LOG = Logger.getLogger( BatchConfiguration.class );
@Bean
public BatchConfigurer configurer(){
return new CustomBatchConfigurer();
}
@Bean
public Job importElementsJob( JobBuilderFactory jobs, Step step1 ){
return jobs.get("importElementsJob")
.incrementer( new RunIdIncrementer() )
.flow( step1 )
.end()
.build();
}
@Bean
public Step step1( StepBuilderFactory stepBuilderFactory, ItemReader<InputElement> reader,
ItemWriter<List<Entity>> writer, ItemProcessor<InputElement, List<Entity>> processor ){
return stepBuilderFactory.get("step1")
.<InputElement, List<Entity>> chunk(100)
.reader( reader )
.processor( processor )
.writer( writer )
.build();
}
@Bean
public ItemReader<InputElement> reader() throws IOException {
return new CustomItemReader();
}
@Bean
public ItemProcessor<InputElement, List<Entity>> processor(){
return new CutsomItemProcessor();
}
@Bean
public ItemWriter<List<Entity>> writer(){
return new CustomItemWriter();
}
}
BatchConfigurer，使用内存数据库：
public class CustomBatchConfigurer extends DefaultBatchConfigurer {
@Override
@Autowired
public void setDataSource( @Qualifier("batchDataSource") DataSource dataSource) {
super.setDataSource(dataSource);
}
}
最后，我的作家：
public class CustomItemWriter implements ItemWriter<List<Entity>> {
private static final Logger LOG = Logger.getLogger( EntityWriter.class );
@Autowired
private EntityRepository entityRepository;
@Override
public void write(List<? extends List<Entity>> items)
throws Exception {
if( items != null && !items.isEmpty() ){
for( List<Entity> entities : items ){
for( Entity entity : entities ){
Entity fromDb = entityRepository.findById( entity.getId() );
// Insert
if( fromDb == null ){
entityRepository.save( entity );
}
// Update
else {
// TODO : entityManager.merge()
}
}
}
}
}
}
EntityRepository接口扩展了JpaRepository。
问题
当我以这种方式分离数据源时，当我调用存储库的save方法时没有任何反应。我在日志中看到来自findById（）调用的select查询。但没有什么可以拯救。我的输出数据库最后是空的。
当我回到一个独特的数据源配置（删除配置文件bean并让Spring Boot单独管理数据源）时，插入查询工作正常。
也许主数据源配置不足以让JPA正确执行插入。但缺少什么？
我终于解决了在Spring类BasicBatchConfigurer的基础上实现我自己的BatchConfigurer的问题，并强制使用基于Map的jobRepository和jobExplorer。没有更多的自定义数据源配置，只有一个我让Spring Boot管理的数据源：这样更容易。
我的自定义BatchConfigurer：
public class CustomBatchConfigurer implements BatchConfigurer {
private static final Logger LOG = Logger.getLogger( CustomBatchConfigurer.class );
private final EntityManagerFactory entityManagerFactory;
private PlatformTransactionManager transactionManager;
private JobRepository jobRepository;
private JobLauncher jobLauncher;
private JobExplorer jobExplorer;
/**
* Create a new {@link CustomBatchConfigurer} instance.
* @param entityManagerFactory the entity manager factory
*/
public CustomBatchConfigurer( EntityManagerFactory entityManagerFactory ) {
this.entityManagerFactory = entityManagerFactory;
}
@Override
public JobRepository getJobRepository() {
return this.jobRepository;
}
@Override
public PlatformTransactionManager getTransactionManager() {
return this.transactionManager;
}
@Override
public JobLauncher getJobLauncher() {
return this.jobLauncher;
}
@Override
public JobExplorer getJobExplorer() throws Exception {
return this.jobExplorer;
}
@PostConstruct
public void initialize() {
try {
// transactionManager:
LOG.info("Forcing the use of a JPA transactionManager");
if( this.entityManagerFactory == null ){
throw new Exception("Unable to initialize batch configurer : entityManagerFactory must not be null");
}
this.transactionManager = new JpaTransactionManager( this.entityManagerFactory );
// jobRepository:
LOG.info("Forcing the use of a Map based JobRepository");
MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean( this.transactionManager );
jobRepositoryFactory.afterPropertiesSet();
this.jobRepository = jobRepositoryFactory.getObject();
// jobLauncher:
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(getJobRepository());
jobLauncher.afterPropertiesSet();
this.jobLauncher = jobLauncher;
// jobExplorer:
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
jobExplorerFactory.afterPropertiesSet();
this.jobExplorer = jobExplorerFactory.getObject();
}
catch (Exception ex) {
throw new IllegalStateException("Unable to initialize Spring Batch", ex);
}
}
}
我的配置类现在看起来像这样：
@Configuration
@EnableBatchProcessing
public class BatchConfiguration {
@Bean
public BatchConfigurer configurer( EntityManagerFactory entityManagerFactory ){
return new CustomBatchConfigurer( entityManagerFactory );
}
[...]
}
我的属性文件：
# DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.url=jdbc:mariadb://localhost:3306/inotr_poc
spring.datasource.username=root
spring.datasource.password=root
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
spring.datasource.max-age=10000
spring.datasource.initialize=true
# JPA (JpaBaseConfiguration, HibernateJpaAutoConfiguration)
spring.jpa.generate-ddl=false
spring.jpa.show-sql=true
spring.jpa.database=MYSQL
# SPRING BATCH (BatchDatabaseInitializer)
spring.batch.initializer.enabled=false
感谢上面的帖子！在过去的几天里，我一直在努力让我的Spring Boot with Batch应用程序与基于内存映射的作业存储库和无资源的事务管理器一起工作。 （我不能让Spring Batch使用我的应用程序数据源用于批处理元数据表，因为我没有DDL访问权限来创建BATCH_表）最后在查看上面的帖子后得到了以下配置并且它工作得很好！ ！
public class CustomBatchConfigurer implements BatchConfigurer {
private static final Logger LOG = LoggerFactory.getLogger(CustomBatchConfigurer.class);
// private final EntityManagerFactory entityManagerFactory;
private PlatformTransactionManager transactionManager;
private JobRepository jobRepository;
private JobLauncher jobLauncher;
private JobExplorer jobExplorer;
/**
* Create a new {@link CustomBatchConfigurer} instance.
* @param entityManagerFactory the entity manager factory
public CustomBatchConfigurer( EntityManagerFactory entityManagerFactory ) {
this.entityManagerFactory = entityManagerFactory;
}
*/
@Override
public JobRepository getJobRepository() {
return this.jobRepository;
}
@Override
public PlatformTransactionManager getTransactionManager() {
return this.transactionManager;
}
@Override
public JobLauncher getJobLauncher() {
return this.jobLauncher;
}
@Override
public JobExplorer getJobExplorer() throws Exception {
return this.jobExplorer;
}
@PostConstruct
public void initialize() {
try {
// transactionManager:
LOG.info("Forcing the use of a Resourceless transactionManager");
this.transactionManager = new ResourcelessTransactionManager();
// jobRepository:
LOG.info("Forcing the use of a Map based JobRepository");
MapJobRepositoryFactoryBean jobRepositoryFactory = new MapJobRepositoryFactoryBean( this.transactionManager );
jobRepositoryFactory.afterPropertiesSet();
this.jobRepository = jobRepositoryFactory.getObject();
// jobLauncher:
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(getJobRepository());
jobLauncher.afterPropertiesSet();
this.jobLauncher = jobLauncher;
// jobExplorer:
MapJobExplorerFactoryBean jobExplorerFactory = new MapJobExplorerFactoryBean(jobRepositoryFactory);
jobExplorerFactory.afterPropertiesSet();
this.jobExplorer = jobExplorerFactory.getObject();
}
catch (Exception ex) {
throw new IllegalStateException("Unable to initialize Spring Batch", ex);
}
}
}
以下是我在作业配置类中添加的bean
@Bean
public BatchConfigurer configurer(){
return new CustomBatchConfigurer();
}
Eria的回答有效！但是，我已将其修改为使用：
org.springframework.batch.support.transaction.ResourcelessTransactionManager
来自Custom BatchConfigurer：
@PostConstruct
public void initialize() {
try {
// transactionManager:
LOGGER.info("Forcing the use of ResourcelessTransactionManager for batch db");
this.transactionManager = new ResourcelessTransactionManager();
//the rest of the code follows...
}