Here we will see how we can configure different datasource for application and batch. By default, Spring batch stores the job details and execution details in database. If separate data source is not configured for spring batch then it will use the available data source in your application if configured and create batch related tables there. Which may be the unwanted burden on application database and we would like to configure separate database for spring batch.
To overcome this situation we will configure the different datasource for spring batch using in-memory database, since we don't want to store batch job details permanently.
Other thing is the configuration of spring cloud task in case of multiple datasource and it must point to the same data source which is pointed by spring batch.
In below sections, we will se how to configure application, batch and cloud task related data sources.
Spring batch job with spring cloud data flow server
How to setup spring cloud data flow server
To overcome this situation we will configure the different datasource for spring batch using in-memory database, since we don't want to store batch job details permanently.
Other thing is the configuration of spring cloud task in case of multiple datasource and it must point to the same data source which is pointed by spring batch.
In below sections, we will se how to configure application, batch and cloud task related data sources.
Application Data Source
Define the data source in application properties or yml configuration to configure the database.spring.datasource.url=jdbc:h2:tcp://localhost:19092/mem:app-data spring.datasource.jdbcUrl=jdbc:h2:tcp://localhost:19092/mem:app-data spring.datasource.driverClassName=org.h2.Driver spring.datasource.username=sa spring.datasource.password= spring.jpa.database-platform=org.hibernate.dialect.H2DialectDefine the spring beans for data source, entity manager factory and transaction manager. We also need to configure JPA repository where we tell the location of our repository classes and domain objects for business data manipulation.
@Configuration @EnableJpaRepositories( entityManagerFactoryRef = "appEntityManagerFactory", transactionManagerRef = "appTransactionManager", basePackages = "com.ttj.app.repository" ) @EnableTransactionManagement public class AppDataSourceConfig { @Bean @ConfigurationProperties(prefix = "spring.datasource") public DataSource appDataSource(){ return DataSourceBuilder.create().build(); } @Bean(name = "appEntityManagerFactory") public LocalContainerEntityManagerFactoryBean appEntityManagerFactory(EntityManagerFactoryBuilder builder, @Qualifier("appDataSource") DataSource appDataSource){ return builder .dataSource(appDataSource) .packages("com.ttj.app.domain") .persistenceUnit("app") .build(); } @Bean(name = "appTransactionManager") public PlatformTransactionManager appTransactionManager(@Qualifier("appEntityManagerFactory") EntityManagerFactory appEntityManagerFactory) { return new JpaTransactionManager(appEntityManagerFactory); } }
Spring Batch Data Source
Configure below data source properties in application properties or yml configuration. Please note if you are going to use this job as cloud task to register with spring cloud data flow server then make sure it is pointing to same database which is used by data flow server.spring.batch.datasource.url=jdbc:h2:tcp://localhost:19092/mem:dataflow spring.batch.datasource.jdbcUrl=jdbc:h2:tcp://localhost:19092/mem:dataflow spring.batch.datasource.driverClassName=org.h2.Driver spring.batch.datasource.username=sa spring.batch.datasource.password=Define the spring beans for batch datasource and transaction manager. Please note that here we are making the datasource and transaction manager as primary beans as spring cloud task looks for some other bean name by default for data source and transaction manager and making it primary makes cloud task to use these beans.
@Configuration public class BatchDataSourceConfig { @Bean(name="batchDataSource") @Primary @ConfigurationProperties(prefix = "spring.batch.datasource") public DataSource batchDataSource(){ return DataSourceBuilder.create().build(); } @Primary @Bean(name = "batchEntityManagerFactory") public LocalContainerEntityManagerFactoryBean batchEntityManagerFactory(EntityManagerFactoryBuilder builder, @Qualifier("batchDataSource") DataSource batchDataSource){ return builder .dataSource(batchDataSource) .packages("com.ttj.batch.domain") .persistenceUnit("batch") .build(); } @Primary @Bean(name = "batchTransactionManager") public PlatformTransactionManager appTransactionManager(@Qualifier("batchEntityManagerFactory") EntityManagerFactory batchEntityManagerFactory) { return new JpaTransactionManager(batchEntityManagerFactory); } @Bean public BatchConfigurer configurer(@Qualifier("batchDataSource") DataSource batchDataSource) { return new DefaultBatchConfigurer(batchDataSource); } }
Spring Cloud Task Data Source
We need to use the same data source which is configured with spring batch and need to define the bean for TaskConfigurer as given below. Also by default it looks for transaction manager with the bean name as "transactionManager" and to use the batch transaction manager we need to annotate batch transaction manager with @Primary which we saw in previous section of spring batch data source configuration.@Configuration public class CloudTaskConfig { @Bean public TaskConfigurer taskConfigurer(@Qualifier("batchDataSource") DataSource batchDataSource){ return new DefaultTaskConfigurer(batchDataSource); } }Other posts you may like to explore:
Spring batch job with spring cloud data flow server
How to setup spring cloud data flow server
This is really nice post, I found and love this content. I will prefer this, thanks for sharing. Business Intelligence Data Analytics.
ReplyDeleteThanks for your kind words.
DeleteI followed you, but its not creating Spring Batch metadata tables automatically
ReplyDeleteexactly what i was looking for, thank you
ReplyDeleteI'm facing Caused by: org.hsqldb.HsqlException: user lacks privilege or object not found: BATCH_JOB_INSTANCE problem. What am I missing here.
ReplyDeleteThere could be several reasons like user for batch doesn't have permission in configured database or it is connecting to incorrect database in case multiple database configured in your application.
Delete
ReplyDeleteIt was such a good post. Visit Hack App Data Pro Apk Download. Thanks for sharing.
Thank you so much for imparting this excellent information to us. Anyone can benefit from this information. Therefore, kindly impart this kind of wisdom to everyone. Thanks. Information about.best pdf signature api
ReplyDeleteThe blog consistently posts new content, and it is a fascinating blog. Thank you for revealing.hire programmers in india
ReplyDeleteThis is a great article you've shared because it offers some great information that will be very helpful to me. I'm grateful that you said that. Work well done; continue.hire coldfusion developers
ReplyDeleteHey i read your blog and find it very helpful for me because i was looking for these type of content online.
ReplyDeleteReasons Why To Hire A Bodyguard in London, UK:| Spetsnaz Security International Fidel Matola
https://www.spetsnazsecurityinternational.co.uk/reasons-to-hire-a-london-bodyguard.html
https://www.spetsnazsecurityinternational.co.uk/reasons-to-hire-a-london-bodyguard.html