Skip to main content

Multiple data source with Spring boot, batch and cloud task

Here we will see how we can configure different datasource for application and batch. By default, Spring batch stores the job details and execution details in database. If separate data source is not configured for spring batch then it will use the available data source in your application if configured and create batch related tables there. Which may be the unwanted burden on application database and we would like to configure separate database for spring batch.
To overcome this situation we will configure the different datasource for spring batch using in-memory database, since we don't want to store batch job details permanently.
Other thing is the configuration of  spring cloud task in case of multiple datasource and it must point to the same data source which is pointed by spring batch.
In below sections, we will se how to configure application, batch and cloud task related data sources.

Application Data Source

Define the data source in application properties or yml configuration to configure the database.

Define the spring beans for data source, entity manager factory and transaction manager. We also need to configure JPA repository where we tell the location of our repository classes and domain objects for business data manipulation.
        entityManagerFactoryRef = "appEntityManagerFactory",
        transactionManagerRef = "appTransactionManager",
        basePackages = ""
public class AppDataSourceConfig {

    @ConfigurationProperties(prefix = "spring.datasource")
    public DataSource appDataSource(){
        return DataSourceBuilder.create().build();

    @Bean(name = "appEntityManagerFactory")
    public LocalContainerEntityManagerFactoryBean appEntityManagerFactory(EntityManagerFactoryBuilder builder,
            @Qualifier("appDataSource") DataSource appDataSource){

        return builder

    @Bean(name = "appTransactionManager")
    public PlatformTransactionManager appTransactionManager(@Qualifier("appEntityManagerFactory") EntityManagerFactory
                                                                        appEntityManagerFactory) {

        return new JpaTransactionManager(appEntityManagerFactory);

Spring Batch Data Source

Configure below data source properties in application properties or yml configuration. Please note if you are going to use this job as cloud task to register with spring cloud data flow server then make sure it is pointing to same database which is used by data flow server.
Define the spring beans for batch datasource and transaction manager. Please note that here we are making the datasource and transaction manager as primary beans as spring cloud task looks for some other bean name by default for data source and transaction manager and making it primary makes cloud task to use these beans.
public class BatchDataSourceConfig {

    @ConfigurationProperties(prefix = "spring.batch.datasource")
    public DataSource batchDataSource(){
        return DataSourceBuilder.create().build();

    @Bean(name = "batchEntityManagerFactory")
    public LocalContainerEntityManagerFactoryBean batchEntityManagerFactory(EntityManagerFactoryBuilder builder,
                                                                          @Qualifier("batchDataSource") DataSource batchDataSource){
        return builder

    @Bean(name = "batchTransactionManager")
    public PlatformTransactionManager appTransactionManager(@Qualifier("batchEntityManagerFactory") EntityManagerFactory
                                                                        batchEntityManagerFactory) {

        return new JpaTransactionManager(batchEntityManagerFactory);

    public BatchConfigurer configurer(@Qualifier("batchDataSource") DataSource batchDataSource) {

        return new DefaultBatchConfigurer(batchDataSource);

Spring Cloud Task Data Source

We need to use the same data source which is configured with spring batch and need to define the bean for TaskConfigurer as given below. Also by default it looks for transaction manager with the bean name as "transactionManager" and to use the batch transaction manager we need to annotate batch transaction manager with @Primary which we saw in previous section of spring batch data source configuration.
public class CloudTaskConfig {
    public TaskConfigurer taskConfigurer(@Qualifier("batchDataSource") DataSource batchDataSource){
        return new DefaultTaskConfigurer(batchDataSource);

Other posts you may like to explore:
Spring batch job with spring cloud data flow server
How to setup spring cloud data flow server


  1. This is really nice post, I found and love this content. I will prefer this, thanks for sharing. Business Intelligence Data Analytics.

  2. I followed you, but its not creating Spring Batch metadata tables automatically

  3. exactly what i was looking for, thank you

  4. I'm facing Caused by: org.hsqldb.HsqlException: user lacks privilege or object not found: BATCH_JOB_INSTANCE problem. What am I missing here.

    1. There could be several reasons like user for batch doesn't have permission in configured database or it is connecting to incorrect database in case multiple database configured in your application.


  5. It was such a good post. Visit Hack App Data Pro Apk Download. Thanks for sharing.


Post a Comment

Popular Posts

Setting up kerberos in Mac OS X

Kerberos in MAC OS X Kerberos authentication allows the computers in same domain network to authenticate certain services with prompting the user for credentials. MAC OS X comes with Heimdal Kerberos which is an alternate implementation of the kerberos and uses LDAP as identity management database. Here we are going to learn how to setup a kerberos on MAC OS X which we will configure latter in our application. Installing Kerberos In MAC we can use Homebrew for installing any software package. Homebrew makes it very easy to install the kerberos by just executing a simple command as given below. brew install krb5 Once installation is complete, we need to set the below export commands in user's profile which will make the kerberos utility commands and compiler available to execute from anywhere. Open user's bash profile: vi ~/.bash_profile Add below lines: export PATH=/usr/local/opt/krb5/bin:$PATH export PATH=/usr/local/opt/krb5/sbin:$PATH export LDFLAGS=&

SpringBoot - @ConditionalOnProperty example for conditional bean initialization

@ConditionalOnProperty annotation is used to check if specified property available in the environment or it matches some specific value so it can control the execution of some part of code like bean creation. It may be useful in many cases for example enable/disable service if specific property is available. Below are the attributes which can be used for property check. havingValue - Provide the value which need to check against specified property otherwise it will check that value should not be false. matchIfMissing - If true it will match the condition and execute the annotated code when property itself is not available in environment. name - Name of the property to be tested. If you want to test single property then you can directly put the property name as string like "" and if you have multiple properties to test then you can put the names like {"prop.name1","prop.name2"} prefix - It can be use when you want to apply some prefix to

Why HashMap key should be immutable in java

HashMap is used to store the data in key, value pair where key is unique and value can be store or retrieve using the key. Any class can be a candidate for the map key if it follows below rules. 1. Overrides hashcode() and equals() method.   Map stores the data using hashcode() and equals() method from key. To store a value against a given key, map first calls key's hashcode() and then uses it to calculate the index position in backed array by applying some hashing function. For each index position it has a bucket which is a LinkedList and changed to Node from java 8. Then it will iterate through all the element and will check the equality with key by calling it's equals() method if a match is found, it will update the value with the new value otherwise it will add the new entry with given key and value. In the same way it check for the existing key when get() is called. If it finds a match for given key in the bucket with given hashcode(), it will return the value other

jaxb2-maven-plugin to generate java code from XSD schema

In this tutorial I will show how to generate the Java source code from XSD schema. I will use jaxb2-maven-plugin to generate the code using XSD file which will be declared in pom.xml to make it part of build, so when maven build is executed it will generate the java code using XSD. Class generation can be controlled in plugin configuration. Maven changes (pom.xml) Include below plugin in your pom.xml. Here we have done some configuration under configuration section as given below. schemaDirectory : This is the directory where I keep my schema (XSD file). outputDirectory : This is the java source location where I want to generate the Java files. If it is not given then by default it will be generate inside target folder. clearOutputDir : If this property is true then it will generate the classes on each build otherwise it will generate only if output directory is empty. <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>jaxb2-maven-plugin</art