Spring Boot :Spring Batch

What is Batch Processing?

Batch processing is the execution of a series of jobs (programs) on a computer without manual intervention. It's commonly used for:

Batch processing

Batch processing is a method of running a series of tasks or processes on a set of data without manual intervention. It is designed to handle large volumes of data efficiently by grouping jobs into batches and processing them sequentially or in parallel.

Key Characteristics

Processes large volumes of data in batches

Email Campaigns
- Example: When sending a large number of emails (e.g., a marketing email campaign), email servers often use batch processing to send emails in groups. This prevents overloading the server and ensures emails are delivered efficiently.
- Use Case: A company sends out promotional emails to thousands of customers. The emails are grouped into batches, and each batch is sent at specific intervals to avoid being flagged as spam.
scheduled to run at specific times (e.g., nightly jobs)

Database Backups
- Example: Many organizations schedule nightly backups of their databases to ensure data is safely stored without impacting user operations during peak hours.
- Use Case: A company schedules a backup job every night at midnight to back up its customer database, including transaction logs, ensuring that it can restore the data in case of any failure.
Automated Software Updates
- Example: Systems and applications may schedule jobs to check for and install software updates at specific times to keep systems secure and up to date.
- Use Case: A company schedules its IT system to run an update check every night at 3 AM, automatically downloading and installing critical security patches for its servers without disrupting users.
Reduces the need for constant manual monitoring

Database Performance Optimization
- Example: Databases can run maintenance jobs, like reindexing or defragmenting, at off-peak hours to maintain performance, reducing the need for manual monitoring or intervention.
- Use Case: A database system is scheduled to automatically optimize performance every night by reorganizing tables and running integrity checks. This ensures that users experience optimal performance without the need for an administrator to manually intervene.
Alert Management
- Example: Automated systems can detect issues (e.g., system errors, performance drops, or security breaches) and generate alerts only when necessary, reducing the need for IT staff to monitor systems constantly.
- Use Case: A monitoring system checks for website uptime every 5 minutes. If the site is down, it automatically sends an email or SMS alert to the support team. This eliminates the need for someone to manually check website status.
Automated Fraud Detection
- Example: Fraud detection systems can run periodic checks to analyze transactions for unusual patterns (e.g., high-value transfers or suspicious locations) without requiring constant human monitoring.
- Use Case: A financial system runs a scheduled task every night to review transactions for potential fraud. If any suspicious activity is detected, an alert is sent to the fraud department for further investigation.
Optimized for high-throughput and low-latency requirements

Real-Time Financial Transactions
- Example: Payment processing systems are designed to handle thousands of transactions per second with minimal delay to ensure that customers’ payments are processed instantly.
- Use Case: A credit card payment gateway is optimized for high throughput, processing thousands of transactions per second while ensuring that each payment authorization happens within milliseconds to avoid user frustration or potential loss of revenue.

Data migration
Report generation
File format conversion
ETL (Extract, Transform, Load) operations
Automated billing systems

Why Spring Batch?

Advantages:

Transaction management
Chunk-based processing
Declarative I/O
Start/Stop/Restart capabilities
Retry/Skip mechanisms

Key Features:

Job scheduling
Monitoring and metrics
Parallel processing
Error handling
Extensible architecture

2. Core Concepts

Main Components

graph TD A[JobLauncher] --> B[Job] B --> C[Step 1] B --> D[Step 2] B --> E[Step N] C --> F[ItemReader] C --> G[ItemProcessor] C --> H[ItemWriter] F --> I[(Data Source)] H --> J[(Data Destination)] B --> K[JobRepository]

Processing Model

graph LR A[ItemReader] --> B[ItemProcessor] B --> C[ItemWriter] A -- Reads --> D[Data] B -- Transforms --> E[Data] C -- Writes --> F[Destination]

Explanation:

Job: The entire batch process. Contains one or more steps.

Step: A stage in a job containing reader, processor, writer.

ItemReader: Reads data from source (file, database, etc.)

ItemProcessor: Processes and transforms data

ItemWriter: Writes processed data to destination

JobRepository: Persists job metadata and execution state

3. Hello World Example

Spring Boot Project with Spring Batch

dependencies in pom.xml.


<dependencies>
    <!-- Spring Boot and Batch Dependencies -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-batch</artifactId>
    </dependency>
   

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
</dependencies>

Let's start with a simple "Hello World" batch job that prints numbers.

Step 1: Create the ItemReader



package com.example.testbach.reader;


import org.springframework.batch.infrastructure.item.ItemReader;
import org.springframework.stereotype.Component;

@Component
public class HelloWorldReader implements ItemReader<String> {

    private final String[] messages = {"Hello", "World", "Spring", "Batch"};
    private int counter = 0;

    @Override
    public String read() {
        if (counter < messages.length) {
            return messages[counter++];
        } else {
            return null; // End of data
        }
    }
}

Explanation:

This reader returns one message at a time from an array. When all messages are read, it returns null to signal end of data.

Step 2: Create the ItemProcessor



package com.example.testbach.processor;


import org.springframework.batch.infrastructure.item.ItemProcessor;
import org.springframework.stereotype.Component;

@Component
public class HelloWorldProcessor implements ItemProcessor<String, String> {

    @Override
    public String process(String item) {
        return "Processed: " + item.toUpperCase();
    }
}

Explanation:

This processor converts each string to uppercase and adds a prefix. The process method takes one item and returns one transformed item.

Step 3: Create the ItemWriter

package com.example.testbach.writer;


import org.springframework.batch.infrastructure.item.Chunk;
import org.springframework.batch.infrastructure.item.ItemWriter;
import org.springframework.stereotype.Component;
import java.util.List;

@Component
public class HelloWorldWriter implements ItemWriter<String>{
    
    @Override
    public void write(Chunk<? extends String> items) {
        for (String item : items) {
            System.out.println(item);
        }
    }

	
}

Explanation:

This writer receives a list of items (chunk) and prints each one. Writers typically work with chunks for efficiency.

Step 4: Configure the Job

package com.example.testbach.config;



import org.springframework.batch.core.job.Job;
import org.springframework.batch.core.job.builder.JobBuilder;
import org.springframework.batch.core.job.parameters.RunIdIncrementer;
import org.springframework.batch.core.step.Step;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.batch.infrastructure.support.transaction.ResourcelessTransactionManager;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import org.springframework.transaction.PlatformTransactionManager;

import com.example.testbach.processor.HelloWorldProcessor;
import com.example.testbach.reader.HelloWorldReader;
import com.example.testbach.writer.HelloWorldWriter;

@Configuration
public class HelloWorldBatchConfig {

	@Bean
	public PlatformTransactionManager transactionManager() {
	    return new ResourcelessTransactionManager();
	}
	
    @Bean
    public Job helloWorldJob(
            JobRepository jobRepository,
            Step step1
    ) {
        return new JobBuilder("helloWorldJob", jobRepository)
                .incrementer(new RunIdIncrementer())
                .start(step1)
                .build();
    }

    @Bean
    public Step step1(
            JobRepository jobRepository,
            PlatformTransactionManager transactionManager,
            HelloWorldReader reader,
            HelloWorldProcessor processor,
            HelloWorldWriter writer
    ) {
        return new StepBuilder("step1", jobRepository)
        		.<String, String>chunk(2)
				.transactionManager(transactionManager)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .build();
    }
}

Explanation:

This configuration defines a job with one step. The chunk size is 2, meaning 2 items are read, processed, and written together.

4. CSV to Database Processing

In this example, we'll read employee data from a CSV file and store it in a database.

Architecture Diagram

graph LR A[CSV File] --> B[FlatFileItemReader] B --> C[EmployeeProcessor] C --> D[JpaItemWriter] D --> E[(Database)]

Step 1: Define the Entity

package com.example.batch.entity;

import javax.persistence.Entity;
import javax.persistence.Id;

@Entity
public class Employee {
    @Id
    private Long id;
    private String firstName;
    private String lastName;
    private String email;
    private String department;
    
    // Constructors, getters and setters
    public Employee() {}
    
    public Employee(Long id, String firstName, String lastName, String email, String department) {
        this.id = id;
        this.firstName = firstName;
        this.lastName = lastName;
        this.email = email;
        this.department = department;
    }
    
    // Getters and setters...
}

Step 2: Create the Reader

package com.example.batch.config;

import com.example.batch.entity.Employee;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;

@Configuration
public class CsvToDatabaseJobConfig {
    
    @Bean
    public FlatFileItemReader<Employee> employeeItemReader() {
        return new FlatFileItemReaderBuilder<Employee>()
                .name("employeeItemReader")
                .resource(new ClassPathResource("employees.csv"))
                .delimited()
                .names(new String[]{"id", "firstName", "lastName", "email", "department"})
                .fieldSetMapper(new BeanWrapperFieldSetMapper<Employee>() {{
                    setTargetType(Employee.class);
                }})
                .build();
    }
}

Explanation:

The FlatFileItemReader reads CSV data and maps columns to Employee object properties using field names.

Step 3: Create the Processor

package com.example.batch.processor;

import com.example.batch.entity.Employee;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.stereotype.Component;

@Component
public class EmployeeItemProcessor implements ItemProcessor<Employee, Employee> {
    
    @Override
    public Employee process(Employee employee) {
        // Convert names to uppercase
        employee.setFirstName(employee.getFirstName().toUpperCase());
        employee.setLastName(employee.getLastName().toUpperCase());
        
        // Validate email
        if (employee.getEmail() == null || !employee.getEmail().contains("@")) {
            throw new IllegalArgumentException("Invalid email: " + employee.getEmail());
        }
        
        return employee;
    }
}

Explanation:

The processor transforms data (uppercase names) and validates it (email format). Invalid data throws exceptions.

Step 4: Create the Writer

package com.example.batch.writer;

import com.example.batch.entity.Employee;
import org.springframework.batch.item.database.JpaItemWriter;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class EmployeeWriterConfig {
    
    @Bean
    public JpaItemWriter<Employee, Employee> employeeItemWriter() {
        JpaItemWriter<Employee, Employee> writer = new JpaItemWriter<>();
        writer.setEntityManagerFactory(entityManagerFactory);
        return writer;
    }
}

Explanation:

JpaItemWriter persists Employee entities to the database using JPA.

5. Database to XML Export

This example demonstrates exporting data from a database to an XML file.

Architecture Diagram

graph LR A[(Database)] --> B[JdbcCursorItemReader] B --> C[ListAggregator] C --> D[StaxEventItemWriter] D --> E[XML File]

Step 1: Define XML Structure

package com.example.batch.entity;

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import java.util.List;

@XmlRootElement(name = "employees")
public class Employees {
    private List<Employee> employees;
    
    public Employees() {}
    
    public Employees(List<Employee> employees) {
        this.employees = employees;
    }
    
    @XmlElement(name = "employee")
    public List<Employee> getEmployees() {
        return employees;
    }
    
    public void setEmployees(List<Employee> employees) {
        this.employees = employees;
    }
}

Step 2: Create Database Reader

package com.example.batch.config;

import com.example.batch.entity.Employee;
import org.springframework.batch.item.database.JdbcCursorItemReader;
import org.springframework.batch.item.database.builder.JdbcCursorItemReaderBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.RowMapper;

import javax.sql.DataSource;
import java.sql.ResultSet;
import java.sql.SQLException;

@Configuration
public class DatabaseToXmlJobConfig {
    
    @Bean
    public JdbcCursorItemReader<Employee> employeeDbReader(DataSource dataSource) {
        return new JdbcCursorItemReaderBuilder<Employee>()
                .name("employeeDbReader")
                .dataSource(dataSource)
                .sql("SELECT id, first_name, last_name, email, department FROM employee")
                .rowMapper(new EmployeeRowMapper())
                .build();
    }
    
    public static class EmployeeRowMapper implements RowMapper<Employee> {
        @Override
        public Employee mapRow(ResultSet rs, int rowNum) throws SQLException {
            Employee employee = new Employee();
            employee.setId(rs.getLong("id"));
            employee.setFirstName(rs.getString("first_name"));
            employee.setLastName(rs.getString("last_name"));
            employee.setEmail(rs.getString("email"));
            employee.setDepartment(rs.getString("department"));
            return employee;
        }
    }
}

Explanation:

JdbcCursorItemReader executes SQL query and maps each row to an Employee object using RowMapper.

Step 3: Create XML Writer

package com.example.batch.writer;

import com.example.batch.entity.Employees;
import org.springframework.batch.item.xml.StaxEventItemWriter;
import org.springframework.batch.item.xml.builder.StaxEventItemWriterBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.FileSystemResource;
import org.springframework.oxm.jaxb.Jaxb2Marshaller;

@Configuration
public class XmlWriterConfig {
    
    @Bean
    public StaxEventItemWriter<Employees> employeeXmlWriter() {
        return new StaxEventItemWriterBuilder<Employees>()
                .name("employeeXmlWriter")
                .resource(new FileSystemResource("output/employees.xml"))
                .marshaller(employeeMarshaller())
                .rootTagName("employees")
                .overwriteOutput(true)
                .build();
    }
    
    @Bean
    public Jaxb2Marshaller employeeMarshaller() {
        Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
        marshaller.setClassesToBeBound(Employees.class, com.example.batch.entity.Employee.class);
        return marshaller;
    }
}

Explanation:

StaxEventItemWriter uses JAXB to convert Employee objects to XML format and writes to file.

6. Parallel Processing

This example shows how to process multiple files in parallel using partitioning.

Architecture Diagram

graph TD A[Master Step] --> B[Partitioner] B --> C[Slave Step 1] B --> D[Slave Step 2] B --> E[Slave Step N] C --> F[File 1] D --> G[File 2] E --> H[File N]

Step 1: Partitioner Implementation

package com.example.batch.partition;

import org.springframework.batch.core.partition.support.Partitioner;
import org.springframework.batch.item.ExecutionContext;

import java.util.HashMap;
import java.util.Map;

public class FilePartitioner implements Partitioner {
    
    private String[] filenames;
    
    public FilePartitioner(String[] filenames) {
        this.filenames = filenames;
    }
    
    @Override
    public Map<String, ExecutionContext> partition(int gridSize) {
        Map<String, ExecutionContext> partitions = new HashMap<>();
        
        for (int i = 0; i < filenames.length; i++) {
            ExecutionContext context = new ExecutionContext();
            context.putString("filename", filenames[i]);
            partitions.put("partition" + i, context);
        }
        
        return partitions;
    }
}

Explanation:

The partitioner divides work by creating separate execution contexts for each file to process.

Step 2: Master-Slave Configuration

package com.example.batch.config;

import com.example.batch.partition.FilePartitioner;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.core.partition.support.Partitioner;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.SimpleAsyncTaskExecutor;

@Configuration
public class ParallelProcessingConfig {
    
    @Autowired
    public JobBuilderFactory jobBuilderFactory;
    
    @Autowired
    public StepBuilderFactory stepBuilderFactory;
    
    @Bean
    public Job parallelProcessingJob() {
        return jobBuilderFactory.get("parallelProcessingJob")
                .incrementer(new RunIdIncrementer())
                .flow(masterStep())
                .end()
                .build();
    }
    
    @Bean
    public Step masterStep() {
        return stepBuilderFactory.get("masterStep")
                .partitioner("slaveStep", partitioner())
                .step(slaveStep())
                .taskExecutor(new SimpleAsyncTaskExecutor())
                .build();
    }
    
    @Bean
    public Partitioner partitioner() {
        String[] filenames = {"file1.csv", "file2.csv", "file3.csv"};
        return new FilePartitioner(filenames);
    }
    
    @Bean
    public Step slaveStep() {
        return stepBuilderFactory.get("slaveStep")
                .<Object, Object>chunk(10)
                .reader(/* your reader */)
                .processor(/* your processor */)
                .writer(/* your writer */)
                .build();
    }
}

Explanation:

The master step uses a partitioner to divide work among slave steps running in parallel threads.

7. Cron Scheduling

Schedule batch jobs to run automatically at specific times using cron expressions.

Architecture Diagram

graph TD A[Cron Scheduler] --> B[@Scheduled Method] B --> C[JobLauncher] C --> D[Job] D --> E[Job Execution]

Using @Scheduled Annotation

package com.example.batch.scheduler;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

import java.time.LocalDateTime;

@Component
public class JobScheduler {
    
    @Autowired
    private JobLauncher jobLauncher;
    
    @Autowired
    private Job importUserJob;
    
    // Run every day at 2 AM
    @Scheduled(cron = "0 0 2 * * ?")
    public void runDailyJob() throws Exception {
        JobParameters params = new JobParametersBuilder()
                .addString("JobID", String.valueOf(System.currentTimeMillis()))
                .addLocalDateTime("runTime", LocalDateTime.now())
                .toJobParameters();
        
        jobLauncher.run(importUserJob, params);
    }
    
    // Run every 30 minutes during business hours
    @Scheduled(cron = "0 0/30 9-17 * * MON-FRI")
    public void runBusinessHoursJob() throws Exception {
        JobParameters params = new JobParametersBuilder()
                .addString("JobID", String.valueOf(System.currentTimeMillis()))
                .toJobParameters();
        
        jobLauncher.run(importUserJob, params);
    }
}

Explanation:

@Scheduled annotation triggers job execution based on cron expressions. Jobs run automatically without manual intervention.

Enable Scheduling in Configuration

package com.example.batch.config;

import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableScheduling;

@Configuration
@EnableScheduling
public class SchedulingConfig {
    // Scheduling is now enabled
}

Cron Expression Examples

Cron Expression	Description
`0 0 12 * * ?`	Run at 12:00 PM every day
`0 15 10 ? * *`	Run at 10:15 AM every day
`0 0/5 14 * * ?`	Run every 5 minutes starting at 2 PM
`0 0 22 ? * SUN`	Run at 10 PM on Sundays
`0 0 0 1 * ?`	Run at midnight on the first day of every month

8. Error Handling

Implement robust error handling strategies for production environments.

Architecture Diagram

graph TD A[ItemReader] --> B{Error?} B -->|Yes| C[Skip Policy] B -->|No| D[ItemProcessor] D --> E{Error?} E -->|Yes| F[Retry Policy] E -->|No| G[ItemWriter] G --> H{Error?} H -->|Yes| I[Dead Letter Queue] H -->|No| J[Success]

Custom Skip Policy

package com.example.batch.exception;

import org.springframework.batch.core.step.skip.SkipLimitExceededException;
import org.springframework.batch.core.step.skip.SkipPolicy;
import org.springframework.batch.item.file.FlatFileParseException;
import org.springframework.stereotype.Component;

@Component
public class CustomSkipPolicy implements SkipPolicy {
    
    @Override
    public boolean shouldSkip(Throwable throwable, int skipCount) throws SkipLimitExceededException {
        if (throwable instanceof FlatFileParseException && skipCount <= 10) {
            return true;
        }
        
        if (throwable instanceof NumberFormatException && skipCount <= 5) {
            return true;
        }
        
        return false;
    }
}

Explanation:

Skip policy determines whether to skip problematic items based on exception type and skip count.

Retry Configuration

package com.example.batch.config;

import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.retry.backoff.FixedBackOffPolicy;

@Configuration
public class RetryConfig {
    
    @Autowired
    private StepBuilderFactory stepBuilderFactory;
    
    @Bean
    public Step retryStep() {
        return stepBuilderFactory.get("retryStep")
                .<Object, Object>chunk(10)
                .reader(/* your reader */)
                .processor(/* your processor */)
                .writer(/* your writer */)
                .faultTolerant()
                .retry(Exception.class)
                .retryLimit(3)
                .backOffPolicy(backOffPolicy())
                .build();
    }
    
    @Bean
    public FixedBackOffPolicy backOffPolicy() {
        FixedBackOffPolicy backOffPolicy = new FixedBackOffPolicy();
        backOffPolicy.setBackOffPeriod(2000); // 2 seconds
        return backOffPolicy;
    }
}

Explanation:

Retry mechanism attempts to reprocess failed items with exponential backoff between attempts.

9. Job & Step Listeners

Implement listeners to monitor and control job execution lifecycle.

Architecture Diagram

sequenceDiagram participant J as Job participant L as Listener participant S as Step J->>L: beforeJob() J->>S: execute Step S->>L: beforeStep() S->>L: beforeChunk() S->>L: afterChunk() S->>L: afterStep() J->>L: afterJob()

Job Execution Listener

package com.example.batch.listener;

import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;
import org.springframework.stereotype.Component;

@Component
public class JobDurationListener implements JobExecutionListener {
    
    @Override
    public void beforeJob(JobExecution jobExecution) {
        System.out.println("Job started at: " + jobExecution.getStartTime());
    }
    
    @Override
    public void afterJob(JobExecution jobExecution) {
        long duration = jobExecution.getEndTime().getTime() - jobExecution.getStartTime().getTime();
        System.out.println("Job finished at: " + jobExecution.getEndTime());
        System.out.println("Job duration: " + duration + " ms");
        System.out.println("Job status: " + jobExecution.getStatus());
    }
}

Explanation:

JobExecutionListener monitors job start and completion events for logging and metrics collection.

Step Execution Listener

package com.example.batch.listener;

import org.springframework.batch.core.ExitStatus;
import org.springframework.batch.core.StepExecution;
import org.springframework.batch.core.StepExecutionListener;
import org.springframework.stereotype.Component;

@Component
public class PerformanceStepListener implements StepExecutionListener {
    
    @Override
    public void beforeStep(StepExecution stepExecution) {
        System.out.println("Starting step: " + stepExecution.getStepName());
    }
    
    @Override
    public ExitStatus afterStep(StepExecution stepExecution) {
        System.out.println("Step completed: " + stepExecution.getStepName());
        System.out.println("Items processed: " + stepExecution.getReadCount());
        return stepExecution.getExitStatus();
    }
}

Explanation:

StepExecutionListener tracks individual step execution for detailed monitoring and statistics.

10. Best Practices

Performance Optimization

Chunk Size

Optimize chunk size (10-1000) based on memory and transaction requirements

Connection Pooling

Use connection pools to minimize database connection overhead

Parallel Processing

Leverage multi-threading for CPU-intensive or I/O-bound operations

Error Handling Strategy

graph TD A[Failed Item] --> B{Retry Possible?} B -->|Yes| C[Retry with Backoff] B -->|No| D{Safe to Skip?} D -->|Yes| E[Skip Item] D -->|No| F[Send to DLQ] C --> G{Retry Limit Reached?} G -->|Yes| D G -->|No| C

Monitoring and Observability

# Enable Actuator endpoints
management.endpoints.web.exposure.include=batch,health,info,metrics
management.endpoint.batch.enabled=true

# Configure logging
logging.level.org.springframework.batch=INFO
logging.level.org.springframework.batch.core=DEBUG

Key Recommendations:

Use appropriate chunk sizes - Balance memory usage and transaction overhead
Implement proper error handling - Use retry/skip policies with limits
Monitor job executions - Enable actuator endpoints for metrics
Secure job endpoints - Protect batch jobs from unauthorized access
Test thoroughly - Include unit tests for readers, processors, and writers

Rappel

Angular (FrontEnd)

Middlewares Orientés Messages

Communication Synchrone vs. Asynchrone API JMS : Java Message Service JMS avec ActiveMQ et HornetQ KAFKA Spring Batch Kafka Streams CQRS Résumé

Architectures Serverless

Architectures Serverless

Spring AI

Spring AI

Spring Security

CRUD

Spring Boot (BackEnd) TPs

Produit Rest API

What is Batch Processing?

Batch processing

Key Characteristics

Processes large volumes of data in batches

Email Campaigns

scheduled to run at specific times (e.g., nightly jobs)

Database Backups

Automated Software Updates

Reduces the need for constant manual monitoring

Database Performance Optimization

Alert Management

Automated Fraud Detection

Optimized for high-throughput and low-latency requirements

Real-Time Financial Transactions

Why Spring Batch?

Advantages:

Key Features:

2. Core Concepts

Main Components

Processing Model

Explanation:

3. Hello World Example

Spring Boot Project with Spring Batch

Step 1: Create the ItemReader

Explanation:

Step 2: Create the ItemProcessor

Explanation:

Step 3: Create the ItemWriter

Explanation:

Step 4: Configure the Job

Explanation:

4. CSV to Database Processing

Architecture Diagram

Step 1: Define the Entity

Step 2: Create the Reader

Explanation:

Step 3: Create the Processor

Explanation:

Step 4: Create the Writer

Explanation:

5. Database to XML Export

Architecture Diagram

Step 1: Define XML Structure

Step 2: Create Database Reader

Explanation:

Step 3: Create XML Writer

Explanation:

6. Parallel Processing

Architecture Diagram

Step 1: Partitioner Implementation

Explanation:

Step 2: Master-Slave Configuration

Explanation:

7. Cron Scheduling

Architecture Diagram

Using @Scheduled Annotation

Explanation:

Enable Scheduling in Configuration

Cron Expression Examples

8. Error Handling

Architecture Diagram

Custom Skip Policy

Explanation:

Retry Configuration

Explanation:

9. Job & Step Listeners

Architecture Diagram

Job Execution Listener