Spring Batch 5: Using MultiResourceItemReader and ItemStreamWriter for Efficient Batch Processing

2 min readJan 29, 2025

In many business scenarios, such as banking systems or catalog data transfers, we often rely on Spring Batch to handle large-scale data processing efficiently.

A typical workflow involves:

Reading files (JSON, text, etc.) from cloud storage.
Processing the data (filtering, validating, enriching, etc.).
Saving the processed data in a different format (CSV, XML, etc.) in another storage location for future use.

To illustrate this, I’ll walk you through a simple example:

We’ll read text files (file1.txt, file2.txt, file3.txt) containing basic customer data.
The processor will filter the data based on specific criteria.
Finally, the filtered data will be written as CSV files (file1.csv, file2.csv, file3.csv).

The Customer Class

First, let’s define the Customer class, which will be used throughout the Spring Batch process:

The Reader

We have three files in the resources folder, each with the prefix "file". To read these files, we'll create a CustomerMultiResourceReader class, which will utilize MultiResourceItemReader.

This class will delegate the reading process to a new customerReader, which will read one file at a time:

The Processor

The processor is simple: it will filter out any firstName that doesn’t have the first letter in uppercase, then it will correct the formatting:

The Writer

The writer step is a bit more involved. It is responsible for saving the filtered data in the desired format:

Putting It All Together

To combine the Reader, Processor, and Writer, we use the Job class in Spring Batch. Here, we wire everything together to form a complete job:

Debugging

For debugging purposes, you can open MySQL Workbench to view the execution messages and verify the process:

Conclusion

This example code demonstrates the basics of setting up a Spring Batch job. However, it doesn’t cover error handling or infrastructure configuration. Be sure to adapt this approach to your specific business needs for better reliability and performance.