Spring Batch: Skip Exception not behaving as expected

Question

I've configured my Spring Batch app below:

<batch:job id="mPortRiskJob">
    <batch:step id="mPortRiskStep">
        <tasklet throttle-limit="10">
           <chunk reader="MPortRiskReader"  processor="MPortRiskProcessor" writer="MPortRiskWriter" commit-interval="10"
            skip-limit="1">
               <batch:skippable-exception-classes>
                   <include class="com.common.exception.ERDException"/>
               </batch:skippable-exception-classes>
           </chunk>
            <batch:no-rollback-exception-classes>
                <include class="com.common.exception.ERDException"/>
            </batch:no-rollback-exception-classes>
        </tasklet>
    </batch:step>   
    <batch:listeners>
        <batch:listener ref="MJobExecutionListener"/>
    </batch:listeners>    
</batch:job>

In my writer, I have a for-loop that inserts records into the database. The code looks like below:

for (String id : keylist) {
  try {
      insertRecord(id);
  } catch (Exception e) {
      throw new ERDException("Failure in write method", e);
  }
}

What I want is if, for instance, the first record throws a DuplicateKeyException, for that record to be skipped, and the next record to be inserted. What's happening is, when the ERDException is thrown, Spring Batch retries all the records, including the duplicate. I want it to discard that particular record and insert the others. Is there a way to accomplish that?

score 0 · Answer 1 · answered Dec 10 '19 at 22:47

0

Well since your exception is thrown in writer with commit-interval as 10 it is retrying all 10 records again because it needs to determine which record throws exception. Once it determines the record it will skip just that record and process others.

Please see this post.

answered Dec 10 '19 at 22:47

Sushil Behera

817
6
20

So does that mean my skip limit should be 10 also? I wanted it to only try once, which is why I set the limit to 1. – CNDyson Dec 10 '19 at 22:50
No. for writer it will try twice. There is nothing much you can do. But it should not try more than twice. – Sushil Behera Dec 10 '19 at 22:58
Would writing my own SkipListener help me achieve what I want? – CNDyson Dec 11 '19 at 15:10

score 0 · Accepted Answer · answered Dec 19 '19 at 18:43

For anyone interested, I solved my problem by combining no-rollback-exception-classes with a skip-policy! I probably went a little overboard with defining my own skip policy - one of the out-of-the-box Spring policies would have been fine. See below:

<bean id="skipPolicy" class="com.trp.erd.batch.ERDSkipPolicy">

<batch:job id="midrPortRiskJob">
    <batch:step id="midrPortRiskStep">
        <tasklet throttle-limit="10">
           <chunk reader="MIDRPortRiskReader"  processor="MIDRPortRiskProcessor" writer="MIDRPortRiskWriter" commit-interval="10"
            skip-limit="1" skip-policy="skipPolicy">
               <batch:skippable-exception-classes>
                   <include class="com.trp.erd.common.exception.ERDException"/>
               </batch:skippable-exception-classes>
           </chunk>
            <batch:no-rollback-exception-classes>
                <include class="com.trp.erd.common.exception.ERDException"/>
            </batch:no-rollback-exception-classes>
        </tasklet>
    </batch:step>   
    <batch:listeners>
        <batch:listener ref="MIDRJobExecutionListener"/>
    </batch:listeners>    
</batch:job>

And the skip policy implementation:

public class ERDSkipPolicy implements SkipPolicy {

    @Override
    public boolean shouldSkip(Throwable t, int skipCount) {
        if (t instanceOf ERDException) {
            return true;
        }
    }    
}

Spring Batch: Skip Exception not behaving as expected

2 Answers2