8

I upgraded my spring boot application from Java 8 and tomcat 8 to java 11 and tomcat 9. Everything seems to work fine except the part that I use parallel streams on lists.

list.addAll(items
                .parallelStream()
                .filter(item -> !SomeFilter.isOk(item.getId()))
                .map(logic::getSubItem)
                .collect(Collectors.toList()));

The previous part of code used to work fine with Java 8 and Tomcat 8 but after Java 9 changes on how the classes are loaded with Fork/Join common pool threads return the system class loader as their thread context class loader.

I know that under the hood parallel streams use ForkJoinPool and I created a custom bean class but still, it is not used by the app. Most probably due to the fact that maybe they are created before this bean.

@Bean
public ForkJoinPool myForkJoinPool() {
    return new ForkJoinPool(threadPoolSize, makeFactory("APP"), null, false);
}

private ForkJoinPool.ForkJoinWorkerThreadFactory makeFactory(String prefix) {
    return pool -> {
        final ForkJoinWorkerThread worker = ForkJoinPool.defaultForkJoinWorkerThreadFactory.newThread(pool);
        worker.setName(prefix + worker.getPoolIndex());
        worker.setContextClassLoader(Application.class.getClassLoader());
        return worker;
    };
}

Lastly, I also tried to wrap it around an instance of my ForkJoinPool but it is done asynchronously and I do not want to. I also do not want to use the submit and get because that means that I have to wrap up all the parallel stream that I have on the application with try/catch and the code will be nasty to read.

forkJoinPool.execute(() -> 
  list.addAll(items
                .parallelStream()
                .filter(item -> !SomeFilter.isOk(item.getId()))
                .map(logic::getSubItem)
                .collect(Collectors.toList())));

Ideally, I would like all my parallel streams that are used in the application to use the class loader from the application and not the system one.

Any idea?

Naman
  • 27,789
  • 26
  • 218
  • 353
PavlMits
  • 523
  • 2
  • 14
  • Which Spring Boot version? As you cannot simply upgrade java you also need a version of Spring Boot which supports it. Another thing upgrading tomcat, does this mean you are deploying an application or the embedded container in Tomcat? – M. Deinum Feb 17 '21 at 12:04
  • 3
    In which regard is your code dependent on the thread’s context loader? – Holger Feb 17 '21 at 12:05
  • @M.Deinum my spring boot version is 2.2.10.RELEASE. But which spring version does support it? – PavlMits Feb 17 '21 at 12:09
  • @Holger there is a place that Hazelcast instance is used and tries to retrieve something from the cache which is an Application class loader and it throws ClassNotFoundException – PavlMits Feb 17 '21 at 12:11
  • That version should be ok. Could you add the stacktrace to the question? You also might want to mention Hazelcast in this question as that might be related (maybe a newer version has a fix for this as well). – M. Deinum Feb 17 '21 at 12:14
  • In the beginning, I was thinking that it was a Hazelcast issue and I posted that question https://stackoverflow.com/questions/66183691/hazelcast-on-deployed-servers-throws-java-lang-classnotfoundexception-and-locall/66188778 but after some debugging, I realised that it was coming from the different class loader among the parallel code in the threads that were used. – PavlMits Feb 17 '21 at 12:20
  • 1
    Stream won't pick up the `@Bean` automatically even if you `@Ordered` it. You will need to inject that bean into your method and invoke `execute`. I think you will have the best chance of someone answering this question if you create a minimal code that reproduces the error. – Aniket Sahrawat Feb 17 '21 at 12:57
  • 3
    …and you’re sure that there is no way to set the class loader for the cache explicitly, instead of relying on a fragile mechanism like the thread context loader? Google found [`Config.setClassLoader(…)`](https://docs.hazelcast.org/docs/4.1.1/javadoc/com/hazelcast/config/Config.html#setClassLoader-java.lang.ClassLoader-): “*Sets the class-loader to be used during de-serialization and as context class-loader of Hazelcast internal threads.*” If that’s applicable, it sounds like the better solution to me. – Holger Feb 17 '21 at 13:55
  • 1
    @Holger Thank you very much. I just used this and it seems to work nicely – PavlMits Feb 18 '21 at 15:08

1 Answers1

1

If using a third-party is an option, you could use the Parallel Collectors library:

list.addAll(items
            .stream()
            .filter(item -> !SomeFilter.isOk(item.getId()))
            .collect(parallel(logic::getSubItem, Collectors.toList(), forkJoinPool)
            .join());
M A
  • 71,713
  • 13
  • 134
  • 174
  • I also checked that but I was thinking of a more general solution like CustomForkJoinWorkerThreadFactory which will automatically be used to the parallel streams anyways. I found this but it states that it does not work with JDK9+. https://stackoverflow.com/questions/49113207/completablefuture-forkjoinpool-set-class-loader/57551188#57551188 – PavlMits Feb 17 '21 at 12:08
  • @PavlMits Where is it stated that it does not work with JDK9? The person who posted the answer says "seems to work well with JDK9-JDK11". Did you try setting the system property `java.util.concurrent.ForkJoinPool.common.threadFactory` to a custom one that sets the context CL? – M A Feb 17 '21 at 13:53
  • @PavlMits Not though that even if the system property works, you should probably avoid such a global solution. Given what you described about the related Hazelcast question, it could be solved with Hazelcast specific config. – M A Feb 17 '21 at 14:04