2

I need to support 2 different versions of the same library (to support a legacy version), es4hadoop for Apache Spark.

Both versions have same dependencies (scala-lang and Spark).

Not sure at all about the naming of that, but I would like something like:

implementation(group: 'org.elasticsearch', name: 'elasticsearch-spark-13_' + scalaVersion, version:'6.2.2') {
     exclude group: "org.scala-lang"  
}

implementation(group: 'org.elasticsearch', name: 'elasticsearch-spark-13_' + scalaVersion, version:'6.3.2') {
     exclude group: "org.scala-lang"
     relocate org.elasticsearch org.elasticsearch6 // ???
}

so I can use both new and old elasticsearch library, in the same project / JVM.

I know already it's possible to relocate lib with the shadowJar plugin, but is it possible to relocate a specific version?

Thomas Decaux
  • 21,738
  • 2
  • 113
  • 124
  • 2
    Yes, assuming they have the same packagenames, you would need to shade/shadow one of the libraries to move the classes to a different package (and change all existing references in classes needing that version), e.g. see https://stackoverflow.com/questions/53060258. This is a very complex topic, so expect a lot of problems, and you need to explain in much more detail what you need to achieve to get a response. – tkruse May 28 '19 at 04:16
  • I see this is very complicate :/ for now, I clone the lib and replace all package names by hand, it's working fine but quite dirty, curious to see there is no a simple solution. – Thomas Decaux May 28 '19 at 07:23

1 Answers1

3

Put one of the elasticsearch-spark dependencies into a subproject aa2 and relocate it. Then the other subproject aa1 can depend on aa2's shadow configuration.

// aa2/build.gradle
dependencies {
    implementation 'org.elasticsearch:elasticsearch-spark-13_2.10:6.2.2'
}

shadowJar {
    relocate "org.elasticsearch", "org.elasticsearch_v6_2_2"
}

// aa1/build.gradle
dependencies {
    implementation 'org.elasticsearch:elasticsearch-spark-13_2.10:6.3.2'
    implementation project(path: ':aa2', configuration: 'shadow')
}

You can now declare the same class in this way:

package com.github.chehsunliu.stackoverflow.q56332118;

import org.elasticsearch.spark.cfg.SparkSettings;

public class App {
  public static void main(String[] args) {
    System.out.println(SparkSettings.class);
    System.out.println(org.elasticsearch_v6_2_2.spark.cfg.SparkSettings.class);
  }
}

However, you should pay more attention to their transitive dependencies. You might also need to relocate them to make the direct dependencies work normally. Sometimes I will decompress the output JAR file and investigate these .class files to ensure the resolution correctness.

References

chehsunliu
  • 1,559
  • 1
  • 12
  • 22