Jenkins 2.0: Running SBT in a docker container

Question

I have the following Jenkinsfile:

def notifySlack = { String color, String message ->
    slackSend(color: color, message: "${message}: Job ${env.JOB_NAME} [${env.BUILD_NUMBER}] (${env.BUILD_URL})")
}

node {
    try {
        notifySlack('#FFFF00', 'STARTED')
        stage('Checkout project') {
            checkout scm
        }
        scalaImage = docker.image('<myNexus>/centos-sbt:2.11.8')
        stage('Test project') {
            docker.withRegistry('<myNexus>', 'jenkins-nexus') {
                scalaImage.inside('-v /var/lib/jenkins/.ivy2:/root/.ivy2') { c ->
                    sh 'sbt clean test'
                }
            }
        }
        if (env.BRANCH_NAME == 'master') {
            stage('Release new version') {
                docker.withRegistry('<myNexus>', 'jenkins-nexus') {
                    scalaImage.inside('-v /var/lib/jenkins/.ivy2:/root/.ivy2') { c ->
                        sh 'sbt release'
                    }
                }
            }
        }
        notifySlack('#00FF00', 'SUCCESSFUL')
    } catch (e) {
        currentBuild.result = "FAILED"
        notifySlack('#FF0000', 'FAILED')
        throw e
    }

}

Unfortunately when I reach the sbt clean test line I end up with the following error:

java.lang.IllegalArgumentException: URI has a query component
    at java.io.File.<init>(File.java:427)
    at sbt.IO$.uriToFile(IO.scala:160)
    at sbt.IO$.toFile(IO.scala:135)
    at sbt.Classpaths$.sbt$Classpaths$$bootRepository(Defaults.scala:1942)
    at sbt.Classpaths$$anonfun$appRepositories$1.apply(Defaults.scala:1912)
    at sbt.Classpaths$$anonfun$appRepositories$1.apply(Defaults.scala:1912)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
    at scala.collection.AbstractTraversable.map(Traversable.scala:105)
    at sbt.Classpaths$.appRepositories(Defaults.scala:1912)
    at sbt.Classpaths$$anonfun$58.apply(Defaults.scala:1193)
    at sbt.Classpaths$$anonfun$58.apply(Defaults.scala:1190)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.EvaluateSettings$MixedNode.evaluate0(INode.scala:175)
    at sbt.EvaluateSettings$INode.evaluate(INode.scala:135)
    at sbt.EvaluateSettings$$anonfun$sbt$EvaluateSettings$$submitEvaluate$1.apply$mcV$sp(INode.scala:69)
    at sbt.EvaluateSettings.sbt$EvaluateSettings$$run0(INode.scala:78)
    at sbt.EvaluateSettings$$anon$3.run(INode.scala:74)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

If I run the simple docker run ... followed by docker exec I get what I want but I would like to work with the defined Jenkins functionality.

So this seems to be an SBT issue. I use version 0.13.16 inside the docker image. From what I understand the classpath contains a query parameter that SBT:

• doesn't like
• doesn't know how to handle
• is illegal

I put no such query parameters myself so I thought that this .inside method does that. I checked the env in the container and found a single entry RUN_CHANGES_DISPLAY_URL=<my_ip>/job/scheduler/job/fix-jenkins-pipeline/23/display/redirect?page=changes. I tried to unset it but didn't manage to.

I'm out of ideas and am not really sure I'm in the right direction. Any help would be appreciated.

Answer 1

So after long and tedious searches what finally worked for me is setting explicitly the .sbt and .ivy2 folder like this inside the docker container:

sbt -Dsbt.global.base=.sbt -Dsbt.boot.directory=.sbt -Dsbt.ivy.home=.ivy2 clean test

That somehow prevents sbt from generating the ? folder and directly puts the aforementioned folders in the root of the directory checkout.

Answer 2

I spent a lot of time tracing this down through the code.

It looks like the easiest solution is to just pass -Duser.home=<path> to sbt, or to set it in the SBT_OPTS environment variable; then all the rest of the directories will be built as if the <path> is the user's home directory.

Answer 3

I fixed this by setting ivys cache directory -> How to override the location of Ivy's Cache?

The problem was that it wasn't set and by default it creates a ? folder which in return can't be handled by sbt itself.

I created a custom Dockerfile to have more control over sbt. Here are the steps I executed to solve the issue:

I created a file called ivysettings.xml with the following contents:

<ivysettings>
    <properties environment="env" />
    <caches defaultCacheDir="/home/jenkins/.ivy2/cache" />
</ivysettings>

and a Dockerfile:

FROM openjdk:8

RUN wget -O- "http://downloads.lightbend.com/scala/2.11.11/scala-2.11.11.tgz" \
    | tar xzf - -C /usr/local --strip-components=1

RUN curl -Ls https://git.io/sbt > /usr/bin/sbt && chmod 0755 /usr/bin/sbt

RUN adduser -u 1000 --disabled-password --gecos "" jenkins

ADD ./files/ivysettings.xml /home/jenkins/.ivy2/ivysettings.xml
RUN chown -R jenkins:jenkins /home/jenkins

USER jenkins

CMD ["sbt"]

I then pushed the image to our private docker repository and our pipeline finally works!

Answer 4

Problem is that jenkins is running a specific user inside the container. But overriding it does the trick.

withDockerContainer(args: "-u root -v ${HOME}/.sbt:/root/.sbt -v ${HOME}/.ivy2:/root/.ivy2 -e HOME=/root",
            image: 'xyz/sbt:v') {