--mount=type=cache in buildkit

Question

I'm trying to get maven offline builds working from within a docker container. I've enabled buildkit. I've run mvn dependency:go-offline -s ~/checkouts/settings.xml to cache the dependencies in /root/.m2 of host machine. I wish to use this inside the container that builds maven project.

Here is my Dockerfile:

#syntax=docker/dockerfile:experimental
FROM maven:3.6.1-jdk-11 AS build
WORKDIR /
COPY . /
RUN --mount=type=cache,target=/root/.m2 mvn -o install 

FROM scratch
COPY --from=build /admin/admin- 
rest/target/admin-rest.war /webapps/ROOT.war

When I try to docker build this Dockerfile, I get the following error:

Plugin org.codehaus.mojo:build-helper-maven-plugin:3.0.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.mojo:build-helper-maven-plugin:jar:3.0.0 has not been downloaded from it before. -> [Help 1]

My Docker version:

Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77
 Built:             Sat May  4 02:35:57 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:19:41 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.5
  GitCommit:        bb71b10fd8f58240ca47fbb579b9d1028eea7c84
 runc:
  Version:          1.0.0-rc6+dev
  GitCommit:        2b18fe1d885ee5083ef9f0838fee39b62d653e30
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Am I right in my expectation that /root/.m2 from my host machine needs to get mounted on to /root/.m2 inside the docker container environment? And when I run mvn -o install it needs to refer to the mounted /root/.m2?

I added tail -f /dev/null, removed second build stage and changed the mvn install to mvn version with the cache-mount instruction intact in order to debug this. I see that nothing gets mounted on to /root/.m2 inside the container. It is empty. (Unsure if this is the right way to debug this as it may get unmounted after the image is built)

I'm not familiar with the `--mount=type=cache,target=…` syntax, but note that you might hit at some point the limitation of `mvn dependency:go-offline` described (and solved) in this SO thread: [Maven/Docker: cache all dependencies](https://stackoverflow.com/a/52942964/9164010) — ErikMD, Aug 20 '19 at 21:54

score 31 · Accepted Answer · edited May 10 '21 at 08:20

31

It's best to think of --mount=type=cache as being like a named volume in docker, managed by BuildKit, and potentially deleted if the BuildKit cache gets full or a prune is requested. The next time you run a build, that same named volume may be available, significiantly reducing the build time spent downloading dependencies. While very useful, that doesn't appear to be what you're looking for here. To use a cache like this, you'd need to include the go-offline as an earlier step in the Dockerfile:

#syntax=docker/dockerfile:experimental
FROM maven:3.6.1-jdk-11 AS build
WORKDIR /
# copy just the pom.xml for cache efficiency
COPY ./pom.xml /
# go-offline using the pom.xml
RUN --mount=type=cache,target=/root/.m2 mvn dependency:go-offline
# now copy the rest of the code and run an offline build
COPY . /
RUN --mount=type=cache,target=/root/.m2 mvn -o install 

FROM scratch
COPY --from=build /admin/admin-rest/target/admin-rest.war /webapps/ROOT.war

To mount a directory into the container from the host, what you appear to be looking for is a bind mount. And with BuildKit's experimental settings, that is available, but only to the build context, not to any arbitrary directory on the build host. For that, you can place your .m2 directory in the build context directory and then use the following line in your Dockerfile:

RUN --mount=type=bind,source=./.m2,target=/root/.m2,rw mvn -o install

Note if any of the dependencies change, then Maven may try to connect over the network again.

edited May 10 '21 at 08:20

Bouke Versteegh

4,097
1
39
35

answered Aug 21 '19 at 00:20

BMitch

231,797
42
475
450

Initially I was thinking why would I do go-offline inside a container which is ephemeral.Thanks @BMitch that clears up my mis-understanding.I'll try this. – A.R.K.S Aug 21 '19 at 13:54
1

@AshwiniR When you run the same command in a later build, docker will reuse the build cache results from before. To avoid breaking the cache, the `COPY . /` is not run until after the `go-offline` step. – BMitch Aug 21 '19 at 13:56
1

The note about mounting the directory from the host is incorrect, `--mount=type=bind,source=./.m2,target=/root/.m2,rw` mounts the directory from either the context, or another container, depending on the `from=` parameter. Unfortunately it doesn't support mounting from the host afaik. – Evgeny Dec 18 '19 at 20:11
2

@Evgeny the answer specifies "but only to the build context, not to any arbitrary directory on the build host". I did my best to indicate the constraints given the OP's request to mount an arbitrary host directory and explain why their command was not working. – BMitch Dec 18 '19 at 20:20
1

The important part here is that **every** `mvn` statement must include the ` --mount=type=cache,target=/root/.m2` mount. Otherwise dependencies will be downloaded again. – membersound Mar 30 '22 at 09:42

--mount=type=cache in buildkit

1 Answers1

Linked