Specify multiple files in ARG to COPY in Dockerfile

Question

Consider the following docker build context:

src/
  hi
  there
  bye

and Dockerfile:

FROM ubuntu

RUN mkdir test
COPY src/hi src/there test/

This works just fine but I would like to make the list of files to copy an ARG, something like:

FROM ubuntu

ARG files

RUN mkdir test
COPY ${files} test/

Unfortunately calling with docker build --build-arg files='src/hi src/there' some_path fails because it treats src/hi src/there as a single item. How can I "expand" the files argument into multiple files to copy?

On a whim I tried specifying the files arg multiple times: docker build --build-arg files='src/hi' --build-arg files='src/there' some_path, but this only copies "there".

You should consider about excluding files into another directory (e.g project_root/docker/....) and copy entire directory. It would be much cleaner. — Tigran, May 06 '20 at 12:45
Appreciate the suggestion but unfortunately that wouldn't work for me since it's a matter of copying some files first then doing some stuff then copying some other files later. The point being to make better use of the cache. The set of files will change depending on the build type — Luis, May 07 '20 at 01:37
Did you try with double quotes? like: _docker build --build-arg files="src/hi src/there" some_path_ — pol92, Oct 16 '20 at 10:25

score 2 · Answer 1 · answered Oct 10 '20 at 14:47

because it treats src/hi src/there as a single item.
How can I "expand" the files argument into multiple files to copy?

That seems unlikely considering the Dockerfile format mentions, regarding arguments:

whitespace in instruction arguments, such as the commands following RUN, are preserved

And that is not limited to RUN.

COPY, however, also includes:

Each <src> may contain wildcards and matching will be done using Go’s filepath.Match rules.

It would not work in your case.

Remain the multi-stage builds approach

COPY src (the all folder)

Remove everything you don't want:

RUN find pip ${files} -maxdepth 1 -mindepth 1 -print | xargs rm -rf

Build your actual image based on the resulting state of the first image.

You can pass as one argument the folders you want to preserve

docker build --build-arg="files=! -path "src/hi" ! -path "src/there"" .

See an example in "Docker COPY files using glob pattern?".

You propose the following two explainations for the problem of variable expansion as distinct command argument: - Spaces are preserved in command arguments: This means spaces in command arguments are not modified, but isn't related to whether spaces withing a variable are interpreted. As a single argument containing spaces or as multiple elements separated by spaces. - Pattern matching follows Go's rule: There is no pattern matching at play here. Those are fully specified files and dirs. Those seems to address related but distinct subject. — Antoine Gallix, Oct 12 '20 at 14:38
@AntoineGallix Still, the point remains: a multi-stage approach with a full copy and a subsequent filter will work. — VonC, Oct 12 '20 at 14:41

Ilyan · Answer 2 · 2021-05-03T14:09:25.117

As a possible workaround, you can try to use .dockerignore file

*
!src/hi
!src/there

with

COPY . test/

https://docs.docker.com/engine/reference/builder/#dockerignore-file

Finally, you may want to specify which files to include in the context, rather than which to exclude. To achieve this, specify * as the first pattern, followed by one or more ! exception patterns.

Update:

Using wildcards in .dockerignore is limited because of bug: https://github.com/moby/moby/issues/30018

score 0 · Answer 3 · answered Oct 15 '20 at 16:57

From the Docker documentation, for your information.

If you have multiple Dockerfile steps that use different files from your context, COPY them individually, rather than all at once. This ensures that each step’s build cache is only invalidated (forcing the step to be re-run) if the specifically required files change.

Specify multiple files in ARG to COPY in Dockerfile

3 Answers3