Conditional COPY/ADD in Dockerfile?

Question

Inside of my Dockerfiles I would like to COPY a file into my image if it exists, the requirements.txt file for pip seems like a good candidate but how would this be achieved?

COPY (requirements.txt if test -e requirements.txt; fi) /destination
...
RUN  if test -e requirements.txt; then pip install -r requirements.txt; fi

or

if test -e requirements.txt; then
    COPY requiements.txt /destination;
fi
RUN  if test -e requirements.txt; then pip install -r requirements.txt; fi

Answer 1

Here is a simple workaround:

COPY foo file-which-may-exist* /target

Make sure foo exists, since COPY needs at least one valid source.

If file-which-may-exist is present, it will also be copied.

NOTE: You should take care to ensure that your wildcard doesn't pick up other files which you don't intend to copy. To be more careful, you could use file-which-may-exist? instead (? matches just a single character).

Or even better, use a character class like this to ensure that only one file can be matched:

COPY foo file-which-may-exis[t] /target

Answer 2

As stated by this comment, Santhosh Hirekerur's answer still copies the file, to achieve a true conditional copy, you can use this method.

ARG BUILD_ENV=copy

FROM alpine as build_copy
ONBUILD COPY file /file

FROM alpine as build_no_copy
ONBUILD RUN echo "I don't copy"

FROM build_${BUILD_ENV}
# other stuff

The ONBUILD instructions ensures that the file is only copied if the "branch" is selected by the BUILD_ENV. Set this var using a little script before calling docker build

Answer 3

2021+, from this answer, using glob pattern, Docker COPY will not fail if it won't find any valid source

COPY requiements.tx[t] /destination

---

2015: This isn't currently supported (as I suspect it would lead to a non-reproducible image, since the same Dockerfile would copy or not the file, depending on its existence).

This is still requested, in issue 13045, using wildcards: "COPY foo/* bar/" not work if no file in foo" (May 2015).
It won't be implemented for now (July 2015) in Docker, but another build tool like bocker could support this.

---

2021:

COPY source/. /source/ works for me (i.e. copies directory when empty or not, as in "Copy directory into docker build no matter if empty or not - fails on "COPY failed: no source files were specified"")

2022

Here is my suggestion:

# syntax=docker/dockerfile:1.2

RUN --mount=type=bind,source=jars,target=/build/jars \
 find /build/jars -type f -name '*.jar' -maxdepth 1  -print0 \
 | xargs -0 --no-run-if-empty --replace=source cp --force source >"${INSTALL_PATH}/modules/"

That works around:

COPY jars/*.jar "${INSTALL_PATH}/modules/"

But copies no *.jar if none is found, without throwing an error.

Answer 4

I think I came up with a valid workaround with this Dockerfile

FROM alpine
COPy always_exist_on_host.txt .
COPY *sometimes_exist_on_host.txt .

The always_exist_on_host.txt file will always be copied to the image and the build won't fail to COPY the sometimes_exist_on_host.txt file when it doesn't exist. Furthermore, it will COPY the sometimes_exist_on_host.txt file when it does exist.

For example:

.
├── Dockerfile
└── always_exist_on_host.txt

build succeeds

docker build . -t copy-when-exists --no-cache
[+] Building 1.0s (7/7) FINISHED                                                                                                                            
 => [internal] load .dockerignore                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                        0.0s
 => [internal] load build definition from Dockerfile                                                                                                   0.0s
 => => transferring dockerfile: 36B                                                                                                                    0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                                                                       1.0s
 => [internal] load build context                                                                                                                      0.0s
 => => transferring context: 43B                                                                                                                       0.0s
 => CACHED [1/2] FROM docker.io/library/alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a                                 0.0s
 => [2/2] COPY always_exist_on_host.txt *sometimes_exist_on_host.txt .                                                                                 0.0s
 => exporting to image                                                                                                                                 0.0s
 => => exporting layers                                                                                                                                0.0s
 => => writing image sha256:e7d02c6d977f43500dbc1c99d31e0a0100bb2a6e5301d8cd46a19390368f4899                                                           0.0s               

---

.
├── Dockerfile
├── always_exist_on_host.txt
└── sometimes_exist_on_host.txt

build still succeeds

docker build . -t copy-when-exists --no-cache
[+] Building 1.0s (7/7) FINISHED                                                                                                                            
 => [internal] load build definition from Dockerfile                                                                                                   0.0s
 => => transferring dockerfile: 36B                                                                                                                    0.0s
 => [internal] load .dockerignore                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                        0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                                                                       0.9s
 => [internal] load build context                                                                                                                      0.0s
 => => transferring context: 91B                                                                                                                       0.0s
 => CACHED [1/2] FROM docker.io/library/alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a                                 0.0s
 => [2/2] COPY always_exist_on_host.txt *sometimes_exist_on_host.txt .                                                                                 0.0s
 => exporting to image                                                                                                                                 0.0s
 => => exporting layers                                                                                                                                0.0s
 => => writing image sha256:4c88e2ffa77ebf6869af3c7ca2a0cfb9461979461fc3ae133709080b5abee8ff                                                           0.0s
 => => naming to docker.io/library/copy-when-exists                                                                                                    0.0s

Answer 5

Copy all files to a throwaway dir, hand pick the one you want, discard the rest.

COPY . /throwaway
RUN cp /throwaway/requirements.txt . || echo 'requirements.txt does not exist'
RUN rm -rf /throwaway

You can achieve something similar using build stages, which relies on the same solution, using cp to conditionally copy. By using a build stage, your final image will not include all the content from the initial COPY.

FROM alpine as copy_stage
COPY . .
RUN mkdir /dir_for_maybe_requirements_file
RUN cp requirements.txt /dir_for_maybe_requirements_file &>- || true

FROM alpine
# Must copy a file which exists, so copy a directory with maybe one file
COPY --from=copy_stage /dir_for_maybe_requirements_file /
RUN cp /dir_for_maybe_requirements_file/* . &>- || true
CMD sh

Answer 6

Work around Solution

I had requirement on copy FOLDER to server based on ENV Variables. I took the empty server image. created required deployment folder structure at in local folder. then added below line to DockerFile copy the folder to container. In last line added entry point to execute init file.sh before docker start the server.

#below lines added to integrate testing framework
RUN mkdir /mnt/conf_folder
ADD install /mnt/conf_folder/install
ADD install_test /mnt/conf_folder/install_test
ADD custom-init.sh /usr/local/bin/custom-init.sh
ENTRYPOINT ["/usr/local/bin/custom-init.sh"]

Then create the custom-init.sh file in local with script something like below

#!/bin/bash
if [ "${BUILD_EVN}" = "TEST" ]; then
    cp -avr /mnt/conf_folder/install_test/* /mnt/wso2das-3.1.0/
else
    cp -avr /mnt/conf_folder/install/* /mnt/wso2das-3.1.0/
fi;

In docker-compose file below lines.

environment: - BUILD_EVN=TEST

These changes copy folder to container during docker build. when we execute docker-compose up it copy or deploy the actual required folder to server before server starts.

Answer 7

Tried the other ideas, but none met our requirement. The idea is to create base nginx image for child static web applications. For security, optimization, and standardization reasons, the base image must be able to RUN commands on directories added by child images. The base image does not control which directories are added by child images. Assumption is child images will COPY resources somewhere under COMMON_DEST_ROOT.

This approach is a hack, but the idea is base image will support COPY instruction for 1 to N directories added by child image. ARG PLACEHOLDER_FILE and ENV UNPROVIDED_DEST are used to satisfy <src> and <dest> requirements for any COPY instruction not needed.

#
# base-image:01
#
FROM nginx:1.17.3-alpine
ENV UNPROVIDED_DEST=/unprovided
ENV COMMON_DEST_ROOT=/usr/share/nginx/html
ONBUILD ARG PLACEHOLDER_FILE
ONBUILD ARG SRC_1
ONBUILD ARG DEST_1
ONBUILD ARG SRC_2
ONBUILD ARG DEST_2
ONBUILD ENV SRC_1=${SRC_1:-PLACEHOLDER_FILE}
ONBUILD ENV DEST_1=${DEST_1:-${UNPROVIDED_DEST}}
ONBUILD ENV SRC_2=${SRC_2:-PLACEHOLDER_FILE}
ONBUILD ENV DEST_2=${DEST_2:-${UNPROVIDED_DEST}}

ONBUILD COPY ${SRC_1} ${DEST_1}
ONBUILD COPY ${SRC_2} ${DEST_2}

ONBUILD RUN sh -x \
    #
    # perform operations on COMMON_DEST_ROOT
    #
    && chown -R limited:limited ${COMMON_DEST_ROOT} \
    #
    # remove the unprovided dest
    #
    && rm -rf ${UNPROVIDED_DEST}

#
# child image
#
ARG PLACEHOLDER_FILE=dummy_placeholder.txt
ARG SRC_1=app/html
ARG DEST_1=/usr/share/nginx/html/myapp
FROM base-image:01

This solution has obvious shortcomings like the dummy PLACEHOLDER_FILE and hard-coded number of COPY instructions that are supported. Also there is no way to get rid of the ENV variables that are used in the COPY instruction.

Answer 8

COPY no longer requires at least one source to exist, and globbing doesn't fail if there are no matches, so you can just

COPY requirements.tx[t] /destination

This will copy requirements.txt if it exists, and won't fail if it doesn't.

Answer 9

I have other workarounds for the same. The idea is to touch the file in the build context and use the copy statement inside the Dockerfile. If the file exists it will just create an empty file and the docker build will not fail. If there is already a file it will just change the time stamp.

touch requirements.txt

and for Dockerfile

FROM python:3.9
COPY requirements.txt .