Is docker caching useless when RUN is involved?

Question

Consider this example from the docs:

# syntax=docker/dockerfile:1
FROM node
WORKDIR /app
COPY package.json yarn.lock .    # Copy package management files
RUN npm install                  # Install dependencies
COPY . .                         # Copy over project files
RUN npm build                    # Run build
By installing dependencies in earlier layers of the Dockerfile, there is no need to rebuild those layers when a project file has changed.

Which layers can be skipped here?

By my understanding, npm command is black-box for docker,
so docker doesn't know what npm's inputs are and what will it produce. If so, then docker has to always run npm install and npm build commands, which means caching is useless here.

What am I missing here?

Docker doesn't know what the RUN commands will do, but it assumes they're pure in terms of the preceding layers. Read the big note boxout about apt-get on the page you're linking to, for example. — jonrsharpe, Aug 10 '23 at 08:25
@jonrsharpe So docker will cache `RUN curl https://bbc.com -o bbc.txt` as well? — IndustryUser1942, Aug 10 '23 at 08:31

score 0 · Answer 1 · answered Aug 10 '23 at 08:25

Is docker caching useless when RUN is involved?

No.

Which layers can be skipped here?

All.

so docker doesn't know what npm's inputs are and what will it produce

Docker exactly knows that - the input is all the files and the image produced from earlier stages, and the output are all the modifications made to the files in the image. The modifications are saved as a new layer.

score 0 · Answer 2 · answered Aug 10 '23 at 08:46

[…] docker doesn't know what npm's inputs are and what will it produce.

Docker indeed doesn't know what npm install means (e.g., you might configure your terminal to alias npm install to echo 42 or anything else). But it does know that running it does not depend on any files.

"What will it produce" depends on its inputs (as @jonrsharpe also said in the comments). So, if inputs aren't changed, the outputs aren't changed either, Docker can infer that.

Compare it with, for example, COPY package.json yarn.lock . which does depend on the contents of package.json and yarn.lock. Conversely, if either of those files is changed, then the COPY … command is re-applied, and thus the subsequent RUN npm install is re-applied as well.

So, to answer the titular question:

Is docker caching useless when RUN is involved?

It's rather the opposite: RUN-ing side-effect'y scripts is useless when Docker caching is involved.

Is docker caching useless when RUN is involved?

2 Answers2

Linked