0

Relatively new to Docker so trying to understand how to accomplish my task.

Locally, I:

  1. Build the image
  2. Push the image to some URL
  3. SSH into Linux VM
  4. docker pull image from URL
  5. docker run image_name

This image, when run, downloads 2 fairly large csv.gz's. When unzipped, the two CSV's are about 15GB each.

I set up /app on the Linux VM to have 200GB available. So, in short, I need to have the Docker image download those 2 CSV's there. However no matter what I've tried within my Dockerfile, I see

'No space left on device' when it gets to the part to download the CSVs.

I've tried to set WORKDIR to /app, but that does not help.

Do I need to use a daemon.json file? Does some sort of Docker setting need to be changed on the Linux VM? Do I need to look into Docker volumes?

Relevant pieces of Dockerfile:

FROM centos/python-36-centos7
USER root
WORKDIR /usr/src/app
COPY . .

As for /usr/src/app, I've never seen anything in there. I normally use /usr/src/app since that's what I use for my Cloud Foundry deployments.

Any insight to point me in the right direction would be appreciated.

Nubtacular
  • 1,367
  • 2
  • 18
  • 38
  • Are you downloading the files during build? If so that would be a bad design. You should be doing the download during the run. Also you should use volumes to make sure `/app` is mapped to outside – Tarun Lalwani Apr 04 '21 at 05:48
  • Nah downloaded during the docker run, not the build. Yea been looking into volumes. I keep seeing the CSVs going to `/var/lib/docker/overlay.../usr/src/app/` using `docker run -d -v myvolume:/app IMAGE python3 main.py ARG_HERE` What do you mean by mapped to outside? – Nubtacular Apr 04 '21 at 06:03
  • Map a host folder and not a named volume. So something like `-v /usr/app:/app ....` – Tarun Lalwani Apr 04 '21 at 06:17
  • Ok that makes sense. The partially downloaded CSVs still showing up in `/usr/src/app` though. Should `WORKDIR` in the Dockerfile still be `/usr/src/app`? – Nubtacular Apr 04 '21 at 16:00
  • WOKRDIR is unrelated, it decides what is the current directory when someone goes inside the container. So that depends on you – Tarun Lalwani Apr 04 '21 at 16:52

1 Answers1

0

Doing the following resolved the issue:

Create (if daemon.json does not exist): /etc/docker/daemon.json

And write

{ 
  “data-root”: “/app” 
}

Looks like by default everything goes to /var, and in my case, /var only has 4GB of space. /app is where the 200GB resides.

You will have to restart docker service when creating/saving daemon.json.

Referenced this answer: the one with 88 upvotes

Nubtacular
  • 1,367
  • 2
  • 18
  • 38