docker service with compose file single node and local image

Question

So I need rolling-updates with docker on my single node server. Until now, I was using docker-compose but unfortunately, I can't achieve what I need with it. Reading the web, docker-swarm seems to be the way to go.

I have found how to run an app with multiple replicas on a single node using swarm:

docker service create --replicas 3 --name myapp-staging myapp_app:latest

myapp:latest being built from my docker-compose.yml:

version: "3.6"

services:
  postgres:
    env_file:
      - ".env"
    image: "postgres:11.0-alpine"
    volumes:
      - "/var/run/postgresql:/var/run/postgresql"
  app:
    build: "."
    working_dir: /app
    depends_on:
      - "postgres"
    env_file:
      - ".env"
    command: iex -S mix phx.server
    volumes:
      - ".:/app"

volumes:
  postgres: {}
  static:
    driver_opts:
      device: "tmpfs"
      type: "tmpfs"

Unfortunately, this doesn't work since it doesn't get the config from the docker-compose.yml file: .env file, command entry etc.

Searching deeper, I find that using

docker stack deploy -c docker-compose.yml <name>

will create a service using my docker-compose.yml config.

But then I get the following error message:

failed to update service myapp-staging_postgres: Error response from daemon: rpc error: code = InvalidArgument desc = ContainerSpec: image reference must be provided

So it seems I have to use the registry and push my image there so that it works. I understand this need in case of a multiple node architecture, but in my case I don't want to do that. (Carrying images are heavy, I don't want my image to be public, and after all, image is here, so why should I move it to the internet?)

How can I set up my docker service using local image and config written in docker-compose.yml?

I could probably manage my way using docker service create options, but that wouldn't use my docker-compose.yml file so it would not be DRY nor maintainable, which is important to me.

docker-compose is a great tool for developers, it is sad that we have to dive into DevOps tools to achieve such common features as rolling updates. This whole swarm architecture seems too complicated for my needs at this stage.

have you tried running ```docker-compose -f docker-compose.yml up``` ? — Bhaskar, Sep 16 '19 at 08:30
also, in your services/app you have not specified an image.. which probably is the reason for this error... — Bhaskar, Sep 16 '19 at 08:32

score 3 · Accepted Answer · answered Sep 16 '19 at 10:14

You don't have to use registeries in your single node setup. you can build your "app" image on your node from a local docker file using this command -cd to the directory of you docker file-

docker build . -t my-app:latest

This will create a local docker image on your node, this image is only visible to your single node which is benefitial in your use case but i wouldn't recommend this in a production setup.

You can now edit the compose file to be:

version: "3.6"

services:
  postgres:
    env_file:
      - ".env"
    image: "postgres:11.0-alpine"
    volumes:
      - "/var/run/postgresql:/var/run/postgresql"
  app:
    image: "my-app:latest"
    depends_on:
      - "postgres"
    env_file:
      - ".env"
    volumes:
      - ".:/app"

volumes:
  postgres: {}
  static:
    driver_opts:
      device: "tmpfs"
      type: "tmpfs"

And now you can run your stack from this node and it will use your local app image and benefit from the usage of the image [updates - rollbacks ...etc]

I do have a side note though on your stack file. You are using the same env file for both services, please mind that swarm will look for the ".env" file relative/next to the ".yml" file, so if this is not intentional please revise the location of your env files.

Also on a side note this solution is only feasable on a single node cluster and if you scale your cluster you will have to use a registery and registeries dont have to be public, you can deploy a private registery on your cluster and only your nodes can access it -or you can make it public- the accessibility of your registery is your choice.

Hope this will help with your issue.

Thanks for your answer, I'll git it a go! So feedback: `this solution is only feasable on a single node cluster` => Yes, that exactly my question :). `i wouldn't recommend this in a production setup` => Why so? For the lack of automation? `if this is not intentional` => I merged postgres and app's environment variables at this stage. Anyway, for now I'm just playing with config so security is not a thing here (yet!). — Augustin Riedinger, Sep 16 '19 at 15:41
In a production environment you probably will have a multi node cluster and since the image is built on one node so the other nodes will not be able to pull the image and run the service -unless you copy it manually which is tideous and inefficient :) -. And for the variables files, merging the variables will work i think :)) good luck. — M.Hassan, Sep 17 '19 at 09:11

vaibhav kharode · Answer 2 · 2019-09-18T05:05:26.587

0

Instead of docker images, you can directly use the docker file there. please check the below example.

version: "3.7"
services:
  webapp:
    build: ./dir

The error is because of compose unable to find an image on the Docker public registry. Above method should solve your issue.

edited Sep 18 '19 at 05:05

answered Sep 12 '19 at 11:17

vaibhav kharode

11
3

1

your docker file must be in dir directory. – vaibhav kharode Sep 12 '19 at 11:17
I can't see how moving the `Dockerfile` to the `dir` directory will change anything. Or did I miss something? – Augustin Riedinger Sep 12 '19 at 11:52
For example, your docker-compose file inside /user/docker-compose.yaml, so keep your dockerfile in /user/dir/dockerfile. – vaibhav kharode Sep 12 '19 at 11:59

Mani · Answer 3 · 2019-09-22T09:19:36.517

Basically you need to use docker images in order to make the rolling update to work in docker swarm. Also I would like to clarify that you can host a private registry and use it instead of public one.

Detailed Explanation: When you try out rolling update how docker swarm works is that it sees whether there is a change in the image which is used for the service if so then docker swarm schedules service updation based on the updation criteria's set up and will work on it. Let us say there is no change to the image then what happens? Simply docker will not apply the rolling update. Technically you can specify --force flag to make it force update the service but it will just redeploy the service.

Hence create a local repo and store the images into that and use that image name in docker-compose file to be used for a swarm. You can secure the repo by using SSL, user credentials, firewall restrictions which is up to you. Refer this for more details on deploying docker registry server.

Corrections in your compose file:

Since docker stack uses the image to create service you need to specify image: "<image name>" in app service like done in postgres service. AS you have mentioned build instruction image-name is mandatory as docker-compose doesn't know what tho name the image as.Reference.
Registry server is needed if you are going to deploy the application in multi-server. Since you have mentioned it's a single node deployment just having the image pulled/built on the server is enough. But private registry approach is the recommended.
My recommendation is that don't club all the services into a single docker-compose file. The reason is that when you deploy/destroy using docker-compose file all the services will be taken down. This is a kind of tight coupling. Of course, I understand that all the other services depend on DB. in such cases make sure DB service is brought up first before other services.
Instead of specifying the env file make it as a part of Docker file instruction. either copy the env file and source it in entry point or use ENV variable to define it.

Also just an update: Stack is just to group the services in swarm.

So your compose file should be:

version: "3.6"

services:
  postgres:
    env_file:
      - ".env"
    image: "postgres:11.0-alpine"
    volumes:
      - "/var/run/postgresql:/var/run/postgresql"
  app:
    build: "."
    image: "image-name:tag" #the image built will be tagged as image-name:tag
    working_dir: /app # note here I've removed .env file
    depends_on:
      - "postgres"
    command: iex -S mix phx.server
    volumes:
      - ".:/app"

volumes:
  postgres: {}
  static:
    driver_opts:
      device: "tmpfs"
      type: "tmpfs"

Dockerfile:

from baseimage:tag
COPY .env /somelocation
# your further instructions go here
RUN ... & \
    ... & \
    ... && chmod a+x /somelocation/.env
ENTRYPOINT source /somelocation/.env && ./file-to-run

Alternative Dockerfile:

from baseimage:tag
ENV a $a
ENV b $b
ENV c $c # here a,b,c has to be exported in the shell befire building the image.
ENTRYPOINT ./file-to-run

And you may need to run

docker-compose build

docker-compose push (optional needed to push the image into registry in case registry is used)]

docker stack deploy -c docker-compose.yml <stackname>

NOTE: Even though you can create the services as mentioned here by @M.Hassan I've explained the ideal recommended way.

docker service with compose file single node and local image

3 Answers3