19

My current docker-compose.yml -

# This docker-compose file uses '.env' file present in the current directory, 
# for database credentials. If you want to change the credentials please 
# change the data in '.env'.
# '.env' file might be hidden as dot files are hidden please unhide to see it.
# Know more about '.env' file: https://docs.docker.com/compose/env-file/

version: '3'

services: 
  postgresdb:
    image: postgres:9.5
    environment: 
      POSTGRES_USER: ${ENV_POSTGRES_USER}
      POSTGRES_PASSWORD: ${ENV_POSTGRES_PASSWORD}
      POSTGRES_DB: ${ENV_POSTGRES_DB}
    volumes: 
      - "../app/volumes/postgres/data:/var/lib/postgresql/data"

  # This is python service. It uses python 3.6 as base image.
  # It will build this service using the Dockerfile present in current directory
  # To modify the values of environment variables please open '.env' file.
  # This service will not run until postgresdb service gets started
  python-app:
    image: python:3.6
    build: .    # Builds using Dockerfile from current directory
    depends_on: 
      - postgresdb
    ports: 
      - "5001:5001"
    tty: true
    volumes: 
      - "../app/volumes/trained_knn_model.clf:/usr/src/app/my-app/trained_knn_model.clf"
      - "../app/volumes/XYPickle.pickle:/usr/src/app/my-app/XYPickle.pickle"
    environment: 
      - POSTGRES_USER=${ENV_POSTGRES_USER}
      - POSTGRES_PASSWORD=${ENV_POSTGRES_PASSWORD}
      - POSTGRES_HOST=${ENV_POSTGRES_HOST}
      - POSTGRES_PORT=${ENV_POSTGRES_PORT}
      - POSTGRES_DB=${ENV_POSTGRES_DB}

My docker-compose.yml file contains 2 services. I have specified postgrasdb service to start before python-app service using depends_on but the docker-compose in not running the services in specified order. How can I get postgrasdb service to be run before python-app service? I am running docker-compose up --build --remove-orphans command.

gravetii
  • 9,273
  • 9
  • 56
  • 75
Sunil
  • 429
  • 1
  • 9
  • 25
  • you have to use `depend_on` with condition – Shoshi Oct 08 '18 at 10:12
  • 1
    @Shoshi depends_on's condition clause is no more valid in version 3. – gravetii Oct 08 '18 at 10:16
  • 2
    sorry, I was thinking of version 2. anyway, I have solved it for my case (version 3) also. you can try https://docs.docker.com/compose/startup-order/ this. I have used `wait-for-it` for my case. – Shoshi Oct 08 '18 at 10:19
  • 1
    Hey @Shoshi, thank you for your comment, can you please show me an example of using that `wait-for-it` – Sunil Oct 08 '18 at 13:24
  • **See Also**: [Docker Compose wait for container X before starting Y](https://stackoverflow.com/q/31746182/1366033) – KyleMit Nov 19 '20 at 23:14

2 Answers2

34

Note that depends_on only waits for the other container to be up, but not for the process it is running to start. The thing that could probably be happening in your case is that you are trying to connect to the postgres process on its specified port while it's still getting started.

There are two ways you can tackle such a scenario -

  1. Specify some sort of restart clause for your python-app container - You're probably seeing your python-app container in failed state and so you have posted this question. restart: on-failure:10 in the docker-compose.yml for your python-app service will restart your container up to 10 times in case it fails connecting to the postgres container. This will ensure that you would have given it enough time before the postgres container is up and running...the process that is.

  2. Use an external tool like dockerize that allows you to wait on other services before starting up the container. This essentially gives you the behavior you desire with depends_on.

gravetii
  • 9,273
  • 9
  • 56
  • 75
  • thank you for your quick response. I used `restart: on-failure:10`, now it starts `postgresdb` service before `pythonapp` service. But when I make the first request I get this log response from `postgresdb` service *postgresdb_1 | LOG: unexpected EOF on client connection with an open transaction*. This would not happen with docker-compose 1.13.0 – Sunil Oct 08 '18 at 12:37
  • Complementing @gravetii's answer, there is another cool tool to achieved this at https://github.com/ufoscout/docker-compose-wait. It gives you some other configuration options to make it more customizable – AnhellO May 18 '20 at 05:23
  • This is interesting to learn about, but it seems this only works if for example the python-app ends up in a failed state from its race condition. `sqlalchemy.exc.DatabaseError`s dont seem to trigger it and enforce restart – Caleb Jay Aug 01 '21 at 05:15
  • @CalebJay the error you're referring to - does it happen after the database service is up? If yes, of course, the fix I've suggested can do little. – gravetii Aug 02 '21 at 04:12
6

depends_on just guaranteed that database service run before web service. To run web service after database is ready, use wait_for_it.sh script https://docs.docker.com/compose/startup-order/ For example, i have a docker-compose.yml file with two services: app and db, and i want to run container app after db service are ready:

docker-compose.yml

version: "2"
services:
  app:
    image: employee-jdbc
    ports:
      - "8080:8080"
    networks:
      - employee-mysql
    depends_on:
      - mysqldb
    container_name: employee-jdbc
  db:
    image: mysql:5.7
    networks:
      - employee-mysql
    environment:
      - MYSQL_ROOT_PASSWORD=root
      - MYSQL_DATABASE=bootdb  
    container_name: mysqldb
networks:
  employee-mysql:
    external: true

In Dockerfile to build employee-jdbc image:

Dockerfile

FROM openjdk:8
COPY ./target/*.jar ROOT.jar
COPY wait-for-it.sh /wait-for-it.sh
RUN chmod +x /wait-for-it.sh
ENTRYPOINT ["./wait-for-it.sh","db:3306","--","java","-jar","ROOT.jar"]

wait_for_it.sh file you can download at: https://github.com/vishnubob/wait-for-it

Văn Huy Nguyễn
  • 61
  • 1
  • 2
  • 1
    Hi. You are linking to external resource for the actual answer. Could you extract the important parts regarding this question to the answer? – Juho Rutila Nov 06 '19 at 11:37
  • Hello, I want to use it without being a root but it says `/usr/local/bin/docker-entrypoint.sh: exec: line 8: ./wait-for: Permission denied`, is there a way to run it with a non-root user – Sofienne Lassoued May 20 '21 at 18:14