Deploy to docker with nginx, django, daphne

Question

I want to deploy my service to docker.

and my service is developed using python+django and django-channels

── myproject ├── myproject │ ├── settings.py │ ├── urls.py │ ├── asgi.py │ ├── ... ├── collected_static │ ├── js │ ├── css │ ├── ... ├── nginx │ ├── Dockerfile │ ├── service.conf ├── requirements.txt ├── manage.py ├── Dockerfile └── docker-compose.yml

myproject/Dockerfile :

FROM python
ENV PYTHONUNBURRERED 1

RUN mkdir -p /opt/myproject
WORKDIR /opt/myproject
ADD . /opt/myproject

RUN pip install -r requirements.txt
RUN python manage.py migrate

myproject/docker-compose.yml:

version: '2'
services:
  nginx:
    build: ./nginx
    networks:
      - front
      - back
    ports:
      - "80:80"
    depends_on:
      - daphne
  redis:
    image: redis
    networks:
      - "back"
    ports:
      - "6379:6379"
  worker:
    build: .
    working_dir: /opt/myproject
    command: bash -c "python manage.py runworker"
    environment:
      - REDIS_HOST=redis
    networks:
      - front
      - back
    depends_on:
      - redis
    links:
      - redis
  daphne:
    build: .
    working_dir: /opt/myproject
    command: bash -c "daphne -b 0.0.0.0 -p 8000 myproject.asgi:channel_layer"
    ports:
      - "8000:8000"
    environment:
      - REDIS_HOST=redis
    networks:
      - front
      - back
     depends_on:
      - redis
     links:
      - redis
  networks:
    front:
    back:

myproject/nginx/Dockerfile

FROM nginx
COPY service.conf /etc/nginx/sites-enabled/

myproject/nginx/service.conf

server {
  listen 80;
  server_name example.com #i just want to hide domain name..
  charset utf-8;
  client_max_body_size 20M;

  location /static/ {
    alias /opt/myproject/collected_static/;
  }

  location / {
    proxy_pass http://0.0.0.0:8000;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";

    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Host $server_name;
  }
}

and i write a command docker-compose up -d, nginx and daphne work well.

but when i connected to example.com:80, i just can see nginx default page.

and when i connected to example.com:8000, i just can see myproject's service page. (but cannot see static files)

I want to link nginx and daphne service! what should I do? please help me.

• when i just deploy with nginx+daphne+django without docker, my service works well.

Answer 1

TLDR;

Nginx is not configured correctly, but also your docker-compose needs some correction:

Nginx

The Nginx website has some helpful tips for deploying with Docker that you should read, including a sample, very simple Dockerfile:

FROM nginx
RUN rm /etc/nginx/conf.d/default.conf
RUN rm /etc/nginx/conf.d/example_ssl.conf
COPY content /usr/share/nginx/html
COPY conf /etc/nginx

which points to some improvements you need to make (see the Docker Compose section for further help with Docker).

Bearing in mind the updates to deployment that we will make below, you will also need to change your Nginx config:

• rename service.conf -> service.template
• change listen ${NGINX_PORT};
• change server_name ${NGINX_HOST};
• change proxy_pass http://${DAPHNE_HOST}:${DAPHNE_PORT};

Docker Compose

Now your Nginx configuration is correct, you need to setup the docker compose directives correctly, thankfully, the Docker Hub Nginx page has an example for docker compose:

Here is an example using docker-compose.yml:

web:
  image: nginx
  volumes:
   - ./mysite.template:/etc/nginx/conf.d/mysite.template
  ports:
   - "8080:80"
  environment:
   - NGINX_HOST=foobar.com
   - NGINX_PORT=80
  command: /bin/bash -c "envsubst < /etc/nginx/conf.d/mysite.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"

The mysite.template file may then contain variable references like this:

listen ${NGINX_PORT};

From r00m's answer

You can make all those improvements, and in fact, without sharing the volumes your static files won't be served correctly.

• Create an image for the project and re-use it
• Add the Volume references to allow static files to be shared
• OPTIONAL: you should also follow the advice about collecting the static files, but your project structure kind of suggests that you've already done that.

Bringing it all together

Finally, we can merge those three improvements to give us the following setup:

myproject/Dockerfile:

FROM python
ENV PYTHONUNBUFFERED 1

RUN mkdir -p /opt/myproject
WORKDIR /opt/myproject
ADD . /opt/myproject

RUN pip install -r requirements.txt
RUN python manage.py migrate # Can this be done during build? i.e. no link to the DB?

VOLUME ["/opt/myproject/collected_static"]

myproject/docker-compose.yml:

version: '2'
services:
  nginx:
    build: ./nginx
    networks:
      - front
      - back
    ports:
      - "80:80"
    volumes_from:
      - "daphne"
    environment:
      - NGINX_HOST=example.com
      - NGINX_PORT=80
      - DAPHNE_HOST=daphne
      - DAPHEN_PORT=8000
    depends_on:
      - daphne
    links:
      - daphne
    command: /bin/bash -c "envsubst < /etc/nginx/conf.d/service.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"
  redis:
    image: redis
    networks:
      - "back"
    ports:
      - "6379:6379"
  daphne:
    build: .
    image: "myproject:latest"
    working_dir: /opt/myproject
    command: bash -c "daphne -b 0.0.0.0 -p 8000 myproject.asgi:channel_layer"
    ports:
      - "8000:8000"
    environment:
      - REDIS_HOST=redis
    networks:
      - front
      - back
     depends_on:
      - redis
     links:
      - redis
  worker:
    image: "myproject:latest"
    working_dir: /opt/myproject
    command: bash -c "python manage.py runworker"
    environment:
      - REDIS_HOST=redis
    networks:
      - front
      - back
    depends_on:
      - redis
    links:
      - redis
  networks:
    front:
    back:

myproject/nginx/Dockerfile

FROM nginx
RUN rm /etc/nginx/conf.d/default.conf
RUN rm /etc/nginx/conf.d/example_ssl.conf
COPY service.template /etc/nginx/conf.d

myproject/nginx/service.template

server {
  listen ${NGINX_PORT};
  server_name ${NGINX_HOST}
  charset utf-8;
  client_max_body_size 20M;

  location /static/ {
    alias /opt/myproject/collected_static/;
  }

  location / {
    proxy_pass http://${DAPHNE_HOST}:${DAPHNE_PORT};
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";

    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Host $server_name;
  }
}

Final thoughts

• I'm not sure what you're trying to achieve with your network directives, but it almost certainly doesn't achieve it, for example nginx shouldn't connect into your backend network (I think...).
• You need to consider whether "migrate" should be done at build time or run time.
• Do you need to be able to change your nginx configuration easily? If so, you should remove the COPY from the nginx build and add in the volumes directive from the Docker Compose section.

Answer 2

You have misconfigured NGINX. Try proxy_pass http://127.0.0.1:8000;

As for the static files, it's because you haven't made the files available to the container. I would suggest the following modifications:

myproject/Dockerfile:

[...]
ADD . /opt/myproject
VOLUME ["/opt/myproject/collected_static"]
[..]
# may I also suggest automatic static file collection?
RUN python manage.py collectstatic --noinput

myproject/docker-compose.yml:

[...]
build: ./nginx
volumes_from:
  - "worker" # or daphne

I would also consider adding image option to daphne and worker services. This will tag the image and allow to reuse it, thus it will be only built once (instead of twice).

myproject:
  build: .
  image: "myproject:latest"
[..]
worker:
  image: "myproject:latest"
[..]
daphne:
  image: "myproject:latest"