Serving static content with Django, Gunicorn and Ngnix in shared development environment

Question

I have seen many great answers to this question here on SO, but either I don't understand them or they don't apply to my particular circumstance. I have looked here: Serving static files with Nginx + Gunicorn + Django and many others. I have followed the recommendation in those answers and I still do not have solution that works. I hope that if I explain exactly what I am doing then maybe someone will tell me where I went wrong.

I developing in a shared environment with several other teams and we share an ngnix server. I have a Django project on this shared server, sre-dev.example.com. The path to the Django project is /apps/capman/capman_port10001/capman.

In my settings.py I have these values set:

STATIC_ROOT = '/apps/capman/capman_port10001/capman/static'
STATIC_URL = '/static/'
INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'corsheaders',
    'rest_framework',
    'rest_framework.authtoken',
    ...
    'django_db_logger',
    'rest_framework_swagger'
]

I have created a /etc/nginx/sites-enabled directory and sym link,

ln -s /apps/capman/capman_port10001/capman/nginx.conf caasa-dev.example.com

I also created alias (CNAME), caasa-dev.example.com in our DNS for sre-dev. And in added a nginx.conf file, /apps/capman/capman_port10001/capman/nginx.conf with the contents of:

server {
    listen 10001;
    server_name caasa-dev.example.com;

    location /static {
        alias /apps/capman/capman_port10001/capman/static/;
    }
}

I have executed ...

python manage.py collectstatic

... several times now and the files are there:

$ pwd
/apps/capman/capman_port10001/capman/static
$ ls
admin  rest_framework  rest_framework_swagger

I have restarted nginx and gunicron. I have a systemd configuration for my gunicorn server so I can do stuff like:

systemctl restart gnuicorn-capman.service

If I do systemctl -l status gnuicorn-capman.service I get this output:

# systemctl -l status gnuicorn-capman.service
● gnuicorn-capman.service - capman gnuicron service
   Loaded: loaded (/etc/systemd/system/gnuicorn-capman.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2018-12-14 12:42:40 GMT; 17min ago
  Process: 14949 ExecStopPost=/bin/rm -rf /run/gnuicorn-capman (code=exited, status=0/SUCCESS)
  Process: 14946 ExecStop=/bin/kill -s TERM $MAINPID (code=exited, status=0/SUCCESS)
  Process: 14955 ExecStartPre=/bin/chown -R sreuser:example /run/gnuicorn-capman /apps/capman/capman_port10001 (code=exited, status=0/SUCCESS)
  Process: 14953 ExecStartPre=/bin/mkdir /run/gnuicorn-capman (code=exited, status=0/SUCCESS)
 Main PID: 14960 (gunicorn)
   CGroup: /system.slice/gnuicorn-capman.service
           ├─14960 /apps/capman/capman_port10001/env/bin/python2.7 /apps/capman/capman_port10001/env/bin/gunicorn --workers 4 capman.wsgi --bind 0.0.0.0:10001 --access-logfile ./access.log --error-logfile ./error.log --timeout 600
           ├─14966 /apps/capman/capman_port10001/env/bin/python2.7 /apps/capman/capman_port10001/env/bin/gunicorn --workers 4 capman.wsgi --bind 0.0.0.0:10001 --access-logfile ./access.log --error-logfile ./error.log --timeout 600
           ├─14968 /apps/capman/capman_port10001/env/bin/python2.7 /apps/capman/capman_port10001/env/bin/gunicorn --workers 4 capman.wsgi --bind 0.0.0.0:10001 --access-logfile ./access.log --error-logfile ./error.log --timeout 600
           ├─14970 /apps/capman/capman_port10001/env/bin/python2.7 /apps/capman/capman_port10001/env/bin/gunicorn --workers 4 capman.wsgi --bind 0.0.0.0:10001 --access-logfile ./access.log --error-logfile ./error.log --timeout 600
           └─14971 /apps/capman/capman_port10001/env/bin/python2.7 /apps/capman/capman_port10001/env/bin/gunicorn --workers 4 capman.wsgi --bind 0.0.0.0:10001 --access-logfile ./access.log --error-logfile ./error.log --timeout 600

Yet still when I attempt to go to caasa-dev.example.com:10001/admin/login/?next=/admin/ I do not get my static content:

I also get messages like this ...

... "GET /static/admin/css/base.css HTTP/1.1" 404 2793 "http://caasa-dev.example.com:10001/admin/login/?next=/admin/" "Mozilla/5.0 (Macintosh; Intel Mac OS X

... in my access.log.

So what have I left out? What am I doing wrong?

Update: If I go to caasa-dev.example.com:10001/static/ I the standard Django 404 error page.

Update 2:

For some reason my log files are empty:

[root@sre-dev sites-enabled]# systemctl -l status nginx
● nginx.service - The NGINX HTTP and reverse proxy server
   Loaded: loaded (/etc/systemd/system/nginx.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2018-12-14 12:42:37 GMT; 56min ago
  Process: 14926 ExecStop=/bin/kill -s QUIT $MAINPID (code=exited, status=0/SUCCESS)
  Process: 14932 ExecStart=/apps/nginx/sbin/nginx (code=exited, status=0/SUCCESS)
  Process: 14930 ExecStartPre=/apps/nginx/sbin/nginx -t (code=exited, status=0/SUCCESS)
 Main PID: 14934 (nginx)
   CGroup: /system.slice/nginx.service
           ├─14934 nginx: master process /apps/nginx/sbin/ngin
           └─14935 nginx: worker process

Dec 14 12:42:37 sre-dev systemd[1]: Starting The NGINX HTTP and reverse proxy server...
Dec 14 12:42:37 sre-dev nginx[14930]: nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
Dec 14 12:42:37 sre-dev nginx[14930]: nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
Dec 14 12:42:37 sre-dev systemd[1]: Started The NGINX HTTP and reverse proxy server.
[root@sre-dev sites-enabled]# ps -ef | grep nginx
root     14934     1  0 12:42 ?        00:00:00 nginx: master process /apps/nginx/sbin/nginx
nobody   14935 14934  0 12:42 ?        00:00:00 nginx: worker process
root     26500  3799  0 13:40 pts/8    00:00:00 grep --color=auto nginx
[root@sre-dev sites-enabled]# vim /etc/nginx/nginx.conf
[root@sre-dev sites-enabled]# grep log /etc/nginx/nginx.conf
error_log /var/log/nginx/error.log;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    access_log  /var/log/nginx/access.log  main;
[root@sre-dev sites-enabled]# ls -l /var/log/nginx/error.log
-rw-r--r--. 1 nginx root 0 Jun  1  2018 /var/log/nginx/error.log
[root@sre-dev sites-enabled]# ls -l /var/log/nginx/access.log
-rw-r--r--. 1 nginx nginx 0 Jun  7  2018 /var/log/nginx/access.log

Update 3:

Thanks Ivan but should my /apps/capman/capman_port10001/capman/nginx.conf be:

server {
    listen 80;
    server_name caasa-dev.example.com;

    location /static {
        alias /apps/capman/capman_port10001/capman/static/;
    }

    location / {
        proxy_pass http://127.0.0.1:10001;
    }
}

or

server {
    listen 10001;
    server_name caasa-dev.example.com;

    location /static {
        alias /apps/capman/capman_port10001/capman/static/;
    }

    location / {
        proxy_pass http://127.0.0.1:80;
    }
}

Update 4:

I just noticed that my /etc/nginx/nginx.conf does not include my sites-enabled directory:

[root@sre-dev nginx]# grep -r include *
nginx.conf:include /usr/share/nginx/modules/*.conf;
nginx.conf:    include             /etc/nginx/mime.types;
nginx.conf:    # See http://nginx.org/en/docs/ngx_core_module.html#include
nginx.conf:    include /etc/nginx/conf.d/*.conf;
nginx.conf:        include /etc/nginx/default.d/*.conf;
nginx.conf:#        include /etc/nginx/default.d/*.conf;
nginx.conf.default:    include       mime.types;
nginx.conf.default:        #    include        fastcgi_params;

Should I add the line ...

include /etc/nginx/sites-enabled/*.conf

... to my /etc/nginx/nginx.conf and if so where in that file?

Try to open static url manually. What is the response, error message? — Ivan Starostin, Dec 14 '18 at 13:17
Nginx config does not seem to be applied - `/static/` is still forwarded to django. See anything at all in nginx logs? — Ivan Starostin, Dec 14 '18 at 13:36

Ivan Starostin · Answer 1 · 2018-12-14T13:48:40.507

1

gunicorn --workers 4 capman.wsgi --bind 0.0.0.0:10001

nginx: listen 10001;

I guess nginx did not even start (see equal port?). All the requests are processed by Django. Change port.

And if this is full Nginx config you have then it's missing proxy_pass block to forward requests to django

edited Dec 14 '18 at 13:48

answered Dec 14 '18 at 13:44

Ivan Starostin

8,798
5
21
39

Thanks Ivan. In what file do I change the port? – Red Cricket Dec 14 '18 at 13:46
if 10001 is supposed to be published port then leave it for nginx and change gunicorn config. – Ivan Starostin Dec 14 '18 at 13:49
10001 is the port my team is suppose to use for our django project so as to not interfere with other dev teams. Everything works find if I run the project with `python manage.py runserver`. Also I new to nginx and gunicron. If you could tell me more explicitly I'd be so greatful. – Red Cricket Dec 14 '18 at 13:54
I cannot know what your config **should be**. I found the major flaw, which port to choose for any of these services is up to you and your team. I'm not sure if you may take 80 for any of these services since you are sharing infrastructure with others. – Ivan Starostin Dec 14 '18 at 14:39

score 1 · Answer 2 · answered Dec 14 '18 at 15:12

The problem is that u are using gunicorn to serve django but nginx is not listening on that port so that reverse proxy ith to you domain name. use this nginx config:

server {
listen 80;
server_name caasa-dev.example.com;

location /static {
    alias /apps/capman/capman_port10001/capman/static;
}

location / {
    proxy_pass http://127.0.0.1:10001;
}
}

In this config, nginx with act as a reverse proxy, and route all traffic to port 80 of your domain, to internal network 127.0.0.1:10001 which is handled by gunicorn.

note you are using gunicorn on address 0.0.0.0. I donot recommend that. change it to 127.0.0.1 instead or use linux .sock files for better config.

Serving static content with Django, Gunicorn and Ngnix in shared development environment

2 Answers2