Setting up CI/CD on Gitlab for django project

Question

This is the first time I've tried adding CI/CD for a Django project on gitlab. I want to set up automatic testing and deploying to the server in the development branch if it is successful. With tests almost everything worked out, dependencies are installed and started python manage.py test but the problem with the testing database. The traceback error is a bit lower and here I don't quite understand how the interaction with the database happens during the tests.

Creating test database for alias 'default'...
 .....
 MySQLdb._exceptions.OperationalError: (2002, "Can't connect to MySQL server on '127.0.0.1' (115)")
 The above exception was the direct cause of the following exception:
 Traceback (most recent call last):
   File "manage.py", line 21, in <module>
     main()
   File "manage.py", line 17, in main
...
     super(Connection, self).__init__(*args, **kwargs2)
 django.db.utils.OperationalError: (2002, "Can't connect to MySQL server on '127.0.0.1' (115)")

In the settings of django settings.py, a connector to the database is obtained through such variables in a .env file.

.env

SECRET_KEY=ja-t8ihm#h68rtytii5vw67*o8=o)=tmojpov)9)^$h%9#16v&
DEBUG=True
DB_NAME=db_name
DB_USER=username
DB_PASSWORD=dbpass
DB_HOST=127.0.0.1

And with the deployment of the project, everything is not clear yet. I'd appreciate your help setting it up.

gitlab-ci.yml

stages:
  - test
  - deploy

test:
  stage: test
  script:
  - apt update -qy
  - apt install python3 python3-pip virtualenvwrapper -qy
  - virtualenv --python=python3 venv/
  - source venv/bin/activate
  - pip install -r requirements.txt
  - python manage.py test 

stage: deploy
  script:
  ...
  ???
  only:
  - develop

UPD Accordingly Ruddra recommendation I added to yml file next line:

services:
- mysql

variables:
  # Configure mysql service (https://hub.docker.com/_/mysql/)
  MYSQL_DATABASE: test
  MYSQL_ROOT_PASSWORD: mysql

connect:
  image: mysql
  script:
  - echo "SELECT 'OK';" | mysql --user=root --password="$MYSQL_ROOT_PASSWORD" --host=mysql "$MYSQL_DATABASE"

And as a result I got connect successful status and test error status with the same traceback as a starting question

You're trying to connect to your localhost database. You need to set up a database for your production environment and make a separate set of environment variables that are read only on production — Snackoverflow, Mar 09 '20 at 09:51
Check https://aws.amazon.com/getting-started/tutorials/create-mysql-db/ to see how to setup a database in AWS. Check this to find more about how to load environment variables on production only: https://stackoverflow.com/questions/10664244/django-how-to-manage-development-and-production-settings — Snackoverflow, Mar 09 '20 at 10:05

ruddra · Accepted Answer · 2020-03-09T18:44:04.393

1

Actually you can run MySQL as service in GitLab. For example:

services:
  - mysql:latest

variables:
  # Configure mysql environment variables (https://hub.docker.com/_/mysql/)
  MYSQL_DATABASE: "db_name"
  MYSQL_ROOT_PASSWORD: "dbpass"
  MYSQL_USER: "username"
  MYSQL_PASSWORD: "dbpass"

Update: In your .env file, update the following settings:

DB_HOST=mysql

Update 2: (Based on this issue on GitLab) You can update the code like this:

variables:
      MYSQL_DATABASE: "db_name"
      MYSQL_ROOT_PASSWORD: "dbpass"
      MYSQL_USER: "username"
      MYSQL_PASSWORD: "dbpass"

test:
  script:
    - apt update -qy
    - mysql --user=$MYSQL_USER --password=$MYSQL_PASSWORD --database=$MYSQL_DATABASE --host=$MYSQL_HOST --execute="SHOW DATABASES; ALTER USER '$MYSQL_USER'@'%' IDENTIFIED WITH mysql_native_password BY '$MYSQL_PASSWORD'"

edited Mar 09 '20 at 18:44

answered Mar 09 '20 at 09:57

ruddra

50,746
7
78
101

I got `django.db.utils.OperationalError: (1045, 'Plugin sha256_password could not be loaded: /usr/lib/x86_64-linux-gnu/mariadb19/plugin/sha256_password.so: cannot open shared object file: No such file or directory')` – Jekson Mar 09 '20 at 10:36
This might help: https://stackoverflow.com/a/50014173/2696165 – ruddra Mar 09 '20 at 10:39
Yes this link is usefull. I changed to - `mysql:5.7` but unfortunately there is now an access error. `django.db.utils.OperationalError: (1045, "Access denied for user 'userlocal'@'172.17.0.3' (using password: YES)")` – Jekson Mar 09 '20 at 10:49
@Jekson also you can look into Update2 section of the answer :) – ruddra Mar 09 '20 at 10:49
I'll try your second advice – Jekson Mar 09 '20 at 10:52
`/bin/bash: line 95: mysql: command not found` . Wrong sintax? – Jekson Mar 09 '20 at 11:02
1

Again, please see the updated answer. (based on this: https://stackoverflow.com/a/46873279/2696165) – ruddra Mar 09 '20 at 11:05
Why is it so complicated! Now I get `Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)` – Jekson Mar 09 '20 at 11:20
Okay, then just use `script` for that. Please see the updated answer – ruddra Mar 09 '20 at 11:47
With `script` the result is the same. – Jekson Mar 09 '20 at 12:29
I will mark that the answer is correct, because in principle the current problem is another question. And just create a new question by the database connection error. – Jekson Mar 09 '20 at 12:30

Setting up CI/CD on Gitlab for django project

1 Answers1