Gitlab CI/CD: New runner has not been connected yet

Question

I am very new to Gitlab CI/CD and I have read its documentation very carefully about creating a new CI/CD process using .gitlab-ci.yml file. As I have found out in order to have Continuous Deployment (also known as CD), it is needed to register a new gitlab-runner on my linux server.

Explanation

Here is my .gitlab-ci.yml file:

stages:
  - build
  - deploy

docker-build:
  image: docker:stable
  services:
    - docker:dind
  stage: build
  only:
    refs:
      - ci-test
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build --pull -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA .
    - echo $CI_REGISTRY_IMAGE
    - echo $CI_COMMIT_SHORT_SHA
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

deploy:
  image: docker:stable
  stage: deploy
  services:
    - docker:dind
  only:
    refs: 
      - ci-test
  when: manual
  except:
    changes:
      - "*.md"
  script:
    - docker build --pull -t $CI_REGISTRY_IMAGE$CI_COMMIT_SHORT_SHA .
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

and this is my Dockerfile:

version: '3'

services:
  app:
#    build: .
    environment:
      - DEFAULT_DATABASE_HOST=${DEFAULT_DATABASE_HOST}
      - DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
      - DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
      - DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
      - DEBUG=${DEBUG}
      - DJANGO_SETTINGS_MODULE=_base.settings
      - CELERY_BROKER_URL=redis://redis:6379/0
#    command: uwsgi --http :8000 --socket /socket/api.sock --chmod-socket=666 --module _base.wsgi --master --processes 5 --threads 2
    command: uwsgi --http :8000 --module _base.wsgi --master --processes 5 --threads 2
#    command: python manage.py runserver 0.0.0.0:8000
    depends_on:
      - redis
    ports:
      - 8000:8000
    image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}
    restart: on-failure
    network_mode: host

  redis:
    image: redis:6.0-alpine
    volumes:
      - /var/lib/redis/redis.dump:
    ports:
      - 6379:6379
#
  celery-beat:
    environment:
      - DEFAULT_DATABASE_HOST=${DEFAULT_DATABASE_HOST}
      - DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
      - DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
      - DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
      - DEBUG=${DEBUG}
      - DJANGO_SETTINGS_MODULE=_base.settings
      - CELERY_BROKER_URL=redis://redis:6379/0
    command: celery -A _base beat -l info
    depends_on:
      - redis
    image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}

  celery-worker-default:
    environment:
      - DEFAULT_DATABASE_HOST=172.17.0.1
      - DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
      - DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
      - DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
      - DEBUG=${DEBUG}
      - DJANGO_SETTINGS_MODULE=_base.settings
      - CELERY_BROKER_URL=redis://redis:6379/0
    command: celery -A _base worker -l INFO -Q clearsight-default --concurrency 1 -n clearsight-default
    depends_on:
      - redis
    image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}


  celery-worker-aws:
    environment:
      - DEFAULT_DATABASE_HOST=172.17.0.1
      - DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
      - DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
      - DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
      - DEBUG=${DEBUG}
      - DJANGO_SETTINGS_MODULE=_base.settings
      - CELERY_BROKER_URL=redis://redis:6379/0
    command: celery -A _base worker -l INFO -Q clearsight-aws --concurrency 1 -n clearsight-default
    depends_on:
      - redis
    image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}

  watchtower:
    image: containrrr/watchtower:1.3.0
    container_name: watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ~/.docker/config.json:/config.json
    restart: always
    command: --interval 60

Problem I have tried the CI pipeline and it works properly with shared runners:

So, I think there isn't any problem in CI side. Therefore, I registered a new gitlab-runner (docker+machine) in the server and it can be seen in the repository CI/CD runners, but as it can be seen in the below picture, it is not connected!

Question How can I resolve the runner issue and make the runner connect to the jobs?

Mostafa Ghadimi · Accepted Answer · 2021-08-16T11:03:02.673

The problem was solved after running the gitlab-runner verify command.

~ ➤ gitlab-runner -h # shows the help and other commands of gitlab-runner
NAME:
   gitlab-runner - a GitLab Runner

USAGE:
   gitlab-runner [global options] command [command options] [arguments...]

VERSION:
   13.11.0 (7f7a4bb0)

AUTHOR:
   GitLab Inc. <support@gitlab.com>

COMMANDS:
     exec                  execute a build locally
     list                  List all configured runners
     run                   run multi runner service
     register              register a new runner
     install               install service
     uninstall             uninstall service
     start                 start service
     stop                  stop service
     restart               restart service
     status                get status of a service
     run-single            start single runner
     unregister            unregister specific runner
     verify                verify all registered runners
     artifacts-downloader  download and extract build artifacts (internal)
     artifacts-uploader    create and upload build artifacts (internal)
     cache-archiver        create and upload cache artifacts (internal)
     cache-extractor       download and extract cache artifacts (internal)
     cache-init            changed permissions for cache paths (internal)
     health-check          check health for a specific address
     read-logs             reads job logs from a file, used by kubernetes executor (internal)
     help, h               Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --cpuprofile value           write cpu profile to file [$CPU_PROFILE]
   --debug                      debug mode [$DEBUG]
   --log-format value           Choose log format (options: runner, text, json) [$LOG_FORMAT]
   --log-level value, -l value  Log level (options: debug, info, warn, error, fatal, panic) [$LOG_LEVEL]
   --help, -h                   show help
   --version, -v                print the version

Now it works fine:

Thanks @WytrzymałyWiktor for his comment on this post. I didn't find anything helpful other than his comment.

P.S After performing the above steps, you may need to run gitlab-runner start in order to resolve your problem!

Perfect 100% if you use windows use this command ".\gitlab-runner.exe verify" — Hesham Magdy, Nov 25 '21 at 12:31
I have faced a similar issue when I used the REST API to register a runner. `curl --request POST "/api/v4/runners --form "xxxx" --form "description=Group-Runner-1"` did register it but the runner was in `not_connected` state. I had to use `curl --request POST "/api/v4/runners/verify" --form "token=yyyy"` to bring it online. I failed to find any documentation on this too. Currently, the runner is active but it is not picking up jobs. I might have to run `gitlab-runner start` so it can start picking jobs. — dragsu, Mar 04 '22 at 20:54
Also resolved the issue "Runner has never contacted this instance." — Josiah Yoder, Aug 11 '22 at 19:17

andersonjwan · Answer 2 · 2022-07-16T14:58:27.583

7

For those that are using GitLab Runner under Docker, use the equivalent command to verify your new instance as provided by @MostafaGhadimi above:

docker exec <container name> /bin/sh -c "gitlab-runner verify"

edited Jul 16 '22 at 14:58

answered Oct 03 '21 at 05:57

andersonjwan

96
2
3

score 7 · Answer 3 · answered Nov 21 '21 at 05:15

7

For me sudo gitlab-runner verify worked. Second command to be executed is sudo gitlab-runner run. For custom runner, it is important to manually execute the runner.

answered Nov 21 '21 at 05:15

B Pruthi

71
1
3

score 3 · Answer 4 · answered Jan 05 '23 at 14:21

3

Running

gitlab-runner verify

makes it connected again.

answered Jan 05 '23 at 14:21

siiraaj

368
3
13

Gitlab CI/CD: New runner has not been connected yet

4 Answers4

Linked