19

I have a GitLab pipeline that I want to:

  1. Build a Java app
  2. Test using docker-compose
  3. Push to my Docker repository

The primary issue I'm having is that this works:

services:
  - docker:dind

docker_test:
  stage: docker_test
  image: docker:latest
  script:
  - docker version

The output is printed as expected:

> gitlab-ci-multi-runner exec docker --docker-privileged docker_test
...
$ docker version
Client:
 Version:      17.06.0-ce
...
Server:
 Version:      17.06.0-ce
...
Build succeeded

While this does not (installation steps for docker-ce omitted):

services:
  - docker:dind

docker_test:
  stage: docker_test
  image: ubuntu:latest       << note change
  script:
  - docker version

It fails with:

$ docker version
Client:
 Version:      17.06.0-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:23:31 2017
 OS/Arch:      linux/amd64
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
ERROR: Build failed: exit code 1
FATAL: exit code 1

How do I make my ubuntu image (or whatever image is going to build my project) connect to the linked Docker-in-Docker service? What is docker:latest doing that I'm not?

I've read up on the GitLab services documentation, but it only makes sense to me from a hostname perspective. (If you have a mysql service, you can connect over mysql:3306.)

Edit: Updating the command to echo $DOCKER_HOST, I see in the docker:latest image:

$ echo $DOCKER_HOST
tcp://docker:2375

And in the ubuntu:latest image I see:

$ echo $DOCKER_HOST
(nothing - but SO doesn't let me add a blank code line)
Craig Otis
  • 31,257
  • 32
  • 136
  • 234
  • Can you please do this in both examples? instead of `docker version`, do this `echo $DOCKER_HOST` – Robert Jul 28 '17 at 12:27
  • Thanks @Robert - answer updated. Looks like `$DOCKER_HOST` is not set in the Ubuntu image. Is `tcp://docker:2375` a constant-enough value that I should export the variable in my Dockerfile, or is there a better/programmatic way of determining it? – Craig Otis Jul 28 '17 at 13:16
  • I've posted the suggestion as an answer. Please tell me whether it works. – Robert Jul 28 '17 at 13:27
  • I think that that will be a fixed value, so no problem to hard-code it. It is not too ugly. – Robert Jul 28 '17 at 21:11
  • Thanks @Robert - but it definitely seems like that's the trick. I'm still wiring things up and double-checking, but will let you know. – Craig Otis Jul 29 '17 at 01:10

2 Answers2

22

As the information you've added, I hope that this does work:

services:
  - docker:dind

docker_test:
  stage: docker_test
  image: ubuntu:latest
  variables:
      DOCKER_HOST: "tcp://docker:2375"
  script:
  - docker version

Alternatively:

services:
  - docker:dind

docker_test:
  stage: docker_test
  image: ubuntu:latest 
  script:
  - export DOCKER_HOST=tcp://docker:2375
  - docker version

It seems that Gitlab does not set the DOCKER_HOST variable for custom images.

Robert
  • 33,429
  • 8
  • 90
  • 94
2

The Gitlab CI runner needs to mount the hosts docker socket in order to excute docker in docker. For example: 

docker exec gitlab-runner gitlab-runner register \
           --non-interactive \
           --url https://gitlab.com/ci \
           --registration-token TOKEN\
           --description "Docker Runner" \
           --tag-list "docker" \
           --executor docker \
           --docker-image "docker:latest" \
           --docker-volumes /var/run/docker.sock:/var/run/docker.sock

As you can see the docker.sock is mounted here as last parameter. However i would not recommend the use of docker-compose for Gitlab CI, since Gitlab CI has its own syntax https://docs.gitlab.com/ce/ci/docker/using_docker_images.html

madnight
  • 420
  • 3
  • 10
  • Few things - first, I don't have control over the `gitlab-runner`, as I'm using the free tier DigitalOcean servers. All I have control over (essentially) is what goes in my `.gitlab-ci.yml`. I will try to rework my build setup to avoid `docker-compose`, but it was extremely nice for the local and GitLab builds to use the same compose file/process. Otherwise things become duplicated in both `.gitlab-ci,yml` and `docker-compose.yml`, like my MySQL instance, my Redis instance, etc. – Craig Otis Jul 26 '17 at 11:31
  • And while I'm familiar with mounting `docker.sock` as a volume to the guest container, it doesn't quite explain how the `docker:latest` container is able to access the `dind` service, while my `ubuntu:latest` image is not. When I run the `docker` command inside the `docker:latest` container, is it communicating with my `dind` service? – Craig Otis Jul 26 '17 at 11:32
  • **This is NOT docker in docker**. Sharing the host's `docker.sock` means jobs will of course use the hosts' docker daemon. The idea of the Gitlab `service:` is to run a separate `dockerd` as a service alongside your job. – istepaniuk Aug 22 '23 at 14:47