2

I am receiving the following error message in my job log:

217 $ docker run -e TWINE_USERNAME \  # collapsed multi-line command
218 docker: invalid reference format.
219 See 'docker run --help'.
223 ERROR: Job failed: exit code 125

Full log is available at https://gitlab.com/oxr463/buildbot_tyrian_theme/-/jobs/360851515#L217.

The .gitlab-ci.yml job is as follows:

pypi:
  type: deploy
  variables:
    TWINE_USERNAME: $TWINE_USERNAME
    TWINE_PASSWORD: $TWINE_PASSWORD
  before_script:
    - |
         docker run -v $(pwd):/opt/buildbot_tyrian_theme \
                    $CI_REGISTRY/oxr463/buildbot_tyrian_theme \
                    python setup.py bdist_wheel sdist --formats gztar
  script:
    - |
         docker run -e TWINE_USERNAME \ 
                    -e TWINE_PASSWORD \
                    -v $(pwd):/opt/buildbot_tyrian_theme \
                    $CI_REGISTRY/oxr463/buildbot_tyrian_theme twine upload dist/*
  only:
    - tags

Source: https://gitlab.com/oxr463/buildbot_tyrian_theme/blob/master/.gitlab-ci.yml

This only occurs when running via GitLab CI/CD; the command works fine if ran locally.

Update:

Instead of a multi-line string for the script command, I collapsed it down to a single line and it worked. However, I would still like to figure out why it was not working despite it passing the GitLab CI/CD linter.

References:

See also:

oxr463
  • 1,573
  • 3
  • 14
  • 34
  • Did you set these variables in the CI settings for this repo? Maybe you have to remove the two variable declarations from your .gitlab-ci.yml. – derkoe Nov 25 '19 at 21:27
  • Yes, they are defined under Settings > CI/CD > Variables. I will try removing those declarations for my next build. – oxr463 Nov 26 '19 at 00:15
  • Nope, that failed too, (See: https://gitlab.com/oxr463/buildbot_tyrian_theme/-/jobs/360998617). – oxr463 Nov 26 '19 at 00:18

3 Answers3

0

Services are created before the other scripts are evaluated, so I'd declare global variables:

#top of .gitlab-ci.yml
variables:

Try echoing TWINE_USERNAME and TWINE_PASSWORD in the script block to verify their values.

theberbie
  • 94
  • 1
  • 6
  • This did not resolve the issue, (See: https://gitlab.com/oxr463/buildbot_tyrian_theme/pipelines/98958866). – oxr463 Nov 27 '19 at 19:48
0

That docker run command is executed into the shell (as you see from the starting $); hence that TWINE_USERNAME is interpreted as string and not as environment variable.

The command should probably be:

$ docker run -e "$TWINE_USERNAME" \ # collapsed multi-line command

This way the env-var value will be expanded into your command.

EDIT: I missed the -e option and gave a wrong answer. I can give another couple of advices.

Try these two:

  1. Ensure the variable is set correctly and is available in the script.
script:
    - echo "TWINE_USERNAME is set to '${TWINE_USERNAME}'"
    - |
         docker run -e TWINE_USERNAME ....
  1. Try to use -e/--env as setter rather than as proxy. Explicitly set its value, sourcing it from the surrounding environment
docker run -e TWINE_USERNAME="$TWINE_USERNAME"

This use of the option should be safer. I honestly am not 100% sure that just specifying the name will proxy the value; I have no experience with it.

But try setting it explicitly first and ensure that the issue isn't somewhere else.

Kamafeather
  • 8,663
  • 14
  • 69
  • 99
  • Hope it helped! – I'm moving my first steps in GitlabCI/CD, so I might be wrong. – **Tip:** when something doesn't work as expected I usually add an initial `before_script` entry with `echo ` and then check the output logs into Gitlab Jobs, to ensure every variable gets expanded correctly (also it makes easy to copy and paste in the local shell to verify that the syntax is correct). – Kamafeather Sep 23 '20 at 10:27
  • My bad, I didn't check the `-e` flag meaning (import/proxy specific envs). – First check the env value is available, `echo`ing as said in the previous comment. If valid, then your code should import it correctly. If it doesn't, try to specify the value explicitly, with `docker run -e TWINE_USERNAME="$TWINE_USERNAME"` -- Btw, in `gitlab-ci.yml` I would also define it as `TWINE_USERNAME: "${TWINE_USERNAME}"` to avoid problems with whitespaces. – Kamafeather Sep 23 '20 at 10:47
  • Sorry for the mess ; I updated this answer and [added another one](https://stackoverflow.com/a/64026615/3088045) with a solution unrelated to the passage of env-variables. – Kamafeather Sep 23 '20 at 11:10
0

Note: I attempt a second answer, to not mix up in a confusing way with the other answer.

Remove the schema from the registry URI

Yesterday I had a similar error message, and the sneaky problem in my case was that the env, with the private registry URI value, was specifying a scheme https://, while Docker complains for it with a generic

invalid reference format.

I spent half afternoon digging it, thinking that the problem was laying somewhere else..

Give a try: echo "$CI_REGISTRY" and ensure that there's no schema into your $CI_REGISTRY. If it's there, remove it. Docker will add it automatically.

Let us know if it resolved your issue!

Kamafeather
  • 8,663
  • 14
  • 69
  • 99
  • `CI_REGISTRY` is a predefined variable, (See: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html). It can be overridden but I am not sure if I should be doing that... – oxr463 Sep 23 '20 at 13:22
  • Yes, you shouldn't. But check what's the value stored in it. I might be wrong, but yesterday my problem was solved ensuring it didn't contain the URL schema. Obtain the command, with all the expanded values, and you can check if copy/pasting into your local shell it gives the same `invalid reference format` error. For what I understood, the *reference* is the full URI to the image tag into the repository on the registry; so something is probably wrong there (in my specific case it was the unnecessary `https://` schema). – Kamafeather Sep 23 '20 at 13:28