Can't share global variable value between jobs in gitlab ci yaml file

Question

I'm trying to build an application using GitLab CI.

The name of the generated file is depending on the time, in this format

DEV_APP_yyyyMMddhhmm

(example: DEV_APP_201810221340, corresponding to the date of today 2018/10/22 13h40).

How can I store this name in a global variable inside the .gitlab-ci.yml file?

Here is my .gitlab-ci.yml file:

image: docker:latest
image: docker:latest
services:
- docker:dind

variables:
  DOCKER_DRIVER: overlay
  SPRING_PROFILES_ACTIVE: gitlab-ci
#   TIME: ""
#  BRANCH: ""
#  REC_BUILD_NAME: ""
  TIME: "timex"
  BRANCH: "branchx"
  DEV_BUILD_NAME: "DEV_APP_x"

stages:
- preparation
- build
- package
- deploy
- manual_rec_build
- manual_rec_package

job_preparation:
  stage: preparation
  script:
  - echo ${TIME}
  - export TIME=$(date +%Y%m%d%H%M)
  - "BRANCH=$(echo $CI_BUILD_REF_SLUG | sed 's/[^[[:alnum:]]/_/g')"
  - "DEV_BUILD_NAME=DEV_APP_${BRANCH}_${TIME}"
  - echo ${TIME}

maven-build:
  image: maven:3-jdk-8
  stage: build
  script:
  - echo ${TIME}
  - "mvn package -B"
  artifacts:
paths:
    - target/*.jar
  only:
  - merge-requests
  - /^feature\/sprint.*$/
  - /^DEV_.*$/
#  when: manual


docker-build:
  stage: package
  script:
  - echo ${TIME}
  - docker build -t registry.gitlab.com/mourad.sellam/actuator-simple .
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
  - docker push registry.gitlab.com/mourad.sellam/actuator-simple
  only:
  - merge-requests
  - /^feature\/sprint.*$/
  - /^DEV_.*$/
  when: manual

    k8s-deploy-production:
  image: google/cloud-sdk
  stage: deploy
  script:
  - echo ${TIME}
  - echo "$GOOGLE_KEY" > key.json
  - gcloud auth activate-service-account --key-file key.json
  - gcloud config set compute/zone europe-west1-c
  - gcloud config set project actuator-sample
  - gcloud config set container/use_client_certificate True
  - gcloud container clusters get-credentials actuator-example
  - kubectl delete secret registry.gitlab.com
  - kubectl create secret docker-registry registry.gitlab.com --docker-server=https://registry.gitlab.com --docker-username=myUserName--docker-password=$REGISTRY_PASSWD --docker-email=myEmail@gmail.com
  - kubectl apply -f deployment.yml --namespace=production
  environment:
    name: production
    url: https://example.production.com
  when: manual

job_manual_rec_build:
  image: maven:3-jdk-8
  stage: manual_rec_build
  script:
  - echo ${TIME}
  - "mvn package -B"
  artifacts:
    paths:
    - target/*.jar
  when: manual
    #   allow_failure: false

job_manual_rec_package:
  stage: manual_rec_package
  variables:
  script:
    - echo ${TIME}
  - echo ${DEV_BUILD_NAME}
  - docker build -t registry.gitlab.com/mourad.sellam/actuator-simple:${DEV_BUILD_NAME} .
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
- docker push registry.gitlab.com/mourad.sellam/actuator-simple
  artifacts:
    paths:
    - target/*.jar
  when: on_success
  #test 1

When I call

echo ${TIME}

It displays "timex".

echo faild

Could you tell me how to store a global variable and set it in each job?

I couldn't pass variables this way either. I manually trigger through API using curl request and send variables through that. This works but you will not get that cool pipeline graph thats all — Macindows, Feb 08 '20 at 12:53

score 59 · Answer 1 · edited Jun 06 '23 at 13:37

Check if GitLab 13.0 (May 2020) could help in your case:

Inherit environment variables from other jobs

Passing environment variables (or other data) between CI jobs is now possible.

By using the dependencies keyword (or needs keyword for DAG pipelines), a job can inherit variables from other jobs if they are sourced with dotenv report artifacts.

This offers a more graceful approach for updating variables between jobs compared to artifacts or passing files.

See documentation and issue.

You can inherit environment variables from dependent jobs.

This feature makes use of the artifacts:reports:dotenv report feature.

Example with dependencies keyword.

build:
  stage: build
  script:
    - echo "BUILD_VERSION=hello" >> build.env
  artifacts:
    reports:
      dotenv: build.env

deploy:
  stage: deploy
  script:
    - echo $BUILD_VERSION # => hello
  dependencies:
    - build

Example with the needs keyword:

build:
  stage: build
  script:
    - echo "BUILD_VERSION=hello" >> build.env
  artifacts:
    reports:
      dotenv: build.env

deploy:
  stage: deploy
  script:
    - echo $BUILD_VERSION # => hello
  needs:
    - job: build
      artifacts: true

Updated documentation: [`artifacts:reports:dotenv`](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportsdotenv) — Andrew Gunsch, Aug 01 '22 at 23:56

score 10 · Answer 2 · answered May 21 '19 at 13:58

10

You can use artifacts for passing data between jobs. Here's example from Flant to check previous pipeline manual decision:

approve:
  script:
    - mkdir -p .ci_status
    - echo $(date +%s) > .ci_status/approved
  artifacts:
    paths:
      - .ci_status/

NOT approve:
  script:
    - mkdir -p .ci_status
    - echo $(date +%s) > .ci_status/not_approved
  artifacts:
    paths:
      - .ci_status/

deploy to production:
  script:
    - if [[ $(cat .ci_status/not_approved) > $(cat .ci_status/approved) ]]; then echo "Need approve from release engineer!"; exit 1; fi
    - echo "deploy to production!"

answered May 21 '19 at 13:58

Dmitry

470
6
9

Thanks. This assumes that all jobs run on the same server / under the same runner. – Leonid Jan 27 '20 at 07:20
Generally, yes. But you can specify path for shared mount for all runners from host. And path at host itself can be mounted from somewhere :) – Dmitry Jan 28 '20 at 08:19
Ah, good point, thanks. With many jobs these paths would need to be well organized as to not collide but I suppose this is outside of the scope of the question. That said, it would result in a giant & fragile mess of a state saved on the file system. Ideally I would want Gitlab to handle in a clean and robust way like git or a DBMS - I don't need to know much about how either is implemented to use them. – Leonid Jan 29 '20 at 08:27
I cannot pass artifacts between successive jobs using different containers. Is this possible with Gitlab-CI? – Paul Rougieux Apr 15 '21 at 16:55
1

@PaulRougieux yep, but it reqires to mount shared volume between jobs. Where you need to mount depends where is your artifacts are. In gitlab-runner container (which executes commands you defined in gitlab-ci.yml) or in your container, where all magic goes. Second is straight forward - you control it. First is tricky - you need define this volume in config.toml for all runners. Beware of concequences. – Dmitry Apr 18 '21 at 09:43

score 2 · Answer 3 · answered Oct 22 '18 at 13:08

2

There's an open issue 47517 'Pass variables between jobs' on Gitlab CE..

CI/CD often needs to pass information from one job to another and artifacts can be used for this, although it's a heavy solution with unintended side effects. Workspaces is another proposal for passing files between jobs. But sometimes you don't want to pass files at all, just a small bit of data.

I have faced the same issue, and workaround this by storing DATA in file, then access to it in other Jobs..

answered Oct 22 '18 at 13:08

FuSsA

4,223
7
39
60

1

It doesn't work for me, the created file is not found because the directory is changing depending on the job i guess – mourad Oct 22 '18 at 15:15
Thank you for your suggestions – mourad Oct 29 '18 at 16:07
It works using pipeline id instead of time, thank you for your suggestions. – mourad Oct 29 '18 at 16:09

Michal · Answer 4 · 2022-09-29T20:45:23.293

You kind of can....The way I went about it:

You can send a POST request to the project to save a variable:

export VARIABLE=secret

curl --request POST --header "PRIVATE-TOKEN: $CI_ACCESS_TOKEN" "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/variables/" --form "key=VARIABLE" --form "value=$VARIABLE"

and cleanup after the work/trigger is finished

curl --request DELETE --header "PRIVATE-TOKEN: $CI_ACCESS_TOKEN" "https://gitlab.seznam.net/api/v4/projects/$CI_PROJECT_ID/variables/VARIABLE"

I'm not sure it suppose to be used this way, but it does the trick. You have the variable accessible for all the following jobs (specially when you use trigger and script in that job is not an option.)

Just please make sure, you run the cleanup job even if previous once fail...

Can't share global variable value between jobs in gitlab ci yaml file

4 Answers4

Inherit environment variables from other jobs

Linked