GitLab CI How to POST JSON data to a url within a CI job using here-document?

Question

Hoping someone can help. I am experiencing difficulty making a curl POST request with JSON data from a gitlab CI job.

The curl request works fine in a local terminal session, (N.B I did not use double quotes in terminal session). If I do not escape double quotes in the gitlab CI yaml I get the error curl: (3) [globbing] nested brace in column 112

If I escape the double quotes in the GitLab CI job, as shown below I get the error: curl: (3) [globbing] unmatched brace in column 1

In all cases I get the error /bin/bash: line 134: warning: here-document at line 134 delimited by end-of-file (wanted `EOF')

Is it possible to POST JSON data using here-documents from a GitLab CI job?

.gitlab-ci.yml job extract

release:
  image: node:12-stretch-slim
  stage: release
  before_script:
    - apt-get update && apt-get install -y curl git jq
  script:
    - version=$(git describe --tags | cut -d'-' -f 1 | sed 's/^v*//')
    - echo "generating release for version ${version}"
    - npm pack
    # - >
    #   url=$(curl
    #   --header "Private-Token: $API_TOKEN"
    #   -F "file=@${CI_PROJECT_DIR}/objectdetection-plugin-${version}.tgz" "https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/uploads"
    #   |
    #   jq '.url')
    - url="http://www.example.com"
    - echo "Retrieved url from file upload as ${url}"
    - echo "The full url would be ${CI_PROJECT_URL}/${url}"
    - >
      curl -X POST https://requestbin.io/1f84by61
      --header 'Content-Type: application/json; charset=utf-8'
      --data-binary @- << EOF
      {
        \"name\": \"Release v${version}\",
        \"tag_name\": \"v${version}\",
        \"ref\": \"v${version}\",
        \"description\": \"Test\",
        \"assets\": {
          \"links\": [
            {
              \"name\": \"objectdetection-plugin-source\",
              \"url\": \"CI_PROJECT_URL/${url}\",
              \"filepath\": \"${url}\",
              \"link_type\": \"other\"
            }
          ]
        }
      }
      EOF
  when: manual
  only:
    - /^release-.*$/

score 9 · Accepted Answer · answered Jun 29 '20 at 15:01

Solved it after reading this

Using |- preserves newlines within the command and does not append a newline at the end of the command string. Used this principle to save the JSON data to a variable and then referenced the variable in the subsequent curl command.

Below I have included the script:

release:
  image: node:12-stretch-slim
  stage: release
  before_script:
    - apt-get update && apt-get install -y curl git jq
  script:
    - git fetch --prune --unshallow
    - version=$(git describe --tags | cut -d'-' -f 1 | sed 's/^v*//')
    - npm pack
    - >
      url=$(curl --silent --show-error
      --request POST
      --header "Private-Token: $API_TOKEN"
      -F "file=@${CI_PROJECT_DIR}/objectdetection-plugin-${version}.tgz" "https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/uploads"
      |
      jq --raw-output --monochrome-output '.url')
    - |-
      PAYLOAD=$(cat << JSON
      {
        "name": "Release v$version",
        "tag_name": "v$version",
        "ref": "v$version",
        "description": "$(sed -zE 's/\r\n|\n/\\n/g' < CHANGELOG.md)",
        "assets": {
          "links": [
            {
              "name": "objectdetection-plugin-source",
              "url": "$CI_PROJECT_URL$url",
              "filepath": "$url",
              "link_type": "other"
            }
          ]
        }
      }
      JSON
      )
    - echo "$PAYLOAD"
    - >
      http_response=$(curl --silent --show-error --write-out "%{http_code}" -o response.txt
      --request POST "https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/releases"
      --header 'Content-Type: application/json'
      --header 'Accept: application/json'
      --header "Private-Token: ${API_TOKEN}"
      --data-binary "${PAYLOAD}")
    - |-
      if [ $http_response != "201" ]; then
        exit 1
      else
        echo "Server returned:"
        cat response.txt    
      fi
  when: manual
  allow_failure: false
  only:
    - /^release-.*$/

score 0 · Answer 2 · answered Oct 03 '22 at 17:48

I was trying to send a message to my Google Chat, WebHook URL, and the above solution didn't work.

So I used this instead

'curl --silent --show-error --location --request POST ${GCHAT_DEVOPS_WEBHOOK} --header ''Content-Type: application/json; charset=UTF-8'' --data-raw ''{"text":"Hey <users/all>: A new build has been deployed to *''${APP_ENVIRONMENT}''* via ''${CI_JOB_NAME}'' - *''${APP_BRANCH}''*"}'''

GitLab CI How to POST JSON data to a url within a CI job using here-document?

2 Answers2