grep failing in gitlab CI

Question

I am trying to finalyze a script in Gitlab CI but struggling with some syntax error

  script: |        
          echo "running jenkins job from user $EMAIL using following settings - $BRANCH /  $TAGS in $ENV enviroment" 
          
          lastbuildNumber=$(curl -s --user ${EMAIL}:${TOKEN} "$JENKINS_URL/$ENV-SmokeTests-UIOnly/lastBuild/api/json"  | jq ".number")

          echo "last build was number ${lastbuildNumber}"
          currentBuild=$((lastbuildNumber + 1 ))  
          
          echo "current build is ${currentBuild}"

          echo "view cucumber report here"
          baseurl="$JENKINS_URL/${ENV}-SmokeTests-UIOnly"
          echo $baseurl
          
          curl -s --user $EMAIL:$TOKEN $JENKINS_URL/$ENV-SmokeTests-UIOnly/ --output cucumber.txt


          cucumber_endpoint=$(cat cucumber.txt | grep -o -m 1 "[a-zA-Z.-]*/cucumber-html-reports_[a-zA-Z0-9.-]*/[a-zA-Z0-9.-]*")

          full_cucumber=$baseurl$cucumber_endpoint

          echo $full_cucumber

The script works fine on my local terminal, but fails in the CI when running

cucumber_endpoint=$(cat cucumber.txt | grep -o -m 1 "[a-zA-Z.-]*/cucumber-html-reports_[a-zA-Z0-9.-]*/[a-zA-Z0-9.-]*")

is for sure something related to quotes but cannot work out what the issue is.

update:

I changed to:

after_script:
    - |
        echo "view cucumber report here"
        baseurl="$JENKINS_URL/job/${ENV}-SmokeTests-UIOnly"

        curl -s --user "$EMAIL":"$TOKEN" $JENKINS_URL/"$ENV"-SmokeTests-UIOnly/ --output cucumber.txt
        cat cucumber.txt 
        
        cucumber_endpoint=$(cat cucumber.txt | grep -o -m 1 '[a-zA-Z.-]*/cucumber-html-reports_[a-zA-Z0-9.-]*/[a-zA-Z0-9.-]*')
        full_cucumber="${baseurl}${cucumber_endpoint}"
        echo "${full_cucumber}"

and I have run the script through 'shellcheck.net'

it's the grep that is not working but is not returning anyerror now.

the result of the cat command are correct, as on my local machine.

proving is not an issue with set -e

#!/bin/bash
set -e
        echo "view cucumber report here"
        baseurl="https://example"

        cucumber_endpoint=$(curl -s --user "$EMAIL":"$TOKEN" ${JENKINS_URL}/"$ENV"-SmokeTests-UIOnly/ | grep -o -m 1 '[a-zA-Z.-]*/cucumber-html-reports_[a-zA-Z0-9.-]*/[a-zA-Z0-9.-]*')
        # cat cucumber.txt 
        
        # cucumber_endpoint=$(cucumber.txt | grep -o -m 1 '[a-zA-Z.-]*/cucumber-html-reports_[a-zA-Z0-9.-]*/[a-zA-Z0-9.-]*')
        full_cucumber="${cucumber_endpoint}"
        echo "${baseurl}${full_cucumber}"

which gets what I want:

 ➜ ./cucumber.sh                                                                                                                                                                                                                                         [16/02/23|11:39:59|]
view cucumber report here
https://example/cucumber-html-reports_fb3a3468-c298-3fb5-ad9a-dacbc0323763/overview-features.html

I'm pretty sure the script runs with `set -e` and your problem is unrelated to quoting. If you find otherwise, please [edit] your question to clarify, and ping me to get it reopened. If there really is a syntax error, what is the precise error message? — tripleee, Feb 16 '23 at 10:24
However, your script contains various quoting errors etc; probably run it through https://shellcheck.net/ — tripleee, Feb 16 '23 at 10:24
I can't see any indication that my guess is incorrect. What's in the `cat` output? (Notice also that the `cat` to `grep` is [useless](https://stackoverflow.com/questions/11710552/useless-use-of-cat).) — tripleee, Feb 16 '23 at 11:23
I need to get and endpoint of an url that matches the regex in my grep call. the cat output is just a html file. and I was just decomposing the commands to debug. — bruvio, Feb 16 '23 at 11:39

score 0 · Accepted Answer · answered Feb 16 '23 at 11:55

0

apparently gitlab ci did not like the -m 1 option in the grep call

so changed to

cucumber_endpoint=$(curl -s --user "$EMAIL":"$TOKEN" ${JENKINS_URL}/"$ENV"-SmokeTests-UIOnly/ | grep -o '[a-zA-Z.-]*/cucumber-html-reports_[a-zA-Z0-9.-]*/[a-zA-Z0-9.-]*'| sort -u)

answered Feb 16 '23 at 11:55

bruvio

853
1
9
30

`sort -u` is a weird attempt to fix it; perhaps `head -n 1` would be better? Or `tail -n 1`? – tripleee Feb 16 '23 at 14:12

grep failing in gitlab CI

1 Answers1