Jenkins "file not found" error with existing Bash script

Question

My goal is to have Jenkins 2 execute alpha integration tests between an express js app and a postgres db. I am to spin up containerized resources locally and test successfully with bash scripts that employ docker-compose. The relevant bash script is scripts/docker/dockerRunTest.sh.

However, when I try to do the same thing via Jenkins, Jenkins claims that the initiating script is not found.

Jenkinsfile

stage('Alpha Integration Tests') {
  agent {
    docker {
      image 'tmaier/docker-compose'
      args '-u root -v /var/run/docker.sock:/var/run/docker.sock --network host'
    }
  }
  steps {
    sh 'ls -lah ./scripts/docker/'
    sh './scripts/docker/dockerRunTest.sh'
  }
}

Output

+ ls -lah ./scripts/docker/
total 36
drwxr-xr-x    2 root     root        4.0K Jan 26 21:31 .
drwxr-xr-x    6 root     root        4.0K Jan 26 20:54 ..
-rwxr-xr-x    1 root     root        2.2K Jan 26 21:31 docker.lib.sh
-rwxr-xr-x    1 root     root         282 Jan 26 21:31 dockerBuildApp.sh
-rwxr-xr-x    1 root     root         289 Jan 26 21:31 dockerBuildTestRunner.sh
-rwxr-xr-x    1 root     root         322 Jan 26 21:31 dockerDown.sh
-rw-r--r--    1 root     root         288 Jan 26 21:31 dockerRestart.sh
-rwxr-xr-x    1 root     root         482 Jan 26 21:31 dockerRunTest.sh
-rwxr-xr-x    1 root     root         284 Jan 26 21:31 dockerUp.sh

+ ./scripts/docker/dockerRunTest.sh
/var/jenkins_home/workspace/project-name@2@tmp/durable-9ac0d23a/script.sh: line 1: ./scripts/docker/dockerRunTest.sh: not found
ERROR: script returned exit code 127

The file clearly exists per the ls output. I have some hazy idea that there may be some conflict between how shell scripts and bash scripts work, but I cannot quite grasp the nuance in how Jenkins is not able to execute a script that clearly exists.

edit (including script contents):
dockerRunTest.sh

#!/bin/bash

MY_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd -P )"
MY_DIR="${MY_DIR:?}"
SCRIPTS_DIR="$(realpath "${MY_DIR}/..")"
ROOT_DIR="$(realpath "${SCRIPTS_DIR}/..")"
TEST_DIR="${ROOT_DIR}/test/integration"
SRC_DIR="${ROOT_DIR}/src"
REPORTS_DIR="${ROOT_DIR}/reports"

. "${SCRIPTS_DIR}/docker/docker.lib.sh"

dockerComposeUp
dockerExecuteTestRunner
dockerComposeDown

docker.lib.sh

#!/bin/bash

CURRENT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd -P )"
CURRENT_DIR="${CURRENT_DIR:?}"
SCRIPTS_DIR="$(realpath "${CURRENT_DIR}/..")"
ROOT_DIR="$(realpath "${SCRIPTS_DIR}/..")"

. "${SCRIPTS_DIR}/lib.sh"

dockerComposeUp() {
  docker-compose build --no-cache
  docker-compose up --detach --force-recreate
  DC_CODE=$?

  if [ ${DC_CODE} -ne 0 ]; then
    # Introspection
    docker-compose logs
    docker-compose ps

    exit ${DC_CODE}
  fi
}

dockerComposeDown() {
  # docker-compose rm: Removes stopped service containers.
  #   -f, --force - Don't ask to confirm removal.
  #   -s, --stop - Stop the containers, if required, before removing.
  #   -v - Remove any anonymous volumes attached to containers.
  docker-compose rm --force --stop -v
}

dockerComposeRestart() {
  dockerComposeDown
  dockerComposeUp
}

dockerBuildTestRunner() {
  docker build -f test/Dockerfile -t kwhitejr/botw-test-runner .
}

dockerExecuteTestRunner() {
  IMAGE_NAME="kwhitejr/botw-test-runner"

  echo "Build new ${IMAGE_NAME} image..."
  dockerBuildTestRunner

  echo "Run ${IMAGE_NAME} executable test container..."
  docker run -it --rm --network container:api_of_the_wild_app_1 kwhitejr/botw-test-runner
}

Can you include the shebang from the script? and is the named interpreter installed on Jenkins? I think that error is subtly different from the names script itself not found. — erik258, Jan 26 '19 at 22:15
Included the relevant bash scripts in the main post. Not sure what a "named interpreter" is? Do you mean Bash vs vanilla shell? — Kwhitejr, Jan 27 '19 at 01:44
Yeah. `/bin/bash` must be available for this to work. Is it? This would prove it beyond all doubt: `ls -l /bin/bash` — erik258, Jan 27 '19 at 14:32
The docker commands would probably go smoother in Jenkins via the Docker Pipeline plugin. — Matthew Schuchard, Jan 27 '19 at 14:35
@MattSchuchard maybe you can offer an answer explaining what you're saying? Otherwise that seems like an arbitrary preference. I'd be interested in seeing how that makes it smoother. One thing I like about the jenkinsfile as it is is that it's pretty clear to modify it and it ties the code repo to the jenkins exec, to which it's inherently coupled anyway. — erik258, Jan 27 '19 at 14:38

Bless · Accepted Answer · 2019-01-27T17:01:06.957

10

tmaier/docker-compose image doesn't have /bin/bash interpreter installed by default since latest tag is an alpine image [1, 2]. This can be confirmed by running:

$ docker run -it --rm tmaier/docker-compose bash
/usr/local/bin/docker-entrypoint.sh: exec: line 35: bash: not found

To get the script working, either install bash in the docker image using apk add bash or change the shebang to #!/bin/sh if the script can be run using ash shell (the default shell in busybox).

[1] https://github.com/tmaier/docker-compose/blob/b740feb61fb25030101638800a609605cfd5e96a/Dockerfile#L2

[2] https://github.com/docker-library/docker/blob/d94b9832f55143f49e47d00de63589ed41f288e7/18.09/Dockerfile#L1

edited Jan 27 '19 at 17:01

answered Jan 27 '19 at 14:39

Bless

5,052
2
40
44

2

Ahh, investigating in the given public docker image - glorious. Very clever. FWIW, "vanilla sh" isn't really valid. Different shells have different syntaxes and several predate bash ( csh, ksh, bourne which this probably closly emulates). The _real_ `sh` on this platform is busybox (try `$ docker run -it --rm --entrypoint sh tmaier/docker-compose --help | head -n 1`) – erik258 Jan 27 '19 at 14:48
1

Good point. I see busybox uses `ash`. Updated the answer to reflect that. Also learnt a few things from here - https://stackoverflow.com/questions/5725296/difference-between-sh-and-bash. – Bless Jan 27 '19 at 14:55
1

I resolved this issue by forking `tmaier/docker-compose` and adding `bash` to the installs: https://github.com/kwhitejr/docker-compose – Kwhitejr Jan 27 '19 at 16:57

score 2 · Answer 2 · answered Nov 11 '19 at 12:55

I have the similar issue but in my case, it is because the shell script file has EOL in Windows format (if you open the file in the terminal using vi, you will see each line ends with ^M)

I can fix this using Notepad++ Edit -> EOL Conversion -> Unix (LF)

Jenkins "file not found" error with existing Bash script

2 Answers2