bash recursion automatically ends after single level

Question

Why is this make_request function ending just after a single traversal?

make_request(){
    path="${1//' '/'%20'}"
    echo $path
    mkdir -p $HOME/"$1"
    $(curl --output $HOME/"$1"/$FILE_NAME -v -X GET $BASE_URL"/"$API_METHOD"/"$path &> /dev/null)
    # sample response from curl
    # {
    #     "count":2,
    #     "items": [
    #         {"path": "somepath1", "type": "folder"},
    #         {"path": "somepath2", "type": "folder"},
    #     ]
    # }
    count=$(jq ".count" $HOME/"$1"/$FILE_NAME)
    for (( c=0; c<$count; c++ ))
    do
        child=$(jq -r ".items[$c] .path" $HOME/"$1"/$FILE_NAME);
        fileType=$(jq -r ".items[$c] .type" $HOME/"$1"/$FILE_NAME);
        if [ "$fileType" == "folder" ]; then
            make_request "$child"
        fi
    done
}

make_request "/"

make_request "/" should give the following output:

/folder
/folder/folder1-1
/folder/folder1-1/folder1-2
/folder/foler2-1
/folder/folder2-1/folder2-2
/folder/folder2-1/folder2-3 ...

but I am getting the following:

/folder
/folder/folder1-1
/folder/folder1-1/folder1-2

Why do you have the `curl` command inside `$()`? `$()` is used to substitute the output of a command back into the command line. But there's no output because you're redirecting to `/dev/null`. — Barmar, Jun 15 '22 at 15:45
@Barmar I am saving the output of curl using --output to some file and reading it afterward as I am also required to save the output. — gg-dev-05, Jun 15 '22 at 15:48
Put `set -x` at the beginning of the script so you see a transcript of all the commands as they're executing. — Barmar, Jun 15 '22 at 15:48
I can see that you're saving the output to a file. So there's no stdout, what do you expect to be substituted with `$()`? — Barmar, Jun 15 '22 at 15:49
@Barmar Thanks, I think `$()` is not required it is working without `$()` also — gg-dev-05, Jun 15 '22 at 15:52
You *aren't* saving the output, `$(curl ...)` would capture the output, then `bash` would try to parse and execute it as another command. The only reason you don't get an error is that you discard `curl`'s output with `>&/dev/null`, and the unquoted expansion produces nothing to parse (as opposed to an empty string which would produce a 'command not found' error). — chepner, Jun 15 '22 at 15:52
@chepner He's saving the output to a file with the `--output` option. — Barmar, Jun 15 '22 at 15:53
@chepner `--output filename` is equivalent to using `> filename`. That's not the problem. — Barmar, Jun 15 '22 at 15:54
[Shellcheck](https://www.shellcheck.net/) finds several problems with the code. — pjh, Jun 15 '22 at 15:54
See [How can I debug a Bash script?](https://stackoverflow.com/q/951336/4154375). — pjh, Jun 15 '22 at 15:58
@pjh None of the issues it finds seem significant unless there are spaces in the variables. — Barmar, Jun 15 '22 at 15:59
Please do not edit solution announcements into the question or title. See [What should I do when someone answers my question?](https://stackoverflow.com/help/someone-answers) You can also create your own answer, and even accept it, if your solution is not yet covered by an existing answer. See [Can I answer my own question?](https://stackoverflow.com/help/self-answer) — jmoerdyk, Jun 15 '22 at 16:23

Socowi · Accepted Answer · 2022-06-15T16:08:01.317

You are using global variables everywhere. Therefore, the inner call changes the loop variables c and count of the outer call, resulting in bogus.

Minimal example:

f() {
  this_is_global=$1
  echo "enter $1: $this_is_global"
  ((RANDOM%2)) && f "$(($1+1))"
  echo "exit $1: $this_is_global"
}

Running f 1 prints something like

enter 1: 1
enter 2: 2
enter 3: 3
exit 3: 3
exit 2: 3
exit 1: 3

Solution: Make the variables local by writing local count=$(...) and so on. For your loop, you have to put an additional statement local c above the for.

markp-fuso · Answer 2 · 2022-06-15T19:55:33.333

As currently written all variables have global scope; this means that all function calls are overwriting and referencing the same set of variables, this in turn means that when a child function returns to the parent the parent will find its variables have been overwritten by the child, this in turn leads to the type of behavior seen here.

In this particular case the loop variable c leaves the last child process with a value of c=$count and all parent loops now see c=$count and thus exit; it actually gets a bit more interesting because count is also changing with each function call. The previous comment to add set -x (aka enable debug mode) before the first function call should show what's going on with each variable at each function call.

What OP wants to do is insure each function is working with a local copy of a variable. The easiest approach is to add a local <variable_list> at the top of the function, making sure to list all variables that should be treated as 'local', eg:

local path count c child fileType

score 0 · Answer 3 · answered Jun 15 '22 at 16:32

0

change variables to have local scope instead of global.

...
local count; # <------ VARIABLE MADE LOCAL
count=$(jq ".count" $HOME/"$1"/$FILE_NAME)
local c; # <------ VARIABLE MADE LOCAL
for (( c=0; c<$count; c++ ))
do
    ....
done
...

answered Jun 15 '22 at 16:32

gg-dev-05

353
8
14

bash recursion automatically ends after single level

3 Answers3