1

I am trying to run the following script, lets call it foo.sh from a different location other than its working directory:

var1="$1"
var2="$2"

for SNR_NR in $var1; do
    for parts in $var2; do
        # echo *-$SNR_NR-$parts
        for file in *-$SNR_NR-$parts; do
        cat "$file" | sed -n '3p' \
                    | sed 's/$/,/' >> tmp
        cat "$file" | sed -n '4p' \
                    | tr ' ' ',' \
                    | sed 's/^.\(.*\).$/\1/' >> tmp
        done
    done
    # Concatenate the above onto one line
    paste -d " "  - - < tmp 
    rm tmp
done

It is simple enough, it runs perfectly well from the working directory as so sh foo.sh 5 500 where it takes two numerical integer arguments.

Now the error I am getting if I try to run it from somewhere other than the working directory is:

/data/RESULTS/foo.sh: 18: /data/RESULTS/foo.sh: cannot create tmp: Is a directory
cat: *-0-500: No such file or directory
/data/RESULTS/foo.sh: 22: /data/RESULTS/foo.sh: cannot create tmp: Is a directory
cat: *-0-500: No such file or directory
paste: -: Is a directory
paste: -: Is a directory
rm: cannot remove ‘tmp’: Is a directory

I have tried adding my working directory to $PATH but it still doesn't work. Running ls -la on foo.sh gives -rw------- but doing chmod +x foo.sh still doesn't work.

The strangest part is that it trying to create a directory called tmp which I cannot quite understand, since it is just a temporary file to store some intermediate results, which it then prints to the shell.

EDIT:

After taking on board suggestions from the comments, the following now produces this error instead, which is an improvement:

var1="$1"
var2="$2"

for SNR_NR in $var1; do
    for parts in $var2; do
        # echo *-$SNR_NR-$parts
        for file in *-$SNR_NR-$parts; do
        cat "$file" | sed -n '3p' \
                    | sed 's/$/,/' >> brian
        cat "$file" | sed -n '4p' \
                    | tr ' ' ',' \
                    | sed 's/^.\(.*\).$/\1/' >> brian
        done
    done
    # Concatenate the above onto one line
    paste -d " "  - - < brian 
    rm brian
done

Run with sh foo.sh 5 500 results in:

cat: *-0-500: No such file or directory
cat: *-0-500: No such file or directory

SOLUTION:

A simple rewrite produced 

var1="$1"
var2="$2"
# Supply results/script directory as argument
var3 = "$3"

# cd into script directory before running it
cd $var3

for SNR_NR in $var1; do
    for parts in $var2; do
        # echo *-$SNR_NR-$parts
        for file in *-$SNR_NR-$parts; do
        cat "$file" | sed -n '3p' \
                    | sed 's/$/,/' >> brian
        cat "$file" | sed -n '4p' \
                    | tr ' ' ',' \
                    | sed 's/^.\(.*\).$/\1/' >> brian
        done
    done
    # Concatenate the above onto one line
    paste -d " "  - - < brian 
    rm brian
done
Astrid
  • 1,846
  • 4
  • 26
  • 48
  • 1
    That's horrible. The entire inner loop should be a single `sed` script; something like `sed -n '3s/$/,/p; !4d; y/ /,/; s/^.\(.*\).$/\1/p; q' "$file"` except with comments. – tripleee Nov 23 '15 at 17:05
  • Right, well the raw files have four lines, and I want the last two ones. In their raw version they look like `-97669.26212165842 \n (0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0)` and I want them to be output on one line as such `-97669.26212165842,0,0,0,0,0,0,..,0,0`. Each line of (0 0 0 .. 0 0 0) has a 1000 entries but for demonstration I added the `...`. Anyway, your script didn't work; `sed: -e expression #1, char 12: unknown command: 4'` – Astrid Nov 23 '15 at 17:31
  • 1
    Sorry, that should be `4!d` not `!4d`. The main point is that using six processes inside when one would trivially do is rather wasteful. `sed` is not particularly easy to read (or write, duh) but your task sounds like an easy one-liner for Awk. – tripleee Nov 24 '15 at 05:28

2 Answers2

3

No, it tried to create a file tmp but tmp was already a directory in that location.

That's why > tmp, < tmp and rm tmp all failed with the is a directory error since none of them can operate on a directory.

If you want a secure temporary file use mktemp.

Etan Reisner
  • 77,877
  • 8
  • 106
  • 148
  • I do not quite follow. Before running the script there is no directory in the working directory called `tmp`. But if it is as you say, how come it works if I run it from the working directory? – Astrid Nov 23 '15 at 15:37
  • @Astrid: IMHO, it should work in any directory that does not contain a `tmp` sub folder, that means at least neither '/', nor '/var' – Serge Ballesta Nov 23 '15 at 15:48
  • 1
    That error very clearly indicates that the working directory **does** have a `tmp` directory already. The working directory of the script is the working directory of the shell/etc. that runs it and not the script's directory (unless you change that in the script itself). Add `pwd` or `echo $PWD` to the script and `ls` or `ls tmp` and see if you get what you expect. – Etan Reisner Nov 23 '15 at 15:48
  • @EtanReisner you are completely right. I confused working directory with script directory. Thx. My error is now instead `cat: *-0-500: No such file or directory cat: *-0-500: No such file or directory` – Astrid Nov 23 '15 at 15:50
  • 1
    That's going to be working directory again. The glob is looking in the working directory and not the script directory. See http://stackoverflow.com/questions/59838/check-if-a-directory-exists-in-a-shell-script?rq=1 for ways to deal with that. – Etan Reisner Nov 23 '15 at 15:51
  • @EtanReisner please excuse my ignorance; but would you be able to explain why the script runs perfectly well from the script directory but not from anywhere else? – Astrid Nov 23 '15 at 15:58
  • 1
    When you have the shell expand `*-0-500` it expands that glob *in the current directory* and matches files from the current directory. The current directory is **not** the directory the script is in. The current directory is the directory of the shell process that is running the script. When you are in a different directory the glob can't find the files you expect it to (and stays `*-0-500`) so you try to `cat` that, which also doesn't exist and things fail. This is why `echo *` outputs different contents in different directories, for example (which you presumably know and expect normally). – Etan Reisner Nov 23 '15 at 16:02
1

You already have a directory named 'tmp', and therefore a file with this name can't be created. You can catch this situation by doing this in the beginning of your script:

if [[ -d tmp ]]
then
  echo I can not execute this script, because a directory named tmp exists
  ls -ld tmp
  exit 1
fi

Of course tmp is not the best name for a temporary file anyway....

user1934428
  • 19,864
  • 7
  • 42
  • 87
  • Ohhhh... I see, that was not obvious to me thanks. Now I getting this instead, after changing the name of the temporary file: `cat: *-0-500: No such file or directory cat: *-0-500: No such file or directory` – Astrid Nov 23 '15 at 15:49
  • I guess there is no file matching the pattern *-$SNR_NR-$parts. – user1934428 Nov 24 '15 at 08:30
  • BTW: If you would use zsh instead of bash, you could add a modifier to the file pattern in the inner for-loop, which would cause the inner loop to be skipped if the pattern does not match. I don't know if such a feature is available in (newer versions) of bash. – user1934428 Nov 24 '15 at 08:32
  • 1
    At least (though not as elegant) you could set the nullglob option for the inner for loop (but only there). See http://wiki.bash-hackers.org/internals/shell_options#nullglob . Or you have your script run on zsh and use `*-$SNR_NR-$parts(N)` for the pattern. – user1934428 Nov 24 '15 at 08:37