0

I have a file file1 with the following contents:

Z 
X
Y

I can use cat to view the file:

$ cat file1
Z
X
Y

I can sort the file:

$ sort -k1,1 file1
X
Y
Z

I can sort it and store the output in a variable:

sorted_file1=$(sort -k1,1 file1)

But when I try to use cat on the variable sorted_file1 I get an error:

$ cat "$sorted_file1" 
cat: X
Y
Z: No such file or directory

I can use echo and it looks about right, but it behaves strangely in my scripts:

$ echo "$sorted_file1" 
X
Y
Z

Why does this happen? How does storing the output of a command change how cat interprets it?

Is there a better way to store the output of shell commands within variables to avoid issues like this?

Reilstein
  • 1,193
  • 2
  • 11
  • 25
  • So I should always use `echo` ? – Reilstein Jan 20 '16 at 22:11
  • `cat file1` prints the contents of the named file, not the name itself. Why would you expect different behavior from `cat "$sorted_file1"`? – chepner Jan 20 '16 at 22:14
  • For what it's worth, you can use `cat <<< "$sorted_file1"` as an alternative to echo. – Corubba Jan 20 '16 at 22:17
  • I expected `cat file1` to print the contents of the variable in the same way that it would print the contents of a named file. Apparently that is incorrect. – Reilstein Jan 20 '16 at 22:17
  • @Corubba Thanks, I will see how that works. @ anubhava I tried printf, but I must have done something wrong further down the script with my variable quoting because it wasn't printing the trailing newline correctly. Thanks though, I normally have luck with printf – Reilstein Jan 20 '16 at 22:20
  • 1
    `cat` doesn't get a variable as its argument; the shell expands the variable to its value, and `cat` receives *that* as its argument. – chepner Jan 20 '16 at 22:29
  • @Reilstein: cat is a filter program. Filter programs (there are many) process the filenames specified on the command-line. If no filenames are given then it uses standard streams, so stdin if it wants to read files (as cat does). Try just typing cat (no parameters) on the command-line, what is happening? By the way, remember that cat is a program which is independent of the shell, it is not part of bash – cdarke Jan 21 '16 at 07:38

2 Answers2

2

cat operates on files. Your invocation of cat (cat "$sorted_file1") expands to the same as cat $'X\nY\nZ', and of course there's no file of that name, hence the error you see.

Shell variables are not files. If you need to make their values available like files, you need to use echo to create a stream:

echo "$sorted_file1" | cat    # portable, STDIN
cat <(echo "$sorted_file1")   # Bash, file
cat <<<"$sorted_file1"        # Bash, STDIN

(obviously cat is pointless here, but the principle applies to other programs that expect their input from files or STDIN).

Toby Speight
  • 27,591
  • 48
  • 66
  • 103
0

Your mixing two concepts, files and variables. Both of these hold data, but they do so in different ways.

I will assume you know what a file is. A variable is like a little data store.

You generally use variables to store little bits of data that you may want to change, use immediately and don't mind losing when your script/program ends.

And you generally use files to store large amounts of data that you want to keep around after your script/program ends.

I believe what you want to do here is sort the file, and store the input in another file. To do this, you need to use redirection, like this

sort -k1,1 file1 > sorted_file1

What this does is sort the file and then outputs the result into a file called "sorted_file1". Then if you do your regular cat sorted_file you will see the sorted contents, as you expect.

You can read a bit more about it here.

pdc
  • 134
  • 1
  • 9
  • Thanks Peter. I actually want to avoid writing this temporary file because I am running this script simultaneously on several different input files, and this file (configuration file) has to be unique each time I run the program. Initially I did what you recommended, but it lead to the program using the configuration file of the previous run for a new run before it had time to remove the file. Thanks again. – Reilstein Jan 20 '16 at 22:39
  • If thats what you want to do, then you can solve that problem directly instead. This is especially true if you are trying to sort a lot of data. Using variables for that might run into other issues e.g. memory allocation problems, especially if you are running several sorts concurrently. For advice on how to create unique temporary files, see [here](https://stackoverflow.com/questions/10982911/creating-temporary-files-in-bash) or [here](https://unix.stackexchange.com/questions/181937/how-create-a-temporary-file-in-shell-script). Also some other [suggested reading](http://xyproblem.info/) – pdc Jan 20 '16 at 23:02
  • Thanks Peter I'll take a look! – Reilstein Jan 21 '16 at 07:40
  • 1
    Hey @Peter just wanted to let you know that I did eventually look through the posts you linked to, and found them very useful. I've been using mktemp to generate temp files in my scripts now, and it works beautifully, so thanks! – Reilstein Feb 02 '16 at 22:02