1

I have a file, with the following content:

2)    Wiegley   Maths      90
3)    Artur     Biology    87
4)    Drew      English    85
5)    Phils     History    89

I store it in a file, called marks.txt.

In shell, I ran some operations with sed and awk (with xfce4 terminal emulator)

awk '{print $3 "\t" $4}' marks.txt | sed -e '/^E/d' > foo.txt | cat foo.txt

When I execute the same command fast (arrow up and enter), it gives me sometimes the output of the file foo.txt. But sometimes it gives me no output, when repeating the same command fast.

I fail to understand this, what it seems as a random behaviour, from my shell. Can anyone explain why cat gives sometimes the output, when I repeat the same command fast? Would it not be waiting on the output of the awk and sed first?

ReneFroger
  • 484
  • 8
  • 20
  • What are expecting `| cat foo.txt` to do? It's almost certainly wrong. – Tom Fenech Mar 25 '16 at 10:34
  • @Tom I'm not talking about the rightness or wrongness. I'm trying to understand why the terminal sometimes display the output, and sometimes not while the command is the same. Feel free to try it out, then you understand what I mean. – ReneFroger Mar 25 '16 at 10:50

2 Answers2

2

The pipe to cat is meaningless because you're not passing any data across it, or attempting to use it (to do this, you would use cat -).

Your command opens three subshells. The > in the middle will truncate the file foo.txt before waiting for any input. cat foo.txt will attempt to access the file foo.txt at more or less the same time. Depending on which of these two things happens first, foo.txt will either contain the contents from the previous time you ran the command or it will be empty.

For the record, your command would be better written as:

awk -v OFS='\t' '$3 !~ /^E/ { print $3, $4 }' marks.txt > foo.txt

If you would like to print the result to standard output and write to the file at the same time, one option would be to use tee.

awk -v OFS='\t' '$3 !~ /^E/ { print $3, $4 }' marks.txt | tee foo.txt

This prints the third and fourth field of any line whose third field doesn't start with E.

Tom Fenech
  • 72,334
  • 12
  • 107
  • 141
  • thanks for your reply. How could I 'measure'/detect if `cat foo.txt` is accessing the file `foo.txt`, while `>` is trying to truncate the file `foo.txt`? It seems to me that sometimes the `cat` command is earlier before the output gets written. I would like to understand in which circumstances the `cat` is earlier. – ReneFroger Mar 25 '16 at 12:20
  • You're basically asking three programs to run at the same time and the OS is trying to honour that request. I'm not sure that the output can be predicted in any reliable way. You could use another tool to [detect whether the file is open](http://superuser.com/q/97844/385589) but by the time this has run, it may well have been closed. – Tom Fenech Mar 25 '16 at 13:10
1

It seems your issue is related to precedence between pipe and redirection.

Splitting a command line into piped commands is done before doing file redirection. File redirection happens second, and if present, has precedence over pipe redirection. (The file redirection always wins.)

From : http://www.cs.colostate.edu/~mcrob/toolbox/unix/redirection

You can found some examples here :

Pipe | Redirection < > Precedence

Community
  • 1
  • 1
SLePort
  • 15,211
  • 3
  • 34
  • 44