How can I redirect input and output to a same file in general? I mean specifically there is -o for the sort command and there might be other such options for various command. But how can I generally redirect input and output to same file without clobbering the file?
For example: sort a.txt > a.txt destroys the a.txt file contents, but I want to store answer in the same file. I know I can use mv and rm after using a temporary file, but is it possible to do it directly?
As mentioned on BashPitfalls entry #13, you can use sponge from moreutils to "soak up" the data before opening the file to write to it.
Example Usage:
sort a.txt | sponge a.txt
While the BashPitfalls page mentions that there could be data loss, the man page for sponge says
It also creates the output file atomically by renaming a temp file into place [...]
This would make it no more dangerous than writing to a temp file and doing a mv.
Credit to Charles Duffy for pointing out the BashPitfalls entry in the comments.
If you're familiar with the POSIX apis, you'll recognize that opening a file has a few possible modes, but the most common ones are read, write and append. You'll recall that if you open a file for writing, you will truncate it, immediately.
The redirects are directly analogous to those common modes.
> x # open x for writing
< x # open x for reading
>> x # open x for appending
There are no shell redirect that are equivalent to modes like O_RDWR, unfortunately.
You can check for this with the noclobber option, but you cannot open a file for both reading and writing using shell redirect operators. You must use a temporary file.
Not if the command doesn't support doing the mv itself after it is finished.
The shell truncates the output file before it even runs the command you told it to run. (Try it with a command that doesn't exist and you'll see it still gets truncated.)
This is why some commands have options to do this for you (to save you from having to use command input > output && mv output input or similar yourself).
I realize this is from a billion years ago, but I came here looking for the answer before I realized tee. Maybe I'll forget and stumble upon this in another 4 years.
$ for task in `cat RPDBFFLDQFZ.tasks | shuf -n 5`; do
echo $task;
grep -v $task RPDBFFLDQFZ.tasks | tee RPDBFFLDQFZ.tasks > /dev/null
done
6551a1fac870
26ab104327af
d6a90cf1720f
9eaa4faea92f
45ebf210a1b6
