dup2 / dup - Why would I need to duplicate a file descriptor?

Question

I'm trying to understand the use of dup2 and dup.

From the man page:

DESCRIPTION

dup and dup2 create a copy of the file descriptor oldfd. After successful return of dup or dup2, the old and new descriptors may be used interchangeably. They share locks, file position pointers and flags; for example, if the file position is modified by using lseek on one of the descriptors, the position is also changed for the other.

The two descriptors do not share the close-on-exec flag, however. dup uses the lowest-numbered unused descriptor for the new descriptor.

dup2 makes newfd be the copy of oldfd, closing newfd first if necessary.

RETURN VALUE

dup and dup2 return the new descriptor, or -1 if an error occurred (in which case, errno is set appropriately).

Why would I need that system call? What is the use of duplicating the file descriptor? If I have the file descriptor, why would I want to make a copy of it? I'd appreciate it if you could explain and give me an example where dup2 / dup is needed.

How would you implement the piping functionality of shells without `dup` or `dup2` ? You need to call `pipe(2)` and then to have one of the file descriptors `dup`-ed to e.g. `STDIN_FILENO` — Basile Starynkevitch, Jul 24 '12 at 23:52
Possible duplicate of [practical examples use dup or dup2](http://stackoverflow.com/questions/1720535/practical-examples-use-dup-or-dup2) — DrTyrsa, Jan 20 '17 at 10:27

Deepthought · Accepted Answer · 2015-07-08T09:12:25.713

52

The dup system call duplicates an existing file descriptor, returning a new one that refers to the same underlying I/O object.

Dup allows shells to implement commands like this:

ls existing-file non-existing-file > tmp1  2>&1

The 2>&1 tells the shell to give the command a file descriptor 2 that is a duplicate of descriptor 1. (i.e stderr & stdout point to same fd).
Now the error message for calling ls on non-existing file and the correct output of ls on existing file show up in tmp1 file.

The following example code runs the program wc with standard input connected to the read end of a pipe.

int p[2];
char *argv[2];
argv[0] = "wc";
argv[1] = 0;
pipe(p);
if(fork() == 0) {
    close(STDIN); //CHILD CLOSING stdin
    dup(p[STDIN]); // copies the fd of read end of pipe into its fd i.e 0 (STDIN)
    close(p[STDIN]);
    close(p[STDOUT]);
    exec("/bin/wc", argv);
} else {
    write(p[STDOUT], "hello world\n", 12);
    close(p[STDIN]);
    close(p[STDOUT]);
}

The child dups the read end onto file descriptor 0, closes the file de scriptors in p, and execs wc. When wc reads from its standard input, it reads from the pipe.
This is how pipes are implemented using dup, well that one use of dup now you use pipe to build something else, that's the beauty of system calls,you build one thing after another using tools which are already there , these tool were inturn built using something else so on .. At the end system calls are the most basic tools you get in kernel

Cheers :)

edited Jul 08 '15 at 09:12

answered Jul 24 '12 at 17:20

Deepthought

2,815
2
28
39

1

So`dup` is helpful for the caller and not the `ls` program itself? Is there any benefit of having `dup` used in program like ls itself if already has access to the file? Here for example, `ls` writes errors to `2` which is is hardcoded, so I have a way to over-ride it as a consumer of `ls`. I think thats a subtle point no? – Nishant Mar 28 '16 at 11:29
3

Your example program *seems* to have a bug; you're calling `dup(p[STDIN])` but then throwing away the result. Did you mean to use `dup2(p[STDIN], 0)`? – Quuxplusone Aug 08 '17 at 18:27
1

@Quuxplusone `dup` returns the "lowest numbered descriptor currently not in use by the process." Since fd 0 was just closed, `dup` should return 0. `dup2` is explicit about which fd should be used, instead of just using the lowest free fd, so I would prefer that. – Wodin Dec 28 '17 at 05:14
@Wodin: Ah, I bet you're right about what OP was thinking. Am I also right, though, that "just closed" is relative, and OP's code could break in the presence of, for example, concurrent threads that might be opening files too? – Quuxplusone Dec 28 '17 at 17:48
@Quuxplusone I suspect you're right, but don't know for sure. But in that case you'll have other problems. If you close stdin because you want to read from somewhere else and then another thread opens a file before you do, it will get fd 0. If you then use dup2, it will close the fd that the other thread opened, so the other thread will now be reading from (and writing to?) the file you open. Anyway, if I remember correctly you shouldn't call `exec*` from a multithreaded process. But I am no threading expert :) – Wodin Dec 31 '17 at 08:48
@Quuxplusone or maybe it was `fork` I was thinking of: http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-using-them – Wodin Dec 31 '17 at 08:54
1

@Wodin JFI: with `dup2` you would not call `close` before. – blueyed Feb 20 '20 at 15:44

score 20 · Answer 2 · answered Jul 24 '12 at 17:53

20

Another reason for duplicating a file descriptor is using it with fdopen. fclose closes the file descriptor that was passed to fdopen, so if you don't want the original file descriptor to be closed, you have to duplicate it with dup first.

answered Jul 24 '12 at 17:53

R.. GitHub STOP HELPING ICE

208,859
35
376
711

`fdopen()` seems don't duplicate a file descriptor, it just create buffer in user space. – Eric May 05 '15 at 05:11
4

You misread my answer. The point is that you may want to `dup` the fd before passing it to `fdopen` since `fclose` will close it. – R.. GitHub STOP HELPING ICE May 05 '15 at 15:30
Why would you call `fclose` if you don't want the file descriptor to be closed? Are you talking about just in the case of like a library function you don't control that calls `fclose`, and you would call `dup` before calling the library function to keep it open? – theferrit32 Oct 31 '18 at 21:00
1

@theferrit32: If you allocate a `FILE` handle to access a preexisting open file via stdio interfaces, you need to call `fclose` to deallocate that `FILE` handle. If you want to keep using the underlying open file, or if your software architecture is such that the original "owner" code for the file descriptor will `close` it, the fact that `fclose` also closes the underlying file descriptor you handed to `fdopen` is a problem. You can avoid this problem by using `dup` to make a new file descriptor for the same open file to pass to `fdopen`, so that `fclose` does not close the original one. – R.. GitHub STOP HELPING ICE Nov 01 '18 at 02:33
3

The point is that fdopen() *moves* ownership of the fd to the `FILE`, rather than *copying* it. That's something users should be aware of. Consumers that need to retain a usable `fd` handle in addition to the `FILE` object must duplicate the `fd`. That's all. – Conrad Meyer Mar 18 '20 at 18:16
1

@ConradMeyer: Yes, that's a very good way of putting it, with a note that there is no operation to "move ownership" away from the `FILE` once you move ownership to it. – R.. GitHub STOP HELPING ICE Mar 18 '20 at 18:50

score 4 · Answer 3 · answered Jul 24 '12 at 16:27

dup is used to be able to redirect the output from a process.

For example, if you want to save the output from a process, you duplicate the output (fd=1), you redirect the duplicated fd to a file, then fork and execute the process, and when the process finishes, you redirect again the saved fd to output.

score 4 · Answer 4 · answered Jul 24 '12 at 19:01

Some points related to dup/dup2 can be noted please

dup/dup2 - Technically the purpose is to share one File table Entry inside a single process by different handles. ( If we are forking the descriptor is duplicated by default in the child process and the file table entry is also shared).

That means we can have more than one file descriptor having possibly different attributes for one single open file table entry using dup/dup2 function.

(Though seems currently only FD_CLOEXEC flag is the only attribute for a file descriptor).

http://www.gnu.org/software/libc/manual/html_node/Descriptor-Flags.html

dup(fd) is equivalent to fcntl(fd, F_DUPFD, 0);

dup2(fildes, fildes2); is equivalent to 

   close(fildes2);
   fcntl(fildes, F_DUPFD, fildes2);

Differences are (for the last)- Apart from some errno value beteen dup2 and fcntl close followed by fcntl may raise race conditions since two function calls are involved.

Details can be checked from http://pubs.opengroup.org/onlinepubs/009695399/functions/dup.html

An Example of use -

One interesting example while implementing job control in a shell, where the use of dup/dup2 can be seen ..in the link below

http://www.gnu.org/software/libc/manual/html_node/Launching-Jobs.html#Launching-Jobs

dup2 / dup - Why would I need to duplicate a file descriptor?

DESCRIPTION

RETURN VALUE

4 Answers4

Linked