Find open handles within a Windows program

Question

In Windows (MinGW), my program is inheriting unwanted handles from the calling process.

The process has no need to have these files open, but because it lives on beyond the lifetime of the parent I get the usual problems with files being held open.

On Linux I fix the problem like this:

// Close all file descriptors
// It's hard to figure out how many are open, but the first 1000 should do
int fd;
for (fd = 0; fd < 1000; fd++)
  close (fd);

This does not appear to work in Windows.

How can I determine which file handles have been inherited? How can I then close them?

The project is written in C (no C++) using MinGW and Windows' Unix compatibility API.

What do you mean by This does not appear to work in Windows? — sara, Sep 12 '13 at 15:32
Have you seen http://stackoverflow.com/questions/13101690/c-get-all-open-file-descriptors? — sara, Sep 12 '13 at 15:33
[This question about enumerating process handles](http://stackoverflow.com/questions/733384/how-to-enumerate-process-handles) might be a starting point. — Mark Wilkins, Sep 12 '13 at 15:34
*it lives on beyond the lifetime of the parent* ... who lives beyond the lifetime of what parent? — Praetorian, Sep 12 '13 at 15:38
@Praetorian: "it" is "The process". The "parent" is another process. — ams, Sep 12 '13 at 15:46
@MarkWilkins: I think I might be able to borrow something out of that heap of C++ code. — ams, Sep 12 '13 at 15:47
@ams: How can you know it runs without errors without checking the result of `close()`? — alk, Sep 12 '13 at 16:07
Why don't you know which file descriptors your program uses/used? — alk, Sep 12 '13 at 16:09
@alk: It closes 1000 descriptors without checking that they even exist. Of course a lot of calls will return a failure. By "without errors", I mean that the program didn't crash. — ams, Sep 12 '13 at 16:09
@alk: read the question more closely: the handles are inherited from the parent process against my will. — ams, Sep 12 '13 at 16:10
So why not just do `for(int i=0, i< 0xffffffff; ++i) CloseHandle(i);` on windows?! :-/ — alk, Sep 12 '13 at 16:12
So you do not have access to the parent process's code? How is this process inheriting the handles created? — alk, Sep 12 '13 at 16:14
@alk, blindly closing a windows handle like that is not a good thing to do. — josh poley, Sep 12 '13 at 16:30
@alk: The parent process does not mark the handles with the `NOINHERIT` flag, so inheritance just happens. This is the normal practice in Unix programs, and MinGW does the same. The problem is that the Unix trick doesn't fix it in MinGW. — ams, Sep 12 '13 at 16:30
If you have control of the parent process, then have it pass FALSE for the `bInheritHandles` flag of `CreateProcess`. — josh poley, Sep 12 '13 at 16:34
@joshpoley that's the wrong place to fix the problem as my process isn't the only one it can call. Even if it were, it uses `_spawnl` so there's no way to stop inheritance. It can't use `CreateProcess` because mixing API leads to bad things (although it sort of works, sometimes). — ams, Sep 12 '13 at 16:40
If having to deal with HANDLEs, havent you then already mixed APIs? — alk, Sep 12 '13 at 16:53
@alk if the handle is inherited then, presumably, the runtime state problems don't exist? — ams, Sep 12 '13 at 17:10
@joshpoley:I'm fully aware of this. It was just the helpless try to make an ironical comment ... — alk, Sep 13 '13 at 05:31

score 1 · Accepted Answer · answered Sep 16 '13 at 11:41

I've now investigated this somewhat, and I've found a solution to the real problem, but not way I had intended.

I had thought that I would be able to find and clean up any undesired open files, but this turns out to be hard. I found a few different tutorials (here, and here) how to do this, but they rely on undocumented APIs. I couldn't make this technique work -- possibly I was doing it wrong, or possibly the API has changed in Windows Server 2012 -- but in any case I'm not sure I want to go there; it's OK for Sysinternals to track this stuff and keep Process Explorer working, but I don't wish to have that maintenance burden on my project.

I now have two choices:

Put some special case code in the parent (calling) process to have it call CreateProcess with inheritance disabled, when appropriate (it currently uses _spawnlp because it's compatible with the Unix-style pipes and file handles, and you can't use CreateProcess with those very reliably).
Have the process immediately call itself with CreateProcess and then exit (or wait indefinitely) in order to kill any unwanted handles.

The first feels more efficient. The second is more flexible (it allows the process to choose for itself).

I think I'm going to choose option one because, for my current needs, it feels like the least worst.

Martin Rosenau · Answer 2 · 2013-09-14T08:10:36.357

First some backginformation - why the loop does not work under Windows:

In Linux the handles have numbers 0...n. The handles passed to "close()" and similar functions are directly passed to the operating system:

void close(int handle)
{
    syscall(SYS_CLOSE,handle);
}

However in Windows the operating system uses its own handles that are similar to pointer addresses and the C library uses some kind of translation table:

void close(int handle)
{
    CloseHandle(table[handle]);
    table[handle]=NULL;
}

If handles remain open the C library does not know (not STDIN, STDOUT, STDERR) these handles will not be in the table.

Now about the actual question:

Unlike Linux Windows mixes up different kinds of handles (File handles, Memory handles, Process handles, ...). If you could get a list of all handles you'll have to treat dem differently.

Another point is that Windows libraries use some handles internally. If you simply close all handles inherited by the parent process you'll risk a crash of your program because the Windows libraries may depend on some of these handles.

So it is definitely up to the parent application to ensure that no handles remain open. By default handles are not inherited in Windows. However the C library wrappers (e.g. "fopen()") will set the "inherit handle" flag so the handles will be inherited.

This answer does not address the question. – ams Sep 13 '13 at 15:06 — ams, Sep 13 '13 at 15:06

Find open handles within a Windows program

2 Answers2