9

The CreateIoCompletionPort function allows the creation of a new I/O completion port and the registration of file handles to an existing I/O completion port.

Then, I can use any function, like a recv on a socket or a ReadFile on a file with a OVERLAPPED structure to start an asynchronous operation.

I have to check whether the function call returned synchronously although it was called with an OVERLAPPED structure and in this case handle it directly. In the other case, when ERROR_IO_PENDING is returned, I can use the GetQueuedCompletionStatus function to be notified when the operation completes.

The question which arise are:

  • How can I remove a handle from the I/O completion port? For example, when I add sockets to the IOCP, how can I remove closed ones? Should I just re-register another socket with the same completion key?

  • Also, is there a way to make the calls ALWAYS go over the I/O completion port and don't return synchronously?

  • And finally, is it possible for example to recv asynchronously but to send synchronously? For example when a simple echo service is implemented: Can I wait with an asynchronous recv for new data but send the response in a synchronous way so that code complexity is reduced? In my case, I wouldn't recv a second time anyways before the first request was processed.

  • What happens if an asynchronous ReadFile has been requested, but before it completes, a WriteFile to the same file should be processed. Will the ReadFile be cancelled with an error message and I have to restart the read process as soon as the write is complete? Or do I have to cancel the ReadFile manually before writing? This question arises in combination with a communication device; so, the write and read should not do problems if happening concurrently.

Aaron Klotz
  • 11,287
  • 1
  • 28
  • 22
Etan
  • 17,014
  • 17
  • 89
  • 148

4 Answers4

12

How can I remove a handle from the I/O completion port?

In my experience you can't disassociate a handle from a completion port. However, you may disable completion port notification by setting the low-order bit of your OVERLAPPED structure's hEvent field: See the documentation for GetQueuedCompletionStatus.

For example, when I add sockets to the IOCP, how can I remove closed ones? Should I just re-register another socket with the same completion key?

It is not necessary to explicitly disassociate a handle from an I/O completion port; closing the handle is sufficient. You may associate multiple handles with the same completion key; the best way to figure out which request is associated with the I/O completion is by using the OVERLAPPED structure. In fact, you may even extend OVERLAPPED to store additional data.

Also, is there a way to make the calls ALWAYS go over the I/O completion port and don't return synchronously?

That is the default behavior, even when ReadFile/WriteFile returns TRUE. You must explicitly call SetFileCompletionNotificationModes to tell Windows to not enqueue a completion packet when TRUE and ERROR_SUCCESS are returned.

is it possible for example to recv asynchronously but to send synchronously?

Not by using recv and send; you need to use functions that accept OVERLAPPED structures, such as WSARecv, WSASend, or alternatively ReadFile and WriteFile. It might be more handy to use the latter if your code is meant to work multiple types of I/O handles, such as both sockets and named pipes. Those functions provide a synchronous mode, so if you use those them you can mix asynchronous and synchronous calls.

What happens if an asynchronous ReadFile has been requested, but before it completes, a WriteFile to the same file should be processed?

There is no implicit cancellation. As long as you're using separate OVERLAPPED structures for each read/write to a full-duplex device, I see no reason why you can't do concurrent I/O operations.

Aaron Klotz
  • 11,287
  • 1
  • 28
  • 22
  • "However, you may disable completion port notification by setting the low-order bit of your OVERLAPPED structure's hEvent field". I could not disable completion port notification for one given socket using this option. My case: (1) create a socket, (2) add to completion port, (3) call WSARecv, (4) if it returns WSA_IO_PENDING - ok, (5) if it returns ERROS_SUCCESS -> I want to close the socket immediately and do not receive a notification about this socket in completion port – DmitriyH Oct 05 '17 at 16:24
9

As I’ve already pointed out there, the commonly held belief that it is impossible to remove handles from completion ports is wrong, probably caused by the abscence of any hint whatsoever on how to do this from nearly all documentation I could find. Actually, it’s pretty easy:

Call NtSetInformationFile with the FileReplaceCompletionInformationenumerator value for FileInformationClass and a pointer to a FILE_COMPLETION_INFORMATION structure for the FileInformation parameter. In this structure, set the Port member to NULL (or nullptr, in C++) to disassociate the file from the port it’s currently attached to (I guess if it isn’t attached to any port, nothing would happen), or set Port to a valid HANDLE to another completion port to associate the file with that one instead.

Community
  • 1
  • 1
Keno
  • 91
  • 1
  • 5
4

First some important corrections.

In case the overlapped I/O operation completes immediately (ReadFile or similar I/O function returns success) - the I/O completion is already scheduled to the IOCP.

Also, according to your questions I think you confuse between the file/socket handles, and the specific I/O operations issued on them.

Now, regarding your questions:

  1. AFAIK there is no conventional way to remove a file/socket handle from the IOCP (usually you just don't have to do this). You talk about removing closed handles from the IOCP, which is absolutely incorrect. You can't remove a closed handle, because it does not reference a valid kernel object anymore!

A more correct question should be how the file/socket should be properly closed. The answer is: just close your handle. All the outstanding I/O operations (issued on this handle) will return soon with an error code (abortion). Then, in your completion routine (the one that calls GetQueuedCompletionStatus in a loop) should perform the per-I/O needed cleanup.

  1. As I've already said, all the I/O completion arrives at IOCP in both synchronous and asynchronous cases. The only situation where it does not arrive at IOCP is when an I/O completes synchronously with an error. Anyway, if you want a unified processing - in such a case you may post an artificial completion data to IOCP (use PostQueuedCompletionStatus).

  2. You should use WSASend and WSARecv (not recv and send) for overlapped I/O. Nevertheless, even of the socket was opened with flag WSA_FLAG_OVERLAPPED - you are allowed to call the I/O functions without specifying the OVERLAPPED structure. In such a case those functions work synchronously. So that you may decide on synchronous/asynchronous modes for every function call.

  3. There is no problem to mix overlapped read/write requests. The only delicate point here is what happens if you try to read the data from the file position where you're currently writing to. The result may depend on subtle things, such as order of completion of I/Os by the hardware, some PC timing parameters and etc. Such a situation should be avoided.

valdo
  • 12,632
  • 2
  • 37
  • 67
  • `If an overlapped operation completes immediately, WSARecv returns a value of zero and the lpNumberOfBytesRecvd parameter is updated with the number of bytes received and the flag bits indicated by the lpFlags parameter are also updated. If the overlapped operation is successfully initiated and will complete later, WSARecv returns SOCKET_ERROR and indicates error code WSA_IO_PENDING. ` why do they mention the case where the operation is successful and completes synchronously then if it is not possible to occur? – Etan Jul 06 '11 at 10:31
  • It **is** possible. This is what actually happens sometimes. Moreover, for file I/O this is what usually happens (asynchronous I/Os are rare under normal conditions). – valdo Jul 06 '11 at 19:08
0

How can I remove a handle from the I/O completion port? For example, when I add sockets to the IOCP, how can I remove closed ones? Should I just re-register another socket with the same completion key?

You've got it the wrong way around. You set the I/O completion port to be used by a file object - when the file object is deleted, you have nothing to worry about. The reason you're getting confused is because of the way Win32 exposes the underlying native API functionality (CreateIoCompletionPort does two very different things in one function).

Also, is there a way to make the calls ALWAYS go over the I/O completion port and don't return synchronously?

This is how it's always been. Only starting with Windows Vista can you customize how the completion notifications are handled.

What happens if an asynchronous ReadFile has been requested, but before it completes, a WriteFile to the same file should be processed. Will the ReadFile be cancelled with an error message and I have to restart the read process as soon as the write is complete?

I/O operations in Windows are asynchronous inherently, and requests are always queued. You may not think this is so because you have to specify FILE_FLAG_OVERLAPPED in CreateFile to turn on asynchronous I/O. However, at the native layer, synchronous I/O is really an add-on, convenience thing where the kernel keeps track of the file position for you and waits for the I/O to complete before returning.

wj32
  • 8,053
  • 3
  • 28
  • 37