7

The .NET / .NET Core Thread Pool uses two different categories of threads internally: worker threads and I/O Completion Port (IOCP) threads. Both are just usual managed threads, but used for different purposes. Via different APIs (e.g. Task.Start or ThreadPool.QueueUserWorkItem) I can start CPU-bound async operations on the worker threads (which shouldn't block, otherwise the Thread Pool would probably create additional worker threads).

But what about performing I/O-bound asynchronous operations? How do the IOCP threads behave exactly in these situations? Specifically, I have the following questions:

  • If I start an async I/O operation (e.g. for file, pipe, or network), I suspect that the current thread dispatches the async request. I also know (via the book "CLR via C#") that the CLR registers to an I/O completion port that is used to perform overlapped async I/O. I suspect that this IOCP is bound to the async operation so that it can queue the async operation result to the Thread Pool later. Thus, is my assumption correct that no IOCP thread is touched when an async request is started?
  • I suspect that when the result of the async I/O operation is reported via the I/O completion port of the CLR, this is the place where IOCP threads come into place. The result is queued to the Thread Pool and an IOCP thread is used to handle it. However, when reading through some forum threads like this one on MSDN, I get the feeling that IOCP threads are actually used to dispatch the request and then block until the result is back. Is this the case? Are IOCP threads blocking while the I/O operation is handled by the opposing system?
  • What about async await and SynchronizationContext? Does an IOCP thread handle the async I/O response and then e.g. queue the continuation on the UI thread (assuming that ConfigureAwait(false) is not called)?
  • What about .NET Core on Linux / MacOS X? There are no I/O completion ports - are they emulated in any kind of way?
feO2x
  • 5,016
  • 2
  • 34
  • 41
  • 3
    [There is no thread](https://blog.stephencleary.com/2013/11/there-is-no-thread.html) is always recommended reading here. – Damien_The_Unbeliever Sep 23 '19 at 07:55
  • 2
    This question is too broad. In general, no, it doesn't take any async code to get an I/O request started. The I/O manager must already deal with multiple processes asking for I/O so it takes care of queuing the driver requests by itself. SynchronizationContext in general plays no role, except for XxxxAsync() methods in early .NET Framework versions that made an effort to get an event to raise on the expected thread. The unixes have their own flavor of it, macOS uses kqueue and Linux uses epoll. – Hans Passant Sep 23 '19 at 08:12
  • @Damien_The_Unbeliever Thanks for the link, Stephen Toub's post answered almost all my questions. – feO2x Sep 23 '19 at 08:25
  • @HansPassant you're right, the question is too broad (or not well-formulated). Thanks for the hints to kqueue and epoll. – feO2x Sep 23 '19 at 08:28
  • 1
    Just for the record, Stephen Toub is the genius who designed a lot of the .NET `async` systems. Stephen Cleary (the "other Stephen") is the guy who just likes to write about it. – Stephen Cleary Sep 24 '19 at 16:41

1 Answers1

6

Damien and Hans pointed me into the right direction in the comments, which I want to sum up in this answer.

Damien pointed to Stephen Cleary's awesome blog post which answers the first three points:

  • The async I/O operation is dispatched on the calling thread. No IOCP thread is involved.
  • Consequently, no IOCP threads block during async I/O.
  • When the result is returned to the .NET application, an IOCP thread is borrowed to mark the task complete. The continuation is queued to the target SynchronizationContext or the thread pool.

Hans pointed out that there are similar mechanisms to IOCP in Linux (epoll) and MacOS (kqueue).

feO2x
  • 5,016
  • 2
  • 34
  • 41
  • 1
    But in this url, https://stackoverflow.com/questions/28690815/iocp-threads-clarification, the second answer have showed `Worker threads: 0, Completion port threads: 30, Total threads: 34` when call I/O operation. ######## There are 30 I/O thread for I/O operation. But this post said, the I/O thread does not waiting for I/O. If it is true, it should show less 30 I/O thread for I/O operation. I can not understand that. – beehuang Jul 15 '21 at 10:24
  • 1
    A completion port thread will only be used briefly to mark the corresponding task in .NET as completed. If you start several `FileStream` instances in a loop, then the thread pool will create more IOCP threads to handle all the completing I/O request packets. However, the IOCP threads will not block during the actual I/O operation. The operation is started on the calling thread, the IOCP thread marks the task as completed once the I/O operation is done, and a continuation will either be performed on a regular background thread or the calling thread if there was a `SynchronizationContext`. – feO2x Jul 15 '21 at 10:38
  • 3
    Thank you and I have a more question. ## you mention that the async I/O operation is dispatched on the calling thread. `No IOCP thread is involved.` And you said that the thread pool will create more IOCP threads to handle all the completing I/O request packets. --> I got that IOCP threads will created when initial I/O operation, and wait for I/O complete. When IOCP threads waiting for I/O complete, `they(IOCP threads) are blocking.` If I have misunderstand, please tell me. thanks a lot. – beehuang Jul 15 '21 at 10:52
  • 1
    No, you're wrong: 1) on your current thread, an I/O Request Packet is created and handed to the OS which starts the actual I/O operation. 2) The device driver is notified by the e.g. network or disk controller once the I/O operation is done. 3) Your .NET process is now notified by the OS that the I/O operation is done. The .NET runtime will only now use an IOCP thread to quickly mark the task as completed and enqueue the continuation (no blocking). 4) The continuation will be executed on the initial thread (if it has a SynchronizationContext) or on another background thread of the Thread Pool. – feO2x Jul 15 '21 at 11:02
  • thanks, I understand it. And I also reference this repo https://github.com/dschenkelman/async-io-talk. – beehuang Jul 15 '21 at 12:52
  • Can you explain how the I/O threads are not blocked? My understanding was that the I/O threads are essentially running in a while(true) loop calling the blocking GetQueuedCompletionStatus function. Is that not the case? – Rob L May 19 '22 at 14:44