I was looking at the very first posts in this section of the folder, and I read that multithreaded blocking socket servers are inefficient. Creating and killing threads for every single HTTP request is very inefficient, but a thread pool where all threads are created and WaitForXXXXX() until the main thread sets an event. There are also new Win2000 thread pooling functions that might help.

I'm just saying this because I can't find a good explanation of async sockets or overlapped I/O.



There has to be a way to use critical sections instead of kernel events, for the sake of speed.
Posted on 2003-06-08 20:57:00 by Paulicles the Philosopher

I was looking at the very first posts in this section of the folder, and I read that multithreaded blocking socket servers are inefficient. Creating and killing threads for every single HTTP request is very inefficient, but a thread pool where all threads are created and WaitForXXXXX() until the main thread sets an event. There are also new Win2000 thread pooling functions that might help.

The problem with using a thread pool and blocking, is that the pool threads will not be active all of the time. For example, if you have 2 threads running and 10 connections, both threads may be blocked on two sends on very slow connections, while the other 8 might be local, high speed connections. In that case, the local connections will have to wait for their turn, waiting on the slow connections to respond. Non-blocking sockets solve this problem because they don't block. Overlapped sockets are even better and I/O completion ports are the most efficient (being able to handle thousands of connections with just two threads). A limitation is also that the wait functions are often limited to 64 objects.

I'm just saying this because I can't find a good explanation of async sockets or overlapped I/O.


For a short intro, see my site. (Free) information about overlapped I/O is not easy to find.. Some links I have found are:
http://fiddle.visc.vt.edu/courses/ece4564/lectures.html
http://www.codeproject.com/internet/winsockiocp.asp
http://www.codeproject.com/internet/iocp.asp
http://www.whiningdog.net/Articles/Programming/Windows/20021115-IOCP/
http://msdn.microsoft.com/msdnmag/issues/1000/winsock/

There has to be a way to use critical sections instead of kernel events, for the sake of speed.

Critical sections are like mutexes, they only allow one thread to access something. Events are a different thing, they notify you about something. So I don't think critical sections are very useful as a replacement of events. And btw, events are pretty fast (see the winsock performance thread from a while ago).
Posted on 2003-06-09 04:57:59 by Thomas
I don't see what the point is in having two working threads in the IOCP model, if it's only running on a single processor machine.

Everything I've read about it seems to infer that having multiple outstanding IO operations on the same socket, even on a single processor is somehow more efficient. And all of the examples I've seen create two threads per CPU.

But in my own tests I've noticed no performance benefit to doing this. And I've also noticed that occasionally with the IO operations, while they do complete in the same order that they were issued, the threads can sometimes handle them in the wrong order. This I suspect is because of the thread-scheduling mechanism, which doesn't guarantee timely processing.

So the solution is to somehow ensure that IO operations get processed in order, which can be a pain to implement. So I've abandoned this in favor of a single-pending-IO per socket, single-thread-per-CPU model, and it works just fine.

I wish I could find docs that address this kind of thing, but as you said, finding any docs on overlapped IO is tough.
Posted on 2003-06-09 08:50:41 by iblis