I've noticed an interesting and rather undocumented behaviour of the ConnectEx function in regards to Overlapped IO via IO Completion Port.
I have scoured the internet for relevant information, and the only reference I could find was another programmer whining about the same issue, with no replies.
If you can shed some light on this issue, I would be very grateful.

The Problem:
When the Connection attempt is SUCCESSFUL, it seems ConnectEx issues TWO completion notifications for the SAME IO operation.
In order to identify 'Pending' operations, I have something like this:


call GetQueuedCompletionStatus
if FALSE
    if pOVERLAPPED != 0
          Deal with failed IO operation
  else
        Check GetLastError() for WAIT_TIMEOUT (no io jobs were queued)
  endif
else
  call WSAGetOverlappedResult ;make sure its not still PENDING completion
  if FALSE
        Deal with failed or pending IO operation
  else
        IO operation completed with success
  endif
endif


I'm seeing my 'outbound connect' io operation complete TWICE, with SUCCESS both times.
That is to say, the above functions are both returning TRUE, and doing so TWICE for the same io job (CompletionKey and pOVERLAPPED the same in both notifications).

I have not noticed any other (WSASend,WSARecv,AcceptEx) function doing this.
Why is it happening? How do I prevent it or deal with it?
Normally when an IO job completes, I want to throw that object back into a pool, or deallocate it, which I can hardly do if the same job is going to complete more than once!

I have checked and rechecked my code to ensure I am not queuing this job twice.
I know I'm not, and my debug spew confirms that.
I issue it ONCE, and it completes TWICE.
WEIRD-O-RAMA.

Posted on 2007-06-19 10:27:19 by Homer
The second notification isn't "data received"?


The following operations are performed when a call to the ConnectEx function completes successfully:

* A new connection is established.
* The first block of data is sent (optional).


Otherwise I don't know, sounds weird.

Posted on 2007-06-19 17:57:32 by f0dder
f0dder - interesting concept.

If I had sent data with the ConnectEx call ('connectandsend'), I would expect to see two notifications, with one of them (the second one) containing #bytes sent.
I supposed that if I had passed a valid buffer to ConnectEx, and told ConnectEx to send zero #bytes, then I would be seeing what I am in fact seeing (two completions, both with zero bytes transferred).
But I'm not sending data with ConnectEx - in fact, I'm not even passing a buffer for that, I'm passing NULL as pSendBuffer, so I expected to only see the actual CONNECT completion notification.

The only api function I feel is similar enough to warrant comparison is AcceptEx, which I am using, and which seems to behave as expected, noting that it offers AcceptAndReceive functionality, very similar to ConnectEx.

Still, perhaps you're onto something here, it's certainly the most rational explanation I've heard.
I might try performing a ConnectAndSend via ConnectEx with a valid sendbuffer and see what kind of notifications I receive.
Posted on 2007-06-20 02:45:29 by Homer
I have performed a ConnectAndSend test, and guess what?
I GET THE SAME RESULT!

In this case, I passed a small buffer to ConnectEx to send when the connection completed.
I received two completion notifications, the only difference to the previous test being that each of these reported the #bytes sent - if I was to naively accept this at face value, I am being told I sent that data twice :|

I didn't get what I had expected, but I did prove this is a genuine issue.
Posted on 2007-06-20 05:40:59 by Homer
Hm, this sounds pretty problematic to me. There's a few inconsistencies here and there with winsock, I remember some relatively erratic behaviour when playing with event-triggering sockets (but that's not really an issue if you design around it).

I'm still most inclined to think there's some weird-ass bug in your code, because if this is "standard" ConnectEx behaviour, it must've been discovered before? Or is nobody but you using ConnectEx? I wonder if it's possible to get in contact with MS about this, but at the very least you'd probably need to be able to duplicate the bug with some really simple proof-of-concept C code...
Posted on 2007-06-20 06:56:51 by f0dder
Well, I've triple-checked my code, read and re-read all the relevant api documentation, and in the end, I settled for a cheesy workaround.
I am absolutely sure this is not a code bug, but it MAY be a threading issue, which is not mentioned in the formal documentation, thus it may be an implementation nuance, however I could be forgiven for assuming that an api designed to work consistently in a multithreaded environment actually does so.
My debug spew is copious, and shows object creation, event issues, event completions, api return values, pretty much everything.

Anyway, for the benefit of the casual observer, I will describe my workaround.
I already had a StateFlag in my Session object (Session is what I called my ClientContext, as my iocp supports both inbound and outbound, client became a confusing term imho, and Session is more general).
I just defined a couple of equates (CONNECTED and NOTCONNECTED)
When I get a Connect Completion notification, I check the Session's flag to see if its NOTCONNECTED, and if so, I set it to CONNECTED and process the event.
When the second notification arrives, I'll see I am already CONNECTED, and disregard the notification.
It seemed like the cheapest and best solution.

Footnote:
Previously I had defined this per-session state flag, but I wasn't using it from the iocp code, I had intended it to be used by 'Protocol Handlers' (overloaded general IO event interface for specific network protocols). Now any classes that derive from 'Protocol' must accept that the values 0 and 1 are reserved to mean not connected , and connected, respectively.


Posted on 2007-06-20 08:37:38 by Homer