Mark,

I hope your ok. Sometimes cheering up a friend can leave you drained.

I do see a need for WSABUF's but only in sending.
You see maximum throughput for the network is governed by the MTU/Packet size. I have a text somewhere on working this out so
at a later point ask me about tuning this.

Anyways, if you send one buffer bigger than the MTU (Maximum Transmission Unit) size the packet has a
"more packets to come" flag set. So the ends of the connection expect to receive and to send more packets and they handshake
on this. If you want fast throughput, try to send data in single non-split packets.
Using a WSABUF buffer for this helps as you can split the data to be sent into MTU size buffers, but still do one send call.

My assumption is that Winsock is smart enought to not try and put all the WSABUF data into one packet. Id have to actually test this.

The other thing is that other users of your code may find a use for the buffer arrangement and forcing them to use one buffer
may not be good. Since WSABUF allows one or multiple buffers, id just use that approach.

With the scaling of resources (threads, buffers and collections) my advice is to scale in percentages that the user can define.
This way they can track the peeks and define a percentage for that. A shrink and a grow percentage.
You dont want to shrink a buffer just to have the next call spike it right up again.

My new version should be posted at some point today (Saturday) or tomorrow.

Rgs, James.
Posted on 2005-06-17 17:57:10 by James_Ladd
I've been a little more worried about "nonconcurrency" when you have >1 worker thread.
Even though IO jobs posted on the IOCP will be processed in the order they are posted, they can complete out of order. This means that serialized IO jobs should be marked with a sequence number..which can in turn be used as an index into a perclient "buffer collection".
I'm not at the point where that issue has raised its head yet, but being aware of it is half way to handling it.

For this reason, I have created two classes : Client and IOJob, and I use the IOJob as the completion key rather than the Client who "owns it".
IOJobs are stored in a Pool, but are each marked with the "client who owns it".
The reason why I chose to use the IOJob as the completionkey rather than the Client is because we now know which asynchronous "partial operation" has completed, and can handle it out-of-order without searching the Client for the matching OVL or anything like that.

James, the "Client" is the object which owns the Socket, and also owns one or more outstanding (recyclable) "IOJob" objects.
These are just like using heap structs, except we can call procedures on a given struct instance...

A class method is simply a procedure with a hidden parameter .. pThis, aka "pStruct" apon which the procedure is to act.

An simple, useless but clear example of what an oop class method looks like if you were masm:

IOJob_SetOperation proc uses esi pIOJob:ptr IOJob, dwOperation:dword
assume esi:ptr IOJob
mov esi,pIOJob
push dwOperation
pop .operation
assume esi:nothing
ret
IOJob_SetOperation endp


What that "crappy method" would like like to a human coding with masm and the OA32 macros:

Method IOJob.SetOperation uses esi dwOperation:dword
SetObject esi
push dwOperation
m2m .operation,dwOperation
MethodEnd


Instances of classes are only a few bytes larger than structs, and nine times out of ten we can forgive this..

Classes which inherit from other classes are compacted during the build process so no bytes are wasted.

There's only one copy of the code no matter how many instances of a class we make.

Unless the methods are "Dynamic", they do not occupy space in the object instances.

I'm going to get you to try this stuff if it kills me because:
A) because your code becomes more portable from project to project as it is modular in nature
B) because it helps you code faster, yet you retain full lowlevel control of the binary code you produce
C) because you can benefit from polymorphism and interact with existing C/C++/VB code, etc.
D) because you have the right background and the right inclination
E) because you get the best of both worlds, can quickly transliterate between them, etc
F) because the overheads are all in your mind
G) because it's too damned cool to ignore

Have a nice day :)
Posted on 2005-06-18 02:56:52 by Homer
EvilHomer,

Once I have a simple example working Ill try an oop example.
But not until then. I know the oop way has advantages. I wouldnt be an oop person everyday if
I didnt understand that. The Smalltalk language is my favourite, so your preching to the converted.

One thing to keep in mind with communications if that event based programming is the way to go.
IOCP does this. And, no order if really guaranteed.

BTW - Thanks for your help.

Rgs, James.
Posted on 2005-06-18 17:28:46 by James_Ladd
Server beta demo was updated.
This time I have implemented a "Pool Class", and derived from it an "IOJob Allocator Class", which manages a "Pool" of "IOJobs". Pool class is threadsafe - meaning the code below it, including Collection and "Your Object", does not have to be threadsafe... and the only requirement of a "Poolable Object" is that it derives from Primer (I manipulate pOwner).
When I say "Pool" I am describing a special collection of "self-recycling objects".
By this I mean that the Pool abstracts the normal Creation and Destruction of the objects it is managing.. it contains "Used and Free lists" of objects, and its job is essentially to ensure that a minimum number of objects are not destroyed but instead these objects (and their resources) are retained and made available through the abstracted Create.
The Pool class is naiive as to the Class of the objects it holds, but in my case, I am pooling a new object called IOJob.
The IOJob object contains wsabuf and overlapped structs, and is formed from code that has mostly been transferred from the Client class, which is now much simpler.
The Server object owns the IOJobs Pool and the Clients Collection.
Each Client can own one or more IOJobs managed by their Owner (Server) IOJobs pool.
When an IOJob is (re)allocated by the Pool, it is (re)marked as being Owned by a given Client.
This way the Clients merely "implicitly own" a number of IOJob objects in the Used part of the Pool.
IOJobs know which Client owns them.. not the other way around.
Since I use ptr to IOJob as my "completionkey" for completed IO jobs, everything falls into place.. the given IOJob which completed knows which Client owns it, and can immediately call methods in the IOJob class and the owner Client class, and via the Client could even call methods in the Server class, which would be silly, I'm merely elaborating on the relationship between the objects.

The updated demo has been written to implement all the above, plus a few other changes.
We no longer create N Clients, and/or call N AcceptEx etc.
We create one initial Client object and one initial Accept socket, and call AcceptEx ONCE (ie, queue ONE IOJob, which is an Accept)..
In the AcceptCompletion handler should be a line to (repeat the above), but I didn't add it yet, so only one client at a time.. I wanted to check the IO stuff was working, since I shifted a lot of code around.

Please betatest :D
Posted on 2005-06-19 02:11:41 by Homer
EvilHomer,

Ill download your version and give it a test tonight.
A new version of FAStServer was posted. It works !!
It can be found here http://www.jamesladdcode.com

Thanks for your help with it.

Rgs, James.
Posted on 2005-06-19 15:57:37 by James_Ladd
I implemented a ClientPool, where Clients own IOJobs from Server's shared IOJobPool, and everything went wrong :(
I'm not even getting AcceptIO's anymore, which seems weird.
I must be doing something critical in the wrong order .. I'll figure it tomorrow, yawn..
Posted on 2005-06-20 09:50:23 by Homer
Homer,
Where can I get the latest to have a look at the issue ?
One suggestion I have is; do the absolute minimum to get the thing done, then add to it.
I have made several versions of FAStServer and then gone back to super simple basics. Because
something invariably goes wrong nad it is easier to work out if things are kept as simple as
they can be to get the job done. If you add the most minute of increments, then it has to be
the 3 or so calls you just added that went wrong. (Almost always).
Rgs, James.
Posted on 2005-06-20 16:06:19 by James_Ladd
The binary update is the same url as previously, but doesn't represent the current state of the source.

I'm hoping to catch you on the messaging thingy soon, I have some questions that would best be answered in realtime.. perhaps you can help :)

Posted on 2005-06-24 10:31:42 by Homer
EvilHomer,
Watch for me on messanger today. Ill be around and eager to talk.
Rgs, James.
Posted on 2005-06-24 19:45:30 by James_Ladd
I think I found the culprit :)
It's to do with the completionkey chosen for the listening socket's OnAccepted completions.
I looked back at your earlier work (one-socket version with acceptex) and noticed something odd.
You call CreateIOCompletionPort with all nulls at first, ok normal, but then when you bind the listen socket to the demo client's acceptio ovl, you were using NULL as the completionkey !!
When I added special casecode for completionkey=null and mimicked your behaviour, everything started working .. really nicely.. zip has been updated :)
Clients are accepted immediately, and the weird "kicking the previous client on accept" is gone.
It seems to be doing everything it's meant to be doing - the source could do with a tidy up now :P
Posted on 2005-06-24 21:08:02 by Homer
So far only receive code had been implemented, and a single recv io per client at that.
When I implemented send using overlapped asynchronous io, something failed - I got winsock error 10045 on my sends.
I've since realized that my send io's completion key is not bound to the iocp.
The best solution seems to me : move the call to CreateIOCompletionPort from the Accepted handler to the IOJobPool_Allocate method, or near that call..

In the meantime, I was messing around with something related: OA32 isn't designed to handle crosscalling of classes across modules, and I've been playing with several simple hacks to get around it. Basically it's a chicken-egg situation I'm trying to resolve, but if I don't find a satisfactory solution I'll simplify the Plugin to be a logic engine which performs no tasks but instead whose methods return values become triggers for operations performed within Server..

Have a nice day :)
Posted on 2005-06-26 03:01:51 by Homer
I didn't find a satisfactory solution to the N chickens / N eggs / N baskets issue.
I therefore spent some time knocking all the meat off the bones of the Plugin module.. I've stripped it down to a very basic callback class which performs only logical operations via a handful of staticabstracts. Then I made an EchoPlugin class which derives from Plugin, and neither of them contain references to other classes.. only to the XOVL struct. This is probably an all-around better solution but at the cost of some call overhead (we can't and don't make cross class calls from Plugin or its derivative(s)).
The idea is that Server calls various callbacks within Plugin, which may only operate on the given XOVL and generally which simply return a value indicating to Server which logical branch to follow.
OK that cleans up the Plugin side of things, and leaves me with a little work to do on Server tomorrow to see these changes through.
It doesn't get me closer to a neat solution to cross-calling within the Server class, however I'm willing to be a little hacky with regards to the bare handful of Plugin method calls..
Posted on 2005-06-26 10:22:24 by Homer
EvilHomer,
Hmmn, sounds like things are moving along.
Dont forget to associate the socket for the acceptex call with the completion port BEFORE you do the accept.
Im sure you have worked this out.
Keep on it.
Rgs, James.
Posted on 2005-06-26 15:55:39 by James_Ladd
James - some potentially sticky questions ..
does it make more sense to you that the CreateIOCP binding to a compkey be performed as soon as an XOVL is (re)allocated? And do you know how many completionkeys can be associated with the same socket?
Finally, rather than binding a client socket and compkey on completion of Accept, is there a reason not to bind new client sockets to the iocp BEFORE they are accepted?

Have a nice day :)
Posted on 2005-06-26 21:43:46 by Homer
Ah, back in the saddle - it's compiling again after only three hours of cutting, pasting and generally manipulating code across a half dozen classes .. it's not WORKING, but it's compiling..

I had a half dozen classes whose methods often called methods in arbitrary other classes.
This not only made the source difficult to build due to N-chicken-egg references, it also made it a little difficult to follow outside of a multiple document interface based ide.

I've been removing a whole bunch of these "cross-calls" so that most of the action now occurs in Server class - indeed IOJob has been almost totally gutted, and Client is looking thin also.

I may have been a little overzealous, we'll see.

Note that I'm generally referring to problems I've created for myself due to my indifferent approach to oop .. it is in my nature to explore possibilities and tread thin ice, and indeed since I am developing public code for the OA32 oop model, it is only right that I tread that ice before anyone else does - but enough messing around, the code is cleaned up and I'll now forge ahead.


Posted on 2005-06-27 00:19:19 by Homer
EvilHomer,

You should bind the socket you will accept on before AcceptEx if you want accept notifications.

To know they are accept notifications you and not some other IO event you should have and
set an operation field in your completion key or extended overlapped struct. I choose to do it in
the completion key struct. eg:    mov operation, OPERATION_ACCEPT.

Then when you do a read, set the to OPERATION_READ. Etc etc.

When the IO routine completes you will get a completion event and the completion key.
This key will tell you which operation the notification is for, however .....

To have multiple operations on the go at one time, you should extend the OVERLAPPED structure as
this is the only one you can specify with each call to WSARecv or WSASend. You could keep the
common stuff in the single completion key associated with the socket, and the changing stuff in
the extended OVERLAPPED structure.

Rgs, James.
Posted on 2005-06-27 03:44:55 by James_Ladd

James,

You should bind the socket you will accept on before AcceptEx if you want accept notifications.


Yep, bind the Listener before issuing AcceptEx - sure - but what about the sockets being accepted?
The examples (which are never truly asynch) always show the accepted client's socket being bound to the iocp during the completion of the accept on said client socket.. often the xovl that was used in the acceptex is rebound for client readIO, which is fine except that we're limited to a single completionkey, which you elect should be a perclient key.. Some examples, mine included, use a per-client-context and a per-io-context, a pointer to the latter being employed as the completionkey.
Is there any reason I can't bind >1 completionkey to an accepted client socket, and is there any reason I can't pre-bind accepted sockets before they are accepted?
Posted on 2005-06-27 09:01:29 by Homer
EvilHomer,

You should never bind the listener socket, only the accept socket !
So yes, you should pre-bind the 'to be accepted' socket before the AcceptEx call.

My example is a simple one, so I use a completion key without problems.
This should be changed to use an extended OVERLAPPED struct since this is the
only way to bind a per-call structure that will be passed back during completion.
The completion key is always the same for every IO completion because its bound
early. For parallelism and true asynchronous IO the OVERLAPPED struct should be
used, so you can specify a different (or same) OVERLAPPED struct on each
WSASend and WSARecv.

I will be updating my example to use the OVERLAPPED struct approach very soon.

Rgs, James.
Posted on 2005-06-27 15:53:27 by James_Ladd
Nevermind, I was talking gibberish.
It really doesn't matter WHAT I use as a completion key, because I do not use the completion key anymore.. I use pOVL (pXOVL) returned from GetQueuedCompletionStatus as a guide to the Client who is its owner via a pointer stored in xovl :)
My XOVL is actually a field of IOJob object class (think of IOJob as a struct with a xovl member), and in turn, XOVL contains a special "backpointer field" called piojob, so that given a pointer to an xovl, we can find the iojob which contains the given xovl..
IOJob also contains a pOwner field indicating which Client class object owns it.

I've reworked the Plugin baseclass and EchoPlugin derivative class which are used in the Protocol plugin DLL .. Plugin is a generic class which the Server thinks its talking to, and ProtocolPlugin derivatives of Plugin class are used to build different Plugin DLLs - that the Server can always talk to.

I got the Plugin class methods hooked up and firing on the Server side, and the EchoPlugin derivation class in the Protocol dll is cooperating .. now I'm considering how to allow the Plugin to queue sends via Server DLL components..perhaps allowing Plugin access to Server isn't a bad idea after all.. I'm not inclined to provide a named export on the Server DLL for asynch sending !!

Also I got WSASend to work, although there's still a small bug in that..
Anyhow, just a heads up to let you know how I'm crawling..
Posted on 2005-06-28 09:20:49 by Homer
EvilHomer,

Sounds like your making good progress.

My initial design for the plugin doing sends was to allow a call a server send function/method.
The server could then use whatever implementation it wanted. Most likely just on call to
WSASend.

Ill be changing things in my example to use the extended overlapped approach.

Rgs, James.
Posted on 2005-06-28 15:52:37 by James_Ladd