Hey :)
I'll be using this thread to describe and to discuss aspects of the NetEngine component which will go public with the next official release of ObjAsm32 - the object oriented programming environment for x86 assembly programmers.
I guess the best place to start is to talk a little about what IOCP actually is, and how IOCP networking differs from regular socket programming.
IOCP stands for IO Completion Port - essentially its a message queue which tells us when an attempted IO operation has completed, and whether it succeeded or failed.
You might wish to think of the IOCP as being a lot like our old friend the Windows Message queue - the key difference is that IO jobs can complete 'out of order' - ie, asynchronously.
You can use an IOCP for anything that supports "overlapped io", including file io - and you can invent other kinds of IO jobs , eg to signal that a chunk of data has finished being decompressed by some other thread... but I will restrict my discussion to network programming :P
So what is an IO job?
There are four main types of IO job (or io operation if you prefer) which network programmers will use.
The are Connect (to a remote host), Accept (an incoming client connection), Read (recv) and Write (send).
When we initiate a socket operation using appropriate api, we will typically get an error "IO PENDING", meaning that the operation could not complete immediately, and that we can expect a message via our IOCP when that operation completes.
In order to service these IOCP messages, we create one of more "worker threads" which essentially sleep until something happens.
This introduces a small problem - although the IOCP messages are serviced in order of completion, the actual SERVICING of IO jobs may complete out of order due to the nature of multithreaded operations!
For example, say we have two Worker threads, and two READ operations complete in order #1, then #2.
Our IOCP alerts the threads "in order" that these receives just completed, but who can say which thread will be serviced first by the thread subsystem? Introducing any kind of mutex here would completely wipe out any benefits gained by using asynchronous multithreaded code in the first place, so that's not a solution - but I digress. The way you handle this will depend a lot apon the kind of work you're trying to do.
In terms of writing a network Server, the major difference is how the Accept api works.
Normally, we would expect that when Accept completes successfully, it returns a Socket for the newly-accepted incoming connection. IOCP does this very differently.
Its very expensive to make a Socket, so to avoid this cost, we create a whole bunch of sockets at the start, which are waiting for Accept to complete.. we can add more "pending accepts" later if we run out, but we'll start to incur that cost I was talking about.
Otherwise, the networking code is very much like any other asynchronous networking code you've seen, with the fundamental difference being that we don't have to pay a cost to receive a notification that an event has occurred.
IOCP is tied closely to the Windows event subsystem, and for this reason, is far more efficient than any other networking scheme on this platform.
IOCP is designed to handle tens of thousands of connections asynchronously, using just a handful of worker threads.. But you can use it for regular networking and still benefit from the low latency.
In my next post, I'll begin to desribe some features of my NetEngine implementation, which I have already used to implement various kinds of servers and clients.
I'll be using this thread to describe and to discuss aspects of the NetEngine component which will go public with the next official release of ObjAsm32 - the object oriented programming environment for x86 assembly programmers.
I guess the best place to start is to talk a little about what IOCP actually is, and how IOCP networking differs from regular socket programming.
IOCP stands for IO Completion Port - essentially its a message queue which tells us when an attempted IO operation has completed, and whether it succeeded or failed.
You might wish to think of the IOCP as being a lot like our old friend the Windows Message queue - the key difference is that IO jobs can complete 'out of order' - ie, asynchronously.
You can use an IOCP for anything that supports "overlapped io", including file io - and you can invent other kinds of IO jobs , eg to signal that a chunk of data has finished being decompressed by some other thread... but I will restrict my discussion to network programming :P
So what is an IO job?
There are four main types of IO job (or io operation if you prefer) which network programmers will use.
The are Connect (to a remote host), Accept (an incoming client connection), Read (recv) and Write (send).
When we initiate a socket operation using appropriate api, we will typically get an error "IO PENDING", meaning that the operation could not complete immediately, and that we can expect a message via our IOCP when that operation completes.
In order to service these IOCP messages, we create one of more "worker threads" which essentially sleep until something happens.
This introduces a small problem - although the IOCP messages are serviced in order of completion, the actual SERVICING of IO jobs may complete out of order due to the nature of multithreaded operations!
For example, say we have two Worker threads, and two READ operations complete in order #1, then #2.
Our IOCP alerts the threads "in order" that these receives just completed, but who can say which thread will be serviced first by the thread subsystem? Introducing any kind of mutex here would completely wipe out any benefits gained by using asynchronous multithreaded code in the first place, so that's not a solution - but I digress. The way you handle this will depend a lot apon the kind of work you're trying to do.
In terms of writing a network Server, the major difference is how the Accept api works.
Normally, we would expect that when Accept completes successfully, it returns a Socket for the newly-accepted incoming connection. IOCP does this very differently.
Its very expensive to make a Socket, so to avoid this cost, we create a whole bunch of sockets at the start, which are waiting for Accept to complete.. we can add more "pending accepts" later if we run out, but we'll start to incur that cost I was talking about.
Otherwise, the networking code is very much like any other asynchronous networking code you've seen, with the fundamental difference being that we don't have to pay a cost to receive a notification that an event has occurred.
IOCP is tied closely to the Windows event subsystem, and for this reason, is far more efficient than any other networking scheme on this platform.
IOCP is designed to handle tens of thousands of connections asynchronously, using just a handful of worker threads.. But you can use it for regular networking and still benefit from the low latency.
In my next post, I'll begin to desribe some features of my NetEngine implementation, which I have already used to implement various kinds of servers and clients.
In the beginning, I designed my implementation to be a network server, so I called the main interface "Server" - I have since extended it to handle both Server and Client connections, but since it also acts as a home for the IOCP Worker threads, I left the name as it stands - it is at heart, a network message notification server.
Most of the actual networking code was pushed out to a class called Client.
A Client object represents a potential network connection, and these are stored in the Server in three "pools" of recyclable objects - it can be outbound, inbound, or "listening".
The Client class represents a socket and a socket address, and implements the functionality we would expect of a network socket - it handles initiation and completion events for connections, accepts, sends and receives.
Some of these are handed on to a so-called "Protocol handler", something I'll speak more of later.
When messages are passed on in this manner, we call it "marshalling"... the events are "marshalled" so that the appropriate "handler" is notified and can react to them.
My implementation of NetEngine allows each and every Client object, whether it represents an inbound or outbound connection, to marshall such events to a particular Protocol handler who is responsible for determining that the session is following the rules of the given protocol.
That means you can have any number of inbound and outbound sessions, each with their own protocol, and all driven by the same IOCP message system!
In my next post, I'll begin to describe the objects involved in my implementation, their relationships, and how they interact. We need to put on our oop teeshirt and get ready to deal with complex object relationships!
Most of the actual networking code was pushed out to a class called Client.
A Client object represents a potential network connection, and these are stored in the Server in three "pools" of recyclable objects - it can be outbound, inbound, or "listening".
The Client class represents a socket and a socket address, and implements the functionality we would expect of a network socket - it handles initiation and completion events for connections, accepts, sends and receives.
Some of these are handed on to a so-called "Protocol handler", something I'll speak more of later.
When messages are passed on in this manner, we call it "marshalling"... the events are "marshalled" so that the appropriate "handler" is notified and can react to them.
My implementation of NetEngine allows each and every Client object, whether it represents an inbound or outbound connection, to marshall such events to a particular Protocol handler who is responsible for determining that the session is following the rules of the given protocol.
That means you can have any number of inbound and outbound sessions, each with their own protocol, and all driven by the same IOCP message system!
In my next post, I'll begin to describe the objects involved in my implementation, their relationships, and how they interact. We need to put on our oop teeshirt and get ready to deal with complex object relationships!
Warning: The following post contains a lot of OOP concepts, I hope you are wearing your oop teeshirt!
Note: You don't need to understand the internals of NetEngine to use it, with the exception of the Network Protocol handler, which is briefly described here. You only need to make a few calls to the main Server class, almost all the interesting stuff happens in your protocol handler.
As mentioned, the major component of the NetEngine is a class called Server.
For most users, the most important two methods of the Server class are called ConnectTo and Listen.
These two methods both require the caller to provide a pointer to a network protocol handling class.
I have provided a class called, strangely enough, NetworkProtocol.
It is a BASE CLASS, you should write your own class which overrides the methods of this class.
In fact, you can handle as many protocols as you like, on a per-session basis.
You simply hand the protocol object you want when making ConnectTo and Listen calls, and NetEngine will use them as 'event sinks'.
NetEngine calls the methods of NetworkProtocol (and your derivatives) in order to inform you of the completion of the various IO jobs that you have issued, or that were issued on your behalf.
You can handle them silently, and/or hand them on to your application - this is the other reason that you need to derive network protocol handler classes for yourself.
You can imagine that they sit inbetween NetEngine and your app.
The Init method of NetworkProtocol is not declared, I suggest you use it to set the Owner of the protocol object to your app window or dialog window so that you can then send your app Window Messages directly from the protocol handler.
As mentioned previously, network sessions (and Listening ports) are represented by a class called Client.
Each client is the Owner of any number of pending io jobs, ie, each io job knows which Client issued it.
But a Client does not STORE io jobs, those are stored in the Server class.
So the Server class is managing all the resources, and the Ownership variable of various objects is just used to make associations between all the objects.
These associations are useful, as they give objects access to methods of related objects.
For example, when an io job completes, it will inform its owner Client, which will inform its protocol handler, which can then inform the application.
The owner of the Client class, in the current implementation, is the handle of the IOCP itself.
This allows a newly-accepted inbound client to bind its socket to the iocp.
I may decide to change this, but since the Client class cannot (and never needs to) make calls to the Server class, I saw no point in making the Server class the owner of arbitrary Clients.
The Client and IOJob classes are managed through classes called ClientPool and IOJObPool.
These two classes derive from a baseclass called Pool, which implements a pool of recyclable objects.
Allocating memory for objects is expensive, why throw away perfectly good objects just because we don't currently need them? Pooling is a way to avoid this, we keep redundant objects in memory, and reinitialize them when we need them again. This is much faster.
However we don't wish to keep an infinite number of dead objects floating in our pools.
The Init method of the Server class allows us to set some sane limits on the minimum and maximum number of pooled resources that we wish to retain at all times, anything outside this range will be going to landfill.
For example, for a Server application, we might wish to have at least 5 pending Accepts, no more than 10 thousand concurrent sessions, and no more than fifty thousand pending io jobs.
For most users, these values can be set to something reasonably small, but if we are coding a MMORPG or other intensive server, we'll want more.
NetEngine will attempt to scale its memory use to suit the demand, but we still need some absolute limits, yes?
For the record, I've tested NetEngine on a 333Mhz machine with 16 meg of ram and a 4 gig hard drive (most of which is consumed by XP), with twenty thousand concurrent connections, it consumed 25 percent of the cpu bandwidth, and packets pinged at no more than 250 ms for each and every client, given that they were talking to localhost.
And a friend of mine tested it on linux under Wine, with very similar results.
In fact, one of the early goals of NetEngine was to implement a massively multi user server that could run on low-end hardware without consuming vast amounts of memory and cpu resources.
I do hope you will support and use NetEngine, help me to identify any remaining bugs and implementation nuances, make suggestions, etc.. its FREE, and its better than anything I've seen on sourceforge or codeguru etc, do the right thing, don't be a packet monkey.
Note: You don't need to understand the internals of NetEngine to use it, with the exception of the Network Protocol handler, which is briefly described here. You only need to make a few calls to the main Server class, almost all the interesting stuff happens in your protocol handler.
As mentioned, the major component of the NetEngine is a class called Server.
For most users, the most important two methods of the Server class are called ConnectTo and Listen.
These two methods both require the caller to provide a pointer to a network protocol handling class.
I have provided a class called, strangely enough, NetworkProtocol.
It is a BASE CLASS, you should write your own class which overrides the methods of this class.
In fact, you can handle as many protocols as you like, on a per-session basis.
You simply hand the protocol object you want when making ConnectTo and Listen calls, and NetEngine will use them as 'event sinks'.
NetEngine calls the methods of NetworkProtocol (and your derivatives) in order to inform you of the completion of the various IO jobs that you have issued, or that were issued on your behalf.
You can handle them silently, and/or hand them on to your application - this is the other reason that you need to derive network protocol handler classes for yourself.
You can imagine that they sit inbetween NetEngine and your app.
The Init method of NetworkProtocol is not declared, I suggest you use it to set the Owner of the protocol object to your app window or dialog window so that you can then send your app Window Messages directly from the protocol handler.
As mentioned previously, network sessions (and Listening ports) are represented by a class called Client.
Each client is the Owner of any number of pending io jobs, ie, each io job knows which Client issued it.
But a Client does not STORE io jobs, those are stored in the Server class.
So the Server class is managing all the resources, and the Ownership variable of various objects is just used to make associations between all the objects.
These associations are useful, as they give objects access to methods of related objects.
For example, when an io job completes, it will inform its owner Client, which will inform its protocol handler, which can then inform the application.
The owner of the Client class, in the current implementation, is the handle of the IOCP itself.
This allows a newly-accepted inbound client to bind its socket to the iocp.
I may decide to change this, but since the Client class cannot (and never needs to) make calls to the Server class, I saw no point in making the Server class the owner of arbitrary Clients.
The Client and IOJob classes are managed through classes called ClientPool and IOJObPool.
These two classes derive from a baseclass called Pool, which implements a pool of recyclable objects.
Allocating memory for objects is expensive, why throw away perfectly good objects just because we don't currently need them? Pooling is a way to avoid this, we keep redundant objects in memory, and reinitialize them when we need them again. This is much faster.
However we don't wish to keep an infinite number of dead objects floating in our pools.
The Init method of the Server class allows us to set some sane limits on the minimum and maximum number of pooled resources that we wish to retain at all times, anything outside this range will be going to landfill.
For example, for a Server application, we might wish to have at least 5 pending Accepts, no more than 10 thousand concurrent sessions, and no more than fifty thousand pending io jobs.
For most users, these values can be set to something reasonably small, but if we are coding a MMORPG or other intensive server, we'll want more.
NetEngine will attempt to scale its memory use to suit the demand, but we still need some absolute limits, yes?
For the record, I've tested NetEngine on a 333Mhz machine with 16 meg of ram and a 4 gig hard drive (most of which is consumed by XP), with twenty thousand concurrent connections, it consumed 25 percent of the cpu bandwidth, and packets pinged at no more than 250 ms for each and every client, given that they were talking to localhost.
And a friend of mine tested it on linux under Wine, with very similar results.
In fact, one of the early goals of NetEngine was to implement a massively multi user server that could run on low-end hardware without consuming vast amounts of memory and cpu resources.
I do hope you will support and use NetEngine, help me to identify any remaining bugs and implementation nuances, make suggestions, etc.. its FREE, and its better than anything I've seen on sourceforge or codeguru etc, do the right thing, don't be a packet monkey.
NetEngine embeds an object called ClientGroup, which represents a Named group of Clients, and gives you a way to organize Clients (think of a Lobby server, for example).
As well as being able to Name groups, you can provide a description string.
You can think of a ClientGroup as a communication channel, chatroom, etc... it can be used to broadcast messages to all of the member Clients of a Group.
ClientGroup supports SUBGROUPS ...
The Server class implements a Root ClientGroup - you can add subgroups to it, and to them, and so create a Tree - a hierarchy of Groups.
Clients can be members of as many Groups as you wish.
Your Protocol class, or your application, can add newly-connected Clients to one or more Groups, but when a Client 'dies', the Server class will automatically remove them from the entire tree.
The ClientGroup.Broadcast method will send data to all member Clients of a given Group and all of its SubGroups, with the exception of the client who is doing the Sending.
The ClientGroup.Send method will send data only to the members of a specific Group, and not its SubGroups (again, with the exception of the Sender).
Although (as mentioned) the Server class will prune dead clients from the ClientGroup tree, its fair to say that ClientGroup is NOT closely tied to the Server class, it was my intention that it work closely with your Protocol class and/or your application, because NetEngine does not care what purpose you use ClientGroups for.
As well as being able to Name groups, you can provide a description string.
You can think of a ClientGroup as a communication channel, chatroom, etc... it can be used to broadcast messages to all of the member Clients of a Group.
ClientGroup supports SUBGROUPS ...
The Server class implements a Root ClientGroup - you can add subgroups to it, and to them, and so create a Tree - a hierarchy of Groups.
Clients can be members of as many Groups as you wish.
Your Protocol class, or your application, can add newly-connected Clients to one or more Groups, but when a Client 'dies', the Server class will automatically remove them from the entire tree.
The ClientGroup.Broadcast method will send data to all member Clients of a given Group and all of its SubGroups, with the exception of the client who is doing the Sending.
The ClientGroup.Send method will send data only to the members of a specific Group, and not its SubGroups (again, with the exception of the Sender).
Although (as mentioned) the Server class will prune dead clients from the ClientGroup tree, its fair to say that ClientGroup is NOT closely tied to the Server class, it was my intention that it work closely with your Protocol class and/or your application, because NetEngine does not care what purpose you use ClientGroups for.
Homer, I love the work you do! I have thrown together a couple versions of my own "oop" version of winsock in masm without it really being "oop" Maybe I should look into the ObjAsm32 project, however the nagoa++ project for nasm had plenty of errors(in the windows inc file) for me to not want to work in nasm as well as be leery of any other "user-based" projects for an assembler which is why I stuck with working in masm32 without too many changes. Every version of my winsock projects seemed to work, but was so dirty that the code seemed like it didn't simplify anything like I wanted it to.
I figure I'll check into the objasm project, however is there a chance you could make a dll version of this framework? Obviously it wouldn't be quite the "Client[56].SendData(lpData,lLength)" but maybe a "invoke SendData,hClient,addr Data, DataLength" I hope you get what I mean.
(I keep explaining things then erasing them because it only makes it more complicated which confuses me more.... gah it's early in the morning I need some sleep. <-- Maybe I should erase this rambling too.. <-- this too... <-- this......................)
I figure I'll check into the objasm project, however is there a chance you could make a dll version of this framework? Obviously it wouldn't be quite the "Client[56].SendData(lpData,lLength)" but maybe a "invoke SendData,hClient,addr Data, DataLength" I hope you get what I mean.
(I keep explaining things then erasing them because it only makes it more complicated which confuses me more.... gah it's early in the morning I need some sleep. <-- Maybe I should erase this rambling too.. <-- this too... <-- this......................)
The earliest version of NetEngine was a DLL core.
I removed that wrapper, because I felt that the framework was immature, perhaps I can change my mind later? Or you can take my code and DLL-wrap it yourself.
You are welcome to do what you like, you are free to, the code is free, there is no license.
If you do that, it would be nice to include some kind of reference to OA32 and myself in the description.
I appreciate your expression of interest, and I will post some actual code real soon, I am currently walking over the code with the originator of OA32 (Biterider), asking him to review my work before it goes public :)
I removed that wrapper, because I felt that the framework was immature, perhaps I can change my mind later? Or you can take my code and DLL-wrap it yourself.
You are welcome to do what you like, you are free to, the code is free, there is no license.
If you do that, it would be nice to include some kind of reference to OA32 and myself in the description.
I appreciate your expression of interest, and I will post some actual code real soon, I am currently walking over the code with the originator of OA32 (Biterider), asking him to review my work before it goes public :)
Well I will await your release!
I'm contemplating adding one final class to NetEngine : an application-layer interface.
This would be a second abstracted class which means you need to write not only at least one Protocol class derivation but also your own Application class derivation, adding complexity and some call cost in return for a lot of flexibility and making it a lot easier to use, obviating the need to understand anything about the internals.
Its a difficult decision, to trade off speed for functionality, especially where that functionality is just something else that the User (programmer) must provide.
Perhaps I should make it optional.
This would be a second abstracted class which means you need to write not only at least one Protocol class derivation but also your own Application class derivation, adding complexity and some call cost in return for a lot of flexibility and making it a lot easier to use, obviating the need to understand anything about the internals.
Its a difficult decision, to trade off speed for functionality, especially where that functionality is just something else that the User (programmer) must provide.
Perhaps I should make it optional.
Hi
Since the Messenger service is not running today, i decided to post the logical processor detection code. It is easier than the Intel proposed code since we don't need all that functionality provided by the original code. If the affinity masks are needed to make sure that each thread will run on a different logical processor then we need to return an affinity mask array (max 32 dwords) and use these masks in combination with the SetThreadAffinityMask api.
Regards,
Biterider
Since the Messenger service is not running today, i decided to post the logical processor detection code. It is easier than the Intel proposed code since we don't need all that functionality provided by the original code. If the affinity masks are needed to make sure that each thread will run on a different logical processor then we need to return an affinity mask array (max 32 dwords) and use these masks in combination with the SetThreadAffinityMask api.
GetLogProcCount proc uses ebx edi esi, pMaskTable:Pointer
local dProcessAffinity:dword, dSystemAffinity:dword, hCurrentProcessHandle:Handle
mov hCurrentProcessHandle, $invoke(GetCurrentProcess)
invoke GetProcessAffinityMask, hCurrentProcessHandle, addr dProcessAffinity, addr dSystemAffinity
.if eax != FALSE
xor edi, edi
xor ebx, ebx
inc edi
mov esi, pMaskTable
@@:
test edi, dSystemAffinity
.if !Zero?
;Check if this logical processor is available to this process
.if $invoke(SetProcessAffinityMask, hCurrentProcessHandle, edi)
.if esi != NULL
mov dword ptr , edi
.endif
invoke Sleep, 0 ;Give OS time to switch CPU
inc ebx
.endif
.endif
shl edi, 1 ;Prepare mask for the next logical processor
jz @F ;Exit if the mask is above 32
cmp edi, dSystemAffinity
jbe @B
@@:
;Set previous processor affinity
invoke SetProcessAffinityMask, hCurrentProcessHandle, dProcessAffinity
mov eax, ebx
.endif
ret
GetLogProcCount endp
Regards,
Biterider
Thanks, that lets me remove one parameter from server init method, since we want one Worker thread per available logical cpu... although it might not seem like a big deal, I think its a really nice addition, its one thing less that the user (programmer) has to think about.
I think this deserves :thumbsup: Two Thumbs Up :thumbsup: !!
I think this deserves :thumbsup: Two Thumbs Up :thumbsup: !!
Thanks Biterider, I implemented that code immediately - NetEngine now automatically creates the optimal number of Worker Threads for the hardware it is running on.
I need to mention something about the Client class.
As mentioned previously, this class is mostly a wrapper for a socket, and can operate in one of three possible ways - it can represent an inbound connection to your server, an outbound connection to a remote host, or it can represent a 'Listening' socket, whose job is to accept inbound connections.
Client class contains a field called pListener.
If this field is set to a non-zero value, it means that this Client represents an inbound connection that your server has accepted, and pListener is a pointer to the Client that represents the Listener.
This is quite a confusing concept - a client field that is pointing to another client object.
Its purpose is to allow newly-accepted clients to be associated with their host "listener" client who accepted them, which in turn allows the newly accepted client to inherit the listener's Protocol.
This lets NetEngine use a common pool of clients for any number of listening ports and protocols.
I may decide to alter this scheme in a future version of NetEngine, but since this is really a deeply internal mechanism, it will only be of interest to those who choose to study the implementation.
I need to mention something about the Client class.
As mentioned previously, this class is mostly a wrapper for a socket, and can operate in one of three possible ways - it can represent an inbound connection to your server, an outbound connection to a remote host, or it can represent a 'Listening' socket, whose job is to accept inbound connections.
Client class contains a field called pListener.
If this field is set to a non-zero value, it means that this Client represents an inbound connection that your server has accepted, and pListener is a pointer to the Client that represents the Listener.
This is quite a confusing concept - a client field that is pointing to another client object.
Its purpose is to allow newly-accepted clients to be associated with their host "listener" client who accepted them, which in turn allows the newly accepted client to inherit the listener's Protocol.
This lets NetEngine use a common pool of clients for any number of listening ports and protocols.
I may decide to alter this scheme in a future version of NetEngine, but since this is really a deeply internal mechanism, it will only be of interest to those who choose to study the implementation.
I am making some last-minute additions to NetEngine in order to support different session behaviours that might be implemented under various protocols, including those defined by the user.
I think the best place to start is to define those behaviours:
PACKET ORIENTED:
NetEngine's current code only allows for packet-oriented protocols, where each packet of data in the stream contains a PacketLength field or other useful means of delimiting the packets... this is important, because we might receive for example two and a half packets, meaning that some data must remain in the buffer while we wait for more.
STREAM ORIENTED:
What if we wish to transfer a file? There will be no way of knowing when the transfer is complete, other than that the session has expired - that is to say, there is no delimiter, session expiry IS the delimiter.
HYBRID:
It is reasonable to suggest a packet-negotiated streaming protocol, where we send a few handshaking packets which are followed by the main body of data being transferred. In this case, the protocol handler would "switch itself" from packet-oriented to stream-oriented at the appropriate moment.
And then there are protocols such as http, where there may be a lot of data transferred, and yet we can still expect a sane delimiter - should we just ignore that fact and wait for session expiry? No, because http 1.1 supports "keep-alive", where the session is NOT terminated when the transfer completes.
Whatever I decide to do, NetEngine is going to need to be able to aggregate received data and retain it at least until we determine that we have a complete protocol packet - which may span several IOJobs.
I will try to implement the necessary changes as quickly as possible.
I think the best place to start is to define those behaviours:
PACKET ORIENTED:
NetEngine's current code only allows for packet-oriented protocols, where each packet of data in the stream contains a PacketLength field or other useful means of delimiting the packets... this is important, because we might receive for example two and a half packets, meaning that some data must remain in the buffer while we wait for more.
STREAM ORIENTED:
What if we wish to transfer a file? There will be no way of knowing when the transfer is complete, other than that the session has expired - that is to say, there is no delimiter, session expiry IS the delimiter.
HYBRID:
It is reasonable to suggest a packet-negotiated streaming protocol, where we send a few handshaking packets which are followed by the main body of data being transferred. In this case, the protocol handler would "switch itself" from packet-oriented to stream-oriented at the appropriate moment.
And then there are protocols such as http, where there may be a lot of data transferred, and yet we can still expect a sane delimiter - should we just ignore that fact and wait for session expiry? No, because http 1.1 supports "keep-alive", where the session is NOT terminated when the transfer completes.
Whatever I decide to do, NetEngine is going to need to be able to aggregate received data and retain it at least until we determine that we have a complete protocol packet - which may span several IOJobs.
I will try to implement the necessary changes as quickly as possible.
Most of the changes have been put into place.
The Server object used to pass recv completion notifications directly to a Client's Protocol handler, bypassing the Client.
Now they are passed to the Client, who can deal with them in its own context, while making calls to its Protocol handler to make decisions on its behalf.
The logic dealing with recv completion has been moved from NetworkProtocol to Client, while the logic dealing with analysing received data to find its protocol-specific delimiters remains in NetworkProtocol.
This makes a lot of sense.
The Client object now inherits from DwordCollection.
This allows Client to store a list of IOJobs which are completely full of data until a delimiter has been found.
It also means that Client REPRESENTS a collection of iojobs that are pending receipt of that delimiter, and can be passed AS a collection to NetworkProtocol (who doesn't actually know what a Client is).
Once a delimiter is found, the last IOJob is also tossed onto the Client's collection, then the Client is passed to NetworkProtocol where the set of IOJob buffers can be processed.
When that call returns, Client can clean up all the consumed IOJobs, possibly leaving some unconsumed data in the last IOJob, which is not part of this protocol-specific receive operation (it's the start of the next one).
This is a rather elaborate use of object inheritance, and shows the true power and flexibility of ObjAsm32, and why OOP can be a Good Thing when used appropriately.
I think the changes I've made cover all possible situations except one.
What about protocols which are only delimited by session expiry?
If a session expires, and we have received data that is pending a delimiter, how can we tell whether the session expiry was the delimiter we sought, or whether the connection was lost prematurely?
It may be necessary to add a flag to NetworkProtocol to alter the behaviour for these "no protocol" transmissions of data (such as raw file transfers or other delimiterless streaming).
In these cases, there is no way we can know that the data sent is complete without sending some kind of preamble that includes the expected size of the complete transfer.. if we don't wish to do that, we'll just have to assume that the data is complete and intact, as there is no protocol in place.
The Server object used to pass recv completion notifications directly to a Client's Protocol handler, bypassing the Client.
Now they are passed to the Client, who can deal with them in its own context, while making calls to its Protocol handler to make decisions on its behalf.
The logic dealing with recv completion has been moved from NetworkProtocol to Client, while the logic dealing with analysing received data to find its protocol-specific delimiters remains in NetworkProtocol.
This makes a lot of sense.
The Client object now inherits from DwordCollection.
This allows Client to store a list of IOJobs which are completely full of data until a delimiter has been found.
It also means that Client REPRESENTS a collection of iojobs that are pending receipt of that delimiter, and can be passed AS a collection to NetworkProtocol (who doesn't actually know what a Client is).
Once a delimiter is found, the last IOJob is also tossed onto the Client's collection, then the Client is passed to NetworkProtocol where the set of IOJob buffers can be processed.
When that call returns, Client can clean up all the consumed IOJobs, possibly leaving some unconsumed data in the last IOJob, which is not part of this protocol-specific receive operation (it's the start of the next one).
This is a rather elaborate use of object inheritance, and shows the true power and flexibility of ObjAsm32, and why OOP can be a Good Thing when used appropriately.
I think the changes I've made cover all possible situations except one.
What about protocols which are only delimited by session expiry?
If a session expires, and we have received data that is pending a delimiter, how can we tell whether the session expiry was the delimiter we sought, or whether the connection was lost prematurely?
It may be necessary to add a flag to NetworkProtocol to alter the behaviour for these "no protocol" transmissions of data (such as raw file transfers or other delimiterless streaming).
In these cases, there is no way we can know that the data sent is complete without sending some kind of preamble that includes the expected size of the complete transfer.. if we don't wish to do that, we'll just have to assume that the data is complete and intact, as there is no protocol in place.
I've written some simple demonstration code consisting of two small applications.
One is a client, one is a server, and they implement an "echo" protocol.
You can use one or more instances of the client to send data to the server.
The server responds by simply sending the data back to the client who sent it.
Not very complex, and not a 'true' network protocol implementation, but enough to prove that NetEngine has no huge bugs, which is nice because there's been a lot of small changes recently.
I guess NetEngine goes public very soon :)
One is a client, one is a server, and they implement an "echo" protocol.
You can use one or more instances of the client to send data to the server.
The server responds by simply sending the data back to the client who sent it.
Not very complex, and not a 'true' network protocol implementation, but enough to prove that NetEngine has no huge bugs, which is nice because there's been a lot of small changes recently.
I guess NetEngine goes public very soon :)
Now I'm working on integrating "transparent" support of automatic port-forwarding using UPNP to control NAT devices. This also means implementing and testing of Clients that use UDP, and protocols which are "connectionless". But for the average user, it will mean they can "run a server", even if they are a LAN client that is behind a router or using ICS or other NAT device that would normally prevent them from being able to accept incoming connections, and it will be handled internally, so there's no extra load placed on the programmer.
Server class was renamed to NetEngine (this avoids confusion when writing client oriented apps).
UDP_Announce was added to the NetEngine class - this method creates a UDP client of given protocol, queues a receive, and sends some initial data - the rest is up to the protocol class.
Server class was renamed to NetEngine (this avoids confusion when writing client oriented apps).
UDP_Announce was added to the NetEngine class - this method creates a UDP client of given protocol, queues a receive, and sends some initial data - the rest is up to the protocol class.
I'm still working on the 'upnpnat' support code.
UPNPNATProtocol is an interesting derived class, because it requires the use of a UDP broadcast followed by TCP stuff, and I've handled it all with a single protocol handler (well, its all part of the same network protocol, but it requires more than one session using more than one carrier protocol).
So far I can find the gateway device (using udp), and query it for more information (using tcp).
I am stuck there because now I need to deal with XML parsing, but since I'm not interested in enumerating the port forwardings or querying them specifically but ONLY in attempting to create a 'new' port forwarding, the job becomes somewhat easier.
I expect to alter NetEngine to use this code to detect the external ip address (that of the gateway machine), and set an internal flag, so that if we try to create a Listening Port (ie a server application), and the local machine is NOT directly connected to the internet, then a suitable port forwarding will be automatically created at that moment - and it would be nice if the port forwarding was automatically removed too, but one step at a time :P
For the average Joe, it means that (assuming that upnp is enabled, the upnp ports allowed past the firewall, and the NAT device is upnp-compatible), anyone can run a Server (accept incoming connections), even if they are sitting in a net cafe with 30 other lan users all sharing a single external ip address.
Obviously this won't always be possible... but in many (most?) cases it will be a blessing, since we don't always have access to the cpanel of the router (or other nat device) but we may have access to upnp (and who would want to manually configure their router if it can be done in code?)
UPNPNATProtocol is an interesting derived class, because it requires the use of a UDP broadcast followed by TCP stuff, and I've handled it all with a single protocol handler (well, its all part of the same network protocol, but it requires more than one session using more than one carrier protocol).
So far I can find the gateway device (using udp), and query it for more information (using tcp).
I am stuck there because now I need to deal with XML parsing, but since I'm not interested in enumerating the port forwardings or querying them specifically but ONLY in attempting to create a 'new' port forwarding, the job becomes somewhat easier.
I expect to alter NetEngine to use this code to detect the external ip address (that of the gateway machine), and set an internal flag, so that if we try to create a Listening Port (ie a server application), and the local machine is NOT directly connected to the internet, then a suitable port forwarding will be automatically created at that moment - and it would be nice if the port forwarding was automatically removed too, but one step at a time :P
For the average Joe, it means that (assuming that upnp is enabled, the upnp ports allowed past the firewall, and the NAT device is upnp-compatible), anyone can run a Server (accept incoming connections), even if they are sitting in a net cafe with 30 other lan users all sharing a single external ip address.
Obviously this won't always be possible... but in many (most?) cases it will be a blessing, since we don't always have access to the cpanel of the router (or other nat device) but we may have access to upnp (and who would want to manually configure their router if it can be done in code?)
Hehe now your server/client framework sounds more interesting... ;)
I've decided to start posting code today.
This code will be subject to change without notice, so consider it as a 'sneak preview' - in future, the latest version of OA32 will always contain the most recent PUBLIC version of the code.
Today we'll take a walk around inside the main object, whose name is NetEngine.
This component is the hub of the engine - it acts as a 'front end' for creating new networking connections, and it is the heart of the resource management.
Let's see that code, then talk about it.
(code attached due to character limit)
It's a lot to take in, so take your time.
NetEngine implements several 'object pools', where Pool is a subclass that supports fast recycling of previously created objects - rather than simply release an object when we are done with it, we toss it into a recycling pool, and when we need a new one, we check if theres any languishing in the pool, preferring to reinitialize a dead object than allocate a new one... this is much quicker, and reduces fragmentation of heap memory (where these guys all live).
Disregarding the structures and equates for the time being, the obvious place to start is the Constructor method, whose name is 'Init'.
This method takes three parameters, which are mainly aimed at server applications.
They are MinClients, MaxClients, and MaxIOJobs.
The Init method creates a half dozen objects which are used to manage various resources, such as pooling objects for IOJobs, inbound clients, outbound clients, and listening clients.
Its also responsible for kickstarting WinSock.
MinClients declares the minimum number of 'pending accepts' for server apps.
MaxClients sets the upper limit of the ClientPools.
MaxIOJobs sets the upper limit of the IOJobPool.
The Init method will check how many 'logical cpu devices' your hardware supports, and create one Worker Thread for each logical cpu device.
I'll be talking more about the Worker Threads later - they're the grunt.
The Init method also creates an object called ClientGroup.
This object gives us a way to organize clients into named groups, which exist in a tree structure.
You can think of a ClientGroup as being a 'communication channel', or if you like, a 'chatroom'.
NetEngine does not care how you use client groups, or what they are for - its just a way to issue data to a bunch of clients at once.
The Destructor method, whose name is 'Done', simply cleans up all these internal resources and shuts down WinSock.
In order to use NetEngine, whether as a client, a server, or both, you must first initialize it via the Init method.
Having done that, you have three major functions that you can call.
Their names are ConnectTo, Listen, and UDP_Broadcast.
All three of these methods take a pointer to a user-supplied 'Protocol Handler'.
This is an object YOU write, which implements the rules of your network protocol, and is derived from the vanilla protocol handler 'NetworkProtocol'.
Each of these three methods, assuming it succeeded, returns a pointer to a Client object.Your application can manage these, and/or add them to a new or existing ClientGroup.
Its important to realize that the Client object is mostly a wrapper for a socket handle, and typically represents a udp or tcp session... however, it can also represent a 'listener', which is a socket that listens for inbound connections.
If you call the Listen method, NetEngine will create a new listening socket bound to a given port, and it will then issue a number of 'pending accepts'.
We use this to make a Server listen on a given port, using a given protocol handler - we can have as many listening ports as we like, but each is associated with just one protocol handler (not necessarily unique - handlers can be shared).
If you call the ConnectTo method, NetEngine will issue a connect attempt to a given remote host ip and port, using a given protocol handler.
And if you call UDP_Broadcast, a udp client is created, and given data is sent to the given broadcast ip / port.
When a network event completes (whether successful or not), one of our Worker Threads will wake up and forward that notification , via a specific Client object, to that Client's protocol handler.
It's up to you what to do with that notification, probably you will want to inform the Application, perhaps via a Windows Message, that something just happened.
The base NetworkProtocol class is fairly well commented, and I'll be posting examples as I build them.
So far we have a working 'echo server' and 'echo client' protocol handlers, and test applications that use them.
Please note that I have previously used this framework to implement a full chatserver with lots of bells and whistles, so I'm not working with code that 'might' work - it works well.
So from the programmer's perspective, in order to support a new protocol, you need only to write a new protocol handler class, which derives from NetworkProtocol.
In the next post, we'll start looking at some of the resource classes, and how they all fit together.
And in particular, we'll be looking at IOJob and the Pooling system.
This code will be subject to change without notice, so consider it as a 'sneak preview' - in future, the latest version of OA32 will always contain the most recent PUBLIC version of the code.
Today we'll take a walk around inside the main object, whose name is NetEngine.
This component is the hub of the engine - it acts as a 'front end' for creating new networking connections, and it is the heart of the resource management.
Let's see that code, then talk about it.
(code attached due to character limit)
It's a lot to take in, so take your time.
NetEngine implements several 'object pools', where Pool is a subclass that supports fast recycling of previously created objects - rather than simply release an object when we are done with it, we toss it into a recycling pool, and when we need a new one, we check if theres any languishing in the pool, preferring to reinitialize a dead object than allocate a new one... this is much quicker, and reduces fragmentation of heap memory (where these guys all live).
Disregarding the structures and equates for the time being, the obvious place to start is the Constructor method, whose name is 'Init'.
This method takes three parameters, which are mainly aimed at server applications.
They are MinClients, MaxClients, and MaxIOJobs.
The Init method creates a half dozen objects which are used to manage various resources, such as pooling objects for IOJobs, inbound clients, outbound clients, and listening clients.
Its also responsible for kickstarting WinSock.
MinClients declares the minimum number of 'pending accepts' for server apps.
MaxClients sets the upper limit of the ClientPools.
MaxIOJobs sets the upper limit of the IOJobPool.
The Init method will check how many 'logical cpu devices' your hardware supports, and create one Worker Thread for each logical cpu device.
I'll be talking more about the Worker Threads later - they're the grunt.
The Init method also creates an object called ClientGroup.
This object gives us a way to organize clients into named groups, which exist in a tree structure.
You can think of a ClientGroup as being a 'communication channel', or if you like, a 'chatroom'.
NetEngine does not care how you use client groups, or what they are for - its just a way to issue data to a bunch of clients at once.
The Destructor method, whose name is 'Done', simply cleans up all these internal resources and shuts down WinSock.
In order to use NetEngine, whether as a client, a server, or both, you must first initialize it via the Init method.
Having done that, you have three major functions that you can call.
Their names are ConnectTo, Listen, and UDP_Broadcast.
All three of these methods take a pointer to a user-supplied 'Protocol Handler'.
This is an object YOU write, which implements the rules of your network protocol, and is derived from the vanilla protocol handler 'NetworkProtocol'.
Each of these three methods, assuming it succeeded, returns a pointer to a Client object.Your application can manage these, and/or add them to a new or existing ClientGroup.
Its important to realize that the Client object is mostly a wrapper for a socket handle, and typically represents a udp or tcp session... however, it can also represent a 'listener', which is a socket that listens for inbound connections.
If you call the Listen method, NetEngine will create a new listening socket bound to a given port, and it will then issue a number of 'pending accepts'.
We use this to make a Server listen on a given port, using a given protocol handler - we can have as many listening ports as we like, but each is associated with just one protocol handler (not necessarily unique - handlers can be shared).
If you call the ConnectTo method, NetEngine will issue a connect attempt to a given remote host ip and port, using a given protocol handler.
And if you call UDP_Broadcast, a udp client is created, and given data is sent to the given broadcast ip / port.
When a network event completes (whether successful or not), one of our Worker Threads will wake up and forward that notification , via a specific Client object, to that Client's protocol handler.
It's up to you what to do with that notification, probably you will want to inform the Application, perhaps via a Windows Message, that something just happened.
The base NetworkProtocol class is fairly well commented, and I'll be posting examples as I build them.
So far we have a working 'echo server' and 'echo client' protocol handlers, and test applications that use them.
Please note that I have previously used this framework to implement a full chatserver with lots of bells and whistles, so I'm not working with code that 'might' work - it works well.
So from the programmer's perspective, in order to support a new protocol, you need only to write a new protocol handler class, which derives from NetworkProtocol.
In the next post, we'll start looking at some of the resource classes, and how they all fit together.
And in particular, we'll be looking at IOJob and the Pooling system.
Before I go ahead and post more code, I just wanted make a brief progress report regarding the implementation of 'port forwarding / nat traversal via upnp'.
I just spent some days trying to translate an obscure XML parser.
Then it occurred to me that I don't need to 'correctly' parse the xml, just be able to parse the stuff I really need.
So I fired up my packetsniffer, and captured all the packets involved, and analyzed them to determine what the heck was actually happening in there.
Then I wrote cheesy parsing code to rip out the stuff that interests me.
I am now able to:
1 - obtain the ip and port for the upnp device controller (this is udp, the rest is tcp)
2 - query the upnp controller for more information, including the url for requesting the EXTERNAL ip address
3 - query that url to obtain the external ip address
Now I think I have everything I need to actually create new port-forwardings (port mappings).
Yes, there are other ways to obtain the external ip address, but those won't help us get the values we need to send commands to the upnp controller, so I followed the upnp way of doing everything.
I'm a few steps closer to implementing transparent forwarding of listening ports :)
The questions on my mind now : should NetEngine.Listen take as params both the internal AND external ports, or should they always be the same? What happens if an external port is already mapped, and we try to map it to 'another' internal port? Does this cause packet duplication? etc.
I just spent some days trying to translate an obscure XML parser.
Then it occurred to me that I don't need to 'correctly' parse the xml, just be able to parse the stuff I really need.
So I fired up my packetsniffer, and captured all the packets involved, and analyzed them to determine what the heck was actually happening in there.
Then I wrote cheesy parsing code to rip out the stuff that interests me.
I am now able to:
1 - obtain the ip and port for the upnp device controller (this is udp, the rest is tcp)
2 - query the upnp controller for more information, including the url for requesting the EXTERNAL ip address
3 - query that url to obtain the external ip address
Now I think I have everything I need to actually create new port-forwardings (port mappings).
Yes, there are other ways to obtain the external ip address, but those won't help us get the values we need to send commands to the upnp controller, so I followed the upnp way of doing everything.
I'm a few steps closer to implementing transparent forwarding of listening ports :)
The questions on my mind now : should NetEngine.Listen take as params both the internal AND external ports, or should they always be the same? What happens if an external port is already mapped, and we try to map it to 'another' internal port? Does this cause packet duplication? etc.
I consider NetEngine to be the 'heart' of the engine, which implies that there is stuff 'below' it, and 'above' it (ie, if NetEngine is the Heart, then there's probably a Brain above it, and perhaps some Legs and some Feet below it). I'll now go on to describe the engine from the bottom up - we can't understand the highest level objects without knowing something about the lowest level objects... suffice to say that low level objects represents resources and resource managers, and high level objects implement the logic that drives everything.
You've seen the NetEngine object, and I've stated that one of its main functions is to manage resources via some lower-level resource management objects.
The other main function of NetEngine is to marshal event notifications to the highest level of the engine (protocol handlers), along with pointers to the resource objects associated with these events.
As mentioned previously, NetEngine uses several 'pools' to recycle objects.
Attached is the Pool class.
This is a baseclass for managing objects that we wish to recycle.
It does not know what kind of objects, and it does not care.
The Pool class simply provides methods to create new recyclable objects, cache 'redundant' objects, and recycle these objects when possible.
Deriving from (built on top of) the Pool class are two classes called IOJobPool and ClientPool.
These are a little more specialized in that they DO know what kind of objects they are managing, and know how to initialize them.
I've attached them as well.
Take a look over these objects, and look at what is going on in there, since they're mentioned a lot in the NetEngine object.
I won't say anything further about these three objects unless you guys ask me to.
They're just resource management.
The really interesting stuff happens in the remaining classes.
Next time I post, we'll be looking closely at the IOJob object.
You've seen the NetEngine object, and I've stated that one of its main functions is to manage resources via some lower-level resource management objects.
The other main function of NetEngine is to marshal event notifications to the highest level of the engine (protocol handlers), along with pointers to the resource objects associated with these events.
As mentioned previously, NetEngine uses several 'pools' to recycle objects.
Attached is the Pool class.
This is a baseclass for managing objects that we wish to recycle.
It does not know what kind of objects, and it does not care.
The Pool class simply provides methods to create new recyclable objects, cache 'redundant' objects, and recycle these objects when possible.
Deriving from (built on top of) the Pool class are two classes called IOJobPool and ClientPool.
These are a little more specialized in that they DO know what kind of objects they are managing, and know how to initialize them.
I've attached them as well.
Take a look over these objects, and look at what is going on in there, since they're mentioned a lot in the NetEngine object.
I won't say anything further about these three objects unless you guys ask me to.
They're just resource management.
The really interesting stuff happens in the remaining classes.
Next time I post, we'll be looking closely at the IOJob object.