Thomas - That routine was copied directly from my own code, so if there are any bugs I guess I'm to blame :)
It works just fine for me though, have you tried turning on Local echo (in case you use the Win9x telnet client)? On winXP this setting is enabled by default.
I must say I agree I didn't code this routine very nicely, and I plan on rewriting it soon. I'll be sure to notify you, Sliver :)
Posted on 2002-04-16 11:38:06 by Qweerdy
I guess it depends on the OS, I'm running win2k SP2 which has local echo off by default, but that shouldn't matter. The problem is that the peek will never give me more than 1 char.. This might not happen on other OSes (and therefore work) but it is valid behaviour, you can't rely on the internal buffers to be filled with a whole string, winsock can choose any buffer size. I think it also has to do with the relatively slow transfer rate from telnet as you're typing the characters..

I'll see if I can find some sample code from my webserver if I have time.

Thomas
Posted on 2002-04-16 12:21:42 by Thomas
Here's part of my HTTP server code that handles the received data buffer. It's a bit lengthy but a lot of stuff is specificly for my own code so you can leave that out. The code isn't optimized yet either.

The ReadDataToBuffer method (it's a class) receives data and adds it to it's internal buffer (.InputBuffer). GetNextBufferLine checks if a full line is found, if it is, it's copied to an output buffer. After you processed that line, it can be deleted from the buffer with RemoveFirstBufferLine.
The pseudo code:


while(connected)
{
wait for incomming data
ReadDataToBuffer
while (GetNextBufferLine finds a line)
{
process line
RemoveFirstBufferLine
}
}




; returns 1 on success (which doesn't have to mean that data has been added to
; the buffer(!).
; returns 0 on failure (dwLastError is set)
CST_ReadDataToBuffer proc uses edi lpTHIS:DWORD
SetObject edi, CServerThread

; Calculate unused buffer part pointer & size
mov eax, ST_INPUT_BUFFER_SIZE
lea ecx, [edi].InputBuffer
sub eax, [edi].InBufferUsed ;nr of unused bytes
add ecx, [edi].InBufferUsed ;pointer to first unused byte

; Check if all bytes are already in use:
.IF eax==0 ;no bytes left?
; If no buffer space is available, disable further notification of
; arriving data. No data is received now. The notification should
; be re-enabled when buffer space has been freed.
and [edi].CurNetEventMask, NOT FD_READ
invoke WSAEventSelect, [edi].hClientSocket, [edi].hNetworkEvent,
[edi].CurNetEventMask
int 3
; Check if errors occurred and quit
.IF eax==SOCKET_ERROR
mov [edi].dwLastError, ST_ERR_INTERNALFUNCTIONFAILED
xor eax, eax
.ELSE
xor eax, eax
inc eax
.ENDIF
ret
.ENDIF

; Receive data and append to buffer:
invoke recv, [edi].hClientSocket, ecx, eax, NULL

; Process return value and quit:
xor ecx, ecx
.IF eax==0
mov [edi].dwLastError, ST_ERR_CONNECTION_CLOSED
.ELSEIF eax==SOCKET_ERROR
invoke WSAGetLastError
.IF eax!=WSAEWOULDBLOCK
mov [edi].dwLastError, ST_ERR_INTERNALFUNCTIONFAILED
.ELSE
inc ecx
.ENDIF
.ELSE
; add number of bytes received (eax) to buffer use counter:
add [edi].InBufferUsed, eax
inc ecx
.ENDIF

mov eax, ecx
ReleaseObject edi
ret
CST_ReadDataToBuffer endp

;------------------------------------------------------------------------------------------

; returns -1 when no lines currently available
; returns -2 on error
; otherwise, the return value is the number of characters copied to the output buffer
; Note that this value may be zero, indicating an empty line
; The output buffer has to have at least the size of the data buffer!!!
; lpLnSrc points to a dword that will hold the size of the line in the original buffer..
; This value may be different from the returnvalue, as LWS are replaced by single spaces
; in the output buffer. Use this value for RemoveFirstBufferLine
CST_GetNextBufferLine proc uses edi esi ebx lpTHIS:DWORD, lpOutBuffer:DWORD, lpLnSrc:DWORD
LOCAL prevChar:BYTE
SetObject edi, CServerThread

; Calculate unused buffer part pointer & size
lea ecx, [edi].InputBuffer
mov edx, ecx
add edx, [edi].InBufferUsed

; Check for empty buffer:
.IF ecx==edx
or eax, -1
ret
.ENDIF

; Check if empty line (CR, LF or CRLF)
mov al, [ecx]

;!!!!!!!!! yet to do: check for other invalid chars
.IF al==0
int 3
;INVALID REQUEST: INVALID REQUEST!!
or eax, -1
dec eax
ret
.ENDIF

.IF al==0Dh ;CR?
.IF [edi].usedEOL==EOL_CR
mov eax, lpOutBuffer
mov byte ptr [eax],0
xor eax, eax
ret
.ELSEIF [edi].usedEOL==EOL_CRLF
inc ecx
.IF ecx==edx ;LF not yet arrived so wait for it (=no lines yet)
or eax, -1
dec eax ;-2
ret
.ELSE
.IF byte ptr [ecx]==0Ah ;LF in CRLF?
mov eax, lpOutBuffer
mov byte ptr [eax],0
xor eax, eax
ret
.ELSE
int 3
;INVALID REQUEST.. CRLF expected but LF is something else!!!
.ENDIF
.ENDIF
.ELSEIF [edi].usedEOL==EOL_LF
int 3
;INVALID REQUEST.. CR used where LF expected!
.ENDIF
.ELSEIF al==0Ah ;LF?
mov eax, lpOutBuffer
mov byte ptr [eax],0
xor eax, eax
ret
.ENDIF


lea ecx, [edi].InputBuffer
and prevChar, 0

;Find line:
.WHILE ecx < edx
mov al, [ecx]
cmp al, " "
je @gnbl001
cmp al, 09h ;tab
je @gnbl001

;if no whitespace:
.IF prevChar==0Dh ;CR?
cmp al, 0Ah ;part of CRLF?
je @gnbl001

mov [edi].usedEOL, EOL_CR
jmp @foundline
.ELSEIF prevChar==0Ah ;LF?
.IF [edi].usedEOL==EOL_CRLF
; note: [ecx-2] is always a valid pointer, as
; [ecx-1] (=prevChar) is LF, and LF can never be
; the first character (this is handled in the empty
; line check at the start of this proc). So there
; HAS to be another character before the LF, at
; [ecx-2].
.IF byte ptr [ecx-2]!=0Dh ; CR in CRLF?
mov [edi].usedEOL, EOL_LF
.ENDIF
jmp @foundline
.ELSEIF [edi].usedEOL==EOL_LF
jmp @foundline
.ELSEIF [edi].usedEOL==EOL_CR
int 3
; INVALID REQUEST.. LF used where CR expected!!!
.ENDIF
.ENDIF
@gnbl001:
mov prevChar, al
inc ecx
.ENDW

;No lines found, check if full buffer used:
or eax, -1
.IF [edi].InBufferUsed==ST_INPUT_BUFFER_SIZE
;BUFFER OVERFLOW (LINE TOO LONG)!!!!!!!!!!!!!
int 3
dec eax
.ENDIF
ret

@foundline:
;here: ecx points to first character after the last character in the
; found line.
;store source length first:
mov edx, lpLnSrc
lea eax, [edi].InputBuffer
neg eax
add eax, ecx
mov [edx], eax

lea edx, [edi].InputBuffer
mov ebx, lpOutBuffer

and prevChar, 0
.WHILE edx<ecx
mov al, [edx]
.IF al==" " || al==09h ;space or tab?
.IF prevChar==0Dh || prevChar==0Ah
mov byte ptr [ebx], " "
inc ebx
;skip all other whitespace at start of line:
.WHILE al==" " || al==09h
; Note: this loop does not need to check if
; edx goes beyond ecx, as the last character of that string
; can never be a space or a tab (only a CR or LF).
inc edx
mov prevChar, al
mov al, [edx]
.ENDW
dec edx
.ELSE
mov byte ptr [ebx], al
inc ebx
.ENDIF
.ELSEIF al!=0Ah && al!=0Dh
mov byte ptr [ebx], al
inc ebx
.ENDIF

inc edx
mov prevChar, al
.ENDW
mov byte ptr [ebx], 0 ;final null terminator
sub ebx, lpOutBuffer
mov eax, ebx
ReleaseObject edi
ret
CST_GetNextBufferLine endp

; returns 0 on failure (dwLastError is set)
; returns 1 on success
CST_RemoveFirstBufferLine proc uses edi esi ebx lpTHIS:DWORD, lnSrc:DWORD
LOCAL prevChar:BYTE
SetObject edi, CServerThread

; Calculate unused buffer part pointer & size
lea ecx, [edi].InputBuffer
mov edx, ecx
add edx, [edi].InBufferUsed

mov ebx, ecx
add ebx, lnSrc


.WHILE ebx<edx
mov al, [ebx]
mov [ecx], al

inc ecx
inc ebx
.ENDW

mov eax, lnSrc
sub [edi].InBufferUsed, eax

xor eax,eax
inc eax

ReleaseObject edi
ret
CST_RemoveFirstBufferLine endp


Thomas
Posted on 2002-04-16 12:31:03 by Thomas
Thomas, what I don't get is this: since I use MSG_PEEK, I get all the data in Winsock's buffer that is available at that moment, right? And that data does not get removed.
In that case, if the data didn't contain a CrLf, I can call recv again, and it should give me the entire buffer again, including the old data I already parsed and the new data that just arrived. At least it works this way on XP, and I always understood it would work this way from MSDN.

I guess your code is more or less what I was aiming at, the code in my example program (telnet server) is just a quick hack to get the network interface working. Can I download the full source somewhere (for this "class", OOP I presume :) )?
Posted on 2002-04-16 13:06:38 by Qweerdy
Thomas, what I don't get is this: since I use MSG_PEEK, I get all the data in Winsock's buffer that is available at that moment, right?

Yes
And that data does not get removed.
In that case, if the data didn't contain a CrLf, I can call recv again, and it should give me the entire buffer again, including the old data I already parsed and the new data that just arrived.

Yes, but what if winsock decides the buffer is full and waits until your program actually received it before adding more data to it's internal buffers? I think this is what happens..
When I trace the code, and 6 chars are in the buffer, it ends up in an endless loop with the peek, and each time the peek only receives one char. When I manually break the loop, and the second recv is called, it receives all characters!


At least it works this way on XP, and I always understood it would work this way from MSDN.

As I said, it probably depends on the OS.


I guess your code is more or less what I was aiming at, the code in my example program (telnet server) is just a quick hack to get the network interface working. Can I download the full source somewhere (for this "class", OOP I presume )?


Sorry, it's for one of my own projects and it relies on many other classes. But you can take out the parts you need, everything is there.

Thomas
Posted on 2002-04-16 13:28:29 by Thomas
Okay, thanks anyway. But in that case I think I'll continue working on my own solution, since it's starting to look better and better. I'd say it was sort-of OOP, since it creates a separate struct for each connection (socket) opened, and requires you to pass a ptr to this struct to all the other procs.

PS: Now that I actually studied your code more closely, I've noticed that you're doing the reading-into-buffer part almost the same way I am (in the new version), only I'm storing the amount of data that's in the buffer, instead of what's left. It may make more sense for me to do it your way though...
I also noticed you're using WSAEventSelect... really nice functions, I agree. But not available on winsock 1.1, and I was trying to make my app compatible with that too. Besides, with a telnet client, performance may not be such a big issue as with your project.
Posted on 2002-04-17 00:40:58 by Qweerdy
Originally posted by Qweerdy
Okay, thanks anyway. But in that case I think I'll continue working on my own solution, since it's starting to look better and better. I'd say it was sort-of OOP, since it creates a separate struct for each connection (socket) opened, and requires you to pass a ptr to this struct to all the other procs.


My code uses several objects (Request, Response, ServerThread) for each connection, but of course that differs for the type of server.


PS: Now that I actually studied your code more closely, I've noticed that you're doing the reading-into-buffer part almost the same way I am (in the new version), only I'm storing the amount of data that's in the buffer, instead of what's left. It may make more sense for me to do it your way though...


I'm doing the same, look at the lnBufferUsed member of the class. I only calculate what's left because you need this to add the new data. You don't want the buffer to overflow because too much data is added.


I also noticed you're using WSAEventSelect... really nice functions, I agree. But not available on winsock 1.1, and I was trying to make my app compatible with that too. Besides, with a telnet client, performance may not be such a big issue as with your project.


Well I consider it safe to use winsock 2. The only OS that doesn't support it is win95, but there's even a patch for that. I don't know about NT, NT 4 probably has it, 2k+ for sure.

Thomas
Posted on 2002-04-17 01:46:44 by Thomas