Hello,

My first asm program (an email client) is progressing well, but I am a little confused over the behaviour of the recv function.

Most of the time, the program will happily download each email header in turn and display various information in a List-View. However, sometimes the recv function does not download all of the header. This seems to occur when the header size is somewhere around 1KB.

The strange thing is, is that if I incorporate a delay (mov eax, 170000000 .while eax!=0 dec eax .endw) between receiving the initial "+OK" and receiving the header, it all works ok.

Fodder mentioned once that "you don't always receive all the data you request, there are multiple reasons why you can receive less (closed socket,not enough data available, etc).".

Microsoft say "calling recv will return as much information as is currently available".

So really, my questions are:

Is this behaviour normal?

Fodder and Microsoft talk about data availability. Why would all the data not be available when the header is over about 1KB?

Is there a simple way to find out if ALL the information requested (i.e. ALL the header and not just part of it) has been received?
Posted on 2002-09-25 16:15:42 by GreenScreenAlternator
It's simply about being as efficient as possible, and since TCP/IP is packet based you will always receive data in chunks. There's no need for winsock to wait for the next packet before returning data to you, since it's not even sure you need that extra data.
The standard way to receive a fixed amount of bytes is something like this:



invoke GlobalAlloc,GMEM_FIXED,AmountOfBytes
mov edi,eax
mov ebx,AmountOfBytes
RecvLoop:
invoke recv,hSock,edi,ebx,0
.if eax==0
;Error
.endif
add edi,eax
sub ebx,eax
jnz RecvLoop
Posted on 2002-09-26 00:41:42 by Qweerdy
Qweerdy,

Thanks for your reply.

Looks like I'll need to make a few modifications to my code!
Posted on 2002-09-26 17:18:36 by GreenScreenAlternator
If you're looking for a easy way to get data one line at a time, you can take a look at my CSOCKET code (click my www button below). Additionally, it supports the MSG_COMPLETE flag for recv, which makes sure it downloads the exact amount of data you asked for (using the code I just posted). ASM source included of course.

It uses the MASM32 OOP model, but it shouldn't be such a pain to figure out even if you don't know anything about OOP.
Posted on 2002-09-27 05:30:01 by Qweerdy
Thanks, but I don't need to read in data line by line at the moment (well, I don't think I do!).

I now have another problem with recv and your code (or more likely, my misunderstanding of your code).

When there is no longer any more data available(i.e. after a couple of recv requests), recv will just sit there forever and wait (something to do with using blocking mode?).

I have temporarily solved this problem by using the ioctlsocket function to find out how much data is available but if I use this immediatly after a "Send" it returns zero. To correct the problem, I need to wait 3/10 ths of a second after "Send" and then use "ioctlsocket".

This can't be the way to go, surely?
Posted on 2002-09-27 08:01:49 by GreenScreenAlternator
My code just downloads a fixed amount of bytes, it doesn't know or care about the amount available at the time. To fix this, set the socket to non-blocking mode (I believe there's a ioctlsocket setting for that; also the WSAAsyncSelect API does this). Then recv will return WSAE_WOULDBLOCK (IIRC) if there's no data available.
Posted on 2002-09-27 08:52:50 by Qweerdy
Hi,

Thanks Qweerdy, may use that later on...

After reading the RFCs again, found out that an email header always ends with "CRLF.CRLF". So, I can now just keep invoking recv until the last 5 bytes are as mentioned, without any risk of data not being available.

Seemed the easiest way to check for the header terminating characters was to use cmpsd and just check for "LF.CRLF".

All working ok now.

Cheers
Posted on 2002-10-03 11:19:07 by GreenScreenAlternator
Hi GreenScreenAlternator, I code email client too, and it's my first 'real' program too! :)
I had similar problems like you had before, I don't know why they happened, anyway
now everything works great:

I use non-blocking sockets, and while receiving the heads just count the periods.
If I request 5 email-headers, and make recv put them all into the same buffer, and for each
finnished recv I check for LF'.'CRLF, when the fifth is found, then I know they all have arrived.

But you don't need to use cmpsb I think, I just check 4 bytes back when recv is done,
like if edi points to the receiving buffer, after a recv:



cmp dword ptr [edi-4], 0A0D2E0Ah
jnz _goOn
dec dwHeadersLeftToReceive
jnz _goOn
; Ok all headers received
Posted on 2002-10-26 19:58:42 by david
Originally posted by david


cmp dword ptr [edi-4], 0A0D2E0Ah
jnz _goOn
dec dwHeadersLeftToReceive
jnz _goOn
; Ok all headers received

Assuming that edi points to the end of your received data, you should add a check if the buffer size is at least 4 bytes as otherwise you will read the memory before the buffer.

Thomas
Posted on 2002-10-27 01:35:45 by Thomas
Thanks Thomas, I didn't think of that. :)
Posted on 2002-10-27 01:49:37 by david
Recently I had weird error but solved it now.

When going to retrieve a list of mail-headers I sent the top commands in a chunck like this string:

'top 1 0'CRLF
'top 2 0'CRLF
'top 3 0'CRLF
etc

All in one send,

Then let recv collect them one by one...
Just that it didnt work as I expected, I got strange errors, sometimes only a couple of headers turned up.
I found that sometimes recv got two headers in the same receive!!

So I changed the method to
send 'top 1 0'
wait for recv and scan it for '.'CRLF
send 'top 2 0'
wait for recv and scan it for '.'CRLF
etc..till all done

and then it worked.
Posted on 2002-10-29 23:07:51 by david
Like qweerdy said, TCP divides your data flow in chunks in order to optimize the bandwidth. *Never* rely on the division winsock makes for the different recv calls. They may (often) have a relation to the protocol (for example, exactly one response line in one recv call), but you can't be sure. Just buffer all the input and parse it properly, and most 'weird' problems will disappear.

Thomas
Posted on 2002-10-30 13:15:58 by Thomas
Thanks for those wise words...it seems it's not possibly to get away with 'dirty hacks' on this one :)
I guess there's some more weeks under the midnight lamp for me...
Posted on 2002-10-30 15:17:09 by david
Is it safe sending a long list of commands in a chunk to server?
like

'recv 1
recv 2
recv 3
recv xx'

instead of sending one command and wait for response, like you would do with telnet?

I noticed the mails or mailheads got sent much faster when I sent list of commands at once, although I didnt make it work right.
I would like it to work if it's safe to do this way.
Would there be problems for server if there was say many new mails on server say, 50 mails, and I sent
a very long list with recv commands for all mails all at once, would all mails be retrieved 100%
Posted on 2002-10-31 00:30:57 by david
I don't think the specifications allow this, you'll have to check the pop3 RFC. It will probably work though. The HTTP specifications do mention the pipelining of requests.

Thomas
Posted on 2002-10-31 01:06:06 by Thomas