hmmm... two problems rolled into one!

hello... i'm working on my HTTP server and i read that message from bitRAKE, i think, about buf. overflow... anyway, my program creates a new thread to handle each incoming connection.

each thread allocates it's own 64K space using the heap... unfortunately, that means that 1024 simul. connections will require 64MBs of RAM... but most HTTP requests only require like 1K at most. is there some way i can quickly allocate RAM /wo it bloating and not have to resize the memory block every time a new packet is recieved?

the second problem is the one related to buffer overflow. i'm using the recv function to retrieve data from the data stream like this:
invoke recv,hSocket,pBuffer,bufferSize,0
is there some function that peeks at the size of the data coming in so that bufferSize can be setup first? or is there some way of reading only one byte at a time from the stream? if i put '1' as the second last parameter, my program doesn't work!

thanks in advance for any assistance.

btw, you don't need to post any source. i'm fine with the name of some functions to look up.
Posted on 2002-04-23 21:46:40 by jademtech
hi jademtech

To find out how much data is waiting use the ioctlsocket function with FIONREAD command.
Posted on 2002-04-24 04:07:27 by Kudos
First, remember that you don't always receive all the data you request,
there are multiple reasons why you can receive less (closed socket,
not enough data available, etc). Second, 64k for a buffer seems a
tad much for a http server recv buffer - I'd probably just use 1k or
so on the stack. There's no reason to recv a byte at a time, keep
reading to buffer, and specify buflen-index as bytes2read.
Stop reading when you get a newline char or the buffer is full. Iirc
http requests are multiline text-only stuff, so each time you get a
LF you can parse some information and store that in other buffers/variables -
then you should move the remaining (if any) parts of the buffer down
and keep reading from the new index. At least this should be overflow
safe, I'm sure Thomas has better ideas or can explain better :).
Posted on 2002-04-24 08:29:42 by f0dder
It depends on how you handle the data. You probably won't need the full request all the time. Just receive what you get and free the memory it uses as soon as you can. For example, the 'host' header: after you verified that the hostname is correct, there's no need to store that header anymore.
Your webserver probably uses a thread for every connection, every thread has it's own stack which by default has the same size as the main thread's stack. You can specify a custom size with CreateThread to limit the amount of memory each thread uses.
For the receiving buffer, I usually set up a buffer of the maximum line length, and let the received data sort of flow through it. As soon as a full line is in the buffer, it is processed and removed.
Have a look at my posts in this thread:

The code comes straight from my own webserver's code, so there's some additional code to handle request lines that span multiple physical lines (ie header lines starting with whitespace are part of the previous line).

Posted on 2002-04-24 14:57:21 by Thomas
thanks :) (btw, i actually solved my first problem, but then this forum was being cleaned up, i guess, so i couldn't send a message saying so)... and i've already created my own struct to contain the header stuff outside the buffer (you may remember my other question about addr of a struct...)

so, thanks again :)
Posted on 2002-04-24 17:15:42 by jademtech