I have very little experience with writing programs to talk on the web.  I want to write a program that will connect to a site periodically, to see if there has been any changes.  I found a program at http://spiff.tripnet.se/~iczelion/files/http15.zip that connects and downloads the html for the site, so I thought this would be a good starting point.  It works perfectly for nearly all sites I have tested except the particular site I'm interested in.  When accessing this site I get the error "Can't assign requested address".  Looking up this error at microsoft only caused more confusion.  The site I'm trying to access is "trstockalerts.com".  I would be grateful if someone would explain what the problem and solution is here.
Posted on 2006-05-08 11:39:52 by JimG
JimG,

The attached application connects to a system and preforms an HTTP GET request. It then prints out the raw HTTP reply. When I tried to connect to 'trstockalerts.com' all I get is a Server error message saying the page wasn't found. But all in all this shows how to do a raw http request and I haven't gotten any "Can't assign requested address" errors with it. I've only tested it twice, I coded it specifically for this thread but it works well enough. Btw, it's a console app:


usage: ghttp <hostname>


Regards,
Bryant Keller
Attachments:
Posted on 2006-05-09 01:29:08 by Synfire
Perhaps trstockalerts.com checks the user-agent field - try setting this to some stock IE identification string.
Posted on 2006-05-09 07:07:55 by f0dder
Bryant-
  Thanks.  When I try it I get-

HTTP/1.1 200 OK
Connection: close
Date: Tue, 09 May 2006 14:23:06 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Content-Length: 600
Content-Type: text/html
Expires: Mon, 08 May 2006 21:43:06 GMT
Set-Cookie: ASPSESSIONIDQARSDSBT=LNMOJKMDKDNFABNFPPFKDOMB; path=/
Cache-control: private

<font size=1 face=Verdana,Arial color=white>Col1=hello ... Col2=there == Col1=he
llo111 ... Col2=there111 == Col1=hello222 ... Col2=there222 == Col1=hello333 ...
Col2=there333 == Col1=hello444 ... Col2=there444 == done...S9</font><font size=
2 face=Verdana,Arial><br><b>Error #1500</b><br><br>The page you requested is tem
porarily unavailable.  Please try again.<br><br>If you are the owner of the site
and you feel you are seeing this page in error, <br>email <a href='mailto:Premi
umSupport@Brinkster.com'>PremiumSupport@Brinkster.com</a>.<br><font size=1 face=
Verdana,Arial color=white>0.09375</font>


I tried many times with the same result.


f0dder-

You may be onto something.  I tried a copy of Opera version 7.23 I had laying around, and it wouldn't connect to the site.  No problem with I.E. or Firefox.  I changed user-agent to "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.3) Gecko/20041002 Firefox/0.10" with no luck. 




Posted on 2006-05-09 09:33:52 by JimG
JimG,
The http15.zip example wont work with trstockalerts.com because this site uses DoS attacks protection scheme.
Basically it is an URI redirection (that is GET reply header returned with code status 302). You will need to extract the redirection path form "Location" field in the reply header, make another request with the obtained path, parse for "Set-Cookie" fields, setup all the cookies, parse for "Location" field once again and follow it.
Perhaps an extension for Firefox/IE that monitors site changes will be an easier solution. Implementing from ground all the parsing, cookies managing, etc stuff could be tricky.
Posted on 2006-05-09 11:19:05 by arafel
Thank you arafel.  Look's like I'm going to have to learn more about this stuff than I wanted.  I'll keep hacking away at it.  I'll probably have more questions when I get a little smarter about reply headers and such.
I'll also see if I can modify the "ReloadEvery" extension for Firefox, looks like a good starting point.
Posted on 2006-05-10 08:26:30 by JimG
jimg, install LiveHTTPHeaders on Firefox and you will find the problem.
Posted on 2006-05-15 10:16:14 by mdp