Here's an update. First of all, accessing www.usfsaonline.org with
the following curl command line statement:

curl -b cookie.txt --trace-ascii www.usfsaonline.org -Lk
www.usfsaonline.org

seems to work fine.
what is happening with my php/curl program though is that it seems to
be generating calls to load gif images but with the wrong url.
e.g.:

http://atlantaopen.info/media/navs/OnlineEventReg2.gif

where is in actual fact that image resides somewhere on the
usfsaonline.org site and not at atlantaopen.info at all.
furthermore, the source at usfsaonline.org calls for those images in
an included php function that to my way of thinking shouldn't even be
executing.
i mean, it isn't executing with the command line curl statement, so
what is it about my php/curl program that's causing this behaviour?

at the very least, why isn't the php/curl program keeping up with the
change in url when it went to the new location directed to by the 302
it encounters when first accessing the usfsaonline.org site?

or in short, how can I coax my php/curl program to behave the same as
my curl command line statement?

On Jan 17, 2008, at 4:04 PM, hugo wrote:

> i've tried runnning it without the proxy on my home server with the
> same result.
> i left the proxy in for the atlantaopen.info/usfsa.php (which is
> hosted by godaddy) so that folks could test it themselves if they
> wanted - and on godaddy it won't work at all without the proxy in.
>
> On Jan 17, 2008, at 3:40 PM, Richard Lynch wrote:
>
>> Perhaps the proxy is pre-fetching and caching the embedded content to
>> make your Internet experience "faster"?...
>>
>> Not sure why it would do so only when using cURL and not when using
>> FF...
>>
>> Could you take the proxy out of the picture temporarily to test?
>>
>> On Thu, January 17, 2008 1:26 pm, hugo wrote:
>>> i'm a newbie, and i'm having a difficult time trying to figure out
>>> the behaviour of my php curl script when accessing a particular
>>> site.
>>>
>>> here is my program:
>>>
>>> <?
>>> $cookiefile = "cookie.txt";
>>> $proxy = "proxy.shr.secureserver.net:3128";
>>> $url = "https://www.usfsaonline.org";
>>>
>>> $ch = curl_init();
>>> curl_setopt($ch, CURLOPT_URL, $url );
>>> curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
>>> curl_setopt($ch, CURLOPT_PROXY,$proxy);
>>> curl_setopt ($ch, CURLOPT_HTTPPROXYTUNNEL, true);
>>> curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiefile );
>>> curl_setopt($ch, CURLOPT_HEADER, 1);
>>> curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
>>> curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false );
>>> $result =curl_exec ($ch);
>>> echo $result;
>>> ?>
>>>
>>> now when i log into the usfsaonline.org site with my firefox browser
>>> Live HTTP Headers gives me the following:
>>>
>>>> http://www.usfsaonline.org/
>>>>
>>>> GET / HTTP/1.1
>>>> Host: www.usfsaonline.org
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: text/xml,application/xml,application/xhtml+xml,text/
>>>> html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Cookie: ASPSESSIONIDAABBSTQD=IMBFJMJDEPDCFGBGEOJGOAPF
>>>>
>>>> HTTP/1.x 302 Object moved
>>>> Date: Thu, 17 Jan 2008 19:10:30 GMT
>>>> Server: Microsoft-IIS/6.0
>>>> X-Powered-By: ASP.NET
>>>> Location: https://www.usfsaonline.org/IntraLock/Login.asp?
>>>> targetpath=%2FDefault%2Easp
>>>> Content-Length: 196
>>>> Content-Type: text/html
>>>> Cache-Control: private
>>>> ----------------------------------------------------------
>>>>
>>>> https://www.usfsaonline.org/IntraLock/Login.asp?targetpath=%
>>>> 2FDefault%2Easp
>>>>
>>>> GET /IntraLock/Login.asp?targetpath=%2FDefault%2Easp HTTP/1.1
>>>> Host: www.usfsaonline.org
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: text/xml,application/xml,application/xhtml+xml,text/
>>>> html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Cookie: ASPSESSIONIDAABBSTQD=IMBFJMJDEPDCFGBGEOJGOAPF
>>>>
>>>>
>>>> HTTP/1.x 200 OK
>>>> Date: Thu, 17 Jan 2008 19:10:31 GMT
>>>> Server: Microsoft-IIS/6.0
>>>> X-Powered-By: ASP.NET
>>>> Content-Length: 53442
>>>> Content-Type: text/html
>>>> Cache-Control: private
>>>>
>>>> ----------------------------------------------------------
>>>>
>>>
>>> but when i execute my php script through the web page it is on
>>> ( http://atlantaopen.info/usfsa.php ) Live HTTP Headers comes back
>>> with a lot of 404 errors that reflect GETs that i never asked for or
>>> issued and that are actually requests for information that is in the
>>> page source of the www.usfsaonline.org site, and not in my code at
>>> all.
>>> This is driving me crazy... does anyone know what i'm doing wrong?
>>>
>>> ( here is truncated output from Live HTTP Headers for my php curl
>>> script )
>>>
>>>
>>>> http://atlantaopen.info/usfsa.php
>>>>
>>>> GET /usfsa.php HTTP/1.1
>>>> Host: atlantaopen.info
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: text/xml,application/xml,application/xhtml+xml,text/
>>>> html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Cookie: 2c9086f9455d6335b54958f99919592a=j2k8cgeaj4c0cihnru7pu0oj54
>>>>
>>>> HTTP/1.x 200 OK
>>>> Date: Thu, 17 Jan 2008 19:17:04 GMT
>>>> Server: Apache
>>>> Keep-Alive: timeout=15, max=100
>>>> Connection: Keep-Alive
>>>> Transfer-Encoding: chunked
>>>> Content-Type: text/html
>>>> ----------------------------------------------------------
>>>> http://atlantaopen.info/media/navs/OnlineEventReg2.gif
>>>>
>>>> GET /media/navs/OnlineEventReg2.gif HTTP/1.1
>>>> Host: atlantaopen.info
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Referer: http://atlantaopen.info/usfsa.php
>>>> Cookie: 2c9086f9455d6335b54958f99919592a=j2k8cgeaj4c0cihnru7pu0oj54
>>>>
>>>>
>>>> HTTP/1.x 404 Not Found
>>>> Date: Thu, 17 Jan 2008 19:17:08 GMT
>>>> Server: Apache
>>>> Keep-Alive: timeout=15, max=99
>>>> Connection: Keep-Alive
>>>> Transfer-Encoding: chunked
>>>> Content-Type: text/html
>>>> ----------------------------------------------------------
>>>>
>>>> http://atlantaopen.info/media/navs/OnlineEventReg1.gif
>>>>
>>>> GET /media/navs/OnlineEventReg1.gif HTTP/1.1
>>>> Host: atlantaopen.info
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Referer: http://atlantaopen.info/usfsa.php
>>>> Cookie: 2c9086f9455d6335b54958f99919592a=j2k8cgeaj4c0cihnru7pu0oj54
>>>>
>>>> HTTP/1.x 404 Not Found
>>>> Date: Thu, 17 Jan 2008 19:17:09 GMT
>>>> Server: Apache
>>>> Keep-Alive: timeout=15, max=98
>>>> Connection: Keep-Alive
>>>> Transfer-Encoding: chunked
>>>> Content-Type: text/html
>>>> ----------------------------------------------------------
>>>> http://atlantaopen.info/media/spacer.gif
>>>>
>>>> GET /media/spacer.gif HTTP/1.1
>>>> Host: atlantaopen.info
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Referer: http://atlantaopen.info/usfsa.php
>>>> Cookie: 2c9086f9455d6335b54958f99919592a=j2k8cgeaj4c0cihnru7pu0oj54
>>>>
>>>> HTTP/1.x 404 Not Found
>>>> Date: Thu, 17 Jan 2008 19:17:09 GMT
>>>> Server: Apache
>>>> Keep-Alive: timeout=15, max=98
>>>> Connection: Keep-Alive
>>>> Transfer-Encoding: chunked
>>>> Content-Type: text/html
>>>> ----------------------------------------------------------
>>>> http://atlantaopen.info/media/shield_top.gif
>>>>
>>>> GET /media/shield_top.gif HTTP/1.1
>>>> Host: atlantaopen.info
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Referer: http://atlantaopen.info/usfsa.php
>>>> Cookie: 2c9086f9455d6335b54958f99919592a=j2k8cgeaj4c0cihnru7pu0oj54
>>>>
>>>> HTTP/1.x 404 Not Found
>>>> Date: Thu, 17 Jan 2008 19:17:09 GMT
>>>> Server: Apache
>>>> Keep-Alive: timeout=15, max=97
>>>> Connection: Keep-Alive
>>>> Transfer-Encoding: chunked
>>>> Content-Type: text/html
>>>> ----------------------------------------------------------
>>>> http://atlantaopen.info/media/USFSA.gif
>>>>
>>>> GET /media/USFSA.gif HTTP/1.1
>>>> Host: atlantaopen.info
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Referer: http://atlantaopen.info/usfsa.php
>>>> Cookie: 2c9086f9455d6335b54958f99919592a=j2k8cgeaj4c0cihnru7pu0oj54
>>>>
>>>> HTTP/1.x 404 Not Found
>>>> Date: Thu, 17 Jan 2008 19:17:09 GMT
>>>> Server: Apache
>>>> Keep-Alive: timeout=15, max=97
>>>> Connection: Keep-Alive
>>>> Transfer-Encoding: chunked
>>>> Content-Type: text/html
>>>>
>>>> ----------------------------------------------------------
>>>> http://atlantaopen.info/media/newLogoUSFSA.gif
>>>>
>>>> GET /media/newLogoUSFSA.gif HTTP/1.1
>>>> Host: atlantaopen.info
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Referer: http://atlantaopen.info/usfsa.php
>>>> Cookie: 2c9086f9455d6335b54958f99919592a=j2k8cgeaj4c0cihnru7pu0oj54
>>>>
>>>> HTTP/1.x 404 Not Found
>>>> Date: Thu, 17 Jan 2008 19:17:09 GMT
>>>> Server: Apache
>>>> Keep-Alive: timeout=15, max=96
>>>> Connection: Keep-Alive
>>>> Transfer-Encoding: chunked
>>>> Content-Type: text/html
>>>> ----------------------------------------------------------
>>>> http://atlantaopen.info/media/navs/Competitions2.gif
>>>>
>>>> GET /media/navs/Competitions2.gif HTTP/1.1
>>>> Host: atlantaopen.info
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Referer: http://atlantaopen.info/usfsa.php
>>>> Cookie: 2c9086f9455d6335b54958f99919592a=j2k8cgeaj4c0cihnru7pu0oj54
>>>>
>>>> HTTP/1.x 404 Not Found
>>>> Date: Thu, 17 Jan 2008 19:17:09 GMT
>>>> Server: Apache
>>>> Keep-Alive: timeout=15, max=96
>>>> Connection: Keep-Alive
>>>> Transfer-Encoding: chunked
>>>> Content-Type: text/html
>>>> ----------------------------------------------------------
>>>>
>>>
>>>
>>> ........... etc. etc. etc ...........
>>>
>>>
>>>> ----------------------------------------------------------
>>>> http://atlantaopen.info/media/navs/viewUpload.gif
>>>>
>>>> GET /media/navs/viewUpload.gif HTTP/1.1
>>>> Host: atlantaopen.info
>>>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:
>>>> 1.8.1.11) Gecko/20071127 Firefox/2.0.0.11
>>>> Accept: image/png,*/*;q=0.5
>>>> Accept-Language: en-us,en;q=0.5
>>>> Accept-Encoding: gzip,deflate
>>>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>>>> Keep-Alive: 300
>>>> Connection: keep-alive
>>>> Referer: http://atlantaopen.info/usfsa.php
>>>> Cookie: 2c9086f9455d6335b54958f99919592a=j2k8cgeaj4c0cihnru7pu0oj54
>>>>
>>>> HTTP/1.x 404 Not Found
>>>> Date: Thu, 17 Jan 2008 19:17:14 GMT
>>>> Server: Apache
>>>> Keep-Alive: timeout=15, max=95
>>>> Connection: Keep-Alive
>>>> Transfer-Encoding: chunked
>>>> Content-Type: text/html
>>>> ----------------------------------------------------------
>>>>
>>>>
>>> _______________________________________________
>>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>>>
>>
>>
>> --
>> Some people have a "gift" link here.
>> Know what I want?
>> I want you to buy a CD from some indie artist.
>> http://cdbaby.com/from/lynch
>> Yeah, I get a buck. So?
>>
>> _______________________________________________
>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2008-01-18