cURL / Mailing Lists / curl-and-php / Single Mail

curl-and-php

Cache and Request Header

From: David Colter <dolan2go_at_yahoo.com>
Date: Mon, 4 Feb 2008 17:10:49 -0800 (PST)

Two questions to resolve the same issue.

I am accessing a frame page within a secure site, that also performs a 'screen-scraper' function to get the latest info from another source. ( I haven't figured out the Java yet to bypass this site ) Problem is, when I use a browser to view the page, which has a time stamp of the current time, it's always updated. When I use my script, the time stamp is a day or two previous version I've seen in the (same) browser.

So, I've tried adding CURLOPT_FRESH_CONNECT, TRUE to all but the first request. There are no cookies being stored in my cookie jar and this seems good, since I don't want to reuse an old connection. But I'd like to know how to see what CURL requests from the server. I can see the response and the CURLOPT_HTTPHEADER that the script creates, but not the rest. The reason I ask is that I can't compare my script to Live HTTP Headers. I did add Keep-Alive and Connection to the request array.

Here's a sample of the first 2 page requests from Live HTTP Headers:

https://www.aapilots.com/ps_login.asp

POST /ps_login.asp HTTP/1.1
Host: www.aapilots.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: https://www.aapilots.com/public/flash/login.asp
Cookie: ASPSESSIONIDAAQRTCRT=HJNC.....OCMDNFJ
Content-Type: application/x-www-form-urlencoded
Content-Length: 62
targetURL=...........x=42&y=11

HTTP/1.x 302 Object moved
Date: Mon, 04 Feb 2008 01:34:44 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Location: private/index.asp
Content-Length: 138
Content-Type: text/html
Set-Cookie: AXCOOKIELOGIN=%3C3.....AXDPmVJMjF2BE%2F; path=/
Cache-Control: private
----------------------------------------------------------
https://www.aapilots.com/private/index.asp

GET /private/index.asp HTTP/1.1
Host: www.aapilots.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.12) Gecko/20070508 Firefox/1.5.0.12
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: https://www.aapilots.com/public/flash/login.asp
Cookie: ASPSESSIONIDAAQRTCRT=HJNC.....OCMDNFJ; AXCOOKIELOGIN=%3C.....AXDPmVJMjF2BE%2F

HTTP/1.x 200 OK
Date: Mon, 04 Feb 2008 01:34:44 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Cache-Control: private, private, private, private, private, no-cache
Expires: 0, 0, 0, 0, Sun, 03 Feb 2008 01:34:44 GMT
Pragma: no-cache
Content-Length: 748
Content-Type: text/html

Any suggestions on how to view what is 'requested' or how to get the true latest page? Apologies for this confusing sounding plea for help.

Thanks,
David Colter

      ____________________________________________________________________________________
Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2008-02-05