cURL / Mailing Lists / curl-users / Single Mail

curl-users

Using curl to download CBOE csv file quotes...help!

From: Yves PETRONIN <y.petronin_at_wanadoo.fr>
Date: Sat, 29 Jan 2011 09:59:33 +0100

I am very far from being an expert. I have hovever become familiar
with such difficulties. I have found most useful to use the "Form"
optin from Firefox, as well as the capabilities of the software to
record the live http headers.
In your case, the apparently simple submission of a ticker correspond
to the following form

<input
id="ctl00_ctl00_AllContent_ContentMain_QuoteTableDownloadCtl1_txtTicker"
  name="ctl00$ctl00$AllContent$ContentMain$QuoteTableDownloadCtl1
$txtTicker" size="10" maxlength="20">

Using the capabilities to record the live transaction made by
Firefox, when submitting the ticker "AAPL", I have several
activities related to cookies, as usual, and the following :

POST /DelayedQuote/QuoteTableDownload.aspx
__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=%
2FwEPDwUKLTc3NTgxOTcwNw9kFgJmD2QWAmYPZBYCAgUPZBYCAgMPZBYEAgUPZBYCAgEPZBY
CAgcPZBYIAgEPZBYCAgIPZBYKAggPFgIeBFRleHQFgAI8SUZSQU1FIHNyYz0iaHR0cDovL2F
kLmRvdWJsZWNsaWNrLm5ldC9hZGkvcXVvdGUuY2JvZS5kYXJ0L3F1b3RlL3F1b3RldGFibGV
kb3dubG9hZDtzej0xMjB4NjA7dGlsZT0yO29yZD0nICsgb3JkICsgJyIgd2lkdGg9IjEyMCI
gaGVpZ2h0PSI2MCIgdmFsaW
......
...... many line omitted
.......

kY3RsMDAkbHZFZHVGZWF0dXJlZEV2ZW50DzwrAAoCBzwrAAQACAIEZCKnBY8jkhgzLm5rm2I
pVY41%2BYEo&__EVENTVALIDATION=%2FwEWBwKJpfylBgLnn5gNAtXW%2F%
2BYLApSZ7dMOArre%2Ba0MArm0noIMAui9nNYP7PowP1dq4P4JaIlPHUsY%2BhoF%2FBg%
3D&ctl00%24ctl00%24AllContent%24ucHeader%24ucCBOEHeaderQuoteBox%
24txtHeaderQuote=Quote&ctl00%24ctl00%24AllContent%24ucHeader%
24CBOEHeaderSearchBox%24txtHeaderSearch=Search&ctl00%24ctl00%
24AllContent%24ContentMain%24QuoteTableDownloadCtl1%
24txtTicker=AAPL&ctl00%24ctl00%24AllContent%24ContentMain%
24QuoteTableDownloadCtl1%24cmdSubmit=Download

I suggest to explore this type of route/tests ... It looks like that
a very simple command will not suffice to mimic all this......and
that perhaps the site is designed in such a way that it is made very
difficult to use other tools than a browser....

Rgds

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-01-29