Here is a user asking the same as what I want:

http://curl.haxx.se/mail/archive-2007-09/0040.html

On Mon, Nov 10, 2008 at 7:27 AM, Jo Anonymous <jojoanon_at_gmail.com> wrote:

> You are right about the redirect.
>
> I just realised, the actual file I am getting is from a redirect after the
> login.
>
> What I meant by prompts is that when I login, I am then redirected from
> their javascript redirect code to the file.
> When this happens, download dialog box pops up.
>
> What I want to do is just download that file automatically to my server
> without any user interaction.
> But now that I see the page is loaded from a redirect, I am wondering if it
> even possible to download the file.
>
> Jojo
>
>
> On Fri, Nov 7, 2008 at 12:43 AM, Thorben Thuermer <
> t.thuermer_at_mediaclipping.de> wrote:
>
>> On Thu, 6 Nov 2008 08:34:42 +1300 "Jo Anonymous" <jojoanon_at_gmail.com>
>> wrote:
>> > Cheers for your replies..
>> > Have tried both methods with no luck.
>> >
>> > Thorben, tried your method but it also didn't work for me, the results
>> said:
>> >
>> > there was no no filename in the headers... Failed writing body
>>
>> well, either the format of the header is different and the regular
>> expression
>> does not match, or there is some kind of other problem... (see below)
>>
>> > The problem I think is that the actual file itself does not exist on the
>> > server. It is just generated, so bascially all I need is a way to bypass
>> the
>> > download prompt and have it somehow let me download without any user
>> > interaction.
>>
>> it does, as far as curl (or for that matter http) is concerened, not make
>> any
>> difference at all if the file is static or dynamically generated.
>>
>> > Will give it another go, any other suggestions are greatly appreciated.
>>
>> also, would you __PLEASE__ __CLARIFY__ what you mean by "download the
>> file"???
>> do you want to store it in a file on your server?
>> do you even care about the name it would be saved as by a browser???
>> or do you maybe want to pass it through to the caller of your php script
>> (which is Stephen now assumes in his response).
>>
>> also, "download to your server without any prompts" does not make any
>> sense
>> at all, because curl (or php for that matter) would have no way to (even
>> if it
>> wanted to!) display such a dialog like a browser would.
>> the browser shows the dialog because the server asks it to (using the
>> Content-Dispostion header), which a non-browser client (curl, wget, ...)
>> will simply ignore.
>>
>>
>> from your current responses, it seems most likely that the server
>> you are requesting the file from sends you a different response when you
>> request the file with curl than when you request it with a browser.
>>
>> this could be due to some authentication scheme (do you need to be logged
>> in?
>> cookies?, maybe limited by IP?),
>> or maybe some user-agent or similar check that is meant to prevent exactly
>> what
>> you are doing?
>>
>> another possibility would be that the server does not actually return data
>> with a
>> content-disposition, but returnsn a redirect to another url (maybe a
>> static file).
>>
>> in the end, the only (and easy!) way to find out is to find out what
>> response
>> curl actually gets from the server (statuscode, headers, and body),
>> instead of
>> just saying that it "doesn't work", while you are not even specifying what
>> your
>> intended result (=definition of "works") is.
>>
>> and before you ask, here is some code to do just that.
>> please run it with your url, and post the output
>> (at least the headers).
>>
>> <?
>> $handle=curl_init("
>> http://blog.ninedays.org/demo/direct-to-download-images-and-pdfs/download.image.php
>> ");
>> curl_setopt($handle,CURLOPT_HEADER,1);
>> curl_exec($handle);
>> curl_close($handle);
>> ?>
>>
>> - Thorben
>>
>> > On Wed, Nov 5, 2008 at 12:01 AM, Thorben Thuermer <
>> > t.thuermer_at_mediaclipping.de> wrote:
>> > > this is actually scarily related to the unanswered question i signed
>> up to
>> > > this
>> > > list for...
>> > >
>> > > proper solution without downloading the file twice:
>> > > (but also using global variables)
>> > >
>> > > <?
>> > > // these should really be stored inside the curl context
>> > > global $filename;
>> > > global $outfile;
>> > >
>> > > function header_cb($handle,$h){
>> > > global $filename;
>> > > if(preg_match("/^Content-Disposition:
>> > > .*filename\=([^\\n|\\r]+)[\\n\\r]/", $h, $m)){
>> > > $filename=$m[1];
>> > > echo "got filename: ".$filename."\n";
>> > > }
>> > > return strlen($h);
>> > > }
>> > >
>> > > function data_cb($handle,$d){
>> > > global $filename;
>> > > global $outfile;
>> > > if (!$outfile){
>> > > $status=curl_getinfo($handle,CURLINFO_HTTP_CODE);
>> > > if ($status != 200){
>> > > echo "bad http status\n";
>> > > return -1;
>> > > }
>> > > if ($filename == null){
>> > > echo "there was no no filename in the headers...\n";
>> > > return -1;
>> > > }
>> > > $outfile=fopen($filename,"w");
>> > > }
>> > > return fwrite($outfile,$d);
>> > > };
>> > >
>> > > // testcase urls found via google:
>> > > $handle=curl_init("
>> > >
>> http://blog.ninedays.org/demo/direct-to-download-images-and-pdfs/download.image.php
>> > > ");
>> > > //$handle=curl_init("
>> > >
>> http://blog.ninedays.org/demo/direct-to-download-images-and-pdfs/original.jpg
>> > > ");
>> > >
>> > > curl_setopt($handle,CURLOPT_WRITEFUNCTION,"data_cb");
>> > > curl_setopt($handle,CURLOPT_HEADERFUNCTION,"header_cb");
>> > > $filename=null;
>> > > if (!curl_exec($handle)){
>> > > echo curl_error($handle)."\n";
>> > > }
>> > > if ($outfile) fclose($outfile);
>> > > curl_close($handle);
>> > >
>> > > ?>
>> > >
>> > > - Thorben
>> > >
>> > > On Mon, 3 Nov 2008 17:44:01 -0500
>> > > "Stephen Pynenburg" <spynenburg_at_gmail.com> wrote:
>> > > > There's probably a way to do it in one request, but I think you
>> could
>> > > just
>> > > > as easily do this:
>> > > > <?PHP
>> > > > $ch = curl_init();
>> > > > curl_setopt($ch, CURLOPT_URL, "site/page.php");
>> > > > curl_setopt($ch, CURLOPT_HEADER, 1);
>> > > > curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
>> > > > $res = curl_exec($ch);
>> > > > preg_match("/filename\=(.*)\\n/", $res, $matches);
>> > > > echo $matches[1]; //the filename
>> > > > ?>
>> > > >
>> > > > Then do another request with the filename gleaned from that request.
>> > > >
>> > > > -Stephen
>> > > >
>> > > > 2008/11/3 Jo Anonymous <jojoanon_at_gmail.com>
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > I am trying to download a csv file which is generated from the
>> url.
>> > > > >
>> > > > > I can login to the website using curl succesfully but I can't
>> download
>> > > the
>> > > > > file I want.
>> > > > >
>> > > > > The download url looks something like this:
>> > > > >
>> > > > > http://www.websitename.com/file.aspx?type=all&download=1
>> > > > >
>> > > > > The website has this line in the header:
>> > > > >
>> > > > > header("Content-Disposition: attachment; filename=dada.csv");
>> > > > >
>> > > > > which brings up a download prompt.
>> > > > >
>> > > > > I just want to download the csv file to my server without any
>> prompts.
>> > > > >
>> > > > > I've been trying for ages to figure this out, there must be a way.
>> > > > >
>> > > > > Many thanks in advance.
>> > > > >
>>
>> _______________________________________________
>> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>>
>
>

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2008-11-09