cURL
Haxx ad
libcurl

curl's project page on SourceForge.net

Sponsors:
Haxx

cURL > Mailing List > Monthly Index > Single Mail

curl-and-php mailing list Archives

Re: curl not scraping Microsoft.com (something wrong?)

From: Daniel Stenberg <daniel-curl_at_haxx.se>
Date: Thu, 14 Apr 2005 16:20:56 +0200 (CEST)

On Thu, 14 Apr 2005 spamiam_at_aroint.org wrote:

> Perhaps I missed where on your website it explains precisely *what* needs to
> be copied from the headers

You should make sure that all headers your curl usage produce look like the
ones you see in the header capture. With as few exceptions as possible.

> and *how* to do it in curl in a step-by-step fashion.

Well, this is a start:

         http://curl.haxx.se/docs/httpscripting.html

And more libcurl-oriented tuturial stuff is here:

         http://curl.haxx.se/libcurl/c/libcurl-tutorial.html

But I'm not a PHP guy so don't expect PHP-related docs from my end.

> I'm not expecting a cookie cutter solution, but it shouldn't be some
> mystical process, either.

It is far from "mystical". You send a HTTP request and headers, and you get
back HTTP headers and a response. It is very simple.

All info about them is found in RFC2616.

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html
Received on 2005-04-14

These mail archives are generated by hypermail.

donate! Page updated November 12, 2010.
web site info

File upload with ASP.NET