cURL / Mailing Lists / curl-and-php / Single Mail

curl-and-php

Re: Tor is not http proxy error with curl script.

From: Stephen Pynenburg <spynenburg_at_gmail.com>
Date: Thu, 5 Mar 2009 21:56:13 -0500

I would imagine that means what it says - you can't use Tor as an HTTP
proxy. If possible, try using something else (I think Apache can be
configured as a proxy) or I'm sure there are other free proxy server
programs.
-Stephen

2009/3/4 yaris dong <yaris684_at_gmail.com>

> heres teh code
>
> <?php
> # Define how your webbot will appear in server logs
> define("WEBBOT_NAME", "USER-AGENT");
>
> # Length of time cURL will wait for a response (seconds)
> define("CURL_TIMEOUT", 25);
>
> # Location of your cookie file. (Must be fully resolved local address)
> define("COOKIE_FILE", "cookie.txt");
>
> # DEFINE METHOD CONSTANTS
> define("HEAD", "HEAD");
> define("GET", "GET");
> define("POST", "POST");
>
> # DEFINE HEADER INCLUSION
> define("EXCL_HEAD", FALSE);
> define("INCL_HEAD", TRUE);
>
> /***********************************************************************
> User interfaces
> -------------------------------------------------------------
> */
>
> /***********************************************************************
> function http_get($target, $ref)
> -------------------------------------------------------------
> DESCRIPTION:
> Downloads an ASCII file without the http header
> INPUT:
> $target The target file (to download)
> $ref The server referer variable
> OUTPUT:
> $return_array['FILE'] = Contents of fetched file, will also
> include the HTTP header if requested
> $return_array['STATUS'] = CURL generated status of transfer
> $return_array['ERROR'] = CURL generated error status
> ***********************************************************************/
> function http_get($target, $ref)
> {
> return http($target, $ref, $method="GET", $data_array="", EXCL_HEAD);
> }
>
> /***********************************************************************
> http_get_withheader($target, $ref)
> -------------------------------------------------------------
> DESCRIPTION:
> Downloads an ASCII file with the http header
> INPUT:
> $target The target file (to download)
> $ref The server referer variable
> OUTPUT:
> $return_array['FILE'] = Contents of fetched file, will also
> include the HTTP header if requested
> $return_array['STATUS'] = CURL generated status of transfer
> $return_array['ERROR'] = CURL generated error status
> ***********************************************************************/
> function http_get_withheader($target, $ref)
> {
> return http($target, $ref, $method="GET", $data_array="", INCL_HEAD);
> }
>
> /***********************************************************************
> http_get_form($target, $ref, $data_array)
> -------------------------------------------------------------
> DESCRIPTION:
> Submits a form with the GET method and downloads the page
> (without a http header) referenced by the form's ACTION variable
> INPUT:
> $target The target file (to download)
> $ref The server referer variable
> $data_array An array that defines the form variables
> (See "Webbots, Spiders, and Screen Scrapers" for
> more information about $data_array)
> OUTPUT:
> $return_array['FILE'] = Contents of fetched file, will also
> include the HTTP header if requested
> $return_array['STATUS'] = CURL generated status of transfer
> $return_array['ERROR'] = CURL generated error status
> ***********************************************************************/
> function http_get_form($target, $ref, $data_array)
> {
> return http($target, $ref, $method="GET", $data_array, EXCL_HEAD);
> }
>
> /***********************************************************************
> http_get_form_withheader($target, $ref, $data_array)
> -------------------------------------------------------------
> DESCRIPTION:
> Submits a form with the GET method and downloads the page
> (with http header) referenced by the form's ACTION variable
> INPUT:
> $target The target file (to download)
> $ref The server referer variable
> $data_array An array that defines the form variables
> (See "Webbots, Spiders, and Screen Scrapers" for
> more information about $data_array)
> OUTPUT:
> $return_array['FILE'] = Contents of fetched file, will also
> include the HTTP header if requested
> $return_array['STATUS'] = CURL generated status of transfer
> $return_array['ERROR'] = CURL generated error status
> ***********************************************************************/
> function http_get_form_withheader($target, $ref, $data_array)
> {
> return http($target, $ref, $method="GET", $data_array, INCL_HEAD);
> }
>
> /***********************************************************************
> http_post_form($target, $ref, $data_array)
> -------------------------------------------------------------
> DESCRIPTION:
> Submits a form with the POST method and downloads the page
> (without http header) referenced by the form's ACTION variable
> INPUT:
> $target The target file (to download)
> $ref The server referer variable
> $data_array An array that defines the form variables
> (See "Webbots, Spiders, and Screen Scrapers" for
> more information about $data_array)
> OUTPUT:
> $return_array['FILE'] = Contents of fetched file, will also
> include the HTTP header if requested
> $return_array['STATUS'] = CURL generated status of transfer
> $return_array['ERROR'] = CURL generated error status
> ***********************************************************************/
> function http_post_form($target, $ref, $data_array)
> {
> return http($target, $ref, $method="POST", $data_array, EXCL_HEAD);
> }
>
> function http_post_withheader($target, $ref, $data_array)
> {
> return http($target, $ref, $method="POST", $data_array, INCL_HEAD);
> }
>
> function http_header($target, $ref)
> {
> return http($target, $ref, $method="HEAD", $data_array="", INCL_HEAD);
> }
>
> /***********************************************************************
> http($url, $ref, $method, $data_array, $incl_head)
> -------------------------------------------------------------
> DESCRIPTION:
> This function returns a web page (HTML only) for a web page through
> the execution of a simple HTTP GET request.
> All HTTP redirects are automatically followed.
>
> IT IS BEST TO USE ONE THE EARLIER DEFINED USER INTERFACES
> FOR THIS FUNCTION
>
> INPUTS:
> $target Address of the target web site
>
> $ref Address of the target web site's referrer
> $method Defines request HTTP method; HEAD, GET or POST
> $data_array A keyed array, containing query string
> $incl_head TRUE = include http header
> FALSE = don't include http header
>
> RETURNS:
> $return_array['FILE'] = Contents of fetched file, will also
> include the HTTP header if requested
> $return_array['STATUS'] = CURL generated status of transfer
> $return_array['ERROR'] = CURL generated error status
> ***********************************************************************/
> function http($target, $ref, $method, $data_array, $incl_head)
> {
> # Initialize PHP/CURL handle
> $ch = curl_init();
>
> # Prcess data, if presented
> if(is_array($data_array))
> {
> # Convert data array into a query string (ie
> animal=dog&sport=baseball)
> foreach ($data_array as $key => $value)
> {
> if(strlen(trim($value))>0)
> $temp_string[] = $key . "=" . urlencode($value);
> else
> $temp_string[] = $key;
> }
> $query_string = join('&', $temp_string);
> }
>
> # HEAD method configuration
> if($method == HEAD)
> {
> curl_setopt($ch, CURLOPT_HEADER, TRUE); // No http
> head
> curl_setopt($ch, CURLOPT_NOBODY, TRUE); // Return
> body
> }
> else
> {
> # GET method configuration
> if($method == GET)
> {
> if(isset($query_string))
> $target = $target . "?" . $query_string;
> curl_setopt ($ch, CURLOPT_HTTPGET, TRUE);
> curl_setopt ($ch, CURLOPT_POST, FALSE);
> }
> # POST method configuration
> if($method == POST)
> {
> if(isset($query_string))
> curl_setopt ($ch, CURLOPT_POSTFIELDS, $query_string);
> curl_setopt ($ch, CURLOPT_POST, TRUE);
> curl_setopt ($ch, CURLOPT_HTTPGET, FALSE);
> }
> curl_setopt($ch, CURLOPT_HEADER, $incl_head); // Include head as
> needed
> curl_setopt($ch, CURLOPT_NOBODY, FALSE); // Return body
> }
>
> curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE); // Cookie
> management.
> curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
> curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
> curl_setopt($ch, CURLOPT_PROXY, "127.0.0.1:9050"); // ENABLE TOR
> curl_setopt($ch, CURLOPT_TIMEOUT, CURL_TIMEOUT); // Timeout
> curl_setopt($ch, CURLOPT_USERAGENT, WEBBOT_NAME); // Webbot name
> curl_setopt($ch, CURLOPT_URL, $target); // Target site
> curl_setopt($ch, CURLOPT_REFERER, $ref); // Referer value
> curl_setopt($ch, CURLOPT_VERBOSE, TRUE); // Minimize logs
> curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // No certificate
> curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects
> curl_setopt($ch, CURLOPT_MAXREDIRS, 4); // Limit
> redirections to four
> curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return in string
>
> # Create return array
> $return_array['FILE'] = curl_exec($ch);
> $return_array['STATUS'] = curl_getinfo($ch);
> $return_array['ERROR'] = curl_error($ch);
>
> # Close PHP/CURL handle
> curl_close($ch);
>
> # Return results
> return $return_array;
> }
> ?>
>
>
>
> On Wed, Mar 4, 2009 at 1:44 AM, yaris dong <yaris684_at_gmail.com> wrote:
>
>> okay so i have a small curl script that just downloads a page.
>>
>> it was working great suddenly today,
>>
>> it throws Tor is not http proxy.
>>
>> i have enabled HTTPTUNNEL. i have enabled SOCKTYPE
>>
>> no avail.
>>
>> what to do ?
>>
>
>
> _______________________________________________
> http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
>
>

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2009-03-06