cURL / Mailing Lists / curl-and-php / Single Mail

curl-and-php

Re: Tor is not http proxy error with curl script.

From: yaris dong <yaris684_at_gmail.com>
Date: Wed, 4 Mar 2009 13:52:47 -0800

heres teh code

<?php
# Define how your webbot will appear in server logs
define("WEBBOT_NAME", "USER-AGENT");

# Length of time cURL will wait for a response (seconds)
define("CURL_TIMEOUT", 25);

# Location of your cookie file. (Must be fully resolved local address)
define("COOKIE_FILE", "cookie.txt");

# DEFINE METHOD CONSTANTS
define("HEAD", "HEAD");
define("GET", "GET");
define("POST", "POST");

# DEFINE HEADER INCLUSION
define("EXCL_HEAD", FALSE);
define("INCL_HEAD", TRUE);

/***********************************************************************
User interfaces
-------------------------------------------------------------
*/

/***********************************************************************
function http_get($target, $ref)
-------------------------------------------------------------
DESCRIPTION:
        Downloads an ASCII file without the http header
INPUT:
        $target The target file (to download)
        $ref The server referer variable
OUTPUT:
        $return_array['FILE'] = Contents of fetched file, will also
                                 include the HTTP header if requested
        $return_array['STATUS'] = CURL generated status of transfer
        $return_array['ERROR'] = CURL generated error status
***********************************************************************/
function http_get($target, $ref)
    {
    return http($target, $ref, $method="GET", $data_array="", EXCL_HEAD);
    }

/***********************************************************************
http_get_withheader($target, $ref)
-------------------------------------------------------------
DESCRIPTION:
        Downloads an ASCII file with the http header
INPUT:
        $target The target file (to download)
        $ref The server referer variable
OUTPUT:
        $return_array['FILE'] = Contents of fetched file, will also
                                 include the HTTP header if requested
        $return_array['STATUS'] = CURL generated status of transfer
        $return_array['ERROR'] = CURL generated error status
***********************************************************************/
function http_get_withheader($target, $ref)
    {
    return http($target, $ref, $method="GET", $data_array="", INCL_HEAD);
    }

/***********************************************************************
http_get_form($target, $ref, $data_array)
-------------------------------------------------------------
DESCRIPTION:
        Submits a form with the GET method and downloads the page
        (without a http header) referenced by the form's ACTION variable
INPUT:
        $target The target file (to download)
        $ref The server referer variable
        $data_array An array that defines the form variables
                      (See "Webbots, Spiders, and Screen Scrapers" for
                      more information about $data_array)
OUTPUT:
        $return_array['FILE'] = Contents of fetched file, will also
                                 include the HTTP header if requested
        $return_array['STATUS'] = CURL generated status of transfer
        $return_array['ERROR'] = CURL generated error status
***********************************************************************/
function http_get_form($target, $ref, $data_array)
    {
    return http($target, $ref, $method="GET", $data_array, EXCL_HEAD);
    }

/***********************************************************************
http_get_form_withheader($target, $ref, $data_array)
-------------------------------------------------------------
DESCRIPTION:
        Submits a form with the GET method and downloads the page
        (with http header) referenced by the form's ACTION variable
INPUT:
        $target The target file (to download)
        $ref The server referer variable
        $data_array An array that defines the form variables
                      (See "Webbots, Spiders, and Screen Scrapers" for
                      more information about $data_array)
OUTPUT:
        $return_array['FILE'] = Contents of fetched file, will also
                                 include the HTTP header if requested
        $return_array['STATUS'] = CURL generated status of transfer
        $return_array['ERROR'] = CURL generated error status
***********************************************************************/
function http_get_form_withheader($target, $ref, $data_array)
    {
    return http($target, $ref, $method="GET", $data_array, INCL_HEAD);
    }

/***********************************************************************
http_post_form($target, $ref, $data_array)
-------------------------------------------------------------
DESCRIPTION:
        Submits a form with the POST method and downloads the page
        (without http header) referenced by the form's ACTION variable
INPUT:
        $target The target file (to download)
        $ref The server referer variable
        $data_array An array that defines the form variables
                      (See "Webbots, Spiders, and Screen Scrapers" for
                      more information about $data_array)
OUTPUT:
        $return_array['FILE'] = Contents of fetched file, will also
                                 include the HTTP header if requested
        $return_array['STATUS'] = CURL generated status of transfer
        $return_array['ERROR'] = CURL generated error status
***********************************************************************/
function http_post_form($target, $ref, $data_array)
    {
    return http($target, $ref, $method="POST", $data_array, EXCL_HEAD);
    }

function http_post_withheader($target, $ref, $data_array)
    {
    return http($target, $ref, $method="POST", $data_array, INCL_HEAD);
    }

function http_header($target, $ref)
    {
    return http($target, $ref, $method="HEAD", $data_array="", INCL_HEAD);
    }

/***********************************************************************
http($url, $ref, $method, $data_array, $incl_head)
-------------------------------------------------------------
DESCRIPTION:
    This function returns a web page (HTML only) for a web page through
    the execution of a simple HTTP GET request.
    All HTTP redirects are automatically followed.

    IT IS BEST TO USE ONE THE EARLIER DEFINED USER INTERFACES
    FOR THIS FUNCTION

INPUTS:
    $target Address of the target web site
    $ref Address of the target web site's referrer
    $method Defines request HTTP method; HEAD, GET or POST
    $data_array A keyed array, containing query string
    $incl_head TRUE = include http header
                 FALSE = don't include http header

RETURNS:
    $return_array['FILE'] = Contents of fetched file, will also
                              include the HTTP header if requested
    $return_array['STATUS'] = CURL generated status of transfer
    $return_array['ERROR'] = CURL generated error status
***********************************************************************/
function http($target, $ref, $method, $data_array, $incl_head)
    {
    # Initialize PHP/CURL handle
    $ch = curl_init();

    # Prcess data, if presented
    if(is_array($data_array))
        {
        # Convert data array into a query string (ie
animal=dog&sport=baseball)
        foreach ($data_array as $key => $value)
            {
            if(strlen(trim($value))>0)
                $temp_string[] = $key . "=" . urlencode($value);
            else
                $temp_string[] = $key;
            }
        $query_string = join('&', $temp_string);
        }

    # HEAD method configuration
    if($method == HEAD)
        {
        curl_setopt($ch, CURLOPT_HEADER, TRUE); // No http
head
        curl_setopt($ch, CURLOPT_NOBODY, TRUE); // Return
body
        }
    else
        {
        # GET method configuration
        if($method == GET)
            {
            if(isset($query_string))
                $target = $target . "?" . $query_string;
            curl_setopt ($ch, CURLOPT_HTTPGET, TRUE);
            curl_setopt ($ch, CURLOPT_POST, FALSE);
            }
        # POST method configuration
        if($method == POST)
            {
            if(isset($query_string))
                curl_setopt ($ch, CURLOPT_POSTFIELDS, $query_string);
            curl_setopt ($ch, CURLOPT_POST, TRUE);
            curl_setopt ($ch, CURLOPT_HTTPGET, FALSE);
            }
        curl_setopt($ch, CURLOPT_HEADER, $incl_head); // Include head as
needed
        curl_setopt($ch, CURLOPT_NOBODY, FALSE); // Return body
        }

    curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE); // Cookie
management.
    curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
    curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
    curl_setopt($ch, CURLOPT_PROXY, "127.0.0.1:9050"); // ENABLE TOR
    curl_setopt($ch, CURLOPT_TIMEOUT, CURL_TIMEOUT); // Timeout
    curl_setopt($ch, CURLOPT_USERAGENT, WEBBOT_NAME); // Webbot name
    curl_setopt($ch, CURLOPT_URL, $target); // Target site
    curl_setopt($ch, CURLOPT_REFERER, $ref); // Referer value
    curl_setopt($ch, CURLOPT_VERBOSE, TRUE); // Minimize logs
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // No certificate
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects
    curl_setopt($ch, CURLOPT_MAXREDIRS, 4); // Limit
redirections to four
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return in string

    # Create return array
    $return_array['FILE'] = curl_exec($ch);
    $return_array['STATUS'] = curl_getinfo($ch);
    $return_array['ERROR'] = curl_error($ch);

    # Close PHP/CURL handle
      curl_close($ch);

    # Return results
      return $return_array;
    }
?>

On Wed, Mar 4, 2009 at 1:44 AM, yaris dong <yaris684_at_gmail.com> wrote:

> okay so i have a small curl script that just downloads a page.
>
> it was working great suddenly today,
>
> it throws Tor is not http proxy.
>
> i have enabled HTTPTUNNEL. i have enabled SOCKTYPE
>
> no avail.
>
> what to do ?
>

_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php
Received on 2009-03-04