curl / Mailing Lists / curl-users / Single Mail

curl-users

Need some help about large data from web

From: <super_big_hero0_at_sina.com>
Date: Thu, 10 Nov 2016 13:12:26 +0800

Hi all
I have a C program which run to get the data from a web API.
And when the web return a large json data, the curl can’t deal with it.
Every time the program return (core dump) error, and I want to know how to deal with the indefinite length data use the libcurl use C.
And my code like this:
size_t write_data(void *ptr, long unsigned int size, long unsigned int nmemb, void *stream)
{

    strcat(stream, (char *)ptr);
    return size*nmemb;
}

char *down_file(char *url_address)
{
    static unsigned char str[64000] = {""};
    char *p2s;

     /* set the download address */
    curl_easy_setopt(curl_handle, CURLOPT_URL, url_address);
     /* set the timeout */
    curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 100);
    /* set the writedata function */
    curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);
    /* set the wirte variable */
    curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, str);
    /* set the user-agent field */
    curl_easy_setopt(curl_handle, CURLOPT_USERAGENT, "libcurl-agent/1.0");

    /* download */
    res = curl_easy_perform(curl_handle);
    //str[MAX_LINE-1] = "\0";
    if(res != CURLE_OK)
    {
        fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
        return NULL; //judge the download if successful
    }
    else
    {
    printf("%lu bytes retrieved\n", strlen(str));
    }
    return str; }

THX
Jay

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ: https://curl.haxx.se/docs/faq.html
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2016-11-10