cURL / Mailing Lists / curl-library / Single Mail

curl-library

Saving data received from http server

From: arnuld uttre <arnuld.mizong_at_gmail.com>
Date: Fri, 12 Nov 2010 17:28:57 +0530

WANTED: save the incoming data from http server.
WHAT I GOT: It works
PROBLEM: A dirty brute-force approach used to save data. Have
explained the problem in comments inside the program. Anyone has
better idea ?

I have written a simple program to receieve data from any HTTP server
(learning to use libcurl in the compnay where I work). I can get the
data easily using this small program written by me. It outputs the
data to stdout (default in libcurl). (VERSION 1). For saving data I
wrote VERSION 2. comment/uncomment to test both compilable & running
versions.

#include <stdio.h>
#include <curl/curl.h>

size_t save_data(void* buffer, size_t size, size_t nmemb, void* incoming_data);

enum {
  SIZE_ERROR_ARR = CURL_ERROR_SIZE * 4
};

int main(void)
{
  CURL* curl;
  CURLcode res;
  char error_arr[SIZE_ERROR_ARR+1] = {0};

  curl = curl_easy_init();
  if(curl)
    {
      /* Use one VERSION by commenting the other */

      /* VERSION 1 : Outputs to stdout */
      curl_easy_setopt(curl, CURLOPT_URL, "http://google.com");
      res = curl_easy_perform(curl);

      /* VERSION 2 : Saves to array
      curl_easy_setopt(curl, CURLOPT_URL, "http://google.com");
      curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, save_data);
      curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, error_arr);
      res = curl_easy_perform(curl);
      */

      if(res)
        {
          printf("IN: %s @%d: Some error occured while curl_easy_perform(),
ERRORBUFFER said: \n", __FILE__, __LINE__);
          printf("%s\n", error_arr);
        }

      /* always cleanup */
      curl_easy_cleanup(curl);
    }

  return 0;
}

size_t save_data(void* buffer, size_t size, size_t nmemb, void* incoming_data)
{
  size_t idx;
  const size_t END_INDEX = size * nmemb;
  char* src = (char*) buffer;

  char dest[100001] = {0};
  /* How to know the size of array in advance.
      char dest[END_INDEX + 1] = {0}; will not work as variable-size
arrays are not allowed in ANSI standard. But using variable-size
      is good solution to the problem ???
      malloc() will be expensive for 10 million http sends/receive
waitings to happen.
      So what to do ???
  */

  for(idx = 0; idx < END_INDEX; ++idx)
    {
      dest[idx] = *src++;
    }

  printf("--------------------------------------\n");
  /* I wonder what 4th argument is used for when it contains garbage :-o */
  printf("incoming_data = %s\n", (char*)incoming_data);
  printf("dest = %s\n", dest);
  printf("--------------------------------------\n");
  return END_INDEX;
}
================ OUTPUT ========================
[arnuld_at_dune Module-cURL]$ gcc -ansi -pedantic -Wall -Wextra
curl-simple.c -lcurl
[arnuld_at_dune Module-cURL]$ ./a.out
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
[arnuld_at_dune Module-cURL]$

-- 
http://uttre.wordpress.com/2008/05/14/the-lost-love-of-mine/
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2010-11-12