cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: Facing issue while fetching large amount of file list using curl.

From: Daniel Stenberg <daniel_at_haxx.se>
Date: Tue, 22 Oct 2013 20:14:24 +0200 (CEST)

On Tue, 22 Oct 2013, Heet Kansagra - Quipment India wrote:

> // Call back function for get file list
> static size_t getFileListCallBackFunction (void *buffer, size_t size, size_t nmemb,
> void *stream)
> {
> printf(" buffer: %s",(char*)buffer);
> return size*nmemb;

The 'buffer' is not necessarily zero terminated so using printf() like that is
not guaranteed to work!

> But if I am trying to get file list from the particular folder with having
> around 4000+ numbers of files, it is getting file names up to around 1199
> files but after that "get file list call back function" get blocked for some
> amount of time and then it gives error, "Could not open remote file for
> reading: Unknown error in libssh2 :: -30".
>
> Please give me some hint where things are going wrong or any solution for
> this issue.

This sounds like a libssh2 problem. Which version of libssh2 are you using and
on what platform? And which libcurl version?

If not the latest libssh2 version, please try that first and see if the
problem remains there.

I think it would be beneficial to try to write up a small libssh2-using
program that lists that directory to see if we can repeat the problem then and
if so continue the debugging over on the libssh2-devel mailing list.

-- 
  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Received on 2013-10-22