cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: sftp download issue (Daniel Stenberg)

From: Sumukh Anantha Manohar <am.sumukh_at_gmail.com>
Date: Thu, 25 Aug 2011 12:19:10 +0530

Hi,
  As i had posted earlier,

> Do you always get this same error (and byte amount) no matter how large
file
> you get? If not, do you get the same error and byte amount for the same
file
> if you retry the exact same transfer?

> Eh, so does it work for _any_ sftp files at all? If so, which?

Even if i try to download a 2MB file with sftp protocol, sometimes it will
download completely and sometimes it will show the error : **curl: (18)
transfer closed with 574 bytes
  remaining to read*. So, this problem is arising for smaller file sizes
also (sometimes). No matter how large the file is , i will get the same
error.

Libraries: libssh2-1.2.9
I am using Curl-7.21.7 package to generate the libcurl on the 64-bit solaris
machine, here in the file lib/sendf.c, there is a piece of code having:

/* If session can pipeline, check connection buffer */
  if(pipelining) {
    size_t bytestocopy = CURLMIN(conn->buf_len - conn->read_pos,
                                 sizerequested);

    /* Copy from our master buffer first if we have some unread data there*/
    if(bytestocopy > 0) {
      memcpy(buf, conn->master_buffer + conn->read_pos, bytestocopy);
      conn->read_pos += bytestocopy;
      conn->bits.stream_was_rewound = FALSE;

      *n = (ssize_t)bytestocopy;
      return CURLE_OK;
    }
    /* If we come here, it means that there is no data to read from the
buffer,
     * so we read from the socket */
    bytesfromsocket = CURLMIN(sizerequested, BUFSIZE * sizeof (char));
    buffertofill = conn->master_buffer;
  }
  else {
    bytesfromsocket = CURLMIN((long)sizerequested,
                              conn->data->set.buffer_size ?
                              conn->data->set.buffer_size : BUFSIZE);
    /* * bytesfromsocket = 10000;* */
   /* *OR* */
   /* * bytesfromsocket = 16384(=BUFSIZE);* */
    buffertofill = buf;
  }

See the else part, where bytesfromsocket is assigned with some values,
normally it will have value like 16384 bytes each time, if the remaining
file size to be downloaded is large. Once the remaining file size to be
downloaded is between 16384 and 10000, bytesfromsocket will be equal to
sizerequested (less than 16384), then i am getting the error as .*curl: (18)
transfer closed with 4456 bytes remaining to read*.
So, i changed bytesfromsocket to a constant value 10000 or to
16384(BUFSIZE) which i have commented, and again rebuilded the library
libcurl.so.4, After doing this, it was perfectly downloading for all the
file sizes. Want to know what i have changed is right or not?

Thanks

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2011-08-25