cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: SFTP download of > 2GB

From: Julien Chaffraix <julien.chaffraix_at_gmail.com>
Date: Sat, 2 Oct 2010 19:15:44 -0700

>> I found several signed <-> unsigned conversion (usually int to size_t) in
>> the security.c code which motivated its rewrite (I fixed some and I tried to
>> flag the remaining conversions). For some reason, the compiler will not
>> output any warnings for these. Such a conversion would not explain why only
>> _some_ servers would show that - that is unless not all the code branches
>> are doing it.
>
> Right, and that's what is most puzzling to me. Unless of course the bug is
> in a specific libssh2 version that the reporter is using - I've only
> personally tried it with the most recent version + the current git.
>
> I kind of get the feeling the problem is in the libssh2 implementation, but
> the code that gets the size from remote is really miniscule.

I have this feeling too as the check you added in ssh.c should not be
hit on linux for such a small value (IIRC curl_off_t is 8 bytes on any
linux platform). It does not mean we are not doing something wrong
though. Looking at this line (ssh.c:1914), there is several flaws that
could bite us later:

curl_off_t size = attrs.filesize;

curl_off_t is a *signed* 64-bit value which can fall back to a 32-bit
value (according to curlbuild.h.dist comments). This means that we can
overflow regardless on the size of curl_off_t on this line (filesize
is an unsigned 64-bit value).

Also the branch containing the check is not always taken - which means
we do not always check for overflow.

>> Also does libcurl+openSSH work in this case?
>
> We've focused on comparing SFTP with libcurl+libssh2 and with the OpenSSH
> tool 'sftp', and as we've found cases where it works with 'sftp' but not
> with libcurl+libssh2 I've been satisfied with that and worked towards trying
> to find out more details exactly when the libcurl+libssh2 combo gets the
> problem.
>
> We don't really have a "libcurl+openSSH" combo, as libcurl only does SFTP
> when powered by libssh2.

OK, I was thinking of any other back-end for SFTP to narrow down the
issue to libssh2 & curl-specific code or escalate to curl.

Thanks,
Julien
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2010-10-03