Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sftp request with range set to "<file size>-" fails with CURLE_BAD_DOWNLOAD_RESUME #359

Closed
tstack opened this issue Jul 30, 2015 · 4 comments
Closed

sftp request with range set to "<file size>-" fails with CURLE_BAD_DOWNLOAD_RESUME #359

tstack opened this issue Jul 30, 2015 · 4 comments
Assignees
@bagder

Comments

@tstack
Copy link

tstack commented Jul 30, 2015

An SFTP request raises an error if you try to do a request with the range set to the file size instead of returning no content. However, the correct behavior is seen if the file is empty. Here is an example when requesting an empty file:

curl -v -r "0-" sftp://stack@localhost/Users/stack/empty > /dev/null

  • Trying 127.0.0.1...
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed
    0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to localhost (127.0.0.1) port 22 (#0)
  • SSH MD5 fingerprint: 89dcc2863281ded6467103a7a3d69ba1
  • SSH host check: 0, key: AAAAB3NzaC1yc2EAAAABIwAAAQEAs19QFrigpKqUboN8oDhfhqIfZZoqB0Kk9fene3zqivp7bdYwRNV4y6BcEt5vEL42zw0Mo2goMn18luPxSEWg+e7VdBaRL/qrYK6PYevCzvKkqr83zorUq1zeOJNSY+dURCZbR6wIH32Ksf1iiok5EiUXTzdYDnyohGbk1QXGrAVrMqDfrAI0e3L1Y3O55ePHicz38cZtAdj9Zkl4rOn9Ok/4gbLQ4FUtQnqzEAbHNXlWgeDmM1URdc+b6i7F2wbaeHCkbsRLikjWIx5OH8MrAtplRwV6nNd2iXWE5CIOxUXPt8Tt084HeggIdQrkUePTRhtO0IZY35wdkOEZELEYrQ==
  • SSH authentication methods available: publickey,keyboard-interactive
  • Using SSH public key file '(nil)'
  • Using SSH private key file '/Users/stack/.ssh/id_rsa'
  • Initialized SSH public key authentication
  • Authentication complete
    { [0 bytes data]
    0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
    0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
  • Connection #0 to host localhost left intact

So, the request completes successfully with zero bytes. But, downloading a two-byte file fails:

curl -v -r "2-" sftp://stack@localhost/Users/stack/twobytes.txt > /dev/null

  • Trying 127.0.0.1...
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed
    0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to localhost (127.0.0.1) port 22 (#0)
  • SSH MD5 fingerprint: 89dcc2863281ded6467103a7a3d69ba1
  • SSH host check: 0, key: AAAAB3NzaC1yc2EAAAABIwAAAQEAs19QFrigpKqUboN8oDhfhqIfZZoqB0Kk9fene3zqivp7bdYwRNV4y6BcEt5vEL42zw0Mo2goMn18luPxSEWg+e7VdBaRL/qrYK6PYevCzvKkqr83zorUq1zeOJNSY+dURCZbR6wIH32Ksf1iiok5EiUXTzdYDnyohGbk1QXGrAVrMqDfrAI0e3L1Y3O55ePHicz38cZtAdj9Zkl4rOn9Ok/4gbLQ4FUtQnqzEAbHNXlWgeDmM1URdc+b6i7F2wbaeHCkbsRLikjWIx5OH8MrAtplRwV6nNd2iXWE5CIOxUXPt8Tt084HeggIdQrkUePTRhtO0IZY35wdkOEZELEYrQ==
  • SSH authentication methods available: publickey,keyboard-interactive
  • Using SSH public key file '(nil)'
  • Using SSH private key file '/Users/stack/.ssh/id_rsa'
  • Initialized SSH public key authentication
  • Authentication complete
  • Offset (2) was beyond file size (2)
    0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
  • Connection #0 to host localhost left intact

curl: (36) Offset (2) was beyond file size (2)

I'm not actually sure if this is curl (7.43.0) complaining, the SSH server (OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011 on OS X 10.10.4), or libssh2 (1.4.3).

I tried doing similar requests with a couple HTTP sites and they behave inconsistently, unfortunately. For example, github just sends back the whole file again if you set the range to the file size, but dropbox does the "right" thing and sends back zero-length content.

The use-case is trying to tail a log file. I've ended up just downloading the last byte of the file since that seems to work everywhere.
@bagder
Copy link
Member

bagder commented Aug 1, 2015

The fact that HTTP range requests are so shaky, it isn't that easy to say for sure what an SFTP resume should do when asked to resume outside of a file size... Just not returning anything and an OK return code doesn't seem entirely desirable either.

@bagder bagder self-assigned this Aug 1, 2015
@tstack
Copy link
Author

tstack commented Aug 5, 2015

I wouldn't say it's outside the file size, though, it's exactly at the end. This is why I mention the request from '0-' for a zero size file. I do think returning an error for a range of "file_size + 1" is correct, requesting the edge oughta work.

I think the variation in HTTP range requests is more due to the number of HTTP server implementations and it's likely that the edge (literally!) cases are not always going to be handled well.

@bagder
Copy link
Member

bagder commented Aug 6, 2015

Oh right!! Looks like an off-by-one error in the file size check. Does this patch make anything different?

diff --git a/lib/ssh.c b/lib/ssh.c
index 7b0e57c..94195a7 100644
--- a/lib/ssh.c
+++ b/lib/ssh.c
@@ -2142,11 +2142,11 @@ static CURLcode ssh_statemach_act(struct connectdata *conn, bool *block)
           }
           if(from < 0) {
             /* from is relative to end of file */
             from += size;
           }
-          if(from >= size) {
+          if(from > size) {
             failf(data, "Offset (%"
                   CURL_FORMAT_CURL_OFF_T ") was beyond file size (%"
                   CURL_FORMAT_CURL_OFF_T ")", from, attrs.filesize);
             return CURLE_BAD_DOWNLOAD_RESUME;
           }

@tstack
Copy link
Author

tstack commented Aug 10, 2015

That patch makes it work. With range "2-" it returns success with no content and "3-" returns an error. Thanks!

@bagder bagder closed this as completed in ade6682 Aug 10, 2015
jgsogo pushed a commit to jgsogo/curl that referenced this issue Oct 19, 2015
@bagder @jgsogo
SFTP: fix range request off-by-one in size check
c58a116 
Reported-by: Tim Stack

Closes curl#359
@lock lock bot locked as resolved and limited conversation to collaborators May 7, 2018
bagder pushed a commit that referenced this issue Dec 11, 2018
@mkauf @bagder
http: don't set CURLINFO_CONDITION_UNMET for http status code 204
c8bf8cc 
The http status code 204 (No Content) should not change the "condition
unmet" flag. Only the http status code 304 (Not Modified) should do
this.

Closes #359
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@bagder bagder

Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tstack @bagder