Bugs item #3215314, was opened at 2011-03-16 12:44
Message generated for change (Comment added) made by mydreamzz
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3215314&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: SCP/SFTP
Group: bad behaviour
>Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Rajesh Naganathan (mydreamzz)
Assigned to: Daniel Stenberg (bagder)
Summary: Post quote operation to rename fails in Windows

Initial Comment:
There is already posting done for this issue(http://curl.haxx.se/mail/lib-2011-02/0322.html) and below is the abstract.

We are using CURLOPT_POSTQUOTE option to rename a file after transfer of the file using sftp protocol.But the POSTQUOTE option for rename is failing when the sftp server is runing in Windows O/S.The same code works perfectly when the sftp server is Linux or Sun OS. Following is the Log from the server side.
  
Complete FTP:
2011-02-23 14:46:31,203 INFO SFTPSubsystemServer [48:temp] Opened /C-drive/.A20110223.0826+0000-XYZ.TBD.0000075BCD15.1 for writing
2011-02-23 14:46:31,844 ERROR SFTPSubsystemServer [48:temp] Failed to move file .A20110223.0826+0000-XYZ.TBD.0000075BCD15.1 => A20110223.0826+0000-XYZ.TBD.0000075BCD15.1: The process cannot access the file because it is being used by another process.
  
Cerebres SFTP :
2011/02/23 15:05:33 [5] File Open command received
2011/02/23 15:05:33 [5] File handle to '//.A20110223.0931+0000-XYZ.TBD.0000075BCD15.1' opened
2011/02/23 15:05:33 [5] Upload started for file 'C:\.A20110223.0931+0000-XYZ.TBD.0000075BCD15.1'
2011/02/23 15:05:34 [5] Unable to rename file 'C:\.A20110223.0931+0000-XYZ.TBD.0000075BCD15.1'
2011/02/23 15:05:34 [5] Upload complete for file 'C:\.A20110223.0931+0000-XYZ.TBD.0000075BCD15.1'
2011/02/23 15:05:34 [5] Handle '//.A20110223.0931+0000-XYZ.TBD.0000075BCD15.1' closed
  
It was suggested that libcurl uses libssh2 to do the SFTP operations and the thing libcurl can do here is to make sure that libssh2_sftp_close() is done on the file properly before it does the POST_QUOTE operations. Since this can be supported by libcurl,it would be better for libcurl to do the libssh2_sftp_close first followed by the execution of the rename operation instead of any server specific implementation.

Currently this works fine in Linux because linux doesnt care about the open files..Suppose Linux also makes sure that the file should be closed to perform the rename operation,the entire post quote operation supported by libcurl will fail on Linux and Windows.

So its better if libcurl could support the behaviour of doing the libssh2_close before doing the postquote operations.

----------------------------------------------------------------------

>Comment By: Rajesh Naganathan (mydreamzz)
Date: 2011-04-13 17:16

Message:
In sftp_done function,the sshc->nextstate is set to SSH_SFTP_CLOSE and
then the state is moved to SSH_SFTP_POSTQUOTE_INIT.But there is no place
before POSTQUOTE_INIT the sftp handle is closed.The fix that is proposed is
to close the handle via libssh2_sftp_close before even entering the
POSTQUOTE_INIT state.We tried to enable debug logs and saw that before
POSTQUOTE_INIT,the sftp handle was never closed using libssh2_sftp_close.If
you feel that its really closed,can u please point to us ?

And regarding the second point ignoring closing of handle during error:
This is the case even in the state machine transition for SSH_SFTP_CLOSE
.Do you suggest if a retry attempt or so should be made?

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2011-04-12 03:23

Message:
Thanks for reporting this issue and helping us improve curl and libcurl.

We're awaiting feedback in this issue. Due to this, I have set the state
of this issue to pending and it will automatically get closed later on
unless we get further info.

Please consider answering the outstanding questions or providing the
missing info so that we can proceed to resolve this issue!

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2011-04-07 04:26

Message:
Well, first we better be sure why the current code doesn't work before we
add more code to do the same thing existing code already attempts to do.

Then, as I already mentioned, the fix you provided doesn't work properly
as it doesn't wait until the function has completed so it very well may not
at all close the handle. It actually could even suggest that what solves
the issue is not the closing at all, but just the little extra time this
operation added between the existing close to the post quote operation.

----------------------------------------------------------------------

Comment By: Rajesh Naganathan (mydreamzz)
Date: 2011-04-06 17:21

Message:
I dont know for sure why the SSH_SFTP_CLOSE didnt cose the handle before
executing postquote.Since the windows server was rejecting the rename
command,we tried to close the handle which will close all file handles in
the server side so that we can start with the POSTQUOTE operation freshly
without any file handles that are in server side.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2011-04-02 14:32

Message:
Thanks for reporting this issue and helping us improve curl and libcurl.

We're awaiting feedback in this issue. Due to this, I have set the state
of this issue to pending and it will automatically get closed later on
unless we get further info.

Please consider answering the outstanding questions or providing the
missing info so that we can proceed to resolve this issue!

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2011-03-22 01:46

Message:
Thanks, but that approach is not working properly.

libssh2_sftp_close() is an operating that very well may return
LIBSSH2_ERROR_EAGAIN due to its non-blocking nature and then the closure
hasn't completed and just discarding the handle at that point seems like a
very weird solution.

Also, I would think that the SSH_SFTP_CLOSE should be run properly before
post-quote operations and that state closes the handle already. Have you
figured out why it doesn't or what we can do to make it do so?

----------------------------------------------------------------------

Comment By: Rajesh Naganathan (mydreamzz)
Date: 2011-03-16 16:41

Message:
I have attached the diff file "mycurlchange.diff" here to show that the
handle can be closed before executing the post quote commands.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2011-03-16 15:18

Message:
Thanks, can you please make a full "proper" diff/patch and attach here?

Here are some hints on how to do it:

http://curl.haxx.se/dev/contribute.html#How_To_Make_a_Patch_without_git

----------------------------------------------------------------------

Comment By: Rajesh Naganathan (mydreamzz)
Date: 2011-03-16 12:49

Message:
Doing the below change in libcurl makes the post quote work in both windows
and linux environment .
 

static CURLcode sftp_done(struct connectdata *conn, CURLcode status,
                               bool premature)
{
  struct ssh_conn *sshc = &conn->proto.sshc;
  int rc;
  struct SessionHandle *data = conn->data;

  if(status == CURLE_OK) {
    /* Before we shut down, see if there are any post-quote commands to
send: */
    if(!status && !premature && conn->data->set.postquote) {

        /* added following part */

        if(sshc->sftp_handle) {
            rc = libssh2_sftp_close(sshc->sftp_handle);

            if(rc == LIBSSH2_ERROR_EAGAIN || rc == 0) {
              sshc->sftp_handle = NULL;
            }
            else if (rc < 0) {
              infof(data, "Failed to close libssh2 file %d\n", rc);
            }
        }
        sshc->nextstate = SSH_SFTP_CLOSE;
        state(conn, SSH_SFTP_POSTQUOTE_INIT);
    }
    else
      state(conn, SSH_SFTP_CLOSE);
  }
  return ssh_done(conn, status);

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3215314&group_id=976
Received on 2011-04-13