Bugs item #2844077, was opened at 2009-08-25 09:15
Message generated for change (Comment added) made by bagder
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=2844077&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: ftp
Group: wrong behaviour
Status: Open
Resolution: None
Priority: 4
Private: No
Submitted By: Mosh_itup (moshitup)
Assigned to: Daniel Stenberg (bagder)
Summary: libcurl (ftp) don't consider CURLOPT_CONNECTTIMEOUT

Initial Comment:
libcurl using FTP don't consider the CURLOPT_CONNECTTIMEOUT
(Tested under Windows XP, curl Version 7.19.6 and also 7.19.5, other versions I did not tried)

After the curl_easy_perform() it takes CURLOPT_TIMEOUT seconds instead of CURLOPT_CONNECTTIMEOUT.

Example-code illustrating the bug:

      gFTP.hFTP[iServerid-1] = (int)curl_easy_init();
      curl_easy_setopt((void*)gFTP.hFTP[iServerid-1],CURLOPT_URL,"ftp://172.99.99.99"); //TEST! only to simulate an ftp-adress which is not reachable
      curl_easy_setopt((void*)gFTP.hFTP[iServerid-1],CURLOPT_PORT,INTERNET_DEFAULT_FTP_PORT);
      curl_easy_setopt((void*)gFTP.hFTP[iServerid-1],CURLOPT_ERRORBUFFER,sErrorBuf);
      curl_easy_setopt((void*)gFTP.hFTP[iServerid-1],CURLOPT_WRITEFUNCTION, writeFunctionConnect);
      curl_easy_setopt((void*)gFTP.hFTP[iServerid-1],CURLOPT_TIMEOUT,300); //transfer operation timeout
      curl_easy_setopt((void*)gFTP.hFTP[iServerid-1],CURLOPT_CONNECTTIMEOUT,30); //connection timeout
      curl_easy_setopt((void*)gFTP.hFTP[iServerid-1],CURLOPT_FTP_FILEMETHOD,CURLFTPMETHOD_SINGLECWD);
      curlReturn = curl_easy_perform((void*)gFTP.hFTP[iServerid-1]);

I debugged into curl in ftp.c.
My proposal for solution:

static long ftp_state_timeout(struct connectdata *conn)
{
  struct SessionHandle *data=conn->data;
  struct ftp_conn *ftpc = &conn->proto.ftpc;
  long timeout_ms=360000; /* in milliseconds */

  if(data->set.ftp_response_timeout )
    /* if CURLOPT_FTP_RESPONSE_TIMEOUT is set, use that to determine remaining
       time. Also, use ftp->response because FTP_RESPONSE_TIMEOUT is supposed
       to govern the response for any given ftp response, not for the time
       from connect to the given ftp response. */
    timeout_ms = data->set.ftp_response_timeout - /* timeout time */
      Curl_tvdiff(Curl_tvnow(), ftpc->response); /* spent time */
  else if(data->set.timeout)
  {
    /* if timeout is requested, find out how much remaining time we have */
    if(ftpc->state == FTP_WAIT220)
        {
          /* we're during connect */
          if(data->set.timeout < data->set.connecttimeout)
                 timeout_ms = data->set.timeout - Curl_tvdiff(Curl_tvnow(), conn->now); /* spent time */
          else
                 timeout_ms = data->set.connecttimeout - Curl_tvdiff(Curl_tvnow(), conn->now); /* spent time */
        }
        else
        {
          timeout_ms = data->set.timeout - Curl_tvdiff(Curl_tvnow(), conn->now); /* spent time */
        }
  }
  else
    /* Without a requested timeout, we only wait 'response_time' seconds for
       the full response to arrive before we bail out */
    timeout_ms = ftpc->response_time -
      Curl_tvdiff(Curl_tvnow(), ftpc->response); /* spent time */

  return timeout_ms;
}

----------------------------------------------------------------------

>Comment By: Daniel Stenberg (bagder)
Date: 2009-08-31 12:57

Message:
I am perhaps "responsible" for libcurl but it is a fully open source
project and we're all volunteers here. None of us have any particular
company funding or payment. We all scratch our own itches.

The solution you have showed before in this bug report is not a fix for
the problem as you describe it. It will make the entire ftp state machine
use that time-out and that's not acceptable.

As I already described for you, the bug happens ONLY when a TCP connect
is performed successfully. There are several ways you can "connect" to a
site that doesn't exist, but in most normal cases you don't and then also
you don't suffer from this bug. You can easily verify this, you don't need
me to tell you this.

I didn't say you were wrong to report this. I'm telling you I won't work
on this bug anytime soon, and you're wrong when you get all upset and
emotional when I point out factual flaws and advice you to work on problems
you experience.

----------------------------------------------------------------------

Comment By: Mosh_itup (moshitup)
Date: 2009-08-31 12:24

Message:
1st) I thought, YOU are responsible for libcurl.
2nd) I fixed it already (-> see Attached Files) or even this post!
3rd) As far as I know, if somebody fixes a bug, ANOTHER person should also
validate,
and not the one who fixes the bug.
4th) It seems, that there is NO TCP-Connect happening, it is while trying
to connect.
Your log says "connected" but this also seems NOT to be correct.
How can you connect to a server which does not exist? (here it is only to
simulate a server which does not response!)
So if server is not reachable (for any reasons) you ALSO wait 5 minutes.
So this bug is triggerd in that case!
Maybe this is seldom, but how do I know before I do the (ftp)connect?
This can happen to anyone. Or should I make a PING-command first? Just
tell me.
5th) So what made I wrong, reporting you a bug?
6th) If you react like that, people won't give you feedback anymore, so if
this is what you want? I hope not.

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-08-31 11:34

Message:
First, why should *I* take my time? Why don't 'YOU* fix this bug if it
bothers you this much?

If a server is *down* there should be nothing to connect to and then this
bug will not be triggered. This "bug" only occurs after a successful TCP
connect but then with a very slow or stalled FTP server taking over from
that moment. In practise this is rare.

What solution are you referring to that you need verification for?

----------------------------------------------------------------------

Comment By: Mosh_itup (moshitup)
Date: 2009-08-31 11:30

Message:
Ok, bagder, take your time.
The bug can be considered as "not very important" as long as all servers
are online.
But beware, if just one server is down, then any "client-Application"
using libcurl
seems to hang! Cause 300 seconds, there is hardly any sign of life!
But anyway, my proposal for solution you have. I tried it out in my
"client-Application" and it works fine. So we need just somebody, who
verifies this.

Another aspect:
Maybe there is a systematic error, concerning set.connecttimeout in other
protocols (not only in ftp)?

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-08-28 15:50

Message:
Just note that I don't consider this bug very important so it isn't likely
that I personally will work on it anytime soon. I have some 10+ bugs
already that I consider more important to work on before... So if you think
this is an important problem, please consider rolling up your own sleeves
and dig in!

----------------------------------------------------------------------

Comment By: Mosh_itup (moshitup)
Date: 2009-08-28 15:16

Message:
yes, bagder, now you get me right!
This is what I wanted you to say ;-)

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-08-28 14:43

Message:
I think I can guess what's happening.

Your TCP connection is done at this point and the code you look at is the
FTP-specific code that follows after the completion of the TCP handshake.
That phase could of course be seen as part of the connection and thus
subject to that particular timeout, but currently isn't...

To make it adhere to the connect timeout, we need to pass on the fact that
it is still in connect phase into the ftp state machine to allow it to use
a different timeout.

----------------------------------------------------------------------

Comment By: Mosh_itup (moshitup)
Date: 2009-08-28 14:35

Message:
Hi bagder,
well, you should know the code better than I ;-)
But just try it out, and you'll see.
If you set a breakpoint into this function while the ftp-connect is in
progress, you'll see, that there is no reference to the
set.connecttimeout-variable but only to the set.timeout.
Therefore it takes 300 seconds instead of 30.
Afterwards, (the loop for connect has ended), then libcurl "detects" that
a timeout has occured, but then it is too late!
So you are convinced the CONNECTTIMEOUT works already, but what I see is
totally different!
Have a nice day!

----------------------------------------------------------------------

Comment By: Daniel Stenberg (bagder)
Date: 2009-08-27 20:04

Message:
Hm, the fix couldn't be in ftp.c since that code is not responsible for the
initial connect (and its timeout).

I'm quite convinced this works already, and the connection code is
protocol agnostic and thus works the same for all (TCP-based) protocols.

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=2844077&group_id=976
Received on 2009-08-31