Bugs item #3285747, was opened at 2011-04-13 13:49
Message generated for change (Comment added) made by bagder
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3285747&group_id=976

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: ftp
Group: hang
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: michal lev (michalev1)
Assigned to: Daniel Stenberg (bagder)
Summary: 100% CPU when FTP server is heavy load

Initial Comment:
Hi,

I'm having a multi threaded application written in c++ that uses curl lib version 7.19.4.
When the FTP server is on heavy load, some of the threads of my application are waiting to the FTP response for the timeout period I defined in CURLOPT_FTP_RESPONSE_TIMEOUT. But during this time of waiting these threads consume lots of CPU and as a result my application consumes most of the machine's CPU.

While digging in the CURL code I can see that these threads are in the "ftp_easy_statemach" function inside the while loop, and they are stuck in this loop until the response timeout is over.
The state of the ftp connection is "FTP_WAIT220"

During the while loop the rc is always 1 and the result is CURLE_OK .
 
on ftp.c:
static CURLcode ftp_easy_statemach(struct connectdata *conn)
{
  curl_socket_t sock = conn->sock[FIRSTSOCKET];
  int rc;
  struct SessionHandle *data=conn->data;
  struct ftp_conn *ftpc = &conn->proto.ftpc;
  CURLcode result = CURLE_OK;

  while(ftpc->state != FTP_STOP) {
    long timeout_ms = ftp_state_timeout(conn);

    if(timeout_ms <=0 ) {
      failf(data, "FTP response timeout");
      return CURLE_OPERATION_TIMEDOUT; /* already too little time */
    }

    rc = Curl_socket_ready(ftpc->sendleft?CURL_SOCKET_BAD:sock, /* reading */
                           ftpc->sendleft?sock:CURL_SOCKET_BAD, /* writing */
                           (int)timeout_ms);

    if(rc == -1) {
      failf(data, "select/poll error");
      return CURLE_OUT_OF_MEMORY;
    }
    else if(rc == 0) {
      result = CURLE_OPERATION_TIMEDOUT;
      break;
    }
    else {
      result = ftp_statemach_act(conn);
      if(result)
        break;
    }
  }

  return result;
}

Is it a known issue?
Is there a workaround ?

I saw this open bug , but it didn't talk about consuming 100% CPU until we get to the timeout.
73. if a connection is made to a FTP server but the server then just never
  sends the 220 response or otherwise is dead slow, libcurl will not
  acknowledge the connection timeout during that phase but only the "real"
  timeout - which may surprise users as it is probably considered to be the
  connect phase to most people. Brought up (and is being misunderstood) in:
  http://curl.haxx.se/bug/view.cgi?id=2844077

Please advise.

Thanks

----------------------------------------------------------------------

>Comment By: Daniel Stenberg (bagder)
Date: 2011-04-17 14:13

Message:
No, it's not a known issue and it is not the known bug #73.

I read the code now and I'm trying to understan how ftp_statemach_act()
can be called many times while being in the FTP_WAIT220 state. Can you?

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=100976&aid=3285747&group_id=976
Received on 2011-04-17